When I run android tests with version 1.0.0 of com.android.support.test:runner following exception is thrown
E/TestExecutor: Fatal exception when running tests
java.lang.NoClassDefFoundError: Failed resolution of: Landroid/support/v4/app/DialogFragment;
at java.lang.Class.classForName(Native Method)
at java.lang.Class.forName(Class.java:400)
at android.support.test.internal.runner.TestLoader.doCreateRunner(TestLoader.java:76)
at android.support.test.internal.runner.TestLoader.getRunnersFor(TestLoader.java:104)
at android.support.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:808)
at android.support.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:481)
at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:367)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1962)
Caused by: java.lang.ClassNotFoundException: Didn't find class "android.support.v4.app.DialogFragment" on path: DexPathList[[zip file "/system/framework/android.test.runner.jar", zip file "/data/app/com.extension.runnercrashapp.test-2/base.apk"],nativeLibraryDirectories=[/data/app/com.extension.runnercrashapp.test-2/lib/arm64, /system/lib64, /vendor/lib64]]
at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
at java.lang.ClassLoader.loadClass(ClassLoader.java:380)
at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
at java.lang.Class.classForName(Native Method)
at java.lang.Class.forName(Class.java:400)
at android.support.test.internal.runner.TestLoader.doCreateRunner(TestLoader.java:76)
at android.support.test.internal.runner.TestLoader.getRunnersFor(TestLoader.java:104)
at android.support.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:808)
at android.support.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:481)
at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:367)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1962)

it happens only if module from gradle dependency is excluded, for example:
implementation ("com.google.android.gms:play-services-analytics:11.0.4") {
exclude group: 'com.android.support', module: 'support-v4'
}

Tested on devices with Android 5 and 7.
With version 0.5 of com.android.support.test:runner it wasn't happening.

To reproduce, run android tests from android studio or gradle connectedCheck
Small example project is available here:
https://github.com/czekoladowyPaczek/RunnerCrashApp