IssueTracker

Using ddmlib 25.3.0-beta3, when a test class is marked as ignored ddmlib calls ITestRunListener#testStarted twice. Once when processing the START code and once when processing the IGNORED code.

With the following test in a test APK:

    @Ignore class ExampleTest {
      @Test public void empty() {}
    }

Invoking instrumentation with:

    IDevice device = // ...
    RemoteAndroidTestRunner runner =
        new RemoteAndroidTestRunner("com.example.test",
            "android.support.test.runner.AndroidJUnitRunner", device);
    runner.addInstrumentationArg("log", "true");

and then calling run() with a listener that logs:

    runner.run(new ITestRunListener() {
      public void testStarted(TestIdentifier test) {
        new Throwable().printStackTrace(System.out);
        System.out.println("[testStarted] test: " + test);
      }

      public void testIgnored(TestIdentifier test) {
        System.out.println("[testIgnored] test: " + test);
      }

      public void testEnded(TestIdentifier test, Map<String, String> testMetrics) {
        System.out.println("[testEnded] test: " + test + ", testMetrics: " + testMetrics);
      }

      // other callbacks...
    });

produces:

    [testStarted] test: com.example.ExampleTest#null
    java.lang.Throwable
        at com.example.Test$1.testStarted(Test.java:33)
        at com.android.ddmlib.testrunner.InstrumentationResultParser.reportResult(InstrumentationResultParser.java:455)
        at com.android.ddmlib.testrunner.InstrumentationResultParser.parseStatusCode(InstrumentationResultParser.java:415)
        at com.android.ddmlib.testrunner.InstrumentationResultParser.parse(InstrumentationResultParser.java:276)
        at com.android.ddmlib.testrunner.InstrumentationResultParser.processNewLines(InstrumentationResultParser.java:247)
        at com.android.ddmlib.MultiLineReceiver.addOutput(MultiLineReceiver.java:100)
        at com.android.ddmlib.AdbHelper.executeRemoteCommand(AdbHelper.java:519)
        at com.android.ddmlib.AdbHelper.executeRemoteCommand(AdbHelper.java:382)
        at com.android.ddmlib.Device.executeShellCommand(Device.java:617)
        at com.android.ddmlib.testrunner.RemoteAndroidTestRunner.run(RemoteAndroidTestRunner.java:259)
        at com.android.ddmlib.testrunner.RemoteAndroidTestRunner.run(RemoteAndroidTestRunner.java:245)
        at com.example.Test.main(Test.java:26)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.intellij.rt.execution.application.AppMain.main(AppMain.java:143)
    [testStarted] test: com.example.ExampleTest#null
    java.lang.Throwable
        at com.example.Test$1.testStarted(Test.java:33)
        at com.android.ddmlib.testrunner.InstrumentationResultParser.reportResult(InstrumentationResultParser.java:479)
        at com.android.ddmlib.testrunner.InstrumentationResultParser.parseStatusCode(InstrumentationResultParser.java:415)
        at com.android.ddmlib.testrunner.InstrumentationResultParser.parse(InstrumentationResultParser.java:276)
        at com.android.ddmlib.testrunner.InstrumentationResultParser.processNewLines(InstrumentationResultParser.java:247)
        at com.android.ddmlib.MultiLineReceiver.addOutput(MultiLineReceiver.java:100)
        at com.android.ddmlib.AdbHelper.executeRemoteCommand(AdbHelper.java:519)
        at com.android.ddmlib.AdbHelper.executeRemoteCommand(AdbHelper.java:382)
        at com.android.ddmlib.Device.executeShellCommand(Device.java:617)
        at com.android.ddmlib.testrunner.RemoteAndroidTestRunner.run(RemoteAndroidTestRunner.java:259)
        at com.android.ddmlib.testrunner.RemoteAndroidTestRunner.run(RemoteAndroidTestRunner.java:245)
        at com.example.Test.main(Test.java:26)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.intellij.rt.execution.application.AppMain.main(AppMain.java:143)
    [testIgnored] test: com.example.ExampleTest#null
    [testEnded] test: com.example.ExampleTest#null, testMetrics: {}

As you can tell from the stack traces, the method is being called from line 455 the first time and 479 the second time. Looking at the source of InstrumentationResultParser we see:

    453:     case StatusCodes.START:
    454:         for (ITestRunListener listener : mTestListeners) {
    455:             listener.testStarted(testId);
    456:         }
    457:         break;

and:

    476:     case StatusCodes.IGNORED:
    477:          metrics = getAndResetTestMetrics();
    478:          for (ITestRunListener listener : mTestListeners) {
    479:              listener.testStarted(testId);
    480:              listener.testIgnored(testId);
    481:              listener.testEnded(testId, metrics);
    482:          }

This occurs at least on API 25 and 21 so it's not new behavior. It seems like ddmlib should omit calling testStarted in the IGNORED case branch.