IssueTracker

The link in the above description doesn't work external to google, expanded link: https://issuetracker.google.com/issues/73450636

This will be fixed in the next release.

any updates here?

I'm also waiting for any updates on this

me too

Nick, is there a way to concatenate all .ec files from individual runs? Alternatively, we could look into passing a directory.

We can definitely collect all files in one dir and pass it back. Not sure about concatenating or merging the .ec files, will need to have a closer look into it.

Guys, do you find any workaround for that? is there any updates from Google side?

Hi is there any update to this issue?

Yes, we're working on resolving this issue for next release.

I added the ability for Orchestrator to generate a coverage file per test. Reassigning this bug to the Studio/Gradle team to utilize this feature when pulling the coverage files from the device.

If I understand correctly, the fix will be included in the next gradle plugin version.
Is that right?

This fix will be included in the next release of Orchestrator for sure.

The Android Studio/Gradle team and the Firebase team will need to adopt this fix in order to make the integration with their respective tools work. I will leave it up to them to provide an estimate on when that's going to be ready.

Using version 1.0.2 of the orchestrator crashes every time if code coverage is enabled:
Sample https://github.com/suarezjulian/unified-code-coverage-android

./gradlew jacocoTestReport
Starting a Gradle Daemon, 1 incompatible Daemon could not be reused, use --status for details
Parallel execution is an incubating feature.
Starting 0 tests on Pixel 2 - 8.1.0
Tests on Pixel 2 - 8.1.0 failed: Instrumentation run failed due to 'Process crashed.'

com.android.builder.testing.ConnectedDevice > No tests found.[Pixel 2 - 8.1.0] FAILED
No tests found. This usually means that your test classes are not in the form that your test runner expects (e.g. don't inherit from TestCase or lack @Test annotations).

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:connectedDebugAndroidTest'.
> There were failing tests. See the report at: {}/unified-code-coverage-android/app/build/reports/androidTests/connected/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 25s

Test orchestrator 1.0.2 does not work yet with the Android Gradle plugin. Fix is underway, and I will update the bug once it has been merged.

The fix should be release in Android Gradle plugin 3.2.0-alpha17.

Fixed in ag/Ib7377ad308b266fb79c753a859c4bf75d43e029f

It seems that in the AndroidTestOrchestrator.java the method:

---------------
static String addTestCoverageSupport(Bundle bundle, String str) {
        if (!shouldRunCoverage(bundle) || !runsInIsolatedMode(bundle)) {
            return null;
        }
        String string = bundle.getString("coverageFilePath");
        if (string == null || string.isEmpty()) {
            throw new IllegalStateException(String.format("The coverage path [%s] is either null or empty", new Object[]{string}));
        } else if (bundle.getString("coverageFile") == null) {
            return new StringBuilder((String.valueOf(string).length() + 3) + String.valueOf(str).length()).append(string).append(str).append(".ec").toString();
        } else {
            throw new IllegalStateException(String.format("Can't use a custom coverage file name [-e %s %s] when running through orchestrator in isolated mode. Since the generated coverage files will overwrite each other. Please consider using [%s] instead.", new Object[]{"coverageFile", bundle.getString("coverageFile"), "coverageFilePath"}));
        }
    }
---------------

Throws an ISE exception if it does not find the coverageFilePath. Is this caused by the fact that no such parameter is specified when executing:

---------------
V/ddms: execute: running CLASSPATH=$(pm path android.support.test.services) app_process / android.support.test.services.shellexecutor.ShellMain am instrument -r -w -e targetInstrumentation com.jpmorgan.am.ipbonline.instr.test/android.support.test.runner.AndroidJUnitRunner   -e coverageFile /data/data/com.jpmorgan.am.ipbonline.instr/coverage.ec -e coverage true -e clearPackageData true android.support.test.orchestrator/android.support.test.orchestrator.AndroidTestOrchestrator
---------------

I just want to confirm if this is the cause of the process crash 'Instrumentation run failed due to 'Process crashed'.'

That's correct, but we already fixed that behaviour in the alpha2 release (androidx.test:runner:1.1.0-alpha1) to be backwards compatible.

Thanks to Google team, JaCoCo is now working with Orchestrator perfectly. (N.B used: gradle: 3.2.0-alpha17, test:runner: 1.0.2)

I had an android project that used orchestrator 1.0.1 and it worked fine in the test case. However in the code coverage it did not generate the right result.
So I changed the orchestrator to 1.0.2 (did not make any changes in the test cases). Gradle can sync but when I tried to generate the report with "createDebugAndroidTestCoverageReport" task, it said "com.android.builder.testing.ConnectedDevice > No tests found"
So in short, if I used an older version of orchestrator, the report returned wrong result
If i used a new version of orchestrator, it has error.

Does anyone have the similar issue? Is that orchestrator 1.0.2 still a little bit buggy or I missed something?

Thanks.

Re #21: I think this might be https://issuetracker.google.com/78108767.

Re #22: The problem seems similar but it does not solve my problem. The project you provided did not even use orchestrator.
I cannot find a fix yet. Any other suggestions?

Thanks.

Re #23: Can you please upload a sample project that reproduces the issue? It looks like the test runner cannot find the test classes in the test APK.

I cannot show you the code of my main project but this is a test project that mimics the same problem in my main project.
If I use orchestrator:1.0.1 (line 112), I can run the test cases with the tasks "createProductionDebugCoverageReport" (./gradlew createProductionDebugCoverageReport).
If I use orchestrator:1.0.2 without changing anything in the test cases, it throws me an error as I mentioned before.

Let me know if you need more information.

As #17 says, you need to use the Android Gradle plugin 3.2.0-alpha17 or higher. This fix is not ported to 3.1 versions of the plugin. Updating your sample project to use 3.2.0-beta04 fixes the issue.

AHHH.. my bad!! I thought it was talking about the gradle version.

It fixed this problem but had a different problem. I will open a new post if I don't find a fix for that.

Thanks man.

Hi
I have used: gradle: 3.2.0-alpha17, test:runner: 1.0.2
distributionUrl=https\://services.gradle.org/distributions/gradle-4.6-all.zip
testInstrumentationRunnerArguments clearPackageData: 'true'
 execution 'ANDROID_TEST_ORCHESTRATOR'
androidTestUtil 'com.android.support.test:orchestrator:1.0.1'

Jacaoco setup -
task jacocoTestReport(type: JacocoReport, dependsOn: ['testDebugUnitTest', 'createDebugCoverageReport']) {

    reports {
        xml.enabled = true
        html.enabled = true
    }
 fileFilter ="***"
 def debugTree = fileTree(dir: "${buildDir}/intermediates/classes/debug", excludes: fileFilter)
    def mainSrc = "${project.projectDir}/src/main/java"

    sourceDirectories = files([mainSrc])
    classDirectories = files([debugTree])
    executionData = fileTree(dir: "$buildDir", includes: [
            "jacoco/testDebugUnitTest.exec",
            "outputs/code-coverage/connected/*coverage.ec"
----

allprojects {
    apply plugin: "jacoco"
    jacoco {
        toolVersion = "0.8.0"
    }
-----
 dependencies {
        classpath 'com.android.tools.build:gradle:3.2.0-alpha17'
        classpath "org.jacoco:org.jacoco.core:0.8.0"

-----


I am getting the Following Error


C:\Development\Projects\BreathMapper\Workspace\Source\BreatheMapper\app\libs\adobeMobileLibrary-4.13.5.jar
Warning:Type `com.google.android.gms.wearable.PutDataMapRequest` was not found, it is required for default or static interface methods desugaring of `void com.adobe.mobile.ConfigSynchronizer.syncData(java.lang.String, com.google.android.gms.wearable.DataMap)`
Warning:Type `com.google.android.gms.wearable.DataMapItem` was not found, it is required for default or static interface methods desugaring of `void com.adobe.mobile.ConfigSynchronizer.restoreConfig(com.google.android.gms.wearable.DataItem)`
Warning:Type `com.google.android.gms.wearable.PutDataMapRequest` was not found, it is required for default or static interface methods desugaring of `com.adobe.mobile.WearableDataResponse com.adobe.mobile.WearableDataConnection.send(com.adobe.mobile.WearableDataRequest)`
Warning:Type `com.google.android.gms.wearable.DataMapItem` was not found, it is required for default or static interface methods desugaring of `void com.adobe.mobile.WearableDataConnection.onDataChanged(com.google.android.gms.wearable.DataEventBuffer)`
Warning:Type `com.google.android.gms.wearable.Asset` was not found, it is required for default or static interface methods desugaring of `com.google.android.gms.wearable.DataMap com.adobe.mobile.WearableDataRequest$Cache.handle(android.content.Context)`
Warning:Interface `com.google.android.gms.common.api.GoogleApiClient$OnConnectionFailedListener` not found. It's needed to make sure desugaring of `com.adobe.mobile.WearableDataConnection` is correct. Desugaring will assume that this interface has no default method.
Warning:Interface `com.google.android.gms.wearable.DataApi$DataListener` not found. It's needed to make sure desugaring of `com.adobe.mobile.WearableDataConnection` is correct. Desugaring will assume that this interface has no default method.
C:\Users\xx\.gradle\caches\transforms-1\files-1.1\runtime-1.1.1.aar\be15bb0ad7b260a4de5c6f23e94be8e7\jars\classes.jar
Warning:Type `android.arch.paging.PositionalDataSource` was not found, it is required for default or static interface methods desugaring of `boolean android.arch.persistence.room.paging.LimitOffsetDataSource.isInvalid()`
C:\Users\xx\.gradle\caches\transforms-1\files-1.1\espresso-core-3.0.2.aar\f93273e747e9ce9e35d6f1173bb60539\jars\classes.jar
Warning:Type `sun.misc.Unsafe` was not found, it is required for default or static interface methods desugaring of `sun.misc.Unsafe android.support.test.espresso.core.internal.deps.guava.cache.Striped64.getUnsafe()`
Warning:Type `sun.misc.Unsafe` was not found, it is required for default or static interface methods desugaring of `void android.support.test.espresso.core.internal.deps.guava.util.concurrent.AbstractFuture$UnsafeAtomicHelper.<clinit>()`
Error:
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
1 warning
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:createDebugAndroidTestCoverageReport'.
> java.io.IOException: No coverage data to process in directory 'C:\Development\Projects\BreathMapper\Workspace\Source\BreatheMapper\app\build\outputs\code-coverage\connected'

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 4m 56s

Re #28:
Can you please file a new issue, and upload a sample project that reproduces it?

Continued Re #28
Full Error log
C:\Development\Projects\MyTest\Workspace\Source\MyTest\app\build\outputs\androidTest-results\connected\TEST-Galaxy_Nexus_API_26(AVD) - 8.0.0-app-.xml. Total tests 48, passed 48,
06:04:55 V/ddms: execute 'CLASSPATH=$(pm path android.support.test.services) app_process / android.support.test.services.shellexecutor.ShellMain am instrument -r -w -e targetInstrumentation com.vn.test.test/android.support.test.runner.AndroidJUnitRunner --no_window_animation  -e coverageFilePath /data/data/com.vn.test/coverage_data/ -e coverage true android.support.test.orchestrator/android.support.test.orchestrator.AndroidTestOrchestrator' on 'emulator-5556' : EOF hit. Read: -1
06:04:55 V/ddms: execute: returning
06:04:55 V/ddms: execute: running echo 'for i in $(run-as com.vn.test ls /data/data/com.vn.test/coverage_data/); do run-as com.vn.test cat /data/data/com.vn.test/coverage_data/$i > /data/local/tmp/com.vn.test-coverage_data/$i; done' > /data/local/tmp/com.vn.test-coverage_data/copyFromUser.sh
06:04:55 V/ddms: execute 'echo 'for i in $(run-as com.vn.test ls /data/data/com.vn.test/coverage_data/); do run-as com.vn.test cat /data/data/com.vn.test/coverage_data/$i > /data/local/tmp/com.vn.test-coverage_data/$i; done' > /data/local/tmp/com.vn.test-coverage_data/copyFromUser.sh' on 'emulator-5556' : EOF hit. Read: -1
06:04:55 V/ddms: execute: returning
06:04:55 V/ddms: execute: running sh /data/local/tmp/com.vn.test-coverage_data/copyFromUser.sh
06:04:55 V/ddms: execute 'sh /data/local/tmp/com.vn.test-coverage_data/copyFromUser.sh' on 'emulator-5556' : EOF hit. Read: -1
06:04:55 V/ddms: execute: returning
06:04:55 V/ddms: execute: running ls /data/local/tmp/com.vn.test-coverage_data > /data/local/tmp/com.vn.test-coverage_data/paths.txt
06:04:55 V/ddms: execute 'ls /data/local/tmp/com.vn.test-coverage_data > /data/local/tmp/com.vn.test-coverage_data/paths.txt' on 'emulator-5556' : EOF hit. Read: -1
06:04:55 V/ddms: execute: returning
06:04:55 D/paths.txt: Downloading paths.txt from device 'emulator-5556'
06:04:55 D/Device: Downloading file from device 'emulator-5556'
06:04:55 V/ddms: execute: running rm -r /data/local/tmp/com.vn.test-coverage_data
06:04:55 V/ddms: execute 'rm -r /data/local/tmp/com.vn.test-coverage_data' on 'emulator-5556' : EOF hit. Read: -1
06:04:55 V/ddms: execute: returning
06:04:55 V/ddms: execute: running pm uninstall com.vn.test.test
06:04:55 V/ddms: execute 'pm uninstall com.vn.test.test' on 'emulator-5556' : EOF hit. Read: -1
06:04:55 V/ddms: execute: returning
06:04:55 V/ddms: execute: running pm uninstall com.vn.test
06:04:56 V/ddms: execute 'pm uninstall com.vn.test' on 'emulator-5556' : EOF hit. Read: -1
06:04:56 V/ddms: execute: returning
:app:createDebugAndroidTestCoverageReport FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:createDebugAndroidTestCoverageReport'.
> java.io.IOException: No coverage data to process in directory 'C:\Development\Projects\MyTest\Workspace\Source\Test\app\build\outputs\code-coverage\connected'

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 3m 14s

@ Re #29:
I have uploaded some more log detail.

Re #29

My setup was working fine with earlier gradle plugin 3.1.2 . with distributionUrl=gradle-4.4-all.zip (.ec file was generating but only coverage data is incomplete ).
But I tried to implement the solution provided in #20
gradle plugin - gradle:3.2.0-alpha17'  with distributionUrl=gradle-4.6-all.zip   But getting #30 error an .ec file not generating

Re #32: Please file a new issue, as it looks different than the original one. We can continue the discussion there.

Re#33: Created. Thanks,  Link : https://issuetracker.google.com/issues/112272957

Hi,
How to test the private methods and greendao database CRUD operations in android robolectric test case.
How to get private method  code coverage in android jacoco library using

I can confirm this is still a problem...

> java.io.IOException: No coverage data to process in directories [[/Users/msh/git/wine-scanner-android/app/build/outputs/code_coverage/worldDebugAndroidTest/connected]]

Only happens when enabling ANDROIDX_TEST_ORCHESTRATOR

Re #36:
Can you please share a project the reproduces the issue?

Not really as its the project at the company where I work. Its a huge project so not small task to maintain - hence the need for test coverage...

I've tried trouble shooting most of the day and so far no luck. I really have no idea where to start looking.

Not sure if it will help but the complete output when running ./gradlew createWorldDebugCoverageReport is attached.

I've tried against an emulator (genymotion) as well as a real device - same result.

I guess the next logical step is to setup a basic project and see if that has the same problem...

I've opened a new issue, https://issuetracker.google.com/123987001, because this differs from the original problem in this issue.

Env. androidx.test:orchestrator:1.1.1
Still error
Execution failed for task ':app:createDebugAndroidTestCoverageReport'.
> java.io.IOException: No coverage data to process in directories /app/build/outputs/code_coverage/debugAndroidTest/connected

Did you remove the clearPackageData option ?

I can confirm, that I also have issue with JaCoCo report with following error log:

> Caused by: org.gradle.api.UncheckedIOException: java.io.IOException: No coverage data to process in directories [[/Users/azizbekian/Projects/XXX/XXX/app/build/outputs/code_coverage/devDebugAndroidTest/connected]]

Using `androidTestUtil androidx.test:orchestrator:1.2.0`
AGP version 3.4.2
Gradle version 5.5
Have specified `testInstrumentationRunnerArguments clearPackageData: 'true'` in `defaultConfig` DSL
Have specified `execution 'ANDROIDX_TEST_ORCHESTRATOR'` in testOptions DSL

Problem is, that I cannot see `coverage.ec` file being created in /code_coverage/devDebugAndroidTest/connected/ directory.

Have any update for this issue? I'm also getting the same issue when run jacoco: java.io.IOException: No coverage data to process in directories

I'm interested in a solution, too. If I cannot use the orchestrator, how can I make sure my tests are independent?

This is ridiculous. How can it not been fixed after more than 2 years?
Running tests and having coverage data is a basic scenario for every developer that has some self-respect.

I'm also running into this issue, can we get an update on this?

But this isn't fixed. When clearPackageData is true, the coverage data is deleted.

Please, is there any update on this?

At it again, Google?  Giving vague answers about things being fixed with no real details to the users subscribed to the issue.  Looks like they "fixed" it by making the "path" option work properly so you can specify the path.  Mind you, I still have not gotten a chance to try this yet as the last time they said they fixed this I spent hours of my life trying to make it work without the required details.  Didn't realize they had this split into two tickets so they could absolutely confuse *two* groups of people.  See my comment on the other issue https://issuetracker.google.com/issues/123987001#comment17

and to you, please provide relevant information to your users when posting "fixed".  Thank you.