command coverage report is not being generated with Test Orchestrator and clearPackageData [126258801]

Assigned

Bug

adexe s nau

Status Update

No update yet.

Description

ra...@gmail.com

created issue #1

Feb 26, 2019 11:54AM

When I use Test Orchestrator with the following command coverage report is not being generated

testInstrumentationRunnerArguments clearPackageData: 'true'

and if I disable testInstrumentationRunnerArguments it is working fine

Distribution: distributions/gradle-4.6-all.zip
Gradle plugin: 'com.android.tools.build:gradle:3.2.0'

Administrators-MacBook-Pro-4:MyApplication2 **$ ./gradlew connectedCheck
04:55:17 D/app-debug.apk: Uploading app-debug.apk onto device '21d964eca40c7ece'
04:55:17 D/Device: Uploading file onto device '21d964eca40c7ece'
04:55:17 D/ddms: Reading file permision of /Users/**/AW/sdk/MyApplication2/app/build/outputs/apk/debug/app-debug.apk as: rw-r--r--
04:55:17 V/ddms: execute: running pm install -r -t "/data/local/tmp/app-debug.apk"

> Task :app:connectedDebugAndroidTest
04:55:17 V/ddms: execute: running am get-config
04:55:17 V/ddms: execute 'am get-config' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:17 V/ddms: execute: returning

04:55:26 V/ddms: execute 'pm install -r -t "/data/local/tmp/app-debug.apk"' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:26 V/ddms: execute: returning
04:55:26 V/ddms: execute: running rm "/data/local/tmp/app-debug.apk"
04:55:26 V/ddms: execute 'rm "/data/local/tmp/app-debug.apk"' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:26 V/ddms: execute: returning
04:55:26 D/orchestrator-1.0.2.apk: Uploading orchestrator-1.0.2.apk onto device '21d964eca40c7ece'
04:55:26 D/Device: Uploading file onto device '21d964eca40c7ece'
04:55:26 D/ddms: Reading file permision of /Users/**/.gradle/caches/modules-2/files-2.1/com.android.support.test/orchestrator/1.0.2/a3e051dc739ac51d6cf2813ad0df598f46d11e2/orchestrator-1.0.2.apk as: rw-r--r--
04:55:26 V/ddms: execute: running pm install -r -t "/data/local/tmp/orchestrator-1.0.2.apk"
04:55:34 V/ddms: execute 'pm install -r -t "/data/local/tmp/orchestrator-1.0.2.apk"' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:34 V/ddms: execute: returning
04:55:34 V/ddms: execute: running rm "/data/local/tmp/orchestrator-1.0.2.apk"
04:55:34 V/ddms: execute 'rm "/data/local/tmp/orchestrator-1.0.2.apk"' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:34 V/ddms: execute: returning
04:55:34 D/test-services-1.0.2.apk: Uploading test-services-1.0.2.apk onto device '21d964eca40c7ece'
04:55:34 D/Device: Uploading file onto device '21d964eca40c7ece'
04:55:34 D/ddms: Reading file permision of /Users/**/.gradle/caches/modules-2/files-2.1/com.android.support.test.services/test-services/1.0.2/5954206f071102ca8543dae923a808b758fe2597/test-services-1.0.2.apk as: rw-r--r--
04:55:34 V/ddms: execute: running pm install -r -t "/data/local/tmp/test-services-1.0.2.apk"
04:55:39 V/ddms: execute 'pm install -r -t "/data/local/tmp/test-services-1.0.2.apk"' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:39 V/ddms: execute: returning
04:55:39 V/ddms: execute: running rm "/data/local/tmp/test-services-1.0.2.apk"
04:55:39 V/ddms: execute 'rm "/data/local/tmp/test-services-1.0.2.apk"' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:39 V/ddms: execute: returning
04:55:39 D/app-debug-androidTest.apk: Uploading app-debug-androidTest.apk onto device '21d964eca40c7ece'
04:55:39 D/Device: Uploading file onto device '21d964eca40c7ece'
04:55:39 D/ddms: Reading file permision of /Users/**/AW/sdk/MyApplication2/app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk as: rw-r--r--
04:55:39 V/ddms: execute: running pm install -r -t "/data/local/tmp/app-debug-androidTest.apk"
04:55:44 V/ddms: execute 'pm install -r -t "/data/local/tmp/app-debug-androidTest.apk"' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:44 V/ddms: execute: returning
04:55:44 V/ddms: execute: running rm "/data/local/tmp/app-debug-androidTest.apk"
04:55:44 V/ddms: execute 'rm "/data/local/tmp/app-debug-androidTest.apk"' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:44 V/ddms: execute: returning
04:55:44 V/ddms: execute: running if [ -d /data/local/tmp/com.boxer.myapplication2-coverage_data ]; then rm -r /data/local/tmp/com.boxer.myapplication2-coverage_data; fi && mkdir /data/local/tmp/com.boxer.myapplication2-coverage_data
04:55:44 V/ddms: execute 'if [ -d /data/local/tmp/com.boxer.myapplication2-coverage_data ]; then rm -r /data/local/tmp/com.boxer.myapplication2-coverage_data; fi && mkdir /data/local/tmp/com.boxer.myapplication2-coverage_data' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:44 V/ddms: execute: returning
04:55:44 V/ddms: execute: running echo 'if [ -d /data/data/com.boxer.myapplication2/coverage_data/ ]; then rm -r /data/data/com.boxer.myapplication2/coverage_data/; fi && mkdir /data/data/com.boxer.myapplication2/coverage_data/' > /data/local/tmp/com.boxer.myapplication2-coverage_data/tmpScript.sh
04:55:44 V/ddms: execute 'echo 'if [ -d /data/data/com.boxer.myapplication2/coverage_data/ ]; then rm -r /data/data/com.boxer.myapplication2/coverage_data/; fi && mkdir /data/data/com.boxer.myapplication2/coverage_data/' > /data/local/tmp/com.boxer.myapplication2-coverage_data/tmpScript.sh' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:44 V/ddms: execute: returning
04:55:44 V/ddms: execute: running run-as com.boxer.myapplication2 sh /data/local/tmp/com.boxer.myapplication2-coverage_data/tmpScript.sh
04:55:44 V/ddms: execute 'run-as com.boxer.myapplication2 sh /data/local/tmp/com.boxer.myapplication2-coverage_data/tmpScript.sh' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:44 V/ddms: execute: returning
04:55:44 I/RemoteAndroidTest: Running CLASSPATH=$(pm path android.support.test.services) app_process / android.support.test.services.shellexecutor.ShellMain am instrument -r -w -e targetInstrumentation com.boxer.myapplication2.test/android.support.test.runner.AndroidJUnitRunner -e coverageFilePath /data/data/com.boxer.myapplication2/coverage_data/ -e coverage true -e clearPackageData true android.support.test.orchestrator/android.support.test.orchestrator.AndroidTestOrchestrator on SM-G965F - 9
04:55:44 V/ddms: execute: running CLASSPATH=$(pm path android.support.test.services) app_process / android.support.test.services.shellexecutor.ShellMain am instrument -r -w -e targetInstrumentation com.boxer.myapplication2.test/android.support.test.runner.AndroidJUnitRunner -e coverageFilePath /data/data/com.boxer.myapplication2/coverage_data/ -e coverage true -e clearPackageData true android.support.test.orchestrator/android.support.test.orchestrator.AndroidTestOrchestrator
04:55:47 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: class=com.boxer.myapplication2.ExampleInstrumentedTest
04:55:47 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: current=1
04:55:47 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: id=AndroidJUnitRunner
04:55:47 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: numtests=2
04:55:47 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: stream=
04:55:47 V/InstrumentationResultParser: com.boxer.myapplication2.ExampleInstrumentedTest:
04:55:47 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: test=useAppContext2
Starting 2 tests on SM-G965F - 9
04:55:47 V/InstrumentationResultParser: INSTRUMENTATION_STATUS_CODE: 1
04:55:47 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: class=com.boxer.myapplication2.ExampleInstrumentedTest
04:55:47 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: current=1
04:55:47 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: id=AndroidJUnitRunner
04:55:47 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: numtests=2
04:55:47 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: stream=.
04:55:47 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: test=useAppContext2
04:55:47 V/InstrumentationResultParser: INSTRUMENTATION_STATUS_CODE: 0
04:55:48 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: class=com.boxer.myapplication2.ExampleInstrumentedTest
04:55:48 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: current=2
04:55:48 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: id=AndroidJUnitRunner
04:55:48 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: numtests=2
04:55:48 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: stream=
04:55:48 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: test=useAppContext
04:55:48 V/InstrumentationResultParser: INSTRUMENTATION_STATUS_CODE: 1
04:55:48 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: class=com.boxer.myapplication2.ExampleInstrumentedTest
04:55:48 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: current=2
04:55:48 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: id=AndroidJUnitRunner
04:55:48 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: numtests=2
04:55:48 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: stream=.
04:55:48 V/InstrumentationResultParser: INSTRUMENTATION_STATUS: test=useAppContext
04:55:48 V/InstrumentationResultParser: INSTRUMENTATION_STATUS_CODE: 0
04:55:49 V/InstrumentationResultParser: INSTRUMENTATION_RESULT: stream=
04:55:49 V/InstrumentationResultParser:
04:55:49 V/InstrumentationResultParser: Time: 2.201
04:55:49 V/InstrumentationResultParser:
04:55:49 V/InstrumentationResultParser: OK (2 tests)
04:55:49 V/InstrumentationResultParser:
04:55:49 V/InstrumentationResultParser:
04:55:49 V/InstrumentationResultParser: INSTRUMENTATION_CODE: -1
04:55:49 V/InstrumentationResultParser:
04:55:49 I/XmlResultReporter: XML test result file generated at /Users/Kumarraghave/AW/sdk/MyApplication2/app/build/outputs/androidTest-results/connected/TEST-SM-G965F - 9-app-.xml. Total tests 2, passed 2,
04:55:49 V/ddms: execute 'CLASSPATH=$(pm path android.support.test.services) app_process / android.support.test.services.shellexecutor.ShellMain am instrument -r -w -e targetInstrumentation com.boxer.myapplication2.test/android.support.test.runner.AndroidJUnitRunner -e coverageFilePath /data/data/com.boxer.myapplication2/coverage_data/ -e coverage true -e clearPackageData true android.support.test.orchestrator/android.support.test.orchestrator.AndroidTestOrchestrator' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:49 V/ddms: execute: returning
04:55:49 V/ddms: execute: running echo 'for i in $(run-as com.boxer.myapplication2 ls /data/data/com.boxer.myapplication2/coverage_data/); do run-as com.boxer.myapplication2 cat /data/data/com.boxer.myapplication2/coverage_data/$i > /data/local/tmp/com.boxer.myapplication2-coverage_data/$i; done' > /data/local/tmp/com.boxer.myapplication2-coverage_data/copyFromUser.sh
04:55:49 V/ddms: execute 'echo 'for i in $(run-as com.boxer.myapplication2 ls /data/data/com.boxer.myapplication2/coverage_data/); do run-as com.boxer.myapplication2 cat /data/data/com.boxer.myapplication2/coverage_data/$i > /data/local/tmp/com.boxer.myapplication2-coverage_data/$i; done' > /data/local/tmp/com.boxer.myapplication2-coverage_data/copyFromUser.sh' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:49 V/ddms: execute: returning
04:55:49 V/ddms: execute: running sh /data/local/tmp/com.boxer.myapplication2-coverage_data/copyFromUser.sh
04:55:49 V/ddms: execute 'sh /data/local/tmp/com.boxer.myapplication2-coverage_data/copyFromUser.sh' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:49 V/ddms: execute: returning
04:55:49 V/ddms: execute: running ls /data/local/tmp/com.boxer.myapplication2-coverage_data > /data/local/tmp/com.boxer.myapplication2-coverage_data/paths.txt
04:55:49 V/ddms: execute 'ls /data/local/tmp/com.boxer.myapplication2-coverage_data > /data/local/tmp/com.boxer.myapplication2-coverage_data/paths.txt' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:49 V/ddms: execute: returning
04:55:49 D/paths.txt: Downloading paths.txt from device '21d964eca40c7ece'
04:55:49 D/Device: Downloading file from device '21d964eca40c7ece'
04:55:49 V/ddms: execute: running rm -r /data/local/tmp/com.boxer.myapplication2-coverage_data
04:55:49 V/ddms: execute 'rm -r /data/local/tmp/com.boxer.myapplication2-coverage_data' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:49 V/ddms: execute: returning
04:55:49 V/ddms: execute: running pm uninstall com.boxer.myapplication2.test
04:55:50 V/ddms: execute 'pm uninstall com.boxer.myapplication2.test' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:50 V/ddms: execute: returning
04:55:50 V/ddms: execute: running pm uninstall com.boxer.myapplication2
04:55:52 V/ddms: execute 'pm uninstall com.boxer.myapplication2' on '21d964eca40c7ece' : EOF hit. Read: -1
04:55:52 V/ddms: execute: returning

> Task :app:createDebugAndroidTestCoverageReport FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:createDebugAndroidTestCoverageReport'.
> java.io.IOException: No coverage data to process in directories [[/Users/**/AW/sdk/MyApplication2/app/build/outputs/code_coverage/debugAndroidTest/connected]]

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at

https://help.gradle.org

BUILD FAILED in 37s
53 actionable tasks: 2 executed, 51 up-to-date

MyApplication2.zip

529 KB

Download

Comments

vi...@google.com <vi...@google.com> #2Mar 1, 2019 12:54PM

Bug is reproducible. For more details, kindly refer -

https://docs.google.com/document/d/1oWGh6swfjFIdli0D-HGKxkk6imI2PAuWgaSs9aXb5vM/edit (internal only link)

ra...@gmail.com <ra...@gmail.com> #3Mar 14, 2019 08:50AM

Can you please share any way to overcome this issue.

gu...@gmail.com <gu...@gmail.com> #4Mar 26, 2019 03:05PM

Hi Google Team -
I'm also facing this issue. Can we please prioritize this issue to fix?I can't get code coverage report with Android Test Orchestrator.

gu...@gmail.com <gu...@gmail.com> #5Mar 26, 2019 03:07PM

As comment # 3 mention, do we have any workaround? Code Coverage is an important parameter in our CI/CD.

ra...@gmail.com <ra...@gmail.com> #6Apr 25, 2019 08:34AM

Tried with gradle 3.4.0 "com.android.tools.build:gradle:3.4.0", still issue is replicable. Is there any work around to generate code coverage with test orchestrator enabled

vi...@google.com <vi...@google.com> #7Apr 26, 2019 08:53AM

Assigned to vi...@google.com.

This bug will be closed in 7 days. Please let us know how to respond to the external bug. Alternatively, please feel free to update the external bug directly. If an update is not provided, the external bug will be closed as infeasible.

vi...@google.com <vi...@google.com> #8Apr 26, 2019 09:01AM

Status: New

Please ignore above comment.

ra...@gmail.com <ra...@gmail.com> #9May 13, 2019 03:32PM

Any update on the issue

vi...@google.com <vi...@google.com> Jul 2, 2019 01:10PM

Assigned to vi...@google.com.

sh...@gmail.com <sh...@gmail.com> #10Mar 25, 2020 06:49AM

I am also facing the same issue , when are we expecting to get it fixed

gu...@gmail.com <gu...@gmail.com> #11Mar 26, 2020 03:13PM

Team google, can you please provide fix or a workaround to generate code coverage report when we are clear app data?

vi...@gmail.com <vi...@gmail.com> #12Mar 27, 2020 07:06AM

Team, This looks to be a critical bug on Android test orchestrator. Kindly increase its priority and provide your comments.

[Deleted User] <[Deleted User]> #13Apr 29, 2020 11:19AM

Hi, I'm facing the same issue. Do you have any update on this?

I'm using:

execution 'ANDROIDX_TEST_ORCHESTRATOR' (version 1.2.0)
testInstrumentationRunnerArguments clearPackageData: 'true'

And the coverage files are not found after the test execution.
I need to use Android Test Orchestrator and clearPackageData to avoid crashes and random failures during the execution. But I also need to get coverage results.

Is there a way to save the coverage files in another location to avoid its removal? I tried to use coverageFilePath but it didn't work.

ja...@google.com <ja...@google.com> #14May 12, 2020 04:36PM

I'm also seeing this error at random on the ToT androidx-master-dev.

ja...@google.com <ja...@google.com> #15May 12, 2020 04:37PM

Here's error snippet.

> Task :support-media2-test-client:createDebugAndroidTestCoverageReport FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':support-media2-test-client:createDebugAndroidTestCoverageReport'.
> A failure occurred while executing com.android.build.gradle.internal.coverage.JacocoReportTask$JacocoReportWorkerAction
   > Unable to generate Jacoco report

tr...@google.com <tr...@google.com> #16May 13, 2020 07:06PM

I am also running into this error in androidx ToT.

As a workaround, I have been using the following command to run connectedCheck:

./gradlew connectedCheck -x createDebugAndroidTestCoverageReport

ra...@gmail.com <ra...@gmail.com> #17May 14, 2020 02:49PM

tried "./gradlew connectedCheck -x createDebugAndroidTestCoverageReport", this command is just ignoring the error, it is not generating code coverage

tr...@google.com <tr...@google.com> #18May 14, 2020 10:54PM

Sorry, #16 was meant to unblock running a connectedCheck, that is, running instrumentation tests on a connected device. This was being blocked by the failure in createDebugAndroidTestCoverageReport. It won't help with generating the code coverage report.

vi...@google.com <vi...@google.com> Jan 7, 2021 09:25AM

Assigned to an...@google.com.

ra...@gmail.com <ra...@gmail.com> #19Jul 7, 2021 10:46AM

Hey,
any update in this ticket

ae...@google.com <ae...@google.com> #20Sep 14, 2023 06:18PM

This has not been updated in over 1 year so is being closed. Please reopen as neccesary.