Status Update
No update yet.
Comments
lu...@google.com
ci...@gmail.com
I see a similar issue using AGP 4.2.0, 4.2.1, 4.2.2, 7.0.0 and 7.0.1 for instrumentation tests. The issue is not observed using AGP 4.1.3.
The generated coverage.ec file's size is 0B. After adding the --debug option to the task createDebugCoverageReport I see the following message:
[DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter] Executing actions for task ':app:connectedDebugAndroidTest'.
[INFO] [org.gradle.api.Task] DeviceConnector 'AOSP on ARM64 - 11': installing /Project/app/build/outputs/apk/debug/app-debug.apk
[INFO] [org.gradle.api.Task] DeviceConnector 'AOSP on ARM64 - 11': installing /Project/app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk
[INFO] [org.gradle.api.Task] run-as: couldn't chdir to package's data directory '/data/user/0/com.obregons.test': Permission denied
[INFO] [org.gradle.api.Task]
I can confirm the tests are recognized and are successful and this issue is happening in 1 out of 3 computers, all with similar configurations (MacOS Catalina, using OpenJDK 11).
I also launched the task and force-stopped it before finishing, checked my connected device and the coverage.ec file in the folder /data/user/0/com.obregons.test is present and non-zero size. The device under test is rooted and no other problems with permissions are reported.
ci...@gmail.com
An update on my previous comment: turns out the issue with run-as came from a missing SELinux policy. After adding the appropriate policy I was able to generate the coverage report
ne...@gmail.com
At first I thought it had something to do with Kotlin 1.5 vs 1.4, but I can also confirm that I'm seeing this issue starting with AGP 4.2 and specifically for Android library modules that run instrumented tests.
pa...@gmail.com
configurations.all {
resolutionStrategy.eachDependency { details ->
if ('org.jacoco' == details.requested.group) {
details.useVersion rootProject.jacocoVersion
}
}
}
This leads me to believe that some library that I might be using, was forcing an older version of jacoco, stopping it from generating the exec file correctly.
By running ./gradlew dependencies I found out that something was using jacoco version 0.8.3 but I couldn't find out which library was using it.
ne...@gmail.com
You must be having a different issue, I tried different versions of Jacoco and they all work to some extend (depending on what the level of support they have for Kotlin generated bytecode), I even raised a bug on their side and they rejected it because it is definitely not something caused by Jacoco:
So you must be having a different issue then. The first comment in this issue #1 perfectly describes the issue: the coverage file is literally empty.
Again: I only see this happening since AGP 4.2, and I only see the coverage file being empty for Android library modules (it seems to work for app modules).
ne...@gmail.com
ne...@gmail.com
pa...@gmail.com
I also created an empty project without importing jacoco, and I still found the androidJacocoAnt task using version 0.8.3, it may be something from gradle-wrapper or indeed kotlin plugin.
ne...@gmail.com
lu...@google.com
lu...@google.com
The reason why the Jacoco executable is empty for libraries is likely due to AGP explicitly excluding AAR classes from being consumed by the JacocoPlugin for unit tests. This was because offline instrumentation would have republished the classes with Jacoco instrumentation and then re-instrumented by the agent. However, as AGP no longer re-publishes offline instrumented code, library classes can now be excluded by the JacocoPlugin for coverage analysis.
I'll update this bug, once there is a fix in AGP.
ne...@gmail.com
lu...@google.com
We're aiming for a release of AGP 7.2-beta01 on the 13th Dec. Unfortunately, I can't provide you with the fixed .exec artifact.
ne...@gmail.com
Any plans for another alpha release that includes this fix?
lu...@google.com
Sorry for the wait - as I mentioned in #11, the fix involved redesigning how coverage data for connected tests was collected; taking a few months. Without avoiding publishing intrumented code in library classes, the library classes from AAR used for unit test coverage would be corrupt. Unfortunately, we do not have any planned alphas before 7.2 beta01.
sa...@google.com
Thank you for your patience while our engineering team worked to resolve this issue. A fix for this issue is now available in:
- Android Studio Chipmunk Canary 6 (2021.2.1.6)
- Android Gradle Plugin 7.2.0-alpha06
We encourage you to try the latest update.
If you notice further issues or have questions, please file a new bug report.
Thank you for taking the time to submit feedback — we really appreciate it!
ha...@gmail.com
Android Studio Chipmunk Canary 6 (2021.2.1.6)
Android Gradle Plugin 7.2.0-alpha06
and the issue is still present.
[Deleted User] <[Deleted User]>
I tried 7.2.0-alpha06 with success (Android Studio version irrelevant, Gradle 7.3.3).
The only thing I had to fix in my outputs/code_coverage/*/connected/*/coverage.ec.
However can we also get this hot-fixed on top of AGP 7.0.4 and as a new beta release for AGP 7.1?
It's kinda strange that this fix is currently only available on a canary version of AGP while it has been broken since 7.0!
mh...@gmail.com
(Android Studio bumblebee, Gradle 7.4 ).
ma...@everis.com
lu...@google.com
Hey mateus.luiz.camilo@, the production release containing this fix, AGP 7.2 is scheduled for final release around 2022-05-05.
ch...@instacart.com
ut...@gmail.com
- distributionUrl=https\://
- "com.android.tools.build:gradle:7.2"
But jacoco exec is not generated :(
ma...@gmail.com
ch...@instacart.com
From 7.x onwards looks like the file moved but is also empty, this was fixed in 7.2.x.
The .exec file moves to build/outputs/unit_test_code_coverage/{flavorName}/test{FlavorName}.exec
Not sure why it moved, but easiest thing to do is change your exec tree to find it like this in the jacoco setup:
```
val execTree = fileTree(project.buildDir) { includes.add("**/*.exec") }
executionData.setFrom(execTree.files)
```
This seems to work for me, gradle 7.4, agp 7.2.1 Java 17.
ut...@gmail.com
>The .exec file moves to build/outputs/unit_test_code_coverage/{flavorName}/test{FlavorName}.exec
yub, but jacoco folder and the jacoco.exec file are not generate, right?
report is always 0%
ch...@instacart.com
ut...@gmail.com
Skipping task ':createDebugCoverageReport' as it has no actions.
:createDebugCoverageReport (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 0.0 secs.
:jacocoTestReport (Thread[Execution worker for ':' Thread 6,5,main]) started.
producer locations for task group 1 (Thread[Execution worker for ':' Thread 7,5,main]) started.
producer locations for task group 1 (Thread[Execution worker for ':' Thread 7,5,main]) completed. Took 0.0 secs.
> Task :jacocoTestReport SKIPPED
Skipping task ':jacocoTestReport' as task onlyIf is false.
:jacocoTestReport (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 0.001 secs.
AAPT2 aapt2-7.2.1-7984345-osx Daemon #0: shutdown
Gralde 7.4.2, AGP 7.2.1, kotlin 1.6.0
[Deleted User] <[Deleted User]>
gradle 7.3.3, AGP7.2.1 with
debug {
testCoverageEnabled true
}
the testDebugUnitTest.exec is put in different folder.
Without it it is put in the build/jacoco/ .
what is this testCoverageEnabled used for?
da...@jpmchase.com
It was primarily geared toward providing instrumenation tests like UIAutomator, Espresso, etc a way to provide code coverage during execution. It pre-instruments all the code with jacoco instrumentation and then bundles this instrumented code in the test apk. However, it really messes with Unit Testing or if you wanted mixed tests in the same build type. You can't mix the two.. Also there is a danger of accidently packaging the instrumented classes.
Google really should split this in into two and remove the old testCoverageEnabled.
instrumentionCoverageEnabled
unitTestCoverageEnabled
Let the instrumentation be used for the Android Test related tests, and for pure unit tests use the just intime instrumentation.
Ideally there should be an option to set where you want the jacoco.exec file be created for both options.
Old assumptions in AGP of how tests were going to be done, are affecting the modern usage of testing.
lu...@google.com
Re comment 31, From AGP 7.3+ we have deprecated testCoverageEnabled and split the option into enableAndroidTestCoverage and enableUnitTestCoverage.
jo...@branch.io
#32 what branch would this be in? I tried switching to 7.3.0-rc01 for both com.android.library and com.android.app for AGP and cannot find anything related to the usage of enableUnitTestCoverage or enabledAndroidTestCoverage.
I also tried checking out studio-master-dev branch:
repo init -u https://android.googlesource.com/platform/manifest -b studio-master-dev
and found no reference to these properties.
Edit:
Ah nvm, it looks like it was misspelled as you can find here:
jo...@branch.io
nu...@traderepublic.com
ak...@gmail.com
Same comment as above 0% in Gradle 7.5.1 and AGP 7.3.1 (Android Studio Dolphin 2021.3.1)
on...@gmail.com
It may break coverage no matter which Gradle version you use. If you have any plugins doing that just disable them in the test (or debug) builds. For example, FirebasePerformance has a property for that.
ga...@linecorp.com
I tried to test with AGP 7.2.1
-
I found
classFileCollectionfromJacocoReportTask build/intermediates/javac/{variant}/classes.
And most of android apps are using kotlin.
So,build/tmp/kotlin-classes/$variantshould be added onclassFileCollection -
Also,
javaSourcescontains java sources only, and all kotlin sources are excluded. -
Some of generated classes/sources (R, BuildConfig, Manifest) are excluded. but other Generated classes (such as ViewBinding, DataBinding, Aidl, and 3P Annotation Processors : Glide ... ) should be excluded from
classFileCollection
So, AGP might provide options to exclude these classes.
ga...@linecorp.com
I submitted 2 issues about
kr...@edwardjones.com
The file is located at "${project.buildDir}/outputs/unit_test_code_coverage/${sourceName}UnitTest/test${sourceName.capitalize()}UnitTest.exec"
Description
This is building on issues discussed here
Now that AGP 7.0.0 is stable, here's the current behavior I'm seeing, which still doesn't seem correct: For library modules, with
testCoverageEnabled true, the exec file generated at<module>/build/outputs/unit_test_code_coveragedoesn't contain any coverage information.This is similar to the behavior on 4.2.0 where, for library modules, the coverage file generated at
<module>/build/jacocodidn't actually contain any coverage information, but then the erroneously created exec file at the root of the library module did contain the correct coverage info. This allowed us to at least then reference that exec at the root, and generate a correct report. It seems now in 7.0.0 we only end up with the empty exec file and can't even fall back on the other.For what it's worth, it appears with
testCoverageEnabled falseon both 4.2.0 and 7.0.0, the exec file created in the build directory contains the proper unit test coverage info. In the 4.2.0 case, the root-level exec file isn't created, and in the 7.0.0 case, the exec file at<module>/build/outputs/unit_test_code_coverageisn't created, and is instead at<module>/build/jacocoAttached is a sample project with comments to be able to switch between 4.2.0 and 7.0.0 easily and contrast their behavior.