Android Gradle Plugin 8.0 ScopedArtifact.CLASSES returns JaCoCo instrumented classes [281266702]

Fixed

Bug

Status Update

No update yet.

Description

ne...@gmail.com

created issue #1

May 7, 2023 10:02AM

In Android Gradle Plugin 8.0 the new artifact ScopedArtifact.CLASSES returns classes (when using toGet() that are already instrumented by JaCoCo when enabling test coverage using enableUnitTestCoverage true or enableAndroidTestCoverage true.

This is potentially a bug as it does not allow users of ScopedArtifact.CLASSES to provide non-instrumented classes to for example the JacocoReport task (org.gradle.testing.jacoco.tasks.JacocoReport.classDirectories).

MultipleArtifact.ALL_CLASSES_DIRS did not have this behavior.

I can reproduce this also on AGP 8.1.0-beta01

Reproduce

Create a new Android Project
Enable test coverage
Create a custom Gradle task that uses ScopedArtifact.CLASSES with toGet()
Run the custom task and observe that the returned class directories are directories that point to JaCoCo instrumented classes.

I created a Minimal Reproducible Example here: https://github.com/Rolf-Smit/Android-ScopedArtifacts-with-JaCoCo

With the sample project the problem can be reproduced in at least 2 ways:

Running the custom task app:debugGetAllClasses which should thrown an error saying

Provided directories contain jacoco instrumented classes!

Running the unit test TestClassUnderTest.test_add(), which will complete successfully but Android Studio will show the following trace:

java.lang.instrument.IllegalClassFormatException: Error while instrumenting org/neotech/example/scopedartifact/ClassUnderTest with JaCoCo 0.8.8.202204050719/5dcf34a.
	at org.jacoco.agent.rt.internal_b6258fc.CoverageTransformer.transform(CoverageTransformer.java:94)
	at java.instrument/java.lang.instrument.ClassFileTransformer.transform(Unknown Source)
	at java.instrument/sun.instrument.TransformerManager.transform(Unknown Source)
	at java.instrument/sun.instrument.InstrumentationImpl.transform(Unknown Source)
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(Unknown Source)
	at java.base/java.security.SecureClassLoader.defineClass(Unknown Source)
	at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(Unknown Source)
	at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(Unknown Source)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(Unknown Source)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown Source)
	at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
	at org.neotech.example.scopedartifact.TestClassUnderTest.test_add(TestClassUnderTest.kt:10)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:108)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:40)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:60)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:52)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at jdk.proxy2/jdk.proxy2.$Proxy5.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Caused by: java.io.IOException: Error while instrumenting org/neotech/example/scopedartifact/ClassUnderTest with JaCoCo 0.8.8.202204050719/5dcf34a.
	at org.jacoco.agent.rt.internal_b6258fc.core.instr.Instrumenter.instrumentError(Instrumenter.java:161)
	at org.jacoco.agent.rt.internal_b6258fc.core.instr.Instrumenter.instrument(Instrumenter.java:111)
	at org.jacoco.agent.rt.internal_b6258fc.CoverageTransformer.transform(CoverageTransformer.java:92)
	... 56 more
Caused by: java.lang.IllegalStateException: Cannot process instrumented class org/neotech/example/scopedartifact/ClassUnderTest. Please supply original non-instrumented classes.
	at org.jacoco.agent.rt.internal_b6258fc.core.internal.instr.InstrSupport.assertNotInstrumented(InstrSupport.java:238)
	at org.jacoco.agent.rt.internal_b6258fc.core.internal.instr.ClassInstrumenter.visitField(ClassInstrumenter.java:56)
	at org.jacoco.agent.rt.internal_b6258fc.asm.ClassVisitor.visitField(ClassVisitor.java:338)
	at org.jacoco.agent.rt.internal_b6258fc.asm.ClassReader.readField(ClassReader.java:1137)
	at org.jacoco.agent.rt.internal_b6258fc.asm.ClassReader.accept(ClassReader.java:739)
	at org.jacoco.agent.rt.internal_b6258fc.asm.ClassReader.accept(ClassReader.java:424)
	at org.jacoco.agent.rt.internal_b6258fc.core.instr.Instrumenter.instrument(Instrumenter.java:91)
	at org.jacoco.agent.rt.internal_b6258fc.core.instr.Instrumenter.instrument(Instrumenter.java:109)
	... 57 more

Version information:

Studio Build: Build #AI-223.8836.35.2231.9923731, built on April 12, 2023
Version of Gradle Plugin: 8.0.1 (also on 8.0.0 and 8.1.0-beta01)
Version of Gradle: 8.0
Version of Java: 1.8 compatibility Java 17

Comments

gm...@gmail.com <gm...@gmail.com> #2May 8, 2023 02:03PM

Also affected by this issue too. I think ScopedArtifact.CLASSES should not be affected by enableAndroidTestCoverage and feels it was done like this more as a workaround than by design, right?

Actually, it would be nice if InternalScopedArtifact.PRE_JACOCO_TRANSFORMED_CLASSES, InternalArtifactType.CODE_COVERAGE and InternalArtifactType.UNIT_TEST_CODE_COVERAGE are promoted to public artifacts as well so 3rd party plugins will be able to consume them without requiring reflection or accessing implementation details.

In my case, I had a plugin that integrates AGP with jacoco-report-aggregation and test-report-aggregation

di...@google.com <di...@google.com> May 12, 2023 11:00AM

Assigned to an...@google.com.

zo...@gmail.com <zo...@gmail.com> #3May 12, 2023 03:10PM

I'm honestly not sure what ScopedArtifact.CLASSES is, but I am seeing similar error messages output to log when running unit tests.

I made a minimal reproducible example here: https://github.com/ZOlbrys/AndroidStudioTestsInstrumentedTestApp/tree/master

On AGP 7.4.2 the test passes and no errors are output. On The AGP 8.0.0 the test still passes, but now I get Please supply original non-instrumented classes type errors printed out.

je...@google.com <je...@google.com> #4May 15, 2023 03:44PM

Reassigned to mi...@google.com.

This is done on purpose AFAIR, we probably need to provide another public type with the non instrumented classes.

gm...@gmail.com <gm...@gmail.com> #5Jun 23, 2023 10:21AM

That will be ideal. Actually, would you be open to considering all intermediates exposed as outgoing variants or to be public API? In my case, I wanted to extend support for androidTest variant coverage. Uninstrumented classes are missing, but also a public artifact to fetch .exec files. I managed to get a reference to it by reflection and expose the variant myself, but that's not good enough of a robust plugin.

Message last modified on Jun 23, 2023 10:56AM

je...@google.com <je...@google.com> #6Jul 20, 2023 04:36AM

Micah, can you bump up this bug in your priority list, this is blocking some external plugins to properly support AGP 8.x

up...@gmail.com <up...@gmail.com> #7Aug 7, 2023 01:17PM

Hi, any updates on this? Preventing us from updating AGP Version.

Message last modified on Aug 7, 2023 01:18PM

bp...@atlassian.com <bp...@atlassian.com> #8Sep 8, 2023 02:50PM

I believe this has a larger impact than just 3rd party plugins.

I can reproduce this by:

1. Create a new compose activity project in Android Studio ( AGP 8.1+)
2. Add a class to the project under src/main/java/...
3. Reference that class in an unit test
4. The test will run but the logs output quite a few errors realting to this

This is basically the setup that #3 above mentioned. In one of our production apps just one of the test files was 52MB from all the stack traces generated from this issue.

je...@google.com <je...@google.com> Nov 1, 2023 03:03PM

Reassigned to am...@google.com.

je...@google.com <je...@google.com> #9Nov 1, 2023 09:29PM

Reassigned to mi...@google.com.

hold-on, this might already be fixed by https://googleplex-android-review.git.corp.google.com/c/platform/tools/base/+/24347429

Micah to confirm.

mi...@google.com <mi...@google.com> Nov 1, 2023 09:31PM

Marked as fixed.

ne...@gmail.com <ne...@gmail.com> #10Nov 1, 2023 09:55PM

Micah,

Or any other Googler working on this, how exactly was this fixed? New API?

mi...@google.com <mi...@google.com> #11Nov 1, 2023 09:59PM

https://android.googlesource.com/platform/tools/base/+/28de65c5d2c6a9052b75c87a551315b209bdca75

The ScopedArtifact.CLASSES was previously representing the Jacoco-instrumented classes, but the workflow was rearranged so that this artifact represents the classes before Jacoco-instrumentation. The Jacoco-instrumented classes are then stored in a new internal artifact.

Message last modified on Nov 1, 2023 10:17PM

zo...@gmail.com <zo...@gmail.com> #12Nov 1, 2023 10:04PM

Hi team,

Thanks for the updates!

When would we expect to see this in production/available to test? The links being posted are Google internal. Is the fix associated with a specific AGP version or something?

ro...@gmail.com <ro...@gmail.com> #13Nov 1, 2023 10:05PM

As far as I can see potentially in an alpha version of 8.3

mi...@google.com <mi...@google.com> #14Nov 1, 2023 10:17PM

It should be available starting with Android Studio Iguana/AGP 8.3 C5. Updated the link above with the public change.

zo...@gmail.com <zo...@gmail.com> #15Nov 3, 2023 02:40AM

I can confirm that this issue is fixed using AGP 8.3 alpha 11. Thanks for the fix/updates here!

gs...@gmail.com <gs...@gmail.com> #16Nov 22, 2023 10:32AM

Any chance to get this fix backported to AGP 8.2 at least? Android Studio Iguana/AGP 8.3 seems to be still far away, and not having proper code coverage reports is a major issue.

mi...@google.com <mi...@google.com> #17Nov 22, 2023 06:01PM

Hi, unfortunately the cutoff to backport to 8.2 has already passed, so this is not possible.