IssueTracker

Thanks Jeffrey, I've asked JetBrains if they have a simple reproduction. We already attempt to filter out module-info.java classes but maybe we are missing the check in one of the entry points.

The filtering happens here: https://r8.googlesource.com/r8/+/master/src/main/java/com/android/tools/r8/utils/FileUtils.java#38

I think we need to update the gradle plugin for this. This is hitting a path where gradle is adding this class file as a byte[]. We do not have a descriptor for those and therefore D8 cannot easily filter these out. Maybe the gradle plugin should filter out all class files under META_INF. Alternatively, at least filter out the module-info.class files.

The fix should be released in 3.2.0-alpha12.

ag/I29c545dae8ee0bf25b9f4d132e35353c3ef8cd75

It seems with gradle plugin 3.2.0-alpha12, the issue is still there. Details can be found here - https://github.com/h2database/h2database/issues/1090

Re #5:

It does not seem the issues are the same. You seem to have duplicate types in your APK, which causes build time issues. To find which jars bring in duplicate types, please follow these instructions:

Sync your project in Android Studio, go Navigate -> Class, make sure "Include non-project classes" checkbox is selected, and type "org.h2.util.Bits". Android Studio should show you jars that contain contain this type. If you are familiar with excluding transitive dependencies, excluding one of Maven dependencies that contain org.h2.util.Bits should fix your issue.

It is not enough to just exclude the module-info.java - you have to exclude all classes present in the META-INF/versions folder. For example h2-1.4.197.jar contains the file META-INF/versions/9/org/h2/util/Bits which is probably what causes dex to fail with MethodHandle.invoke and MethodHandle.invokeExact are only supported starting with Android O (--min-api 26) as described here: https://github.com/h2database/h2database/issues/1317

Re #7:

The Android Gradle plugin already filters module-info and classes under meta-inf for dexing. Which version of the plugin are you using? Can you please c/p the full stacktrace?

It seems to be fixed using android plugin 3.3.0-alpha04 with java target 1.7 only. Nitrite database - https://github.com/dizitart/nitrite-database is able to use latest mvstore version now with android.

Seeing this failure persist with kotlinx.coroutines 0.25, AGP 3.3 alpha 08, and R8 enabled.

org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':frontend:android:transformClassesAndResourcesWithR8ForRelease'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:110)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:77)
        at org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51)
        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59)
        at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101)
        at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44)
        at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91)
        at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:59)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.run(EventFiringTaskExecuter.java:51)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:300)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:292)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:174)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:46)
        at org.gradle.execution.taskgraph.LocalTaskInfoExecutor.execute(LocalTaskInfoExecutor.java:42)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareWorkItemExecutor.execute(DefaultTaskExecutionGraph.java:277)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareWorkItemExecutor.execute(DefaultTaskExecutionGraph.java:262)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:135)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:130)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.execute(DefaultTaskPlanExecutor.java:200)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.executeWithWork(DefaultTaskPlanExecutor.java:191)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.run(DefaultTaskPlanExecutor.java:130)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
Caused by: java.lang.RuntimeException: com.android.tools.r8.CompilationFailedException: Compilation failed to complete
        at com.android.builder.profile.Recorder$Block.handleException(Recorder.java:55)
        at com.android.builder.profile.ThreadRecorder.record(ThreadRecorder.java:104)
        at com.android.build.gradle.internal.pipeline.TransformTask.transform(TransformTask.java:230)
        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73)
        at org.gradle.api.internal.project.taskfactory.IncrementalTaskAction.doExecute(IncrementalTaskAction.java:50)
        at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:39)
        at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:26)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:131)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:300)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:292)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:174)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:120)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:99)
        ... 31 more
Caused by: com.android.tools.r8.CompilationFailedException: Compilation failed to complete
        at com.android.tools.r8.utils.ExceptionUtils.withCompilationHandler(ExceptionUtils.java:70)
        at com.android.tools.r8.utils.ExceptionUtils.withR8CompilationHandler(ExceptionUtils.java:48)
        at com.android.tools.r8.R8.run(R8.java:130)
        at com.android.builder.dexing.R8Tool.runR8(r8Tool.kt:183)
        at com.android.build.gradle.internal.transforms.R8Transform.transform(R8Transform.kt:232)
        at com.android.build.gradle.internal.pipeline.TransformTask$2.call(TransformTask.java:239)
        at com.android.build.gradle.internal.pipeline.TransformTask$2.call(TransformTask.java:235)
        at com.android.builder.profile.ThreadRecorder.record(ThreadRecorder.java:102)
        ... 44 more
Caused by: com.android.tools.r8.utils.AbortException: Error: Program type already present: kotlinx.coroutines.experimental.AwaitAll$AwaitAllNode
        at com.android.tools.r8.utils.Reporter.failIfPendingErrors(Reporter.java:116)
        at com.android.tools.r8.utils.Reporter.fatalError(Reporter.java:74)
        at com.android.tools.r8.utils.ExceptionUtils.withCompilationHandler(ExceptionUtils.java:64)
        ... 51 more

I've uploaded https://r8-review.googlesource.com/c/r8/+/26200 to fix this in R8.

Project: r8
Branch: master

commit a18e9e292df5ed6df4ff10fc5e68f4a5e4e0b2c1
Author: Ivan Gavrilovic <gavra@google.com>
Date:   Tue Sep 04 07:27:58 2018

    Ignore class files under meta-inf
   
    When processing multi-release jars, ignore class files
    under meta-inf.
   
    BUG: 77587908
    Test: ArchiveClassFileProviderTest
    Change-Id: I53d9127df4db9e6617bcf3ded6df5c367b8a6b79

M       src/main/java/com/android/tools/r8/utils/ZipUtils.java
M       src/test/java/com/android/tools/r8/ArchiveClassFileProviderTest.java

https://r8-review.googlesource.com/26200

This issue is a duplicate of https://issuetracker.google.com/issues/70118537. I am not sure how to add it to the list of duplicates.

Thanks Ralph, I just did. :)

Ignoring classes under META-INF will only fix the problem for library dependencies that put module-info.class into META-INF/versions/X, for X >= 9. More commonly, at least in the near future (until the 6-month release cadence starts creating a lot of JDK compatibility creep between different projects, necessitating multi-release jars in a large number of cases), a lot of library jarfiles will put the module-info.class file into the root of the jar. Therefore, the above patch is insufficient to fix the problem.

A module-info.class file is more or less a standard classfile, with a number of disallowed items (such as methods and fields), some extra attributes (which will be skipped by a lot of classfile parsers), and one glaring difference: the superclass name has a constant pool index of 0, indicating a null string (typically constant pool indices have to be greater than 0, for all classes other than Object). It is this constant pool index of 0 that causes most classfile parsing tools to break. Just fixing that one thing typically means that the classfile parsing code just ends up with a class with a weird name ("module-info"), and with no superclass, no fields, and no methods.

A lot of projects across the JVM ecosystem are seeing problems of this nature right now. Sometimes it is caused by custom classfile scanning (e.g. Jetty -- https://github.com/eclipse/jetty.project/issues/1692 ), but more often than not it is caused by a dependency upon an old version of ObjectWeb ASM, and this can be fixed by updating ObjectWeb ASM, if you depend directly on it, or overriding the version of a dependency of a dependency, if you depend upon it only transitively ( https://stackoverflow.com/questions/49398894/unable-to-compile-simple-java-10-project-with-maven/49398936#49398936 ).

A user of a library I wrote (ClassGraph) is reporting the following stacktrace from the Android build tools, which indicates ObjectWeb ASM involement in at least part of the problem (although it is possible there are other scanning or classfile processing steps in the Android build process that also need to be fixed):

Caused by: java.lang.RuntimeException
        at org.objectweb.asm.ClassVisitor.visitModule(ClassVisitor.java:148)
        at org.objectweb.asm.ClassReader.readModule(ClassReader.java:731)
        at org.objectweb.asm.ClassReader.accept(ClassReader.java:632)
        at org.objectweb.asm.ClassReader.accept(ClassReader.java:500)
        at com.android.builder.desugaring.DesugaringClassAnalyzer.analyze(DesugaringClassAnalyzer.java:144)
        at com.android.builder.desugaring.DesugaringClassAnalyzer.analyzeJar(DesugaringClassAnalyzer.java:92)
        at com.android.builder.desugaring.DesugaringClassAnalyzer.analyze(DesugaringClassAnalyzer.java:63)
        at com.android.build.gradle.internal.transforms.DesugarIncrementalTransformHelper.lambda$getInitalGraphData$4(DesugarIncrementalTransformHelper.java:150)
        at java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1424)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

Re #15:
Starting with the Android Gradle Plugin 3.3.0-alpha01, both META-INF/*.class and module-info.class are ignored, so you should not see any issues. We are considering cherry-picking this change for 3.2.2 release as well.

Re: #16:
So, already 3.3.0-rc01 version issued.
Still you considering cherry-picking this change for 3.2.2 release?
Have you information about how soon the 3.2.2 release will be released?

I am not sure if there is going to be a 3.2.2 release, considering 3.3 is already in rc release stage. In case we do release 3.2.2, fix will be included.