DexingNoClasspathTransform (minSdk >= 24) with Java 11 target fails due to missing nest members [266599585]

Fixed

Bug

Status Update

No update yet.

Description

hu...@google.com

created issue #1

Jan 24, 2023 08:17PM

(I'm creating this bug based on Internal Issue 234414461.)

Steps to reproduce

In app, set minSdk >= 24 (when minSdk < 24, dexing transforms are not incremental)
In lib, set compileOption.targetCompatibility = "11" (in lower Java versions, nest members do not exist)
In lib, write a Java class with nested classes
Copy this project to another location. Run ./gradlew :app:mergeLibDexDebug --build-cache at that location.
Come back to the current project, run ./gradlew :app:mergeLibDexDebug --build-cache. Many tasks/transforms should have cache hits.
Make a change to the Java class above or its nested classes.
Run ./gradlew :app:mergeLibDexDebug --build-cache again. It will fail with:

* What went wrong:
Execution failed for task ':app:mergeLibDexDebug'.
> Could not resolve all files for configuration ':app:debugRuntimeClasspath'.
   > Failed to transform debug (project :lib) to match attributes {artifactType=android-dex, asm-transformed-variant=NONE, com.android.build.api.attributes.AgpVersionAttr=8.1.0-dev, com.android.build.api.attributes.BuildTypeAttr=debug, com.android.build.gradle.internal.attributes.VariantAttr=debug, dexing-enable-desugaring=true, dexing-enable-jacoco-instrumentation=false, dexing-is-debuggable=true, dexing-min-sdk=24, org.gradle.category=library, org.gradle.jvm.environment=android, org.gradle.libraryelements=classes, org.gradle.usage=java-runtime, org.jetbrains.kotlin.platform.type=androidJvm}.
      > Execution failed for DexingNoClasspathTransform: MyApplication/lib/build/intermediates/runtime_library_classes_dir/debug.
         > Error while dexing.
...
Caused by: com.android.builder.dexing.DexArchiveBuilderException: Error while dexing.
        at com.android.builder.dexing.D8DexArchiveBuilder.getExceptionToRethrow(D8DexArchiveBuilder.java:186)
        at com.android.builder.dexing.D8DexArchiveBuilder.convert(D8DexArchiveBuilder.java:125)
        at com.android.build.gradle.internal.dependency.BaseDexingTransform.process(DexingTransform.kt:316)
        at com.android.build.gradle.internal.dependency.BaseDexingTransform.processIncrementally(DexingTransform.kt:227)
        at com.android.build.gradle.internal.dependency.BaseDexingTransform.doTransform(DexingTransform.kt:137)
        at com.android.build.gradle.internal.dependency.BaseDexingTransform.transform(DexingTransform.kt:107)
...
Caused by: com.android.tools.r8.CompilationFailedException: Compilation failed to complete, origin: MyApplication/lib/build/intermediates/runtime_library_classes_dir/debug/com/example/lib/JavaClass.class
        at Version.fakeStackEntry(Version_8.1.8-dev.java:0)
...
        at com.android.builder.dexing.D8DexArchiveBuilder.convert(D8DexArchiveBuilder.java:123)
        ... 125 more
Caused by: com.android.tools.r8.utils.b: Compilation of classes com.example.lib.JavaClass requires its nest mates com.example.lib.JavaClass$NestedClass (unavailable) to be on program or class path.
...

Root cause

Dexing transforms use a desugaring graph to store dependencies among class files for incremental dexing/desugaring.

The desugaring graph's nodes are currently absolute paths, which means after a remote cache hit, it will not be usable and will result in subsequent failed incremental builds.

We were aware of this issue when designing the graph and used a workaround, but (I think) at some point Gradle changed the behavior and it no longer worked.

Potential fix

Make the desugaring graph relocatable. It is a bit difficult because there isn't one single root path to normalize the absolute file paths (instead, there are multiple root paths).

(If we can't work out a solution soon, the temporary measure is to ensure in the first build after a remote cache hit, affected transforms will be non-incremental.)

Comments

sh...@gmail.com <sh...@gmail.com> #2Mar 10, 2023 04:45PM

Comment has been deleted.

Message last modified on Mar 10, 2023 04:45PM

sh...@pinterest.com <sh...@pinterest.com> #3Mar 10, 2023 04:46PM

Almost 2 months later and this is still broken

hu...@google.com <hu...@google.com> #4Mar 10, 2023 06:40PM

Since there is no progression, I wanted to share our quick-fix for the issue.

#sdkmanager --package_file=${PATH_WORKSPACE}/packages
while read p; do echo "y" | sdkmanager "${p}"; done <${PATH_WORKSPACE}/packages

sh...@pinterest.com <sh...@pinterest.com> #5Mar 10, 2023 08:39PM

jb...@google.com What is the update on this?

hu...@google.com <hu...@google.com> #6Mar 13, 2023 05:02PM

What is the status of this item?

as...@gmail.com <as...@gmail.com> #7Mar 15, 2023 05:23PM

This has been fixed on master today (internal ref: ag/2945015) and will be available in the next SDK release.

hu...@google.com <hu...@google.com> #8Mar 17, 2023 05:47PM

Any ETA on next release?

as...@gmail.com <as...@gmail.com> #9Mar 20, 2023 07:57AM

Still broken and not updated? --package_file argument is not usable in it's current form on 26.1.1 straight from the developer site.

hu...@google.com <hu...@google.com> #10Mar 20, 2023 12:41PM

Comfirmed that this seems to still be broken. Can we have an update please?

```
(15:58:11) C02W513SHTD8:files aso$ /opt/android-sdk-macosx/tools/bin/sdkmanager --version
26.1.1

(15:58:17) C02W513SHTD8:files aso$ /opt/android-sdk-macosx/tools/bin/sdkmanager --install --package_file=package_file
Warning: Unknown argument --package_file=package_file
```

sh...@pinterest.com <sh...@pinterest.com> #11Mar 21, 2023 10:47PM

Hi, is there any update to this issue? Thanks.

Message last modified on Mar 22, 2023 08:18PM

hu...@google.com <hu...@google.com> #12Mar 22, 2023 01:19PM

Hi Google. You claim it's been fixed on master, but we haven't had a new release since the broken version 26.1.1. Can you please release the fix?

sh...@pinterest.com <sh...@pinterest.com> #13Mar 22, 2023 08:19PM

Yeah, still not fixed --'

hu...@google.com <hu...@google.com> #14Apr 6, 2023 10:55AM

Marked as fixed.

Can't believe this still isn't fixed 2 years later for a command line utility that sits on the main dev site.

as...@gmail.com <as...@gmail.com> #15Apr 11, 2023 12:45PM

Any updates on this? The help for this command clearly states this argument is supported.

hu...@google.com <hu...@google.com> #16Apr 11, 2023 01:42PM

Has anyone re-tried it? We switched back to RUN sdkmanager --package_file=$ANDROID_HOME/packages.txt in our Dockerfile back in March of 2021.

as...@gmail.com <as...@gmail.com> #17Apr 18, 2023 04:34PM

For what it's worth, I did a quick test with the latest CLI: 11076708 (https://developer.android.com/studio#downloads) on OSX and tried it with a simple:

./sdkmanager --sdk_root="../sdk" --package_file=deps.txt

Deps.txt:

platform-tools
extras;google;instantapps
build-tools;35.0.0-rc3

So perhaps this is now resolved? I haven't tried it with more packages

Message last modified on Apr 18, 2023 04:35PM

an...@google.com <an...@google.com> #18Apr 18, 2024 09:05PM

Thank you for your patience while our engineering team worked to resolve this issue. A fix for this issue is now available in:

Android Studio Koala | 2024.1.1 Canary 6
Android Gradle Plugin 8.5.0-alpha06

We encourage you to try the latest update.

If you notice further issues or have questions, please file a new bug report.

Thank you for taking the time to submit feedback — we really appreciate it!

hu...@google.com <hu...@google.com> #19Apr 19, 2024 10:16AM

FYI: Comment #18 is an automated comment, referring to a small improvement to the fix in comment #14. It will be available in AGP 8.5.0-alpha06+: https://android.googlesource.com/platform/tools/base/+/8473e5fa2f36600947d29a49b6cce17ee46dbc45

Message last modified on Apr 19, 2024 10:17AM

ma...@gmail.com <ma...@gmail.com> #20Jun 22, 2024 04:09AM