AGP 7.4.0-rc01 breaks Variant API with "Querying the mapped value of map(provider(java.util.Set)) before task '...' has completed is not supported" [261329823]

Fixed

Bug

Status Update

No update yet.

Description

nc...@meta.com

created issue #1

Dec 5, 2022 01:43PM

This bug should be moved under:
> Android Public Tracker > App Development > Android Studio > Gradle > Android Gradle Plugin
but I don't have permission to do so.

## Expected Result

AGP Variant API works well between 7.4.0-beta05 and 7.4.0-rc01

## Observed Results

Bumping AGP from 7.4.0-beta05 to 7.4.0-rc01 results in a build failure with the following message:

```
> Task :example:mapDebugSourceSetPaths FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':example:mapDebugSourceSetPaths'.
> Querying the mapped value of map(provider(java.util.Set)) before task ':example:reproducerdebugTask' has completed is not supported
```

It seems like AGP is querying the value of the `@OutputDirectory` of the task I created `example:reproducerdebugTask`.

## Reproducer

You can find a small reproducer here:

https://github.com/cortinico/reproducer-agp-74rc01

Specifically this commit shows a working setup with AGP 7.4 beta5
-

https://github.com/cortinico/reproducer-agp-74rc01/commit/89ad099f44ecbb2bed53ece979c2f1bd241fe31f

Bumping to RC01 causes the failure:
-

https://github.com/cortinico/reproducer-agp-74rc01/commit/6fc570e9492b4413a013d9031aba5ceb80f08a65

I've also configured the CI so you have a clean environment where you can easily reproduce.
The command to reproduce upon cloning is `./gradlew :example:assembleDebug`.

## Further Context

This issue was noticed upstream on React Native here:

https://github.com/facebook/react-native/pull/35522
as we're using a similar pattern inside the React Native Gradle Plugin.

Comments

vi...@google.com <vi...@google.com> Dec 5, 2022 01:50PM

Status: New

ga...@google.com <ga...@google.com> #2Dec 5, 2022 06:42PM

Assigned to je...@google.com.

Will be this fix also part of 4.2 please?

je...@google.com <je...@google.com> Dec 5, 2022 07:32PM

Accepted by je...@google.com.

je...@google.com <je...@google.com> #3Dec 6, 2022 05:57PM

Marked as fixed.

for the "out" bundle issue, this seems indeed an issue. for the second one reported adding "debugUpdateAPKArtifact" to the path, this WAI as there can more more than one transformer and we need to automatically separate them so they do not use the same output folder. for #3, I will look at it but it might be difficult to change it at this time.

nc...@meta.com <nc...@meta.com> #4Dec 6, 2022 06:16PM

I understand that if there is more transformer is necessary to make every folder different, but there is the main problem, that the final one (last transform) is inside of the intermediate folder. The last transformation should end in the standard location, right? Maybe also with the default name not?

je...@google.com <je...@google.com> #5Dec 6, 2022 06:30PM

Hello, this is a P1 S1 issue for a while now, and it's blocking us from upgrading to AGP 4.2 and further. What's the current status? And do you have plans to backport it to 4.2.x?

[Deleted User] <[Deleted User]> #6Dec 8, 2022 02:00PM

no we don't have a plan to backport this. why is this blocking you from upgrading ?

le...@gmail.com <le...@gmail.com> #7Dec 8, 2022 02:16PM

We sign apk's and bundles via internal service, and i don't know another way to interact with artifacts; Yesterday i ended up with some ugly workaround: https://github.com/avito-tech/avito-android/pull/1085/files#diff-b898cc20fedebe394439bb37ffb91d3366a1f9a6d359dbdebca79af8a079f911 I'd be glad if you'll point me out how can it be done better.

ca...@gmail.com <ca...@gmail.com> #8Dec 15, 2022 08:14PM

I commented on the merge request but to summarize :

you can set your signing tasks output folder to wherever you need to be :

artifacts.use(target.tasks.signedApkTaskProvider(this)).configure { signedDirProperty.set(File("/path/to/where/you/want/your/signed/files")) }

if you don't set it, then we will set a directory automatically.

je...@google.com <je...@google.com> #9Dec 16, 2022 05:49PM

Thanks, Jerome!

nc...@meta.com <nc...@meta.com> #10Dec 22, 2022 12:55PM

I checked that we do have correct behavior for FILE based artifacts like bundle :

> Task :app:debugUpdateArtifact
originalArtifact: /usr/local/google/home/jedo/src/studio-main/out/apiTests/Kotlin/bugTest/app/build/intermediates/bundle/debug/signDebugBundle/app-debug.aab
updatedArtifact: /usr/local/google/home/jedo/src/studio-main/out/apiTests/Kotlin/bugTest/app/build/outputs/bundle/debug/app-debug.aab

however, we still seem to not be consistent for the DIRECTORY based artifacts like APK:

> Task :app:debugUpdateAPKArtifact
Input folder: /usr/local/google/home/jedo/src/studio-main/out/apiTests/Kotlin/bugTest/app/build/outputs/apk/debug
Output folder: /usr/local/google/home/jedo/src/studio-main/out/apiTests/Kotlin/bugTest/app/build/intermediates/apk/debug
Input file: /usr/local/google/home/jedo/src/studio-main/out/apiTests/Kotlin/bugTest/app/build/outputs/apk/debug/app-debug.apk

Alex, can you have a look ?

ga...@google.com <ga...@google.com> Dec 22, 2022 01:13PM

Status: Assigned (reopened)

xa...@google.com <xa...@google.com> #11Dec 22, 2022 05:11PM

correction, I was not correct about FILE being correct.

Instead of :

updatedArtifact: /usr/local/google/home/jedo/src/studio-main/out/apiTests/Kotlin/bugTest/app/build/outputs/bundle/debug/app-debug.aab

it should be

updatedArtifact: /usr/local/google/home/jedo/src/studio-main/out/apiTests/Kotlin/bugTest/app/build/outputs/app-debug.aab
``

ga...@google.com <ga...@google.com> #12Dec 23, 2022 05:28PM

Marked as fixed.

Two requests are created as they are API changes b/241581686 (set artifact name for transformation) and b/241582907 (change string to OutputFile)

pe...@gmail.com <pe...@gmail.com> #13Dec 29, 2022 12:11PM

Fix will be available in Android Studio F/AGP 8.0 Canary 2 Release

ro...@sentry.io <ro...@sentry.io> #14Jan 13, 2023 12:19AM

I tried to use it and at first look, it looks like everything works as expected. Thanks!

Message last modified on Jan 13, 2023 12:20AM

nc...@meta.com <nc...@meta.com> #15Jan 13, 2023 12:50AM

The big issue is, that the current AGP 7.4.0 is broken. I tried to use this API and it is still broken in 7.4.0, it is probably really in AGP 8 :-(

pe...@gmail.com <pe...@gmail.com> #16Jan 13, 2023 01:15PM

* now waiting

ha...@gmail.com <ha...@gmail.com> #17Jan 24, 2023 09:25AM

Any update on when 7.4.1 will be out?

Everything is jammed till then

je...@google.com <je...@google.com> #18Jan 24, 2023 05:43PM

So far, it is scheduled for Feb 1st which looks likely at this point.

[Deleted User] <[Deleted User]> #19Feb 2, 2023 08:27AM

The error does not seem to be fixed in 7.4.1. Can you confirm this is indeed released?

Message last modified on Feb 2, 2023 08:29AM

nc...@meta.com <nc...@meta.com> #20Feb 2, 2023 02:37PM

Actually the issue looks like fixed on our end. We're bumping React Native to use AGP 7.4.1 and the addGeneratedSourceDirectory API here: https://github.com/facebook/react-native/pull/36039

Thanks for working on this folks even if it took longer to land than expected

je...@google.com <je...@google.com> #21Feb 2, 2023 06:01PM

#19, I am sorry, there was a misunderstanding with the release team. they had an even more urgent fix to push which was 7.4.1 and the normal dot release that contains this fix got pushed to 7.4.2 which is looking to be couple of weeks away (still working the date with them). So sorry for the wrong info, It surprised me too.

nc...@meta.com <nc...@meta.com> #22Feb 2, 2023 06:30PM

That's confusing as the issue seems effectively fixed in 7.4.1, also in the reproducer I linked: https://github.com/cortinico/reproducer-agp-74rc01

pe...@gmail.com <pe...@gmail.com> #23Feb 2, 2023 07:06PM

I tested some more today. First I got the error like mentioned in #19 (looked like it at least), and 2 hours later it seemed to build, at least the app was deployed on the emulator.
For a workmate it also failed on his first try. We might see tomorrow.
Not sure yet, but it might be flaky now, depending on some cache or other internals.

I'll report back if I know more but it's good to know about the new target of 7.4.2. Really sad this kind of thing can't make it into a .0 release as prio 0 bug, especially since it *was* ready.
Maybe you guys can take this as a feedback and improve something, feels like it just hit a process / politics wall.

ar...@gmail.com <ar...@gmail.com> #24Mar 7, 2023 12:45PM

This still seems to be plaguing 7.4.2 as well

je...@google.com <je...@google.com> #25Mar 7, 2023 10:27PM

this might be due to a third party plugin. do you have a small project to reproduce the issue ?

can you provide a stack trace ?

ar...@gmail.com <ar...@gmail.com> #26Mar 14, 2023 05:21AM

my bad, it was due to react native dependencies, had to update those as well