aapt2FromMaven task input prevents build cache from being shared across different OSes [126775254]

Fixed

Bug

Reviewed (L1)

Status Update

No update yet.

Description

da...@robinhood.com

created issue #1

Feb 28, 2019 11:06PM

The aapt2FromMaven task input (on tasks such as `LinkAndroidResForBundleTask`, `LinkLibraryAndroidResourcesTask`, `ProcessAndroidAppResourcesTask` from what I can tell) prevents build caches from being shared across different OSes, such as a Linux CI server and Mac developer machine.

Gradle version: 5.2.1
Android Plugin Version: 3.4.0-beta05
Module Compile SDK Version: 28
Module Build Tools Version: 28.0.3
Android SDK Tools version: 26.1.1

Using org.gradle.debug.caching=true, a sample task from a Mac:
```
> Task :lib-utils:packageExternalDebugResources
Appending taskImplementation to build cache key: com.android.build.gradle.tasks.MergeResources_Decorated@e12697eb158f1775566cb99466820a3b
Appending actionImplementation to build cache key: _BuildScript_$_run_closure12$_closure36$_closure37_c85dff5d029b744ca8c0d5613125abbe@edeb91f3c59fae47d4c97178ba353293
Appending actionImplementation to build cache key: com.android.build.gradle.tasks.MergeResources_Decorated@e12697eb158f1775566cb99466820a3b
Appending inputValuePropertyHash for 'buildToolsVersion' to build cache key: 7d35fa9db500a15cb0bcdb8cc065fa9c
Appending inputValuePropertyHash for 'crunchPng' to build cache key: c06857e9ea338f3f3a24bb78f8fbdf6f
Appending inputValuePropertyHash for 'flags' to build cache key: 7161b3446440a25765674e65e88bdac2
Appending inputValuePropertyHash for 'generatedDensities' to build cache key: 9a29b6f8002e6a19e897f5837201b01e
Appending inputValuePropertyHash for 'minSdk' to build cache key: 91e6b2f4b53f9e7b3eadb03039173cc5
Appending inputValuePropertyHash for 'processResources' to build cache key: c06857e9ea338f3f3a24bb78f8fbdf6f
Appending inputValuePropertyHash for 'pseudoLocalesEnabled' to build cache key: c06857e9ea338f3f3a24bb78f8fbdf6f
Appending inputValuePropertyHash for 'validateEnabled' to build cache key: f6d7ed39fe24031e22d54f3fe65b901c
Appending inputValuePropertyHash for 'vectorSupportLibraryUsed' to build cache key: c06857e9ea338f3f3a24bb78f8fbdf6f
Appending inputFilePropertyHash for 'aapt2FromMaven' to build cache key: 0671cc197976280e6ac013599d270a5b
Appending inputFilePropertyHash for 'extraGeneratedResFolders' to build cache key: 5fd1e7396e8de4cb5c23dc6aadd7787a
Appending inputFilePropertyHash for 'generatedResOutputDir' to build cache key: 4fd5bfd658339311850833b13545918d
Appending inputFilePropertyHash for 'libraries' to build cache key: 5fd1e7396e8de4cb5c23dc6aadd7787a
Appending inputFilePropertyHash for 'microApkResDirectory' to build cache key: 5fd1e7396e8de4cb5c23dc6aadd7787a
Appending inputFilePropertyHash for 'renderscriptResOutputDir' to build cache key: 729b3548b9cd12752e34124746197fda
Appending inputFilePropertyHash for 'resources' to build cache key: fb5bba780c71c1cca1335d37cdf5f80b
Appending inputFilePropertyHash for 'sourceFolderInputs' to build cache key: fb5bba780c71c1cca1335d37cdf5f80b
Appending outputPropertyName to build cache key: generatedPngsOutputDir
Appending outputPropertyName to build cache key: incrementalFolder
Appending outputPropertyName to build cache key: outputDir
Appending outputPropertyName to build cache key: publicFile
Build cache key for task ':lib-utils:packageExternalDebugResources' is 361260430e5e87077160f6e21b4155e6
```
aapt2FromMaven 0671cc197976280e6ac013599d270a5b: [$GRADLE_HOME/caches/transforms-2/files-2.1/cf7d0196c489f6c2a6db1113f68c7c9c/aapt2-3.4.0-beta05-5303311-osx]

And the same task from an Ubuntu CI server:
```
> Task :lib-utils:packageExternalDebugResources
Appending taskImplementation to build cache key: com.android.build.gradle.tasks.MergeResources_Decorated@e12697eb158f1775566cb99466820a3b
Appending actionImplementation to build cache key: _BuildScript_$_run_closure12$_closure36$_closure37_c85dff5d029b744ca8c0d5613125abbe@edeb91f3c59fae47d4c97178ba353293
Appending actionImplementation to build cache key: com.android.build.gradle.tasks.MergeResources_Decorated@e12697eb158f1775566cb99466820a3b
Appending inputValuePropertyHash for 'buildToolsVersion' to build cache key: 7d35fa9db500a15cb0bcdb8cc065fa9c
Appending inputValuePropertyHash for 'crunchPng' to build cache key: c06857e9ea338f3f3a24bb78f8fbdf6f
Appending inputValuePropertyHash for 'flags' to build cache key: 7161b3446440a25765674e65e88bdac2
Appending inputValuePropertyHash for 'generatedDensities' to build cache key: 9a29b6f8002e6a19e897f5837201b01e
Appending inputValuePropertyHash for 'minSdk' to build cache key: 91e6b2f4b53f9e7b3eadb03039173cc5
Appending inputValuePropertyHash for 'processResources' to build cache key: c06857e9ea338f3f3a24bb78f8fbdf6f
Appending inputValuePropertyHash for 'pseudoLocalesEnabled' to build cache key: c06857e9ea338f3f3a24bb78f8fbdf6f
Appending inputValuePropertyHash for 'validateEnabled' to build cache key: f6d7ed39fe24031e22d54f3fe65b901c
Appending inputValuePropertyHash for 'vectorSupportLibraryUsed' to build cache key: c06857e9ea338f3f3a24bb78f8fbdf6f
Appending inputFilePropertyHash for 'aapt2FromMaven' to build cache key: dc0bd7515983a946c274fe8806691c04
Appending inputFilePropertyHash for 'extraGeneratedResFolders' to build cache key: 5fd1e7396e8de4cb5c23dc6aadd7787a
Appending inputFilePropertyHash for 'generatedResOutputDir' to build cache key: 4fd5bfd658339311850833b13545918d
Appending inputFilePropertyHash for 'libraries' to build cache key: 5fd1e7396e8de4cb5c23dc6aadd7787a
Appending inputFilePropertyHash for 'microApkResDirectory' to build cache key: 5fd1e7396e8de4cb5c23dc6aadd7787a
Appending inputFilePropertyHash for 'renderscriptResOutputDir' to build cache key: 729b3548b9cd12752e34124746197fda
Appending inputFilePropertyHash for 'resources' to build cache key: fb5bba780c71c1cca1335d37cdf5f80b
Appending inputFilePropertyHash for 'sourceFolderInputs' to build cache key: fb5bba780c71c1cca1335d37cdf5f80b
Appending outputPropertyName to build cache key: generatedPngsOutputDir
Appending outputPropertyName to build cache key: incrementalFolder
Appending outputPropertyName to build cache key: outputDir
Appending outputPropertyName to build cache key: publicFile
Build cache key for task ':lib-utils:packageExternalDebugResources' is 14626b2cb3a29772ed7c0c8f542e3609
```
aapt2FromMaven dc0bd7515983a946c274fe8806691c04: [$GRADLE_HOME/caches/transforms-2/files-2.1/6843ee0839fffc0ab98c5c7898e122ed/aapt2-3.4.0-beta05-5303311-linux]

Comments

xa...@google.com <xa...@google.com> #2Feb 28, 2019 11:37PM

Assigned to cm...@google.com.

My guess is that we use the aapt tool itself as an @InputFile to invalidate the cache if the tool change but this breaks cross-OS cache.

I guess we should stop doing this and use version of the tool instead.

je...@google.com <je...@google.com> Mar 1, 2019 01:54AM

Reassigned to st...@google.com.

ku...@gmail.com <ku...@gmail.com> #3May 2, 2019 11:18PM

This input also stops the MergeResources task from being a remote cache hit:

https://android.googlesource.com/platform/tools/base/+/studio-master-dev/build-system/gradle-core/src/main/java/com/android/build/gradle/tasks/MergeResources.java#608

which is the most expensive task of our multi-module project. Any plan to fix this in AGP 3.5?

[Deleted User] <[Deleted User]> #4Jun 24, 2019 05:52PM

As I understand it, the aapt2 binary must have the same behavior on different OS (arguments -> output files).
So I don't see the reasons to use it as Input in terms of Gradle.

@InputFiles // <---- Can it be @Internal ?
@PathSensitive(PathSensitivity.RELATIVE)
public FileCollection getAapt2FromMaven() {
return aapt2FromMaven;
}

xa...@google.com <xa...@google.com> #5Jun 24, 2019 07:01PM

the path to aapt should be @Internal but we need to replace it with an @Input on the aapt version.

je...@google.com <je...@google.com> #6Jun 24, 2019 07:17PM

I think most of it is already implemented. Steven ?

st...@google.com <st...@google.com> #7Jun 24, 2019 07:45PM

Marked as fixed.

Apologies: I forgot to update this bug when I merged my change.

All tasks with an `aapt2FromMaven` Input now have that property marked as @Internal and have a new `@Input String getAapt2Version` as recommended.

Change-Id: I30cb9d47ef5d1a6b750d25813e89cb439333d14a

ku...@gmail.com <ku...@gmail.com> #8Jun 24, 2019 07:59PM

Thanks for fixing. Can we expect to see this in the next 3.5 release?