Status Update
Comments
mm...@google.com <mm...@google.com>
mk...@google.com <mk...@google.com> #2
gavra@, do you know how the status of filtering in library consumer rules?
ga...@google.com <ga...@google.com> #3
In
However, this issue about something else. From #1 it seems like your library specifies consumerProguardFiles
. If there is an application that depends on this library (in the same Gradle build), AGP will use library consumerProguardFiles
when running R8 on the application. This is expected behavior as we want to provide consistent output when you consume this library directly (e.g. as project(':library')
and as and AAR (e.g. com.foo:bar:1.0
).
This matches Compile the AAR code in the AAR module and then have the APK module pull in the compiled AAR code and use the Consumer ProGuard file.
scenario from your first comment, but we avoid building the full AAR as it is unnecessary, and we expose individual components directly (classes/resources/consumerProguardRules etc.).
With you snippet in #1 you are causing different behavior when you consume your library as a Gradle subproject vs. when it is consumer externally. Is that intentional, and if so can you please provide a bit more details on why you need that?
tl...@gmail.com <tl...@gmail.com> #4
Thanks for your response, hopefully this will make sense.
Project info
We have a large multi-module project that consists of many Library modules and 2 App modules that serve as "demo" apps to be able to easily run/test the Library code without needing to publish AAR files. We have a 2nd multi-module project that contains a bunch of App modules that consume various combinations of the Library modules from the 1st project. So basically, the Library modules are consumed directly by the 1st project and as AARs by the 2nd. We also have other 3rd parties that have Apps that consume the AARs.
Module Info
Most of the Library modules are part of a "plugin" type architecture that are "discovered" by one of the Library modules. This "plugin" architecture is not part of the public API (thus we want to obfuscate it) and the Release variants are dependent on some classes being mapped to specific names during obfuscation in order for the plugin architecture to work (think of it like how Activities are defined in XML and the platform needs to find the classes).
Because the "plugin" AARs are already obfuscated we provide consumer rules to keep all the obfuscated code (which is technically all of it). This then causes an issue for our Demo apps since they are NOT obfuscating any of the Library code due to the consumer rules saying to keep it all. Thus Release variants of the Demo apps break unless we can exclude the consumer rules in order to allow the Demo apps to obfuscate the Library code and apply a mapping file to name certain classes to specific names.
ga...@google.com <ga...@google.com> #5
I'm still not sure if I understand the use case. I do understand there are some classes (e.g. com.foo.Bar
) in your libraries that you'd like to avoid renaming, and you'd also like to make sure that apps that consume those libraries keep com.foo.Bar
in the final APK.
However, I fail to understand why you need to avoid applying these keep rules and you need to apply manual mapping file when library is consumed from an app. You mention ... since they are NOT obfuscating any of the Library code due to the consumer rules saying to keep it all.
, but isn't that what you need in order for plugin discovery mechanism to work? I.e. your com.foo.Bar
must keep that name in the release APK.
Can you please create a small sample project with both AAR and local project consumption to illustrate the issue?
tl...@gmail.com <tl...@gmail.com> #6
In our Library code the "plugin" class is named something more robust so developers know what it is and does. Since this class is not part of a public API we obfuscate the class name to something that is not human readable to make things difficult for anyone decompiling our code. Our plugin libraries are "add on functionality" and are not directly used by consuming application code, which is why we fully obfuscate them.
So the actual plugin class may be named MyCoolPluginThatDoesMagic
but when obfuscated it's mapped to something like MM
. We have a res/raw/[moduleid].properties
file in the plugin Library that has the class name so the Library being plugged into can find and instantiate the class. So the properties file for the Debug variant needs plugin.class=com.my.package.MyCoolPluginThatDoesMagic
because its not obfuscated and the Release variant has plugin.class=com.my.package.MM
because it is.
The Library consumer rules are needed in order to prevent the plugin class from getting renamed by an AAR consumer. However, this prevents our Demo app (direct library consumer) from being able to obfuscate the plugin class name so that it matches what is in the Library's properties file.
On a side note: We used to minify the Debug variants just to rename the plugin classes, but after switching from ProGuard to R8 our debug variants are no longer able to be obfuscated, thus breaking our code.
tl...@gmail.com <tl...@gmail.com> #7
I should also mention that we did not have consumer rules previously when using ProGuard. We are trying to incorporate them now so we do not need to provide rules to 3rd parties that use our AAR files.
Hopefully #6 provides enough clarification. If not let me know and I can put together a sample project as requested in #5.
ga...@google.com <ga...@google.com> #8
To recap, there are:
-
:library
which has classsrc/main/java/MyCoolPluginThatDoesMagic.java
with classMyCoolPluginThatDoesMagic
a)
src/debug/res/raw/library.properties
withplugin.class=MyCoolPluginThatDoesMagic
b)
src/release/res/raw/library.properties
withplugin.class=MM
c) debug variant with
minifyEnabled false
, release variant withminifyEnabled true
d)
defaultConfig { consumerProguardFiles 'config.pro' } }
e)
config.pro
has-keep class MM
(among other things) -
:app
which depends on `:librarya) debug variant has
minifyEnabled false
b) release variant has
minifyEnabled true
In 2.a), debug application variant should have MyCoolPluginThatDoesMagic
in its runtime classpath, with plugin.class=MyCoolPluginThatDoesMagic
, so nothing should break.
In 2.b), release application variant should have MM
in its runtime classpath, with plugin.class=MM
. Because consumer keep rules prevent R8 from rewriting MM
to something else, things should just work.
If you can point to an incorrect assumption I made, or even better if you can share a sample project, that would definitely help. Thanks!
tl...@gmail.com <tl...@gmail.com> #9
All of your assumptions are correct except 1.e and 2.b
1.e) config.pro
has -keep class com.my.package.** {*;}
. This is because we need all classes to be kept since shrinking will remove them since none of the classes are directly referenced.
2.b) This is the use case that is currently broken. MM
is not in the classpath because of config.pro
's -keep
statement which prevents :app
from obfuscating any of the :library
classes.
More info
:library
also has buildTypes { release { proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'} }
proguard-rules.pro
has -applymapping mapping.txt
and -keep,allowobfuscation class com.my.package.** {*;}
mapping.txt
has com.my.package.MyCoolPluginThatDoesMagic -> com.my.package.MM
I'll share a sample project as soon as I can find time to throw one together, hopefully today.
tl...@gmail.com <tl...@gmail.com> #10
Here is a crude sample project that demonstrates the issue with the class name.
tl...@gmail.com <tl...@gmail.com> #11
Additional thoughts on #1
- Let the APK module compile all the code and ignore the AAR Consumer ProGuard settings
This option should not only ignore the consumer rules of the AAR module, but should technically INCLUDE the normal proguard rules of the AAR module since the APK module is compiling the AAR code.
If any proguard rules are applying mapping files then the mapping files will need to be merged together.
ga...@google.com <ga...@google.com> #12
Thank you for sharing the sample project, I can now see what's going on.
Running with ./gradlew :app:assembleRelease --debug
, logs show [R8] Program classes: ...
and one of the entries is MyApplication/library-plugin/build/intermediates/runtime_library_classes_jar/release/classes.jar
.
Then:
unzip -l /usr/local/google/home/gavra/Downloads/MyApplication/library-plugin/build/intermediates/runtime_library_classes_jar/release/classes.jar
Archive: /usr/local/google/home/gavra/Downloads/MyApplication/library-plugin/build/intermediates/runtime_library_classes_jar/release/classes.jar
Length Date Time Name
--------- ---------- ----- ----
0 1981-01-01 01:01 com/
0 1981-01-01 01:01 com/my/
0 1981-01-01 01:01 com/my/library/
0 1981-01-01 01:01 com/my/library/core/
0 1981-01-01 01:01 com/my/library/core/plugin/
0 1981-01-01 01:01 com/my/library/core/plugins/
0 1981-01-01 01:01 com/my/library/core/plugins/sample/
495 1981-01-01 01:01 com/my/library/core/plugin/BuildConfig.class
605 1981-01-01 01:01 com/my/library/core/plugins/sample/MyCoolPluginThatDoesMagic.class
Also ./gradlew :lib-plug:outgoingVariants --all
shows:
- Variant : android-classes-jar
- Attributes
- com.android.build.api.attributes.AgpVersionAttr = 7.3.1
- com.android.build.api.attributes.BuildTypeAttr = release
- com.android.build.gradle.internal.attributes.VariantAttr = release
- org.gradle.usage = java-runtime
- Artifacts
- build/intermediates/runtime_library_classes_jar/release/classes.jar (artifactType = android-classes-jar)
confirming that AGP does not publish runtime classes correctly.
ze...@google.com <ze...@google.com> #13
DBC: I'd like to caution against using applymapping as part of any library distribution. R8 has very limited support for apply mapping and it is mostly focused on enabling testing of a library/APK by allowing test references to be remapped. We don't have any official support for distributing mapping files for clients of a library to then use as part of their build. The interactions between -applymapping
and other keep rules and debug flags is largely undefined.
In other words, it is fine to use -applymapping
as part of building various artifacts to ensure consistency of your artifacts, but -applymapping
should never be part of AAR rules. If names need stable values when distributed to clients those should have fixed names via -keepX
rules.
tl...@gmail.com <tl...@gmail.com> #14
Agreed, this is what I meant when I mentioned the mapping files in #11.
In other words, it is fine to use
-applymapping
as part of building various artifacts to ensure consistency of your artifacts, but-applymapping
should never be part of AAR rules. If names need stable values when distributed to clients those should have fixed names via-keepX
rules.
ga...@google.com <ga...@google.com>
bi...@google.com <bi...@google.com>
bi...@google.com <bi...@google.com> #15
Ivan, how does agp not publish runtime classes correctly? I am not sure what is the expected behavior vs current behavior based on your
je...@google.com <je...@google.com> #16
I have not looked at the sample application, but based on
ga...@google.com <ga...@google.com> #17
When library project uses R8, jar that it publishes to the runtimeElements
should be the minified one. At the moment, we publish non-minified classes, which is incorrect. See
As an additional note, library apiElements
should contain non-minified classes as we'd expect external API surface of the library to be the same before and after minification. This is already the case, and no work is required here.
bi...@google.com <bi...@google.com> #18
In debug build our obfuscation
and optimization
are disabled. Tree-shaking
is still enabled as it is not affected by this CompilationMode
.
So in order to test obfuscation, we need to use release build.
bi...@google.com <bi...@google.com> #19
A question to Ian, I find the class file output by r8 is not being obfuscated. I guess that is not expected, right? The configuration for r8 looks correct to me. I remember there is a way to share all the details of a r8 invocation right? zerny@
ze...@google.com <ze...@google.com> #20
Regarding debug build, it does indeed disable obfuscation and optimization as both break stack-traces / stepping behavior. Tree shaking is still enabled without explicitly disabling it.
You can generate a BaseCompilerCommand.Builder.dumpInputToFile
method on the builder, but it is not a public API so you may need to use a non-lib version of R8 to access it).
bi...@google.com <bi...@google.com> #21
I think we should remove this from 8.1 beta blocking release list and probably address it in early canary because 1) it is not a regression, 2) the fix would potentially break some users because they need to add additional keep rules for a library module to make sure the consumer(app or another library) module are able to get those classes from the library module.
je...@google.com <je...@google.com> #22
this should be fixed in one of the canary, so blocking beta only.
bi...@google.com <bi...@google.com> #23
Some updates regarding this issue:
As we want to make consuming a library as project dependency consistent with consuming the same library that has been published to maven, we need to add the local file dependencies when doing the inter-project publishing for library modules.
This becomes a problem because from the consumer side, we also include file dependencies of library modules. It is part of the ArtifactScope.FILE
The challenge is how to exclude transitive file dependencies from library modules.
Option 1. Do not add this to runtimeElements when publishing -> don't seen to work, see
Option 2. Compute transitive file dependencies in the application by subtracting computeLocalFileDependencies
-> it works for some cases but not sure it can work for all because computeLocalFileDependencies
doesn't support attributes
Option 3. Publish file dependencies from library and use FilterSpec
to exclude it in application, similar to packagedDependenciesClasspath
-> I feel this might be a proper fix
bi...@google.com <bi...@google.com> #24
Talked with Jerome, the consensus is to remove this from blocking list. The next step is to see if there is a simpler way to support this use case. (e.g. using included build and publish the library to local maven)
ga...@google.com <ga...@google.com> #25
It does make sense to add support for the common use-case (library projects without local file dependencies) and to have a reasonable fallback for the other case (library projects with local file dependencies).
xa...@google.com <xa...@google.com> #26
Stumbled upon this while looking at the list of 8.3 Beta bugs
As an additional note, library apiElements should contain non-minified classes as we'd expect external API surface of the library to be the same before and after minification. This is already the case, and no work is required here.
Is that really always the case? if you publish an AAR with obfuscation, you really want the API jar (<AAR>/api.jar
) to contains the obfuscated code. I would think that the inter-project publication should be the same.
ga...@google.com <ga...@google.com> #27
I agree that ideally we'd also compile against obfuscated classes. However, I'm concerned about blocking the compilation of downstream modules on R8.
Also, there is probably a debug
variant of the consumer that will see non-obfuscated classes on the compile classpath. The IDE is also not helping as it does not know about obfuscation. So even if we do push for compile classpath to block on R8 and have obfuscated classes, users may still hit issues.
Proper seems to be introduction of api
component that allows developers to specify the library API and consumers would compile only against sources in that component.
xa...@google.com <xa...@google.com> #28
That makes sense for inter-project dependencies where the edit-build-deploy cycle would be impacted. I agree the api
component would really be useful here.
bi...@google.com <bi...@google.com> #29
Here is the doc I shared with Gradle about this issue
jedo@, can you evaluate this ticket again? (this one seems to caused quite a lot efforts but the impact is limited, not sure what is the best way moving forward)
je...@google.com <je...@google.com> #30
bi...@google.com <bi...@google.com> #32
Soren, that is exactly this bug is about.
xa...@google.com <xa...@google.com> #33
Moving this to P1. We really need to fix this.
pa...@unissey.com <pa...@unissey.com> #34
I'm the person who asked the question Soren referenced in Stack Overflow. I'm taking note that this is a known bug that's gonna get fixed later on, and I'm glad to read that.
In the meantime, I found a workaround that I've described in my answer on SO:
bi...@google.com <bi...@google.com> #35
Fixed with I5b773131437720291fe02a2897d4171596e33acd
lo...@gmail.com <lo...@gmail.com> #36
Cool! Thank you!
Is that an AS or an AGP fix? Which version(s) will first get it?
bi...@google.com <bi...@google.com> #37
It is AGP fix and you should be able to get it in AGP 8.4-alpha05
an...@google.com <an...@google.com> #38
Thank you for your patience while our engineering team worked to resolve this issue. A fix for this issue is now available in:
- Android Studio Jellyfish | 2023.3.1 Canary 4
- Android Gradle Plugin 8.4.0-alpha04
We encourage you to try the latest update.
If you notice further issues or have questions, please file a new bug report.
Thank you for taking the time to submit feedback — we really appreciate it!
lo...@gmail.com <lo...@gmail.com> #39
@
ni...@gmail.com <ni...@gmail.com> #40
ni...@gmail.com <ni...@gmail.com> #41
bi...@google.com <bi...@google.com> #42
Yeah, feel free to try it out in 8.4.0-alpha04
pa...@unissey.com <pa...@unissey.com> #43
I've just installed the release version of AGP 8.4.0 and can confirm that it resolves the issue. I can safely remove my workaround now, thank you.
bi...@google.com <bi...@google.com> #44
Glad to know that it works out! Thanks for confirming.
Description
Consumer ProGuard files should be handled better when a library module is used as a dependency in a multi-module project.
For example
my-app/build.gradle
contains the following dependency:implementation project(':my-library')
One of the following should happen.
Currently a hack workaround needs to be added to the consuming module in order to remove the consumer ProGaurd files.