IssueTracker

Input sizes are the same:

$ unzip -d /tmp/8.1 ~/Downloads/r8_8_1_dump.zip  > /dev/null
$ unzip -d /tmp/8.3 ~/Downloads/r8_8_3_dump.zip  > /dev/null
$ ls -l /tmp/8.1
total 946048
-rw-r--r--@ 1 ricow  wheel        407 Jan  1  1970 build.properties
-rw-r--r--@ 1 ricow  wheel         22 Jan  1  1970 classpath.jar
-rw-r--r--@ 1 ricow  wheel    5292598 Jan  1  1970 library.jar
-rw-r--r--@ 1 ricow  wheel  460932086 Jan  1  1970 program.jar
-rw-r--r--@ 1 ricow  wheel    1874319 Jan  1  1970 proguard.config
-rw-r--r--@ 1 ricow  wheel         26 Jan  1  1970 r8-version
-rw-r--r--@ 1 ricow  wheel    6902099 Jan  1  1970 startup-profile-1.txt
$ ls -l /tmp/8.3
total 928896
-rw-r--r--@ 1 ricow  wheel        407 Jan  1  1970 build.properties
-rw-r--r--@ 1 ricow  wheel         22 Jan  1  1970 classpath.jar
-rw-r--r--@ 1 ricow  wheel    5292598 Jan  1  1970 library.jar
-rw-r--r--@ 1 ricow  wheel  460932086 Jan  1  1970 program.jar
-rw-r--r--@ 1 ricow  wheel    1874319 Jan  1  1970 proguard.config
-rw-r--r--@ 1 ricow  wheel         26 Jan  1  1970 r8-version
-rw-r--r--@ 1 ricow  wheel    6902099 Jan  1  1970 startup-profile-1.txt

Marker

$ tools/extractmarker.py --no-build ~/Downloads/r8_8_3.apk 
/Users/ricow/Downloads/r8_8_3.apk: ~~R8{"backend":"dex","compilation-mode":"release","has-checksums":false,"min-api":21,"pg-map-id":"9840a01","r8-mode":"full","sha-1":"engineering","version":"8.3.21"}
/Users/ricow/Downloads/r8_8_3.apk: ~~R8{"compilation-mode":"release","has-checksums":false,"pg-map-id":"1c326f6","r8-mode":"compatibility","version":"2.2.64"}
/Users/ricow/Downloads/r8_8_3.apk: ~~R8{"compilation-mode":"release","has-checksums":false,"pg-map-id":"e8a35cb","r8-mode":"compatibility","version":"2.2.71"}
/Users/ricow/Downloads/r8_8_3.apk: ~~R8{"compilation-mode":"release","pg-map-id":"91046de","version":"1.5.68"}

$ tools/extractmarker.py --no-build ~/Downloads/r8_8_1.apk 
Running: /Users/ricow/src/r8/third_party/openjdk/jdk-11/osx/Contents/Home/bin/java -ea -jar /Users/ricow/src/r8/build/libs/r8.jar extractmarker /Users/ricow/Downloads/r8_8_1.apk
/Users/ricow/Downloads/r8_8_1.apk: ~~R8{"backend":"dex","compilation-mode":"release","has-checksums":false,"min-api":21,"pg-map-id":"bdf8d82","r8-mode":"full","sha-1":"engineering","version":"8.1.71"}
/Users/ricow/Downloads/r8_8_1.apk: ~~R8{"compilation-mode":"release","has-checksums":false,"pg-map-id":"1c326f6","r8-mode":"compatibility","version":"2.2.64"}
/Users/ricow/Downloads/r8_8_1.apk: ~~R8{"compilation-mode":"release","has-checksums":false,"pg-map-id":"e8a35cb","r8-mode":"compatibility","version":"2.2.71"}
/Users/ricow/Downloads/r8_8_1.apk: ~~R8{"compilation-mode":"release","pg-map-id":"91046de","version":"1.5.68"}

DexSegments:

$ tools/dexsegments.py --no-build ~/Downloads/r8_8_1.apk 
Segments in dex application (name: size / items):
 - EncodedArrays: 6312 / 472
 - Header: 896 / 8
 - DebugInfo: 329540 / 7379
 - Fields: 3693232 / 461654
 - AnnotationSetRefs: 1920 / 115
 - Strings: 1613108 / 403277
 - Maps: 1748 / 8
 - Protos: 1776720 / 148060
 - Methods: 3717728 / 464716
 - Types: 746772 / 186693
 - StringData: 8466179 / 403277
 - ClassData: 3604781 / 75381
 - TypeLists: 1301662 / 104814 (TypeItems: 882406 / 441203)
 - AnnotationsDirectory: 407696 / 10553
 - Annotation: 427094 / 25580
 - AnnotationSets: 211192 / 23751
 - ClassDefs: 2572992 / 80406
 - Code: 39404636 / 316871

$ tools/dexsegments.py --no-build ~/Downloads/r8_8_3.apk 
Segments in dex application (name: size / items):
 - EncodedArrays: 6371 / 475
 - Header: 1008 / 9
 - DebugInfo: 272115 / 3317
 - Fields: 4113048 / 514131
 - AnnotationSetRefs: 1960 / 118
 - Strings: 1799220 / 449805
 - Maps: 1968 / 9
 - Protos: 1805376 / 150448
 - Methods: 4376480 / 547060
 - Types: 941140 / 235285
 - StringData: 8704261 / 449805
 - ClassData: 5169936 / 141183
 - TypeLists: 1318242 / 111871 (TypeItems: 870758 / 435379)
 - AnnotationsDirectory: 407464 / 10546
 - Annotation: 431122 / 25851
 - AnnotationSets: 213192 / 24001
 - ClassDefs: 4683808 / 146369
 - Code: 39785417 / 425116

From a size perspective, the code section did not grow that much, but we have way more code objects and methods

Dex file distributsion:

$ zipinfo ~/Downloads/r8_8_1.apk | grep classes
-rw----     2.0 fat  6385348 bl defN 10-Jan-01 00:00 classes.dex
-rw----     2.0 fat  8574892 bl defN 10-Jan-01 00:00 classes2.dex
-rw----     2.0 fat  8778548 bl defN 10-Jan-01 00:00 classes3.dex
-rw----     2.0 fat  9313912 bl defN 10-Jan-01 00:00 classes4.dex
-rw----     2.0 fat  8945540 bl defN 10-Jan-01 00:00 classes5.dex
-rw----     2.0 fat  8824588 bl defN 10-Jan-01 00:00 classes6.dex
-rw----     2.0 fat  7379332 bl defN 10-Jan-01 00:00 classes7.dex
-rw----     2.0 fat 10082048 bl defN 10-Jan-01 00:00 classes8.dex
$ zipinfo ~/Downloads/r8_8_3.apk | grep classes
-rw----     2.0 fat  6205352 bl defN 10-Jan-01 00:00 classes.dex
-rw----     2.0 fat  8579808 bl defN 10-Jan-01 00:00 classes2.dex
-rw----     2.0 fat  8134600 bl defN 10-Jan-01 00:00 classes3.dex
-rw----     2.0 fat  9127084 bl defN 10-Jan-01 00:00 classes4.dex
-rw----     2.0 fat  8216176 bl defN 10-Jan-01 00:00 classes5.dex
-rw----     2.0 fat  8466532 bl defN 10-Jan-01 00:00 classes6.dex
-rw----     2.0 fat  8535776 bl defN 10-Jan-01 00:00 classes7.dex
-rw----     2.0 fat  8509564 bl defN 10-Jan-01 00:00 classes8.dex
-rw----     2.0 fat  8257236 bl defN 10-Jan-01 00:00 classes9.dex

I will start bisecting this.

Caused by https://r8.googlesource.com/r8/+/e98da634de940e3d0098e0467d08af80c532139d.

FYI, we have a few ideas to mitigate this. Assigning to Rico who is currently testing solutions.

Ben: if you take 8.3.25 and change:

-if class * {
  @com.google.gson.annotations.SerializedName <fields>;
}
-keepclasseswithmembers,allowobfuscation class <1> {
  <init>(...);
}

to

-if class *
-keepclasseswithmembers,allowobfuscation class <1> {
  @com.google.gson.annotations.SerializedName <fields>;
  <init>(...);
}

You should be back below 8.1 sizes as a work around

The current fixes are for not pinning on -if class *, and also not with member specifications, but the annotation <fields> is somehow tripping us up (and does not easily reproduce with a simple example) I still need to figure out why that is, the size differences are pretty massive

I don't think we'll be able to make that change due to this: https://partnerissuetracker.corp.google.com/issues/280658459#comment4

re #16: Not sure I follow, in what case would the first keep rule keep more than the second?

Iirc, The issue we ran into using the rule you suggested was that we were using the type in a generic for Gson deserialization but never referencing the type directly. so the type would be erased in the generic to Object leading to the Gson code to crash.

Morten suggested the rule we have now which somehow ensures the type wasn't removed. The rule is quite complex so I'm not exactly sure why it works better. I think there's a test case that captures this behavior I believe

Benjamin is right that

-if class *
-keepclasseswithmembers,allowshrinking,allowobfuscation class <1> {
  <init>(...);
  @com.google.gson.annotations.SerializedName <fields>;
}

require that both at least one <init> and a @com.google.gson.annotations.SerializedName annotated field is present, whereas the other will keep the <init>'s for all classes with a @com.google.gson.annotations.SerializedName annotated field.

Note that you should probably change <init>(...) to <init>() to match the rules landing in the next version of Gson. Gson only ever use the no-args constructor reflectively.

It seems we mark 184,827 classes as ineligible for class merging due to -if rules. Many of these classes are completely unrelated to gson (e.g., androidx.annotation.Keep). I will try to diagnose why these classes are being marked as ineligible.

Ben, can you check if updating to 8.3.28 resolves the issue?

You may still see a small regression, since https://r8.googlesource.com/r8/+/e98da634de940e3d0098e0467d08af80c532139d is fixing a situation where R8 would not always evaluate -if rules correctly, but hopefully not anywhere near the regression you reported here.

With R8 8.3.28, we observe app size reduced by 90KB.

Btw, Soren when I replaced our Gson rules with the rules from upstream, our app size increases by 600KB.

Here are the rules we are using for GSON:

##---------------Begin: proguard configuration for Gson  ----------
# Gson uses generic type information stored in a class file when working with fields. Proguard
# removes such information by default, so configure it to keep all of it.
-keepattributes Signature

# For using GSON @Expose annotation
-keepattributes *Annotation*

# Prevent proguard from stripping interface information from TypeAdapterFactory,
# JsonSerializer, JsonDeserializer instances (so they can be used in @JsonAdapter)
-keep,allowobfuscation class * implements com.google.gson.TypeAdapterFactory
-keep,allowobfuscation class * implements com.google.gson.JsonSerializer
-keep,allowobfuscation class * implements com.google.gson.JsonDeserializer

# @SerializedName annotation
-keep @interface com.google.gson.annotations.SerializedName { *; }

# Do not strip any field that is annotated with @SerializedName
-keepclassmembers,allowobfuscation class * {
    @com.google.gson.annotations.SerializedName <fields>;
}

# Ensure init methods are preserved if class has @SerializedName annotations
# Ref: https://partnerissuetracker.corp.google.com/issues/280658459#comment4
-if class * {
  @com.google.gson.annotations.SerializedName <fields>;
}
-keepclasseswithmembers,allowobfuscation class <1> {
  <init>(...);
}
-if class *
-keepclasseswithmembers,allowshrinking,allowobfuscation class <1> {
  @com.google.gson.annotations.SerializedName <fields>;
}

##---------------End: proguard configuration for Gson  ----------

Note, when doing this comparison I didn't switch the <init>(...) with <init>() yet.

Thanks for the update. I will mark this as fixed then.

Thank you for your patience while our engineering team worked to resolve this issue. A fix for this issue is now available in:

• Android Studio Iguana | 2023.2.1 Beta 2
• Android Gradle Plugin 8.3.0-beta02

We encourage you to try the latest update.

If you notice further issues or have questions, please file a new bug report.

Thank you for taking the time to submit feedback — we really appreciate it!

The fixes for this issue are now also available in:

• Android Studio Hedgehog | 2023.1.1 Patch 2
• Android Gradle Plugin 8.2.2

We encourage you to try the latest update.

If you notice further issues or have questions, please file a new bug report.

Comment added by automation: A postmortem has been automatically requested for this issue. Please author a postmortem on IRM. See go/android-postmortem-guidance for more context and details, including how to handle the case when this issue does not need a postmortem. If you have any questions not answered by go/android-postmortem-guidance, please email android-hygiene-tpm@, instead of reassigning the fixed bug. Thank you.

Further fixes for this issue are now available in:

• Android Studio Iguana | 2023.2.1 RC 1
• Android Gradle Plugin 8.3.0-rc01

We encourage you to try the latest update.

If you notice further issues or have questions, please file a new bug report.