IssueTracker

I attached a repro project. This is the stacktrace printed from the Retrofit error (I included the keep rules for the Kodein issue):

java.lang.ClassCastException: java.lang.Class cannot be cast to java.lang.reflect.ParameterizedType
        at retrofit2.HttpServiceMethod.parseAnnotations(HttpServiceMethod.java:365)
        at retrofit2.Retrofit.loadServiceMethod(Retrofit.java:4)
        at retrofit2.Retrofit$1.invoke(Retrofit.java:7)
        at java.lang.reflect.Proxy.invoke(Proxy.java:1006)
        at $Proxy1.getAllBreeds(Unknown Source)
        at com.example.api.DogBreedApiImpl.getAllBreeds(DogBreedApi.kt:5)
        at com.example.kodein7.MainActivity$onCreate$1.invokeSuspend(MainActivity.kt:5)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:3)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:18)
        at android.os.Handler.handleCallback(Handler.java:938)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:223)
        at android.app.ActivityThread.main(ActivityThread.java:7660)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)

R8 fullmode will remove all attributes for classes that are not kept. Attributes and annotations can only be read by reflection and for full mode we require an explicit keep rule to keep the signatures (and other attributes).

As you point to, adding keep rules for the types in kodein will force R8 to keep the signatures.

For retrofit, you will have to keep the api classes. Interestingly, you also have to keep the kotlin.coroutines.Continuation as well. Adding the below rules fixes the issue in the sample for me (note, you should refine the keep rule for the api to only keep what is needed).

-keep class com.example.api.** { *; }
-keep class kotlin.coroutines.Continuation

I will make a pull request on retrofit to add kotlin.coroutines.Continuation to the rules we consume.

Will R8 strip the signature even if I have -keepattributes Signature set in my rules?

If so, is there a rule that could be used to keep the signature just for the api classes?

Yes, R8 in full-mode will strip all signatures for not-kept classes if using -keepattributes Signature. The only way to keep the generic signature for a class is if you keep it.

Having a rule for only keeping certain attributes would actually be quite nice, sort of like:

-keep,Signature class com.example.api.* { *; }

would allow developers to more fine-grained describe what they would like to keep. It would break compatibility though.

We are btw removing all attributes for not kept classes, such as InnerClasses, EnclosingMethod etc.

I found that -keep, allowobfuscation, allowoptimization class com.example.api.**Endpoints { *; } fixes the issue (at least in the sample project).

Is this new behavior for 7.0.0 (or whatever version of R8 is used with it)? In AGP 4.2.0 this wasn't happening.

Maybe somewhat related, but it looks like consumerProguardFile isn't working on AGP 7.0.0-beta01. I tried adding these rules to the consumer-rules.pro file in their respective modules in my real project, and it didn't work unless I put it into my app module's proguard file.

The library gradle file (technically a buildSrc precompiled script) looks like:

plugins {
  id("com.android.library")
}

android {
  compileSdk = project.compileSdkVersion

  defaultConfig {
    consumerProguardFile(project.file("consumer-rules.pro"))

    minSdk = project.minSdkVersion

    testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
  }

  ...
}

Is this new behavior for 7.0.0

Yes. It has been on our road-map for ages, but we did not model generic signatures correctly before and we therefore needed inner-classes and enclosing method attributes to have a somewhat correct output. Now that we model them we can remove a lot more attributes.

I am actually unsure how the whole build setup is when the library is local to the project but I guess it should not make a difference. Can I use the same project as before to reproduce your problem?

Oh that's slightly unfortunate (for me I guess). I've been using full-mode for years, but I think I need to turn it off now because it's affecting too much of my code (it's a very large project). Every time I fix one issue, there's another one that pops up right after it (looks like any class that uses generics with reflection).

Although, I just tested with android.enableR8.fullMode=false and removed the new keep rules, and I'm still seeing these issues.

I'll try to repro the consumerProguardFile and the fullMode=false in the sample project.

Oh actually it looks like with fullMode=false I still need:

-keep, allowobfuscation, allowoptimization class org.kodein.type.TypeReference
-keep, allowobfuscation, allowoptimization class org.kodein.type.JVMAbstractTypeToken$Companion$WrappingTest

-keep, allowobfuscation, allowoptimization class * extends org.kodein.type.TypeReference
-keep, allowobfuscation, allowoptimization class * extends org.kodein.type.JVMAbstractTypeToken$Companion$WrappingTest

but none of the other keep rules I had to define for my project (including the retrofit endpoint classes or Continuation).

Still looking at the consumerProguardFile in the sample project.

I managed to get all of the rules needed to keep fullMode=true. It wasn't fun, and I'm sure this will cause issues in the future if there is a new type that requires its signature being kept for reflection, etc...

This was the final set of rules I needed to add:

-keep class kotlin.coroutines.Continuation

-keep, allowobfuscation, allowoptimization class retrofit2.Response

-keep, allowobfuscation, allowoptimization class com.myapp.api.**Endpoints { *; }

-keep, allowobfuscation, allowoptimization class org.kodein.type.TypeReference
-keep, allowobfuscation, allowoptimization class org.kodein.type.JVMAbstractTypeToken$Companion$WrappingTest

-keep, allowobfuscation, allowoptimization class * extends org.kodein.type.TypeReference
-keep, allowobfuscation, allowoptimization class * extends org.kodein.type.JVMAbstractTypeToken$Companion$WrappingTest

-keep, allowobfuscation, allowoptimization class com.squareup.sqldelight.ColumnAdapter
-keep, allowobfuscation, allowoptimization class * extends com.squareup.sqldelight.ColumnAdapter

-keep, allowobfuscation, allowoptimization class com.myapp.epoxy.GenericListEvent

Is there documentation anywhere about what full-mode does? That probably would've been helpful while debugging this. Also an override for keeping attributes in full-mode would be nice.

Now that I got that out of the way, I'll start looking at the consumerProguardFile issue, and see if I can repro it.

Looks like the consumerProguardFile issue must've been transient while I was figuring out all the other issues. It's working fine now.

Thanks for all the help!

It also seems to work for me. I am not a gradle expert because I can never figure out exactly when something is run or evaluated. The problems could perhaps come from the mutating of the list:

        consumerProguardFiles.clear()
        consumerProguardFiles += file("consumer-rules.pro")

Setting the rules should just work.

        consumerProguardFiles ("consumer-rules.pro")

I understand your frustration with having things that worked before no longer working after upgrading. However, no matter the mode (compat or full) these issues all arise from R8 not knowing that things should be kept. All optimizations we do are only allowed to be done if the program behavior is the same - except for reflective use and stack-traces. When there is reflective use one has to specify keep rules to ensure proper behavior. It is extremely difficult for developers that use libraries with reflective behavior to figure out what is going on if they do not specify proguard rules or incorrect rules. On the other hand, if they specify to broad rules (like -keep class * { *; }, R8 will not be able to shrink anything. That is also why we try to assist as much as we can with updating rules for libraries.

If you switch to non-fullmode you may not see any issues or you may see some issues if there are keep rules that are "missing". As you point to in b/188703877#comment10 we may end up optimizing classes in non-fullmode because R8 has not been told we cannot do that. If we merge two classes there is no reliable way of keeping the generic signatures thus we may have to strip the generic signature. If the program relies on the generic signature of the one of the merged classes it would now fail. In some sense you are just "delaying" what may end up as a failure later. I would not be surprised if the cause of the merging issue is also due to missing keep rules.

Regarding removing signatures and other attributes of non-kept members in full mode. Say we only need one method in our app that is only used to look at the generic signature. Adding -keepattributes Signature in compat mode will force the keep of all signatures in your entire app and there is no way to avoid it. The generic signatures takes up quite a lot of code, so this is just not ideal.

I understand. It's just that these issues are very hard to debug, because it's kind of like finding a needle in a haystack (although I learned some new tricks this time around :)

That's why I think it would be helpful to have -keepattributes Signature work in full mode. Even though it will result in a lot of code being kept that doesn't need to be, it can be helpful when running into an issue like this one. Debugging it took quite a few days, and blocked me from some other work. It would've been nice to use -keepattributes Signature so that I could've fine tuned it when I had some time.

Also as discussed in #6 the ability to specify specific things to keep (like Signature) on a rule by rule basis would be very helpful.

As an aside, is there any documentation outlining the differences between compat and full mode?

I think retrofit already includes the necessary rules, except for the ones I added in the pull request: https://github.com/square/retrofit/pull/3563

The rules are:

# Retain service method parameters when optimizing.
-keepclassmembers,allowshrinking,allowobfuscation interface * {
    @retrofit2.http.* <methods>;
}

and

# With R8 full mode, it sees no subtypes of Retrofit interfaces since they are created with a Proxy
# and replaces all potential values with null. Explicitly keeping the interfaces prevents this.
-if interface * { @retrofit2.http.* <methods>; }
-keep,allowobfuscation interface <1>

Instead of keeping things like **EndPoint, which will hit anything that is called that, these rules keep members of classes if they are annotated with @GET, @POST etc. If adding the rules in b/188703877#comment4 is not working, feel free to open a new issue with an example that shows the problem.

Hello!

Same issue here where newer versions of R8 break ktor 😞

To me, it's more a regression rather than "Intended Behavior".

Do you already have a reproducer or should I make one and attach it there?

Louis, it would be great if you could share a sample project that uses ktor and breaks with R8.

In relation to this we updated the Tivi app (which is using Retrofit) to use R8 3.1 in full mode. The app also works with R8 3.0 in full mode.

The changes to enable full mode are:

• https://github.com/chrisbanes/tivi/commit/308a7f518ec81866cdcb8404489bb3e73db9daff
• https://github.com/chrisbanes/tivi/commit/472cef6f8ebbde6743789ef7cbe5e8f26cfa1426

The changes include an extra rule needed for Retrofit to work, which has now been included in Retrofit's consumer rules:

• https://github.com/square/retrofit/commit/9bcde1549a15e2a1c3e59b3a8b469f5157ceb686

If you are still having issues with R8 3.0/3.1 in full mode, please don't hesitate to reopen this bug.

I think this issue not only with Retrofit but also other libraries, e.g. ktor, firebase.

For me,

• this rule still need to keep
• Firebase database getValue(object : GenericTypeIndicator<T>() {}) still doesn't work
• Algolia, ktor still doesn't work, I use these rules as workaround

  • -keep, allowobfuscation, allowoptimization class com.algolia.search.endpoint.** { *; }
    -keep, allowobfuscation, allowoptimization class io.ktor.util.reflect.** { *; }
    

Indeed, this is not specific to Retrofit, but to code that accesses generic signatures using reflection.

Would it be possible for you to share a reproduction of the issues your are seeing (or refer to an open source app that uses these libraries and don't work when updating to R8 >=3.0)?

It is not the same problem, but I believe it is related. I have Type class of GSON:

new TypeToken<LocalDate>(){}.getType();

AGP 7.0 removed all type attributes. So, I added keep rule as mentioned in this issue. However, the type information is still not completed. Smali code:

Actual:

.annotation system Ldalvik/annotation/Signature;
    value = {
        "Lcom/google/gson/reflect/TypeToken;"
    }
.end annotation

Old behavior and expected one:

.annotation system Ldalvik/annotation/Signature;
    value = {
        "Lcom/google/gson/reflect/TypeToken<",
        "Lj$/time/LocalDate;",
        ">;"
    }
.end annotation

Rule I used:

-keep class com.fatboyindustrial.gsonjavatime.Converters** { *; }

What is the single rule to keep all Type information for all classes, which extends TypeToken? Would be nice to put this rule to gson itself, because it should be a very common problem.

Actually, it seems my problem is the same as #24. I made a reproduction.

Thanks for taking the time to share a reproduction.

The app is reflectively accessing the generic signatures of com.google.gson.reflect.TypeToken and its subclasses.

After removing the rule -keep class com.fatboyindustrial.gsonjavatime.Converters** { *; } and adding the following rules instead, the app prints 'works' upon launch.

# Keep generic signature of TypeToken class and its subclasses.
-keep,allowobfuscation,allowshrinking class com.google.gson.reflect.TypeToken
-keep,allowobfuscation,allowshrinking class * extends com.google.gson.reflect.TypeToken

I'll send out a pull request to add these rules to https://github.com/google/gson/blob/master/examples/android-proguard-example/proguard.cfg.

Pull request opened here: https://github.com/google/gson/pull/1930.

Looks like this still isn't fixed on the GSON side because of https://github.com/google/gson/pull/1929

The fix on the GSON side should be available in the next release

Can I keep parameter generic signatures?

Yes, you should be able to achieve this by adding a -keep rule for the corresponding method. If this does not work for you, please open a new issue in the R8 component at https://issuetracker.google.com/issues/new?component=326788&template=1025938. If you can share a reproduction of your issue that would be great.