IssueTracker

Thank you for reporting an issue and trying out R8.  UI testing can reflectively use part of your APP which one needs to explicitly keep. It may also be different problem :) Can I ask you to include the full stack-trace of the error to allow us to better trace the issue?

I can include it, but it won't tell you much. It's a crash regarding a class we have in our projects that is referenced explicitly in the project. It looks like the class in obfuscated but then when running the ui tests the TestApplication doesn't know the new name of that class, and therefore crashes with a `ClassNotFoundException `

The reason I ask is because the link you posted from stack-overflow has the the underlying error:

Suppressed: java.io.IOException: Failed to open dex files from /data/app/com.your.pkg-GEdf7lpr_eG0NIkHShudhA==/base.apk because: Failure to verify dex file '/data/app/com.your.pkg-GEdf7lpr_eG0NIkHShudhA==/base.apk': Out-of-order entry types: 21a8 then 21a8

There is a big difference if (1) the class your test app cannot find is because it is simply not there and you need to add a keep-rule for it, or (2) we minify the name of the class during the first compilation and fail to rename the class in the test compilation or (3) we produce an invalid dex file and that is causing errors. The other nice thing the stack-trace gives me is the version of R8 used, because it is printed before each R8 task.

If possible, can you send the following such that we can investigate the problem:
1) The produced APK with minify enabled with the proguard configuration and mapping file
2) The produced test apk
3) The proguard-test-rules.pro
4) The full stack-trace

You can send it privately to mkroghj@google.com. If it is not possible to share all of the above, please ensure that the class exists in the original class (either with its original name or mapped) and copy the entire stack trace here. Then we proceed from there :)

I don't think I'm able to share it, I'll need to ask to some people in my company before. But I've investigated it further and figured out few things. First, let's start with the stack trace:

```
 java.lang.NoClassDefFoundError: Failed resolution of: Lcom/clearscore/mobile/core/utils/EmojiDecoder;
        at com.clearscore.mobile.di.DaggerAppTestComponent.injectClearScoreTestApplication(DaggerAppTestComponent.java:2239)
        at com.clearscore.mobile.di.DaggerAppTestComponent.inject(DaggerAppTestComponent.java:2191)
        at com.clearscore.mobile.ClearScoreTestApplication.buildAppComponent(ClearScoreTestApplication.kt:30)
        at com.clearscore.mobile.ClearScoreApplication.buildDependencies(ClearScoreApplication.kt:118)
        at com.clearscore.mobile.ClearScoreApplication.onCreate(ClearScoreApplication.kt:106)
        at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1182)
        at androidx.test.runner.MonitoringInstrumentation.callApplicationOnCreate(MonitoringInstrumentation.java:413)
        at android.app.ActivityThread.handleBindApplication(ActivityThread.java:6460)
        at android.app.ActivityThread.access$1300(ActivityThread.java:219)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1859)
        at android.os.Handler.dispatchMessage(Handler.java:107)
        at android.os.Looper.loop(Looper.java:214)
        at android.app.ActivityThread.main(ActivityThread.java:7356)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:930)
     Caused by: java.lang.ClassNotFoundException: Didn't find class "com.clearscore.mobile.core.utils.EmojiDecoder" on path: DexPathList[[zip file "/system/framework/android.test.runner.jar", zip file "/system/framework/android.test.mock.jar", zip file "/data/app/com.clearscore.mobile.test-IjEj3tpI7xr_PpTSmrvhxw==/base.apk", zip file "/data/app/com.clearscore.mobile.debug-75TecSms9JvNI_QoM9ZHDA==/base.apk"],nativeLibraryDirectories=[/data/app/com.clearscore.mobile.test-IjEj3tpI7xr_PpTSmrvhxw==/lib/x86, /data/app/com.clearscore.mobile.debug-75TecSms9JvNI_QoM9ZHDA==/lib/x86, /data/app/com.clearscore.mobile.test-IjEj3tpI7xr_PpTSmrvhxw==/base.apk!/lib/x86, /data/app/com.clearscore.mobile.debug-75TecSms9JvNI_QoM9ZHDA==/base.apk!/lib/x86, /system/lib, /system/product/lib]]
        at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:196)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:379)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
        at com.clearscore.mobile.di.DaggerAppTestComponent.injectClearScoreTestApplication(DaggerAppTestComponent.java:2239) 
        at com.clearscore.mobile.di.DaggerAppTestComponent.inject(DaggerAppTestComponent.java:2191) 
        at com.clearscore.mobile.ClearScoreTestApplication.buildAppComponent(ClearScoreTestApplication.kt:30) 
        at com.clearscore.mobile.ClearScoreApplication.buildDependencies(ClearScoreApplication.kt:118) 
        at com.clearscore.mobile.ClearScoreApplication.onCreate(ClearScoreApplication.kt:106) 
        at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1182) 
        at androidx.test.runner.MonitoringInstrumentation.callApplicationOnCreate(MonitoringInstrumentation.java:413) 
        at android.app.ActivityThread.handleBindApplication(ActivityThread.java:6460) 
        at android.app.ActivityThread.access$1300(ActivityThread.java:219) 
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1859) 
        at android.os.Handler.dispatchMessage(Handler.java:107) 
        at android.os.Looper.loop(Looper.java:214) 
        at android.app.ActivityThread.main(ActivityThread.java:7356) 
```

Now, this class, EmojiDecoder, is referenced explicitly in the project, so it's not stripped out, it's just obfuscated. Using the Apk Analyser bundled with Android Studio, I can see the bytecode of the class:
```
.class public interface abstract Lcom/clearscore/mobile/m/v/h;
.super Ljava.lang.Object;
.source "EmojiDecoder.kt"
....
```
This class is injected using Dagger in our `Application` class. For UI tests, we have a `TestApplication` class which is a subclass `Application`, so of course `EmojiDecoder` needs to be injected here as well. What I can see though in the bytecode of `TestApplication_MembersInjector` is:
```
.field private final emojiDecoderProvider:Ljavax.inject.Provider;
    .annotation system Ldalvik/annotation/Signature;
        value = {
            "Ljavax/inject/Provider<",
            "Lcom/clearscore/mobile/core/utils/EmojiDecoder;",
            ">;"
        }
    .end annotation
.end field
```
So it looks like that the Dagger generated code isn't obfuscated properly and uses the class as it wasn't obfuscated. Let me know if you need any more info about it, in the next few days I'll be on holiday so I might be slow to answer but I'll try to answer.

This looks like an older problem with applymapping - specifically, it seems like an issue that was solved here:
https://r8-review.googlesource.com/c/r8/+/38556

Can I ask you to try out version 1.6.42 by adding the following to your top-level build.gradle file:

buildscript {

    repositories {
        maven {
            url 'http://storage.googleapis.com/r8-releases/raw'
        }
    }

    dependencies {
        classpath 'com.android.tools:r8:1.6.42'          // Must be before the Gradle Plugin for Android.
        classpath 'com.android.tools.build:gradle:X.Y.Z' // Your current AGP version.
     }
}

Looks like updating R8 fixes that issue but creates a very similar one:
```
java.lang.AbstractMethodError: abstract method "void com.clearscore.mobile.p.m.a(com.clearscore.mobile.ClearScoreApplication)"
        at com.clearscore.mobile.ClearScoreApplication.buildDependencies(ClearScoreApplication.kt:120)
        at com.clearscore.mobile.ClearScoreApplication.onCreate(ClearScoreApplication.kt:106)
        at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1182)
        at androidx.test.runner.MonitoringInstrumentation.callApplicationOnCreate(MonitoringInstrumentation.java:413)
        at android.app.ActivityThread.handleBindApplication(ActivityThread.java:6460)
        at android.app.ActivityThread.access$1300(ActivityThread.java:219)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1859)
        at android.os.Handler.dispatchMessage(Handler.java:107)
        at android.os.Looper.loop(Looper.java:214)
        at android.app.ActivityThread.main(ActivityThread.java:7356)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:930)
```

Looks like this is the inject method of the Dagger component which has been obfuscated. But looking at the code of the generated TestComponent, I can see that this contains:
```
.method public inject(Lcom/clearscore/mobile/ClearScoreApplication;)V
```
Which I assume is the reason of the failure, as the obfuscated method from the interface isn't implemented in the obfuscated implementation

I am not quite following the example because I cannot distinguish the APP source from the test source. As I am reading the stack-trace, we go into:

com.clearscore.mobile.ClearScoreApplication.buildDependencies(ClearScoreApplication.kt:120) which I assume is in the base.

This will do an invoke to:

void com.clearscore.mobile.p.m.a(com.clearscore.mobile.ClearScoreApplication)

on some object, which should happen here ClearScoreApplication.kt:120. Is the object it is doing the invoke on a class originating from the TestComponent (a sub-class or an implementation of an original class in the APP)? And is that the one you say still have the signature:
method public inject(Lcom/clearscore/mobile/ClearScoreApplication;)V

If the above is in-fact what is happening, then check the following:
1) That the base-class or interface from your main APK has the name and method in the produced proguard-map.
2) That the tests are compiled with R8 as well - the task name is on the form transformClassesAndResourcesWithR8ForDebugAndroidTest
3) That you do not have any keep rules in proguard-test-rules that prevent the name from being changed.

That is, if you in your base have an interface:

interface I {
  void inject(Foo);
}

and you allow it to be renamed and it gets obfuscated, it will be in the map (if it still exists) as such:

I -> a:
  void inject(Foo) -> a

And if you then have your proguard-test-rules as such:

-keep interface I { *; }

Then we cannot rename the interface method inject in the test.

If the above is not the case, see if you can provide a small example that will showcase the problem.

Sorry for the time taken to reply, I wanted to investigate it more and I have successfully made a workaround that is preventing the bug. So, first, let me be more clear about the issue:

In my app code, I have an interface with a method:

interface Component {
   fun inject(app: Application)
}

This `inject` method is what is then transformed to `void com.clearscore.mobile.p.m.a(com.clearscore.mobile.ClearScoreApplication)`. Then in my test code, I have:

class TestApplication: Application()

and

interface TestComponent {
    fun inject(app: TestApplication)
}

It looks like that calling this inject method as `testComponent.inject(this as TestApplication)` works fine but `testComponent.inject(this)` is throwing the error above (abstract method "void com.clearscore.mobile.q.n.a(com.clearscore.mobile.ClearScoreApplication)"). This invocation is actually something we don't need, so removing it has fixed the problem, but I still believe this should be considered as a bug on R8

Thank you for going a bit more in depth with the exception and I am happy that you found a work around. I would be very happy to fix the bug, if any, but I am still not able to parse your example fully:

1) Do Component.inject have a default implementation that you rely on? If yes, what is the min-api you are compiling for?
2) Does TestComponent extend Component? Otherwise, how would else would testComponent.inject(Application) work?
3) Does TestComponent.inject have a default implementation?
4) is testComponent an object that implements TestComponent? Like:

A implements TestComponent {

   ...
}

new A().inject(this)?

Ok, I've managed to create a sample project that reproduce the issue, hope it's useful. Just open it and run ui tests either from Android Studio or with `./gradlew connectedCheck`. You should then see the crash in the logcat.

https://drive.google.com/file/d/1efXC2Dr5dVc4dFsr8OVXKcXz6OwTRiM8/view?usp=drive_web (R8bug.zip)

Thank you for your reproduction. I was able to create a smaller one from that sample and a fix for your problem is under review here https://r8-review.googlesource.com/c/r8/+/46444

However, your sample is somewhat "lucky" to work with the fix and let me explain why:

Running the test relies on

interface ApplicationComponent {
  inject(App);
}

The proguard keep rules specified for the main app compilation are:
            proguardFiles getDefaultProguardFile("proguard-android.txt"), 'proguard-rules.pro',  'proguard-test-rules.pro'

proguard-rules.pro is empty.
proguard-test-rules.pro contains:
-keepclassmembers class com.lorenzo.r8bug.App { *; }

In general, every reference your test is relying on from your app should have a -keep,allowobfuscation rule in proguard-rules.pro. Here, you have a reference on ApplicationComponent, so I would expect an entry in proguard-rules.pro that contained:
-keep,allowobfuscation package.ApplicationComponent { *; }

The reason for adding these rules in proguard-rules.pro is that R8 tries really hard to remove interfaces and the only reason why ApplicationComponent is not completely removed is because getDefaultProguardFile("proguard-android.txt") contains -dontoptimize. That means, if you were to use proguardFiles getDefaultProguardFile("proguard-android-optimize.txt"), which is now default in Android Studio, your tests would stop working.

Before the fix for this issue lands, the error would be the same, but you would be guaranteed that R8 will not try to remove ApplicationComponent if you ever upgraded to getDefaultProguardFile("proguard-android-optimize.txt")

A final note on your configuration, then the following makes little sense:
            proguardFiles getDefaultProguardFile("proguard-android.txt"), 'proguard-rules.pro',  'proguard-test-rules.pro'
            testProguardFiles getDefaultProguardFile("proguard-android.txt"), 'proguard-rules.pro', 'proguard-test-rules.pro'

The app is compiled first, where the proguard configuration for that compilation is passed by setting proguardFiles. When that compilation is finished, R8 will produce a map with all that has been renamed. This map, is then passed to the test compilation to update references into the app. The configuration passed to the test compilation is specified as testProguardFiles ... but it always the the following rules passed in additionally:

-keep class * {
  *;
}
-keep interface * {
  *;
}
-keep enum * {
  *;
}

-applymapping <path_to_mapping_file>

That means, all the rules in getDefaultProguardFile("proguard-android.txt"), 'proguard-rules.pro', 'proguard-test-rules.pro' has no effect (except -dontwarn). And actually, if the above rules where not automatically added, R8 would remove all tests since no keep rule is specified for the test. To not confuse where rules are used, you should completely remove proguard-test-rules.pro (add the entries in there to proguard-rules.pro) and do not pass in testProguardFiles in .gradle.

Project: r8
Branch: master

commit e87708d6260fdc9fa904092eefcacf19684dc3c3
Author: Morten Krogh-Jespersen <mkroghj@google.com>
Date:   Thu Dec 05 11:34:15 2019

    Add test for adding mapped members from parent interfaces to classes
   
    This is a reproduction that shows we do not take renamed interface
    methods into account if we cannot locate a class mapping for it's
    direct interface.
   
    Bug: 144151805
    Change-Id: I351efac27404707d7a19c76bde977ad734171fb5

A       src/test/java/com/android/tools/r8/naming/applymapping/ApplyMappingExtendInterfaceTest.java

https://r8-review.googlesource.com/46443

Project: r8
Branch: master

commit 17859c3b4835cc76e78f987850223ea0399c1297
Author: Morten Krogh-Jespersen <mkroghj@google.com>
Date:   Thu Dec 05 13:15:57 2019

    Fix mapping of members in indirect interfaces
   
    We should continue searching up in the interface hierarchy for renamed
    members.
   
    Bug: 144151805
    Change-Id: Iccaac2a14823a5bc52162da2fd74edfe66c77cf3

M       src/main/java/com/android/tools/r8/naming/ProguardMapMinifier.java
A       src/test/java/com/android/tools/r8/naming/applymapping/ApplyMappingExtendEmptyInterfaceTest.java
M       src/test/java/com/android/tools/r8/naming/applymapping/ApplyMappingExtendInterfaceTest.java

https://r8-review.googlesource.com/46444

Thanks for you answer. I started working on this project recently and I understand the current rules aren't perfect, hopefully I will have some time to update them in the future, I just wanted to migrate to R8 as first step. However, about:
"In general, every reference your test is relying on from your app should have a -keep,allowobfuscation rule in proguard-rules.pro."

Should it? Somehow Proguard didn't need those rules, but at the same time I guess those unused methods in the prod apk were ending up in the final release.

Also, could you let me know when a new release of R8 is out with the fix? I'd love to try it in the real project and see if it works.

Project: r8
Branch: 1.6

commit fc946340a3548b2e300ee4cd1716b6b96d674d7c
Author: Morten Krogh-Jespersen <mkroghj@google.com>
Date:   Tue Dec 17 17:30:59 2019

    Version 1.6.54
   
    Cherry pick: Add test for adding mapped members from parent interfaces
    to classes
    CL: https://r8-review.googlesource.com/46443
   
    Cherry pick: Fix mapping of members in indirect interfaces
    https://r8-review.googlesource.com/46444
   
    Bug: 144151805
    Change-Id: Iefdcd2ec42b03b209756e982f24b3437ca3aaa9e

M       src/main/java/com/android/tools/r8/Version.java
M       src/main/java/com/android/tools/r8/naming/ProguardMapMinifier.java
A       src/test/java/com/android/tools/r8/naming/applymapping/ApplyMappingExtendEmptyInterfaceTest.java
A       src/test/java/com/android/tools/r8/naming/applymapping/ApplyMappingExtendInterfaceTest.java

https://r8-review.googlesource.com/46900

Project: r8
Branch: 1.6

commit c37182b22e4262875974c62848931c98f05c6ee8
Author: Morten Krogh-Jespersen <mkroghj@google.com>
Date:   Wed Dec 18 09:44:04 2019

    Version 1.6.55
   
    Fix the ApplyMappingExtend tests such that they compile on release
    branch.
   
    Bug: 144151805
    Change-Id: I0ecfac80f0e99a26abb7c9ec6f462fbc8b656288

M       src/main/java/com/android/tools/r8/Version.java
M       src/test/java/com/android/tools/r8/naming/applymapping/ApplyMappingExtendEmptyInterfaceTest.java
M       src/test/java/com/android/tools/r8/naming/applymapping/ApplyMappingExtendInterfaceTest.java

https://r8-review.googlesource.com/46901

Sorry for the late reply. If you want to try out the version, you can add the following to your top-level gradle file:

buildscript {

    repositories {
        maven {
            url 'http://storage.googleapis.com/r8-releases/raw'
        }
    }

    dependencies {
        classpath 'com.android.tools:r8:1.6.55'          // Must be before the Gradle Plugin for Android.
        classpath 'com.android.tools.build:gradle:X.Y.Z' // Your current AGP version.
     }
}

The version is being rolled into Android Studio 3.6 RC 2 and hopefully made the cut.

Regarding the need for keep rules when making tests, Proguard will maybe need them as well, it is all depending on how elaborate tests you make. Say that you would like to test some view around a provider:

public interface I {
  void doSomething();
}

In your app, you of course have your normal provider:

public class AppI implements I {
  void doSomething() { ... }
}

Now, if no rules are specified, R8 will merge I into AppI, because there is no need to have the interface around.

If you then in a test have:

public class TestI implements I {

  void doSomething() { ... }
}

then running the test on a minified app will no longer work, since I do not exist. So in general, every interface or abstract class used by the tests should have a keep rule. The above example is of course R8 specific, but there are also plenty of examples where this is required for running tests with Proguard.

Hope the above explanation helped a bit - and thank you for reporting an issue and trying out R8.