IssueTracker

The issue is that Jetifier modifies jars and does not remove signatures. Below is the original class:

Classfile /tmp/non-jetified/org/bouncycastle/jce/provider/ExtCRLException.class
  Last modified Mar 1, 2015; size 362 bytes
  MD5 checksum 3968a716f9e60e1ad36c34157bd0239f
class org.bouncycastle.jce.provider.ExtCRLException extends java.security.cert.CRLException
  minor version: 0
  major version: 49
  flags: ACC_SUPER
Constant pool:
   #1 = Utf8               <init>
   #2 = Utf8               cause
   #3 = Utf8               getCause
   #4 = Utf8               java/security/cert/CRLException
   #5 = Utf8               org/bouncycastle/jce/provider/ExtCRLException
   #6 = Class              #4             // java/security/cert/CRLException
   #7 = Class              #5             // org/bouncycastle/jce/provider/ExtCRLException
   #8 = Utf8               Ljava/lang/Throwable;
   #9 = Utf8               (Ljava/lang/String;)V
  #10 = Utf8               ()Ljava/lang/Throwable;
  #11 = Utf8               (Ljava/lang/String;Ljava/lang/Throwable;)V
  #12 = NameAndType        #2:#8          // cause:Ljava/lang/Throwable;
  #13 = NameAndType        #1:#9          // "<init>":(Ljava/lang/String;)V
  #14 = Fieldref           #7.#12         // org/bouncycastle/jce/provider/ExtCRLException.cause:Ljava/lang/Throwable;
  #15 = Methodref          #6.#13         // java/security/cert/CRLException."<init>":(Ljava/lang/String;)V
  #16 = Utf8               Code
{
  java.lang.Throwable cause;
    descriptor: Ljava/lang/Throwable;
    flags:

  org.bouncycastle.jce.provider.ExtCRLException(java.lang.String, java.lang.Throwable);
    descriptor: (Ljava/lang/String;Ljava/lang/Throwable;)V
    flags:
    Code:
      stack=2, locals=3, args_size=3
         0: aload_0
         1: aload_1
         2: invokespecial #15                 // Method java/security/cert/CRLException."<init>":(Ljava/lang/String;)V
         5: aload_0
         6: aload_2
         7: putfield      #14                 // Field cause:Ljava/lang/Throwable;
        10: return

  public java.lang.Throwable getCause();
    descriptor: ()Ljava/lang/Throwable;
    flags: ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: getfield      #14                 // Field cause:Ljava/lang/Throwable;
         4: areturn
}

One processed with Jetifier:

Classfile /tmp/jetified/org/bouncycastle/jce/provider/ExtCRLException.class
  Last modified Oct 14, 2019; size 362 bytes
  MD5 checksum 76bc557706ee3480e53d88b21316913f
class org.bouncycastle.jce.provider.ExtCRLException extends java.security.cert.CRLException
  minor version: 0
  major version: 49
  flags: ACC_SUPER
Constant pool:
   #1 = Utf8               org/bouncycastle/jce/provider/ExtCRLException
   #2 = Class              #1             // org/bouncycastle/jce/provider/ExtCRLException
   #3 = Utf8               java/security/cert/CRLException
   #4 = Class              #3             // java/security/cert/CRLException
   #5 = Utf8               cause
   #6 = Utf8               Ljava/lang/Throwable;
   #7 = Utf8               <init>
   #8 = Utf8               (Ljava/lang/String;Ljava/lang/Throwable;)V
   #9 = Utf8               (Ljava/lang/String;)V
  #10 = NameAndType        #7:#9          // "<init>":(Ljava/lang/String;)V
  #11 = Methodref          #4.#10         // java/security/cert/CRLException."<init>":(Ljava/lang/String;)V
  #12 = NameAndType        #5:#6          // cause:Ljava/lang/Throwable;
  #13 = Fieldref           #2.#12         // org/bouncycastle/jce/provider/ExtCRLException.cause:Ljava/lang/Throwable;
  #14 = Utf8               getCause
  #15 = Utf8               ()Ljava/lang/Throwable;
  #16 = Utf8               Code
{
  java.lang.Throwable cause;
    descriptor: Ljava/lang/Throwable;
    flags:

  org.bouncycastle.jce.provider.ExtCRLException(java.lang.String, java.lang.Throwable);
    descriptor: (Ljava/lang/String;Ljava/lang/Throwable;)V
    flags:
    Code:
      stack=2, locals=3, args_size=3
         0: aload_0
         1: aload_1
         2: invokespecial #11                 // Method java/security/cert/CRLException."<init>":(Ljava/lang/String;)V
         5: aload_0
         6: aload_2
         7: putfield      #13                 // Field cause:Ljava/lang/Throwable;
        10: return

  public java.lang.Throwable getCause();
    descriptor: ()Ljava/lang/Throwable;
    flags: ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: getfield      #13                 // Field cause:Ljava/lang/Throwable;
         4: areturn
}


Changes in the constants pool cause changes in the code (indices change), which causes different MD5 checksums. This means that signatures will not match any more.

This started happening in 3.6.0-beta01 because we are using Jetifier with copyUnmodifiedLibsAlso=true flag (previously it was false).

@Chris You can work around this by updating gradle.properties with:

android.jetifier.blacklist=.*bcprov.*

This will avoid Jetifying the bouncycastle library.

Thanks Ivan!

Pending fix in: https://android-review.googlesource.com/c/platform/frameworks/support/+/1151460

did this fix make it in for 3.6?

Jetifier 1.0.0-beta08 was released and has a fix for this.

To avoid waiting for new Android Studio you can workaround this in the meantime by forcing usage of a new version of jetifier in gradle:
buildscript {
   dependencies {
       classpath 'com.android.tools.build:gradle:x.y.z'
       classpath 'com.android.tools.build.jetifier:jetifier-processor:1.0.0-beta08' // <- Add this
   }
}

Jetifier 1.0.0-beta08, which has the fix for this issue, will be integrated into AGP 3.6.0-rc01 and 4.0.0-alpha06: https://android.googlesource.com/platform/tools/base/+/ef5e56702b626b6be642ecce7d0078b7b1a81bcb

I've succesfully tested the blacklist config with:
* AGP 3.3.0-rc01
* Gradle 5.1-rc-1

I just wanted to add a comment about what I had to blacklist, so that perhaps jetifier can handle this automatically without a rule.

TL;DR: jetifier should probably not even try to open *.so files.

Details:
My project depends (transitively) on a jar which contains only native *.so files.
On one particular machine (docker container) with apparently not enough memory, I was getting this error:

Execution failed for task ':libon-lite-client:checkDebugClasspath'.
> Could not resolve all files for configuration ':libon-lite-client:debugCompileClasspath'.
   > Failed to transform artifact 'liblinphone-debug-symbols.jar (org.liblinphone:liblinphone:4.0.1)' to match attributes {artifactType=android-classes, org.gradle.usage=java-runtime-jars}
      > Execution failed for JetifyTransform: /root/.gradle/caches/modules-2/files-2.1/org.liblinphone/liblinphone/4.0.1/871c6e76e93a7fa1bbafb5eb450e45c2330e866/liblinphone-4.0.1-debug-symbols.jar.
         > Java heap space

(Note that with gradle 4.10.2/AGP 3.2.1, I didn't have this detailed info, but just simply "Transformation hasn’t been executed yet" (https://issuetracker.google.com/issues/119951660).  I got this additional error message with gradle 5.1-rc-1/AGP 3.2.1.)

Now with gradle 5.1-rc1/AGP-3.3.0-rc01, I have this rule:
android.jetifier.blacklist=liblinphone-4.0.1-debug-symbols.jar

And I no longer get the error.

I suppose jetifier shouldn't even try to look into binary *.so files.

Re #comment9:
Jetifier ignores *.so files. However it has to extract each archive to the memory and check each file and recreate a new zip file to the filesystem. So loading the archive to the memory has to be done no matter if there are .so files or not.

@Filip: I wonder if Jetifier should check the zip entries' meta data first (without loading their contents) and return immediately if there are no entries that need jetifying. (Returning is good enough, Jetifier doesn't need to copy the file to somewhere else. And even when it needs to copy some large entries from one zip file to another, we can probably use streams/Zip64 so that it doesn't load the entire zip file/zip entry into memory---see Issue 113559507.)

Is anyone else having issues with AGP 3.3.0-rc01+ or 3.4.0-alpha04+ where the project compiles and runs fine, but cannot resolve the R file? I tried invalidating android studio caches, cleaning, deleting .idea, deleting .gradle, and nothing works except reverting to android plugin 3.2.1...which I can't do because I need the jetifier blacklist workaround.

@12 Sounds like a different issue, possibly around light R classes handling in the IDE, would you mind filing a bug for it and describing the problem in more detail or provide a repro project?

It seems to be working fine in Android Studio 3.3 so I think it might not be an issue in the future. Thanks for the response!

Hi I was wondering the status of the blacklist? Is this official or on the roadmap for 3.4?

@15: The android.jetifier.blacklist property is available to use on AGP 3.3.0 and above. However, please note that it is only a workaround.

We are working a on better solution that does not require developers to use the workaround. This bug is left open to keep track of that work.

Hi. could you backport blacklist option to 3.2.x release?

Since the blacklist property is already available on 3.3.0 stable, I'm afraid that we won't be releasing 3.2.2 only to include that. I would recommend updating to 3.3.0 stable instead, and file a bug with us if you think something is blocking you from updating.

I have some gradle plugin that doesn't work with 3.3.0.
The problem is that Robolectric can't be used with 3.2.1 when Jetifier is enabled because it tries to re-write 'org.ow2.asm:asm' jar which has class files of version > 52.
So I would like to blacklist this jar from being processed.

I see, but again I'm afraid it is too late for 3.2 at this point. One thing you could try is to disable Jetifier (set android.enableJetifier=false), and run the Jetifier standalone tool (https://developer.android.com/studio/command-line/jetifier) on individual libraries and import the jetified ones instead of the original ones. I know it's not pretty, but hopefully it will work for you.

Any ideas about this one -
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':app:kaptGenerateStubsDebugKotlin'
...
Caused by: java.lang.RuntimeException: Failed to transform '/Users/user/.gradle/caches/modules-2/files-2.1/org.xerial/sqlite-jdbc/3.20.1/df3068e837e9490a9554212fcab40a2c55faf0a3/sqlite-jdbc-3.20.1.jar' using Jetifier. Reason: invalid entry size (expected 618164 but got 636977 bytes). (Run with --stacktrace for more details.) To disable Jetifier, set android.enableJetifier=false in your gradle.properties file.
        at com.android.build.gradle.internal.dependency.JetifyTransform.transform(JetifyTransform.kt:204)

Can you please file a new bug with us and link it here so I can follow up?

Sure

Hey, deleting and cloning the repo again fixes this issue, do you still want me to file this?

No need for now (as there is too little information for it to be actionable), but we'll keep that in mind and please do file a bug if you see it again. Thanks!

Hi,

I have this error but I cannot fix if with blacklist, is it the same issue ? Could `android.jetifier.blacklist` property fix it ?

09:08:56 > Could not resolve all files for configuration ':app:releaseUnitTestCompileClasspath'.
09:08:56    > Failed to transform artifact 'full.jar (project :api)' to match attributes {artifactType=processed-jar, com.android.build.api.attributes.BuildTypeAttr=release, com.android.build.api.attributes.VariantAttr=release, com.android.build.gradle.internal.dependency.AndroidTypeAttr=Aar, org.gradle.usage=java-api, org.jetbrains.kotlin.platform.type=androidJvm}

Thanks

This is a different issue. Please file a new bug and let me know the bug ID. Thanks!

Done here : https://issuetracker.google.com/issues/128469625

Jetifier really brings up some headaches. We upgraded AGP from 3.5.2 to 3.6.0-beta04 and found our Cucumber tests failing afterwards. The issue Cucumber reports was

   cucumber.runtime.CucumberException: No backends were found. Please make sure you have a backend module on your CLASSPATH.

Cucumber reports that when an artifact like `cucumber-java` (or any other language backend) is not present. Cucumber tries to service-load this automatically when present in the classpath (see https://github.com/cucumber/cucumber-jvm/search?q=JavaBackendProviderService&unscoped_q=JavaBackendProviderService for references), but apparently fails.

Interestingly, the tests continued to run without issues in Android Studio, they were just failing when executed from command line. A test run with --debug showed me that the Gradle run however used jetified versions of the cucumber libraries in the application classpath. Adding `.*cucumber.*` to the jetifier blacklist solved the issue.

I'm completely unaware of the black magic that Jetifier involves, but I see that it has some very serious consequences when applied broadly on a project, things that are rather hard to track down because subtle things like service locators suddenly stop working.

Thank you for the feedback! We are looking to remove edge cases like this one, and bug reports like this help us do that.

If possible, could you attach the original version of the cucumber library and the jetified version, so that we can investigate what was actually modified by Jetifier?

Starting with AGP 4.0.0-alpha07 (using Jetifier 1.0.0-beta06+), Jetifier will not jetify libraries that already support AndroidX: https://android-review.googlesource.com/c/platform/frameworks/support/+/952620/. The blacklist workaround shouldn't be needed anymore.

After updating to AGP 4.0.0-alpha07+, please remove the android.jetifier.blacklist property if you have been using it. And if you still run into issues after doing that, please file a bug with us at http://issuetracker.google.com/issues/new?component=460323.


To close the loop on the 3 different approaches outlined at comment #1:

> We need a way to determine whether a third-party library should be jetified or not. There are a couple of approaches:
>    1. The library adds some metadata stating that it is AndroidX-ready and therefore should not be jetified.
    => Infeasible

>    2. Jetifier automatically detects the presence of AndroidX (e.g., import statements and strings containing references to AndroidX), and if AndroidX is present, assumes that the library is AndroidX-ready and skips converting it.
    => Implemented

>    3. The users need to specify a blacklist of libraries that should not be jetified.
    => Used previously as a workaround (see comment #5), but not recommended now, as approach #2 should remove the need for this workaround. If you still find yourself having to use the blacklist, it is most likely a bug, please tell us at http://issuetracker.google.com/issues/new?component=460323.