Cannot jetify library with invalid commented out proguard rules [134100420]

Fixed

Bug

Status Update

No update yet.

Description

ca...@gmail.com

created issue #1

May 31, 2019 11:37AM

OS: macOS 10.14.5
Java version "1.8.0_121"
Jetifier version: 1.0.0-beta04

Let there be a Library Foo with the following 'proguard-rules.txt' file:

# DBFlow
-keep class * extends com.raizlabs.android.dbflow.config.DatabaseHolder { *; }
-dontwarn com.raizlabs.android.dbflow.**
# Note: the configuration keeps the entry point 'com.raizlabs.android.dbflow.config.FlowManager$GlobalDatabaseHolder { FlowManager$GlobalDatabaseHolder(com.raizlabs.android.dbflow.config.FlowManager$1); }', but not the descriptor class 'com.raizlabs.android.dbflow.config.FlowManager$1'
-keep class com.raizlabs.android.dbflow.config.FlowManager$*

When trying to jetify this library:

./jetifier-standalone -i /Users/someuser/Downloads/foo/foo.aar -o newfoo.aar

The Jetifier fails with the following output:

Exception in thread "main" java.util.regex.PatternSyntaxException: Illegal repetition near index 147
com/raizlabs/android/dbflow/.*
# Note: the configuration keeps the entry point 'com/raizlabs/android/dbflow/config/FlowManager$GlobalDatabaseHolder { FlowManager$GlobalDatabaseHolder(com/raizlabs/android/dbflow/config/FlowManager$1); }'
^
at java.util.regex.Pattern.error(Pattern.java:1955)
at java.util.regex.Pattern.closure(Pattern.java:3157)
at java.util.regex.Pattern.sequence(Pattern.java:2134)
at java.util.regex.Pattern.expr(Pattern.java:1996)
at java.util.regex.Pattern.compile(Pattern.java:1696)
at java.util.regex.Pattern.<init>(Pattern.java:1351)
at java.util.regex.Pattern.compile(Pattern.java:1028)
at com.android.tools.build.jetifier.core.type.TypesMap.matchOldProguardForNewTypes(TypesMap.kt:112)
[...]

If i remove the notes, the new 'proguard-rules.txt' looks like this:

# DBFlow
-keep class * extends com.raizlabs.android.dbflow.config.DatabaseHolder { *; }
-dontwarn com.raizlabs.android.dbflow.**
-keep class com.raizlabs.android.dbflow.config.FlowManager$*

And this runs through the jetifier just fine. Please provide a fix for this issue, since i have to patch my dependencies currently in order to make AndroidX work for me.

Comments

do...@traveloka.com <do...@traveloka.com> #2Jul 9, 2019 09:39AM

The issue is that Jetifier modifies jars and does not remove signatures. Below is the original class:

Classfile /tmp/non-jetified/org/bouncycastle/jce/provider/ExtCRLException.class
Last modified Mar 1, 2015; size 362 bytes
MD5 checksum 3968a716f9e60e1ad36c34157bd0239f
class org.bouncycastle.jce.provider.ExtCRLException extends java.security.cert.CRLException
minor version: 0
major version: 49
flags: ACC_SUPER
Constant pool:
#1 = Utf8 <init>
#2 = Utf8 cause
#3 = Utf8 getCause
#4 = Utf8 java/security/cert/CRLException
#5 = Utf8 org/bouncycastle/jce/provider/ExtCRLException
#6 = Class #4 // java/security/cert/CRLException
#7 = Class #5 // org/bouncycastle/jce/provider/ExtCRLException
#8 = Utf8 Ljava/lang/Throwable;
#9 = Utf8 (Ljava/lang/String;)V
#10 = Utf8 ()Ljava/lang/Throwable;
#11 = Utf8 (Ljava/lang/String;Ljava/lang/Throwable;)V
#12 = NameAndType #2:#8 // cause:Ljava/lang/Throwable;
#13 = NameAndType #1:#9 // "<init>":(Ljava/lang/String;)V
#14 = Fieldref #7.#12 // org/bouncycastle/jce/provider/ExtCRLException.cause:Ljava/lang/Throwable;
#15 = Methodref #6.#13 // java/security/cert/CRLException."<init>":(Ljava/lang/String;)V
#16 = Utf8 Code
{
java.lang.Throwable cause;
descriptor: Ljava/lang/Throwable;
flags:

org.bouncycastle.jce.provider.ExtCRLException(java.lang.String, java.lang.Throwable);
descriptor: (Ljava/lang/String;Ljava/lang/Throwable;)V
flags:
Code:
stack=2, locals=3, args_size=3
0: aload_0
1: aload_1
2: invokespecial #15 // Method java/security/cert/CRLException."<init>":(Ljava/lang/String;)V
5: aload_0
6: aload_2
7: putfield #14 // Field cause:Ljava/lang/Throwable;
10: return

public java.lang.Throwable getCause();
descriptor: ()Ljava/lang/Throwable;
flags: ACC_PUBLIC
Code:
stack=1, locals=1, args_size=1
0: aload_0
1: getfield #14 // Field cause:Ljava/lang/Throwable;
4: areturn
}

One processed with Jetifier:

Classfile /tmp/jetified/org/bouncycastle/jce/provider/ExtCRLException.class
Last modified Oct 14, 2019; size 362 bytes
MD5 checksum 76bc557706ee3480e53d88b21316913f
class org.bouncycastle.jce.provider.ExtCRLException extends java.security.cert.CRLException
minor version: 0
major version: 49
flags: ACC_SUPER
Constant pool:
#1 = Utf8 org/bouncycastle/jce/provider/ExtCRLException
#2 = Class #1 // org/bouncycastle/jce/provider/ExtCRLException
#3 = Utf8 java/security/cert/CRLException
#4 = Class #3 // java/security/cert/CRLException
#5 = Utf8 cause
#6 = Utf8 Ljava/lang/Throwable;
#7 = Utf8 <init>
#8 = Utf8 (Ljava/lang/String;Ljava/lang/Throwable;)V
#9 = Utf8 (Ljava/lang/String;)V
#10 = NameAndType #7:#9 // "<init>":(Ljava/lang/String;)V
#11 = Methodref #4.#10 // java/security/cert/CRLException."<init>":(Ljava/lang/String;)V
#12 = NameAndType #5:#6 // cause:Ljava/lang/Throwable;
#13 = Fieldref #2.#12 // org/bouncycastle/jce/provider/ExtCRLException.cause:Ljava/lang/Throwable;
#14 = Utf8 getCause
#15 = Utf8 ()Ljava/lang/Throwable;
#16 = Utf8 Code
{
java.lang.Throwable cause;
descriptor: Ljava/lang/Throwable;
flags:

org.bouncycastle.jce.provider.ExtCRLException(java.lang.String, java.lang.Throwable);
descriptor: (Ljava/lang/String;Ljava/lang/Throwable;)V
flags:
Code:
stack=2, locals=3, args_size=3
0: aload_0
1: aload_1
2: invokespecial #11 // Method java/security/cert/CRLException."<init>":(Ljava/lang/String;)V
5: aload_0
6: aload_2
7: putfield #13 // Field cause:Ljava/lang/Throwable;
10: return

public java.lang.Throwable getCause();
descriptor: ()Ljava/lang/Throwable;
flags: ACC_PUBLIC
Code:
stack=1, locals=1, args_size=1
0: aload_0
1: getfield #13 // Field cause:Ljava/lang/Throwable;
4: areturn
}

Changes in the constants pool cause changes in the code (indices change), which causes different MD5 checksums. This means that signatures will not match any more.

This started happening in 3.6.0-beta01 because we are using Jetifier with copyUnmodifiedLibsAlso=true flag (previously it was false).

pa...@gmail.com <pa...@gmail.com> #3Oct 17, 2019 09:40AM

@Chris You can work around this by updating gradle.properties with:

android.jetifier.blacklist=.*bcprov.*

This will avoid Jetifying the bouncycastle library.

pa...@gmail.com <pa...@gmail.com> #4Oct 17, 2019 09:44AM

Thanks Ivan!

ga...@google.com <ga...@google.com> #5Oct 17, 2019 11:41AM

Assigned to pa...@google.com.

Pending fix in:

https://android-review.googlesource.com/c/platform/frameworks/support/+/1151460

pa...@google.com <pa...@google.com> #6Oct 25, 2019 03:40PM

Accepted by pa...@google.com.

did this fix make it in for 3.6?

ap...@google.com <ap...@google.com> #7Oct 31, 2019 06:24PM

Jetifier 1.0.0-beta08 was released and has a fix for this.

To avoid waiting for new Android Studio you can workaround this in the meantime by forcing usage of a new version of jetifier in gradle:
buildscript {
dependencies {
classpath 'com.android.tools.build:gradle:x.y.z'
classpath 'com.android.tools.build.jetifier:jetifier-processor:1.0.0-beta08' // <- Add this
}
}

pa...@google.com <pa...@google.com> #8Nov 26, 2019 02:34PM

Jetifier 1.0.0-beta08, which has the fix for this issue, will be integrated into AGP 3.6.0-rc01 and 4.0.0-alpha06:

https://android.googlesource.com/platform/tools/base/+/ef5e56702b626b6be642ecce7d0078b7b1a81bcb

pa...@gmail.com <pa...@gmail.com> #9Nov 26, 2019 02:40PM

Thanks pa..., it's really nice to receive updates and useful details (root cause, CL, workaround) about the issue. I wish everyone was like this!

hu...@google.com <hu...@google.com> #10Nov 27, 2019 05:34PM

Marked as fixed.