Status Update
Comments
sg...@google.com <sg...@google.com> #2
Thank you for the report. Is it possible for you to share a sample that demonstrates this issue? One option is to
bs...@googlemail.com <bs...@googlemail.com> #3
If you need reproduction steps to face the runtime error I can give them as well.
ze...@google.com <ze...@google.com> #4
Thanks, we will have a look at the project. Since it is open, then attaching a
sg...@google.com <sg...@google.com> #5
Thank you. I can see on the branch that you have enabled R8 in debug mode as well. Are you seeing the issue for debug builds as well, or was that just an experiment? And, if you can also explain how to get the runtime error that would also be helpful.
sg...@google.com <sg...@google.com> #6
Added dump from running:
ANDROID_SDK_ROOT=~/Android/Sdk ./gradlew :android:minifyErgomainnetReleaseWithR8
ANDROID_SDK_ROOT=~/Android/Sdk ./gradlew :android:cleanMinifyErgomainnetReleaseWithR8
ANDROID_SDK_ROOT=~/Android/Sdk ./gradlew :android:minifyErgomainnetReleaseWithR8 -Dcom.android.tools.r8.dumpinputtofile=mydump.zip --no-daemon
sg...@google.com <sg...@google.com> #7
Running dex2oat
on the APK reveals the verification error:
tools/dex2oat.py android-ergomainnet-release-unsigned.apk
Running: /usr/local/google/home/sgjesse/prj/r8/ws1/tools/linux/art/bin/dex2oat --android-root=/usr/local/google/home/sgjesse/prj/r8/ws1/tools/linux/art/product/angler/system --runtime-arg -Xnorelocate --dex-file=/usr/local/google/home/sgjesse/prj/ergo-wallet-app/android/build/outputs/apk/ergomainnet/release/android-ergomainnet-release-unsigned.apk --oat-file=/tmp/tmplgzz2tvs/out.oat --instruction-set=arm64
dex2oat I 03-09 15:17:04 1352571 1352571 dex2oat.cc:3108] /usr/local/google/home/sgjesse/prj/r8/ws1/tools/linux/art/bin/dex2oat --android-root=/usr/local/google/home/sgjesse/prj/r8/ws1/tools/linux/art/product/angler/system --runtime-arg -Xnorelocate --dex-file=/usr/local/google/home/sgjesse/prj/ergo-wallet-app/android/build/outputs/apk/ergomainnet/release/android-ergomainnet-release-unsigned.apk --oat-file=/tmp/tmplgzz2tvs/out.oat --instruction-set=arm64
dex2oat W 03-09 15:17:07 1352571 1352613 method_verifier.cc:467] Verification error in scala.collection.immutable.Iterable[] scala.collection.immutable.TrieIterator.getElems(scala.collection.immutable.Iterable)
dex2oat W 03-09 15:17:07 1352571 1352613 method_verifier.cc:467] scala.collection.immutable.Iterable[] scala.collection.immutable.TrieIterator.getElems(scala.collection.immutable.Iterable) failed to verify: scala.collection.immutable.Iterable[] scala.collection.immutable.TrieIterator.getElems(scala.collection.immutable.Iterable): [0x15] returning 'Reference: scala.collection.AbstractIterable[]', but expected from declaration 'Reference: scala.collection.immutable.Iterable[]'
dex2oat W 03-09 15:17:07 1352571 1352613 method_verifier.cc:467]
dex2oat W 03-09 15:17:07 1352571 1352609 class_linker.cc:3896] Rejecting class scala.collection.immutable.HashMap$HashTrieMap$$anon$1 that attempts to sub-type erroneous class scala.collection.immutable.TrieIterator in /usr/local/google/home/sgjesse/prj/ergo-wallet-app/android/build/outputs/apk/ergomainnet/release/android-ergomainnet-release-unsigned.apk!classes3.dex
dex2oat W 03-09 15:17:07 1352571 1352636 class_linker.cc:3896] Rejecting class scala.collection.immutable.HashSet$HashTrieSet$$anon$1 that attempts to sub-type erroneous class scala.collection.immutable.TrieIterator in /usr/local/google/home/sgjesse/prj/ergo-wallet-app/android/build/outputs/apk/ergomainnet/release/android-ergomainnet-release-unsigned.apk!classes3.dex
dex2oat W 03-09 15:17:07 1352571 1352609 class_linker.cc:3896] Rejecting class scala.collection.immutable.TrieIterator$$anon$1 that attempts to sub-type erroneous class scala.collection.immutable.TrieIterator in /usr/local/google/home/sgjesse/prj/ergo-wallet-app/android/build/outputs/apk/ergomainnet/release/android-ergomainnet-release-unsigned.apk!classes3.dex
dex2oat W 03-09 15:17:07 1352571 1352667 class_linker.cc:3896] Rejecting class scala.collection.immutable.TrieIterator$DupIterator that attempts to sub-type erroneous class scala.collection.immutable.TrieIterator in /usr/local/google/home/sgjesse/prj/ergo-wallet-app/android/build/outputs/apk/ergomainnet/release/android-ergomainnet-release-unsigned.apk!classes3.dex
dex2oat W 03-09 15:17:08 1352571 1352651 compiler_driver.cc:614] Compilation of sigmastate.Values$Value sigmastate.lang.SigmaTyper.assignType(scala.collection.immutable.Map, sigmastate.Values$Value, scala.Option) took 148.840ms
dex2oat I 03-09 15:17:10 1352571 1352571 dex2oat.cc:2808] dex2oat took 5.386s (76.797s cpu) (threads: 96) arena alloc=44MB (46826544B) java alloc=30MB (32452192B) native alloc=80MB (84577808B) free=77MB (81547760B)
bs...@googlemail.com <bs...@googlemail.com> #8
Do you still need some if the information asked for above?
bs...@googlemail.com <bs...@googlemail.com> #9
sg...@google.com <sg...@google.com> #10
Sorry, but we have just not had the time to look further into this. Will prioritize to look into it this week.
bs...@googlemail.com <bs...@googlemail.com> #11
sg...@google.com <sg...@google.com> #12
Looking the input code for scala.collection.immutable.TrieIterator.getElems
is:
private scala.collection.immutable.Iterable<T>[] getElems(scala.collection.immutable.Iterable<T>);
descriptor: (Lscala/collection/immutable/Iterable;)[Lscala/collection/immutable/Iterable;
flags: (0x0002) ACC_PRIVATE
Code:
stack=3, locals=5, args_size=2
0: aload_1
1: instanceof #46 // class scala/collection/immutable/HashMap$HashTrieMap
4: ifeq 24
7: aload_1
8: checkcast #46 // class scala/collection/immutable/HashMap$HashTrieMap
11: astore_2
12: aload_2
13: invokevirtual #49 // Method scala/collection/immutable/HashMap$HashTrieMap.elems:()[Lscala/collection/immutable/HashMap;
16: checkcast #51 // class "[Lscala/collection/AbstractIterable;"
19: astore 4
21: goto 45
24: aload_1
25: instanceof #53 // class scala/collection/immutable/HashSet$HashTrieSet
28: ifeq 51
31: aload_1
32: checkcast #53 // class scala/collection/immutable/HashSet$HashTrieSet
35: astore_3
36: aload_3
37: invokevirtual #56 // Method scala/collection/immutable/HashSet$HashTrieSet.elems:()[Lscala/collection/immutable/HashSet;
40: checkcast #51 // class "[Lscala/collection/AbstractIterable;"
43: astore 4
45: aload 4
47: checkcast #33 // class "[Lscala/collection/immutable/Iterable;"
50: areturn
51: new #58 // class scala/MatchError
54: dup
55: aload_1
56: invokespecial #62 // Method scala/MatchError."<init>":(Ljava/lang/Object;)V
59: athrow
LocalVariableTable:
Start Length Slot Name Signature
0 60 0 this Lscala/collection/immutable/TrieIterator;
0 60 1 x Lscala/collection/immutable/Iterable;
LineNumberTable:
line 41: 0
line 40: 0
line 42: 24
line 40: 45
StackMapTable: number_of_entries = 3
frame_type = 24 /* same */
frame_type = 254 /* append */
offset_delta = 20
locals = [ top, top, class "[Lscala/collection/AbstractIterable;" ]
frame_type = 248 /* chop */
offset_delta = 5
Signature: #396 // (Lscala/collection/immutable/Iterable<TT;>;)[Lscala/collection/immutable/Iterable<TT;>;
and it is translated to
.method private getElems(Lscala/collection/immutable/Iterable;)[Lscala/collection/immutable/Iterable;
.locals 1
.annotation system Ldalvik/annotation/Signature;
value = {
"(",
"Lscala/collection/immutable/Iterable<",
"TT;>;)[",
"Lscala/collection/immutable/Iterable<",
"TT;>;"
}
.end annotation
.line 40
instance-of v0, p1, Lscala/collection/immutable/HashMap$HashTrieMap;
if-eqz v0, :cond_0
check-cast p1, Lscala/collection/immutable/HashMap$HashTrieMap;
invoke-virtual {p1}, Lscala/collection/immutable/HashMap$HashTrieMap;->elems()[Lscala/collection/immutable/HashMap;
move-result-object p1
goto :goto_0
.line 42
:cond_0
instance-of v0, p1, Lscala/collection/immutable/HashSet$HashTrieSet;
if-eqz v0, :cond_1
check-cast p1, Lscala/collection/immutable/HashSet$HashTrieSet;
invoke-virtual {p1}, Lscala/collection/immutable/HashSet$HashTrieSet;->elems()[Lscala/collection/immutable/HashSet;
move-result-object p1
:goto_0
return-object p1
.line 40
:cond_1
new-instance v0, Lscala/MatchError;
invoke-direct {v0, p1}, Lscala/MatchError;-><init>(Ljava/lang/Object;)V
throw v0
.end method
The cast to [Lscala/collection/AbstractIterable
is removed because R8 wrongly concludes that the it is not needed. The element type of the value returned is
@Nullable (@Nullable scala.collection.immutable.HashMap {java.io.Serializable, scala.collection.CustomParallelizable, scala.collection.immutable.Map}[])
and the return element type (and cast element type) is
@Nullable (@Nullable scala.collection.AbstractIterable {scala.collection.Iterable}[])
Part of the class hierachy is:
public class scala.collection.immutable.HashMap extends scala.collection.immutable.AbstractMap implements scala.Serializable, scala.collection.CustomParallelizable
public abstract class scala.collection.immutable.AbstractMap extends scala.collection.AbstractMap implements scala.collection.immutable.Map
public abstract class scala.collection.AbstractMap extends scala.collection.AbstractIterable implements scala.collection.Map
public interface scala.collection.Map extends scala.collection.Iterable, scala.collection.GenMap, scala.collection.MapLike
So the element types of return type array is assignable to the element type of the returned type. We will have to check the array assignability. According to
isJavaAssignable(arrayOf(X), arrayOf(Y)) :-
compound(X), compound(Y), isJavaAssignable(X, Y).
sg...@google.com <sg...@google.com> #13
This can be reduced to the code below, where R8 will remove the cast to I[]
in getArrayOfIWithCast
. The cast can safely be removed on the JVM, but on all Art and Dalvik versions removal cause a VerificationError
(
interface I {}
static class Impl1 implements I {}
static class Impl2 implements I {}
static class TestClass {
public static Impl1[] getArrayOfImpl1() {
return new Impl1[] { new Impl1() };
}
public static Impl2[] getArrayOfImpl2() {
return new Impl2[] { new Impl2() };
}
public static I[] getArrayOfIWithCast(boolean b) {
Object[] arrayOfI;
if (b) {
arrayOfI = TestClass.getArrayOfImpl1();
} else {
arrayOfI = TestClass.getArrayOfImpl2();
}
return (I[]) arrayOfI;
}
public static void main(String[] args) {
getArrayOfIWithCast(args.length == 0);
}
}
}
ap...@google.com <ap...@google.com> #14
Branch: main
commit 9600543f4691278df3abdc27c80f62879399bb10
Author: Søren Gjesse <sgjesse@google.com>
Date: Tue May 03 09:35:27 2022
Add a reproduction of the Scala libraries interface array cast issue
Bug:
Change-Id: I56caa6331bd1eb501d356f1f4660899251ae7feb
A src/test/java/com/android/tools/r8/ir/optimize/checkcast/InterfaceArrayCheckCastTest.java
A src/test/java/com/android/tools/r8/ir/optimize/checkcast/ArrayInterfaceArrayCheckCastTest.java
ap...@google.com <ap...@google.com> #15
Branch: main
commit 8ec45a2335f62f9521bbfbf976c4032295d034d8
Author: Søren Gjesse <sgjesse@google.com>
Date: Tue May 03 10:59:06 2022
Don't remove casts to arrays of interface types
All Dalvik and Art runtimes can reject code with VerifyError if when
joining arrays of interface types.
Bug:
Change-Id: If47782d76b706152f6e14b1238114dc89bc23119
M src/test/java/com/android/tools/r8/ir/optimize/checkcast/InterfaceArrayCheckCastTest.java
M src/test/java/com/android/tools/r8/ir/optimize/checkcast/ArrayInterfaceArrayCheckCastTest.java
M src/main/java/com/android/tools/r8/ir/optimize/CodeRewriter.java
ap...@google.com <ap...@google.com> #16
Branch: 3.3
commit 8af811f40b0ae35db32fae3ee6a0ffbff8b98b48
Author: Søren Gjesse <sgjesse@google.com>
Date: Tue May 03 14:29:13 2022
Version 3.3.36
Bug:
Change-Id: Ic85b9f2e6fec991e117b1a9c405e06352b4b18af
M src/main/java/com/android/tools/r8/Version.java
ap...@google.com <ap...@google.com> #17
Branch: 3.3
commit b82face4da1b9bfbb0c12f6754a565199d196ffd
Author: Søren Gjesse <sgjesse@google.com>
Date: Tue May 03 14:27:53 2022
Don't remove casts to arrays of interface types
All Dalvik and Art runtimes can reject code with VerifyError if when
joining arrays of interface types.
Bug:
Change-Id: If47782d76b706152f6e14b1238114dc89bc23119
M src/test/java/com/android/tools/r8/ir/optimize/checkcast/InterfaceArrayCheckCastTest.java
M src/test/java/com/android/tools/r8/ir/optimize/checkcast/ArrayInterfaceArrayCheckCastTest.java
M src/main/java/com/android/tools/r8/ir/optimize/CodeRewriter.java
ap...@google.com <ap...@google.com> #18
Branch: 3.3
commit b3e66f6513d099c8de81d733352b31d2df1fa704
Author: Søren Gjesse <sgjesse@google.com>
Date: Tue May 03 14:26:53 2022
Add a reproduction of the Scala libraries interface array cast issue
Bug:
Change-Id: I56caa6331bd1eb501d356f1f4660899251ae7feb
A src/test/java/com/android/tools/r8/ir/optimize/checkcast/InterfaceArrayCheckCastTest.java
A src/test/java/com/android/tools/r8/ir/optimize/checkcast/ArrayInterfaceArrayCheckCastTest.java
ap...@google.com <ap...@google.com> #19
Branch: 3.1
commit 4d25053d92089deff0616d98e0c982ceb34715df
Author: Søren Gjesse <sgjesse@google.com>
Date: Tue May 03 14:36:24 2022
Version 3.1.75
Bug:
Change-Id: I1b2b87b3a43ce75fb0945dfb2d85651ad8cdd704
M src/main/java/com/android/tools/r8/Version.java
ap...@google.com <ap...@google.com> #20
Branch: 3.1
commit ab28ed0f89ef6eb110d2cc519215eaeebf0364a9
Author: Søren Gjesse <sgjesse@google.com>
Date: Tue May 03 14:35:06 2022
Don't remove casts to arrays of interface types
All Dalvik and Art runtimes can reject code with VerifyError if when
joining arrays of interface types.
Bug:
Change-Id: If47782d76b706152f6e14b1238114dc89bc23119
M src/test/java/com/android/tools/r8/ir/optimize/checkcast/InterfaceArrayCheckCastTest.java
M src/test/java/com/android/tools/r8/ir/optimize/checkcast/ArrayInterfaceArrayCheckCastTest.java
M src/main/java/com/android/tools/r8/ir/optimize/CodeRewriter.java
ap...@google.com <ap...@google.com> #21
Branch: 3.1
commit 1dff43386d76218e0136528c31e1f9dc4e704f39
Author: Søren Gjesse <sgjesse@google.com>
Date: Tue May 03 14:34:27 2022
Add a reproduction of the Scala libraries interface array cast issue
Bug:
Change-Id: I56caa6331bd1eb501d356f1f4660899251ae7feb
A src/test/java/com/android/tools/r8/ir/optimize/checkcast/InterfaceArrayCheckCastTest.java
A src/test/java/com/android/tools/r8/ir/optimize/checkcast/ArrayInterfaceArrayCheckCastTest.java
ap...@google.com <ap...@google.com> #22
Branch: 3.2
commit 993b6d6d1c19811db5b6c654fc82ed928672371c
Author: Søren Gjesse <sgjesse@google.com>
Date: Tue May 03 14:33:00 2022
Version 3.2.61
Bug:
Change-Id: I618e02d188f619dcf3d06a0203047bfaa0071403
M src/main/java/com/android/tools/r8/Version.java
ap...@google.com <ap...@google.com> #23
Branch: 3.2
commit 85ea0af317ea02c343b6f35165a21f6d5503739e
Author: Søren Gjesse <sgjesse@google.com>
Date: Tue May 03 14:31:55 2022
Don't remove casts to arrays of interface types
All Dalvik and Art runtimes can reject code with VerifyError if when
joining arrays of interface types.
Bug:
Change-Id: If47782d76b706152f6e14b1238114dc89bc23119
M src/test/java/com/android/tools/r8/ir/optimize/checkcast/InterfaceArrayCheckCastTest.java
M src/test/java/com/android/tools/r8/ir/optimize/checkcast/ArrayInterfaceArrayCheckCastTest.java
M src/main/java/com/android/tools/r8/ir/optimize/CodeRewriter.java
ap...@google.com <ap...@google.com> #24
Branch: 3.2
commit bc24725b87b446e4242f3f7a03f9ac4a48f3c39d
Author: Søren Gjesse <sgjesse@google.com>
Date: Tue May 03 14:31:11 2022
Add a reproduction of the Scala libraries interface array cast issue
Bug:
Change-Id: I56caa6331bd1eb501d356f1f4660899251ae7feb
A src/test/java/com/android/tools/r8/ir/optimize/checkcast/InterfaceArrayCheckCastTest.java
A src/test/java/com/android/tools/r8/ir/optimize/checkcast/ArrayInterfaceArrayCheckCastTest.java
ap...@google.com <ap...@google.com> #25
Branch: 3.1
commit ab28ed0f89ef6eb110d2cc519215eaeebf0364a9
Author: Søren Gjesse <sgjesse@google.com>
Date: Tue May 03 14:35:06 2022
Don't remove casts to arrays of interface types
All Dalvik and Art runtimes can reject code with VerifyError if when
joining arrays of interface types.
Bug:
Change-Id: If47782d76b706152f6e14b1238114dc89bc23119
M src/test/java/com/android/tools/r8/ir/optimize/checkcast/InterfaceArrayCheckCastTest.java
M src/test/java/com/android/tools/r8/ir/optimize/checkcast/ArrayInterfaceArrayCheckCastTest.java
M src/main/java/com/android/tools/r8/ir/optimize/CodeRewriter.java
ap...@google.com <ap...@google.com> #26
Branch: 3.1
commit 1dff43386d76218e0136528c31e1f9dc4e704f39
Author: Søren Gjesse <sgjesse@google.com>
Date: Tue May 03 14:34:27 2022
Add a reproduction of the Scala libraries interface array cast issue
Bug:
Change-Id: I56caa6331bd1eb501d356f1f4660899251ae7feb
A src/test/java/com/android/tools/r8/ir/optimize/checkcast/InterfaceArrayCheckCastTest.java
A src/test/java/com/android/tools/r8/ir/optimize/checkcast/ArrayInterfaceArrayCheckCastTest.java
sg...@google.com <sg...@google.com> #27
This is now fixed, and the fix had been cherry-picked to the 3.1, 3.2 and 3.3 branches. For AGP 7.1, 7.2 and 7.3 the R8 versions with the fix are 3.1.75, 3.2.61 and 3.3.36 respectively.
Building on your r8-enabled
branch with this patch:
@@ -11,10 +11,13 @@ buildscript {
repositories {
google()
gradlePluginPortal()
+ maven {
+ url = uri("https://storage.googleapis.com/r8-releases/raw")
+ }
}
dependencies {
// Android Gradle Plugin 4.2.0: problems minifying; 7.x: enableR8=false removed
- classpath 'com.android.tools:r8:3.1.51' //Must be before the Gradle Plugin for Android. - Or any other version
+ classpath 'com.android.tools:r8:3.1.75' //Must be before the Gradle Plugin for Android. - Or any other version
classpath "com.android.tools.build:gradle:7.1.2"
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
classpath("androidx.navigation:navigation-safe-args-gradle-plugin:$nav_version")
no longer removes the check-cast
causing the verification error on Android, as the code for scala.collection.immutable.TrieIterator.getElems
looks like this:
.method private getElems(Lscala/collection/immutable/Iterable;)[Lscala/collection/immutable/Iterable;
.locals 1
.annotation system Ldalvik/annotation/Signature;
value = {
"(",
"Lscala/collection/immutable/Iterable<",
"TT;>;)[",
"Lscala/collection/immutable/Iterable<",
"TT;>;"
}
.end annotation
.line 40
instance-of v0, p1, Lscala/collection/immutable/HashMap$HashTrieMap;
if-eqz v0, :cond_0
check-cast p1, Lscala/collection/immutable/HashMap$HashTrieMap;
invoke-virtual {p1}, Lscala/collection/immutable/HashMap$HashTrieMap;->elems()[Lscala/collection/immutable/HashMap;
move-result-object p1
goto :goto_0
.line 42
:cond_0
instance-of v0, p1, Lscala/collection/immutable/HashSet$HashTrieSet;
if-eqz v0, :cond_1
check-cast p1, Lscala/collection/immutable/HashSet$HashTrieSet;
invoke-virtual {p1}, Lscala/collection/immutable/HashSet$HashTrieSet;->elems()[Lscala/collection/immutable/HashSet;
move-result-object p1
.line 40
:goto_0
check-cast p1, [Lscala/collection/immutable/Iterable;
return-object p1
:cond_1
new-instance v0, Lscala/MatchError;
invoke-direct {v0, p1}, Lscala/MatchError;-><init>(Ljava/lang/Object;)V
throw v0
.end method
bs...@googlemail.com <bs...@googlemail.com> #28
I am not familiar how you do releases - is this release good to use in production or should I wait for the integration until it is available on the main google maven repo?
sg...@google.com <sg...@google.com> #29
Thank you for confirming that the issues has been fixed.
These versions are for production. Most likely there will not be any more patch releases of AGP 7.1, so explicitly using R8 3.1.75 is the only way to get this fix into AGP 7.1. For AGP 7.2 this fix will also most likely not be in the first release and until AGP 7.2.1 explicitly using R8 3.2.61 will also be required. For AGP 7.3 the fix will most likely be in Beta 2.
de...@google.com <de...@google.com> #30
Thank you for your patience while our engineering team worked to resolve this issue. A fix for this issue is now available in:
- Android Studio Dolphin Beta 2 (2021.3.1.11)
- Android Gradle Plugin 7.3.0-beta02
We encourage you to try the latest update.
If you notice further issues or have questions, please file a new bug report.
Thank you for taking the time to submit feedback — we really appreciate it!
Description
I am using Scala 2.11 libraries in my Android project. With Proguard, shrinking worked well. When switching to R8, I got a VerifyError on calling some methods using Scala collection classes:
java.lang.VerifyError: Rejecting class scala.collection.immutable.HashMap$HashTrieMap$$anon$1 that attempts to sub-type erroneous class scala.collection.immutable.TrieIterator (declaration of 'scala.collection.immutable.HashMap$HashTrieMap$$anon$1'
I have
-keep class scala.collection.** { *; }
in my proguard-rules.pro
Any help is appreciated since using Proguard is not possible with APG 7.x