ktor VerifyError [237567012]

Verified

Bug

Status Update

No update yet.

Description

bi...@mail.de

created issue #1

Jun 30, 2022 08:52AM

When I shrink my shadow jar with r8, r8 messes up. I created a sample project where r8 is told not to do anything, still the jar contains invalid bytecode, leading to the VerifyError: https://github.com/binarynoise/ktor-test

As far as I can tell, my toolchain is up-to-date. Also happens with the CIO implemenatation of Ktor. As the not-minified shadow jar works as intended, I suspect r8 to be buggy.

This is my build log:

C:\Users\binarynoise\Documents\src\ktor-test>.\gradlew.bat shadowJarMinified

> Task :shadowJarMinified
Info: Stripped invalid locals information from 2 methods.
Info in C:\Users\binarynoise\Documents\src\ktor-test\build\libs\ktor-test-shadow.jar:kotlinx/coroutines/flow/FlowKt__BuildersKt$asFlow$$inlined$unsafeFlow$10.class:
Methods with invalid locals information:
  java.lang.Object kotlinx.coroutines.flow.FlowKt__BuildersKt$asFlow$$inlined$unsafeFlow$10.collect(kotlinx.coroutines.flow.FlowCollector, kotlin.coroutines.Continuation)
  Attempt to define local of type long as element$iv:java.lang.Object
Info in C:\Users\binarynoise\Documents\src\ktor-test\build\libs\ktor-test-shadow.jar:kotlinx/coroutines/flow/FlowKt__BuildersKt$asFlow$$inlined$unsafeFlow$9.class:
Methods with invalid locals information:
  java.lang.Object kotlinx.coroutines.flow.FlowKt__BuildersKt$asFlow$$inlined$unsafeFlow$9.collect(kotlinx.coroutines.flow.FlowCollector, kotlin.coroutines.Continuation)
  Attempt to define local of type int as element$iv:java.lang.Object
Info: Some warnings are typically a sign of using an outdated Java toolchain. To fix, recompile the source with an updated toolchain.

BUILD SUCCESSFUL in 12s
6 actionable tasks: 5 executed, 1 up-to-date
C:\Users\binarynoise\Documents\src\ktor-test>java -jar build\libs\ktor-test-shadow-minified.jar
Exception in thread "main" java.lang.VerifyError: Stack map does not match the one at exception handler 540
Exception Details:
  Location:
    io/ktor/client/engine/java/JavaHttpResponseBodyHandler$JavaHttpResponseBodySubscriber$1.invokeSuspend(Ljava/lang/Object;)Ljava/lang/Object; @540: astore_2
  Reason:
    Type top (current frame, locals[4]) is not assignable to 'java/lang/Throwable' (stack map, locals[4])
  Current Frame:
    bci: @533
    flags: { }
    locals: { 'io/ktor/client/engine/java/JavaHttpResponseBodyHandler$JavaHttpResponseBodySubscriber$1', 'java/lang/Object', 'java/lang/Throwable', integer, top, null, top, 'java/lang/Object' }
    stack: { 'java/lang/Throwable' }
  Stackmap Frame:
    bci: @540
    flags: { }
    locals: { 'io/ktor/client/engine/java/JavaHttpResponseBodyHandler$JavaHttpResponseBodySubscriber$1', 'java/lang/Object', top, integer, 'java/lang/Throwable', top, top, 'java/lang/Object' }
    stack: { 'java/lang/Throwable' }
  Bytecode:
    0000000: 2ab8 004b 4db4 004d aa00 0000 0000 001c
    0000010: 0000 0000 0000 0002 0000 00a9 0000 0058
    0000020: 0000 0026 bb00 4f59 1251 b700 54bf 2b2a
    0000030: 4b4c 033e 0336 0401 3a05 2ab4 0058 c000
    0000040: 2a3a 062a b400 5ac0 005c 3a07 1907 3a07
    0000050: 2ab4 005e c000 603a 082b b800 66a7 01a6
    0000060: 2b2a 4b4c 033e 0336 0401 3a05 2ab4 0058
    0000070: c000 2a3a 062a b400 5ac0 005c 3a07 1907
    0000080: 3a07 2ab4 005e c000 603a 082b b800 6619
    0000090: 0819 0719 062c 2b1d 1905 1504 3605 3a06
    00000a0: 3604 4d4e 3a07 3a08 3a09 a700 ea4d a701
    00000b0: 672b 2a2b b800 664b 4c2a b400 5ec0 0060
    00000c0: 4e00 2ab4 003a 3a04 1904 b800 6c3a 0419
    00000d0: 042a b400 3a3a 043a 0503 3606 013a 0703
    00000e0: 3608 2d15 0819 0419 0519 0715 063e 3a05
    00000f0: 3a07 3a06 3604 3a08 1908 b800 7236 0915
    0000100: 0999 0105 1906 b800 6c3a 0919 09b9 0075
    0000110: 0100 3a09 1909 b800 7a3a 0919 09c0 007c
    0000120: 3a09 1909 3a09 1909 c700 a919 06b4 007f
    0000130: 3a09 1909 c000 813a 0919 09c6 000b 1909
    0000140: 0ab9 0085 0300 1906 b800 6c3a 0919 092a
    0000150: 1908 b500 5e2a 1907 b500 5a2a 1906 b500
    0000160: 582a 04b5 004d 2ab9 0089 0200 3a09 1909
    0000170: 2ca6 0005 2cb0 1909 1908 1907 1906 2c2b
    0000180: 1d19 0515 0436 053a 0636 044d 4e3a 073a
    0000190: 083a 094c 2bc0 007c 4c2b 4c2b 4c19 072b
    00001a0: 1909 1908 2d2c 1504 1906 1505 3604 3a05
    00001b0: 3e4c 4d3a 073a 083a 093a 06a7 0016 4c2b
    00001c0: 1906 1908 1504 2c4c 3e3a 073a 054d a700
    00001d0: 4719 06b8 008d 3a0a 190a 2a19 08b5 005e
    00001e0: 2a19 07b5 005a 2a19 06b5 0058 2a05 b500
    00001f0: 4d19 092a b900 9303 003a 0a19 0a2c a600
    0000200: 052c b0a7 fef5 00b2 0099 5719 0719 05b8
    0000210: 009f a700 152c 4d2c 3a04 2cbf 4d2c 1907
    0000220: 1904 b800 9fbf 57b2 0099 b0
  Exception Handler Table:
    bci [89, 96] => handler: 173
    bci [139, 143] => handler: 173
    bci [193, 198] => handler: 550
    bci [200, 205] => handler: 550
    bci [207, 213] => handler: 550
    bci [248, 253] => handler: 173
    bci [260, 265] => handler: 173
    bci [267, 274] => handler: 173
    bci [276, 281] => handler: 173
    bci [283, 288] => handler: 173
    bci [299, 304] => handler: 173
    bci [306, 311] => handler: 173
    bci [318, 331] => handler: 173
    bci [333, 364] => handler: 173
    bci [404, 408] => handler: 446
    bci [465, 470] => handler: 173
    bci [472, 505] => handler: 173
    bci [518, 522] => handler: 173
    bci [523, 530] => handler: 550
    bci [533, 540] => handler: 540
    bci [541, 550] => handler: 550
  Stackmap Table:
    chop_frame(@36,2)
    append_frame(@46,Object[#5],Object[#86],Object[#86])
    same_frame(@96)
    full_frame(@173,{Object[#5],Object[#86],Top,Integer,Top,Null,Top,Object[#86]},{Object[#104]})
    full_frame(@177,{Object[#5],Object[#86],Object[#86]},{})
    full_frame(@248,{Object[#5],Object[#86],Object[#86],Integer,Integer,Null,Object[#42],Object[#86],Object[#96]},{})
    same_frame_extended(@326)
    append_frame(@374,Object[#86])
    full_frame(@404,{Object[#5],Object[#86],Object[#86],Object[#86],Integer,Integer,Null,Object[#42],Object[#86],Object[#96]},{})
    full_frame(@446,{Object[#5],Top,Object[#86],Top,Integer,Top,Null,Top,Object[#86]},{Object[#104]})
    full_frame(@465,{Object[#5],Object[#86],Object[#86],Integer,Integer,Null,Object[#42],Object[#86],Object[#96],Object[#124]},{})
    chop_frame(@515,1)
    full_frame(@518,{Object[#5],Object[#86],Top,Integer,Integer,Null,Top,Object[#86],Object[#96]},{})
    full_frame(@533,{Object[#5],Object[#86],Object[#104],Integer,Top,Null,Top,Object[#86]},{})
    full_frame(@540,{Object[#5],Object[#86],Top,Integer,Object[#104],Top,Top,Object[#86]},{Object[#104]})
    full_frame(@550,{Object[#5],Object[#86]},{Object[#161]})
    same_frame(@551)

        at io.ktor.client.engine.java.JavaHttpResponseBodyHandler$JavaHttpResponseBodySubscriber.<init>(JavaHttpResponseBodyHandler.kt:60)
        at io.ktor.client.engine.java.JavaHttpResponseBodyHandler.apply(JavaHttpResponseBodyHandler.kt:26)
        at java.net.http/jdk.internal.net.http.Http1Exchange.readBodyAsync(Http1Exchange.java:373)
        at java.net.http/jdk.internal.net.http.Exchange.readBodyAsync(Exchange.java:185)
        at java.net.http/jdk.internal.net.http.MultiExchange.lambda$responseAsync0$4(MultiExchange.java:353)
        at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1150)
        at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
        at java.base/java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:614)
        at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:653)
        at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482)
        at java.net.http/jdk.internal.net.http.HttpClientImpl$DelegatingExecutor.execute(HttpClientImpl.java:157)
        at java.base/java.util.concurrent.CompletableFuture$UniCompletion.claim(CompletableFuture.java:572)
        at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:642)
        at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
        at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147)
        at java.net.http/jdk.internal.net.http.Http1Response$HeadersReader.handle(Http1Response.java:695)
        at java.net.http/jdk.internal.net.http.Http1Response$HeadersReader.handle(Http1Response.java:621)
        at java.net.http/jdk.internal.net.http.Http1Response$Receiver.accept(Http1Response.java:612)
        at java.net.http/jdk.internal.net.http.Http1Response$HeadersReader.tryAsyncReceive(Http1Response.java:668)
        at java.net.http/jdk.internal.net.http.Http1AsyncReceiver.flush(Http1AsyncReceiver.java:233)
        at java.net.http/jdk.internal.net.http.common.SequentialScheduler$LockingRestartableTask.run(SequentialScheduler.java:205)
        at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:149)
        at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:230)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)

Comments

bi...@mail.de <bi...@mail.de> #2Jun 30, 2022 08:54AM

Fixed in r1313.

sg...@google.com <sg...@google.com> Jun 30, 2022 08:55AM

Reassigned to mk...@google.com.

mk...@google.com <mk...@google.com> #3Jul 1, 2022 04:11PM

[Empty comment from Monorail migration]

mk...@google.com <mk...@google.com> #4Jul 5, 2022 07:44AM

[Empty comment from Monorail migration]

bi...@mail.de <bi...@mail.de> #5Jul 5, 2022 05:17PM

This issue was migrated from

crbug.com/nativeclient/245?no_tracker_redirect=1

[Monorail components: TrustedRT]

mk...@google.com <mk...@google.com> #6Jul 5, 2022 06:09PM

Yes, stack map verification is only done when running on the JVM, not even by javac. Normally all code passed through D8/R8 is for running on ART.

ap...@google.com <ap...@google.com> #7Aug 4, 2022 10:13AM

Project: r8
Branch: main

commit 77a6d243cf1e02beda644d5896fc789c9964d918
Author: Morten Krogh-Jespersen <mkroghj@google.com>
Date: Thu Aug 04 09:37:46 2022

Add test for invalid stackmap for debug locals and trycatch

Bug:

b/237567012

Change-Id: I0edc7aa8bd827db46cae428e48964c47a99165f0

A src/test/java/com/android/tools/r8/cf/CfDebugLocalStackMapVerificationTest.java

https://r8-review.googlesource.com/69950

mk...@google.com <mk...@google.com> Aug 4, 2022 10:15AM

Assigned to ze...@google.com.

ap...@google.com <ap...@google.com> #8Aug 5, 2022 10:28AM

Project: r8
Branch: main

commit fe051981a30c1ed914955d18e1d310cd43b76a3c
Author: Ian Zerny <zerny@google.com>
Date: Fri Aug 05 12:14:23 2022

Small regression test for invalid exceptional transfers.

Bug:

b/237567012

Change-Id: I8ac44f7df25f0a0ca28ed8a02c4a2a2946043bd9

M src/main/java/com/android/tools/r8/cf/CfRegisterAllocator.java
M src/test/java/com/android/tools/r8/cf/CfDebugLocalStackMapVerificationTest.java
M src/main/java/com/android/tools/r8/utils/InternalOptions.java

https://r8-review.googlesource.com/70067

ap...@google.com <ap...@google.com> #9Aug 9, 2022 05:49AM

Marked as fixed.

Project: r8
Branch: main

commit f787c4dbcf81efc7594560aa1665434ff55f0ed4
Author: Ian Zerny <zerny@google.com>
Date: Mon Aug 08 15:42:48 2022

Split block after implicit store via DebugLocalWrite.

Fixes:

b/237567012

Change-Id: I6023e442a97838dfa816ad778b49a96d99b608ac

M src/main/java/com/android/tools/r8/ir/code/DebugLocalWrite.java
M src/main/java/com/android/tools/r8/cf/LoadStoreHelper.java
M src/test/java/com/android/tools/r8/cf/CfDebugLocalStackMapVerificationTest.java

https://r8-review.googlesource.com/70120

ze...@google.com <ze...@google.com> #10Aug 9, 2022 05:56AM

Status: Assigned (reopened)

This should be cherry picked to release branches too.

ap...@google.com <ap...@google.com> #11Aug 9, 2022 09:12AM

Project: r8
Branch: 3.2

commit c1f5a649e6b174dc5c977090c70d2cb0cc4caf58
Author: Ian Zerny <zerny@google.com>
Date: Tue Aug 09 09:32:15 2022

Version 3.2.77

Bug:

b/237567012

Change-Id: I03b64963a474c2ad8709fffe6349769c8788df41

M src/main/java/com/android/tools/r8/Version.java

https://r8-review.googlesource.com/70143

ap...@google.com <ap...@google.com> #12Aug 9, 2022 09:12AM

Project: r8
Branch: 3.2

commit 0d5e3c0c6516de5047c1747fda8ea5c5ac212bdf
Author: Ian Zerny <zerny@google.com>
Date: Tue Aug 09 09:30:51 2022

Split block after implicit store via DebugLocalWrite.

Bug:

b/237567012

https://r8-review.googlesource.com/70142

ap...@google.com <ap...@google.com> #13Aug 9, 2022 09:12AM

Project: r8
Branch: 3.2

commit 1cdabd9d14c033400d9bcfeb40b720523500283a
Author: Ian Zerny <zerny@google.com>
Date: Tue Aug 09 09:06:42 2022

Small regression test for invalid exceptional transfers.

Bug:

b/237567012

https://r8-review.googlesource.com/70141

ap...@google.com <ap...@google.com> #14Aug 9, 2022 09:12AM

Project: r8
Branch: 3.2

commit 028f9e7d5b2a94b8bc8e3ba1f2c0f3eecc795564
Author: Ian Zerny <zerny@google.com>
Date: Tue Aug 09 08:55:57 2022

Add test for invalid stackmap for debug locals and trycatch

Bug:

b/237567012

Change-Id: I0edc7aa8bd827db46cae428e48964c47a99165f0

A src/test/java/com/android/tools/r8/cf/CfDebugLocalStackMapVerificationTest.java

https://r8-review.googlesource.com/70140

ap...@google.com <ap...@google.com> #15Aug 10, 2022 10:48AM

Project: r8
Branch: 3.3

commit b46c3d970800b208b33ee6506d167444c434c7fb
Author: Ian Zerny <zerny@google.com>
Date: Tue Aug 09 12:32:58 2022

Version 3.3.72

Bug:

b/237567012

Change-Id: Ie4d364afbb9fa5fbc3d9e0a691747d3988351aec

M src/main/java/com/android/tools/r8/Version.java

https://r8-review.googlesource.com/70153

ap...@google.com <ap...@google.com> #16Aug 10, 2022 10:48AM

Project: r8
Branch: 3.3

commit 1cbd003b35f979a6c95ed13ccf03922ac32bf883
Author: Ian Zerny <zerny@google.com>
Date: Tue Aug 09 12:19:24 2022

Split block after implicit store via DebugLocalWrite.

Bug:

b/237567012

https://r8-review.googlesource.com/70150

ap...@google.com <ap...@google.com> #17Aug 10, 2022 10:48AM

Project: r8
Branch: 3.3

commit d86c058a65371a93762a10ee81b028e987c11263
Author: Ian Zerny <zerny@google.com>
Date: Tue Aug 09 12:19:13 2022

Small regression test for invalid exceptional transfers.

Bug:

b/237567012

https://r8-review.googlesource.com/70145

ap...@google.com <ap...@google.com> #18Aug 10, 2022 10:48AM

Project: r8
Branch: 3.3

commit ee6fe4992c5267d5dfc3e9b178dcacb4de66bfc8
Author: Ian Zerny <zerny@google.com>
Date: Tue Aug 09 12:19:01 2022

Add test for invalid stackmap for debug locals and trycatch

Bug:

b/237567012

Change-Id: I0edc7aa8bd827db46cae428e48964c47a99165f0

A src/test/java/com/android/tools/r8/cf/CfDebugLocalStackMapVerificationTest.java

https://r8-review.googlesource.com/70144

ze...@google.com <ze...@google.com> #19Aug 10, 2022 10:56AM

Verified by ze...@google.com.

Thanks for reporting this issue. The fix for it has landed on our main branch and release branches for 3.2 and 3.3.

You can locally use a newer version of R8 by updating your settings.gradle (or settings.gradle.kts) file with the below template:

pluginManagement {
    buildscript {
        repositories {
            mavenCentral()
            maven {
                url = uri("https://storage.googleapis.com/r8-releases/raw")
            }
        }

        dependencies {
            classpath("com.android.tools:r8:<r8version>")   // This should be 3.2.77 for AGP 7.2 or 3.3.72 for AGP 7.3
        }
    }
}

ap...@google.com <ap...@google.com> #20Aug 10, 2022 01:08PM

Project: r8
Branch: 3.3

commit 77bf094c51e2ca9398e3ba36030efa8c2f923e1a
Author: Morten Krogh-Jespersen <mkroghj@google.com>
Date: Wed Aug 10 15:06:47 2022

Version 3.3.74

Bug:

b/237567012

Change-Id: I1251991627fa1028c60698aa2f5cfa5b75076c9a

M src/main/java/com/android/tools/r8/Version.java

https://r8-review.googlesource.com/70202

bi...@mail.de <bi...@mail.de> #21Aug 10, 2022 03:26PM

Thank you for fixing!

bi...@mail.de <bi...@mail.de> #22Aug 10, 2022 03:28PM

What about b/237567012 ?

Issue 237567012

Description

Issue summary

Comments

bi...@mail.de <bi...@mail.de> #2Jun 30, 2022 08:54AM

sg...@google.com <sg...@google.com> Jun 30, 2022 08:55AM

mk...@google.com <mk...@google.com> #3Jul 1, 2022 04:11PM

mk...@google.com <mk...@google.com> #4Jul 5, 2022 07:44AM

bi...@mail.de <bi...@mail.de> #5Jul 5, 2022 05:17PM

mk...@google.com <mk...@google.com> #6Jul 5, 2022 06:09PM

ap...@google.com <ap...@google.com> #7Aug 4, 2022 10:13AM

mk...@google.com <mk...@google.com> Aug 4, 2022 10:15AM

ap...@google.com <ap...@google.com> #8Aug 5, 2022 10:28AM

ap...@google.com <ap...@google.com> #9Aug 9, 2022 05:49AM

ze...@google.com <ze...@google.com> #10Aug 9, 2022 05:56AM

ap...@google.com <ap...@google.com> #11Aug 9, 2022 09:12AM

ap...@google.com <ap...@google.com> #12Aug 9, 2022 09:12AM

ap...@google.com <ap...@google.com> #13Aug 9, 2022 09:12AM

ap...@google.com <ap...@google.com> #14Aug 9, 2022 09:12AM

ap...@google.com <ap...@google.com> #15Aug 10, 2022 10:48AM

ap...@google.com <ap...@google.com> #16Aug 10, 2022 10:48AM

ap...@google.com <ap...@google.com> #17Aug 10, 2022 10:48AM

ap...@google.com <ap...@google.com> #18Aug 10, 2022 10:48AM

ze...@google.com <ze...@google.com> #19Aug 10, 2022 10:56AM

ap...@google.com <ap...@google.com> #20Aug 10, 2022 01:08PM

bi...@mail.de <bi...@mail.de> #21Aug 10, 2022 03:26PM

bi...@mail.de <bi...@mail.de> #22Aug 10, 2022 03:28PM

Add comment

Issue metadata