IssueTracker

Something is off with the method leading to this situation.

Can you provide us with a dump of the compilation so we can debug? Here is the documentation on how to make a dump https://r8.googlesource.com/r8/+/refs/heads/main/doc/compilerdump.md

Alternatively we would need a simple project to reproduce the issue.

It's a bit unclear to me what's going on. The SimpleInliningConstraintAnalysis is recursive but that should fine since all seen blacks are remembered and not reprocessed. It could be that the code you have just have a lot of branches leading to something which looks like infinite recursion?

I captured a dump and tried to reproduce the original error, but now I'm getting a different stack trace:

Error: java.lang.AssertionError: During NARROWING, @NotNull java.lang.Object {com.google.android.gms.internal.ads.zzmb} < @NotNull java.lang.Object {} at Assume               v1206 <- v51; not null (context: void com.google.android.gms.internal.ads.zzkb.<init>(com.google.android.gms.internal.ads.zziv, com.google.android.gms.internal.ads.zzco))
Exception in thread "main" com.android.tools.r8.CompilationFailedException: Compilation failed to complete
	at Version.fakeStackEntry(Version_8.5.33.java:0)
	at com.android.tools.r8.T.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:5)
	at com.android.tools.r8.internal.to.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:82)
	at com.android.tools.r8.internal.to.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:32)
	at com.android.tools.r8.internal.to.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:31)
	at com.android.tools.r8.R8.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:79)
	at com.android.tools.r8.R8.run(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:3)
	at com.android.tools.r8.utils.CompileDumpCompatR8.main(CompileDumpCompatR8.java:266)
Caused by: java.lang.AssertionError: During NARROWING, @NotNull java.lang.Object {com.google.android.gms.internal.ads.zzmb} < @NotNull java.lang.Object {} at Assume               v1206 <- v51; not null (context: void com.google.android.gms.internal.ads.zzkb.<init>(com.google.android.gms.internal.ads.zziv, com.google.android.gms.internal.ads.zzco))
	at com.android.tools.r8.internal.ul0.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:291)
	at com.android.tools.r8.internal.rj0.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:134)
	at com.android.tools.r8.internal.rj0.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:11)
	at com.android.tools.r8.internal.rj0.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:78)
	at com.android.tools.r8.internal.rj0.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:12)
	at com.android.tools.r8.ir.optimize.a.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:12)
	at com.android.tools.r8.ir.optimize.a.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:1)
	at com.android.tools.r8.ir.optimize.p0.c(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:410)
	at com.android.tools.r8.ir.optimize.q0.b(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:3)
	at com.android.tools.r8.internal.Gd.b(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:2)
	at com.android.tools.r8.internal.Gd.c(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:5)
	at com.android.tools.r8.internal.Gd.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:2)
	at com.android.tools.r8.internal.Fh0.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:63)
	at com.android.tools.r8.internal.Gd.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:3)
	at com.android.tools.r8.internal.Hd.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:3)
	at com.android.tools.r8.internal.Tt.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:611)
	at com.android.tools.r8.internal.Tt.d(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:29)
	at com.android.tools.r8.internal.Tt.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:401)
	at com.android.tools.r8.internal.to.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:135)
	at com.android.tools.r8.internal.Tt.c(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:5)
	at com.android.tools.r8.internal.Tt.b(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:16)
	at com.android.tools.r8.internal.kY.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:217)
	at com.android.tools.r8.internal.jY.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:48)
	at com.android.tools.r8.internal.dh0.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:2)
	at com.android.tools.r8.internal.dh0.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:17)
	at java.base/java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1448)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
	Suppressed: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.AssertionError
		at com.android.tools.r8.internal.to.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:132)
		at com.android.tools.r8.R8.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:1313)
		at com.android.tools.r8.R8.b(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:1)
		at com.android.tools.r8.internal.to.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:28)
		at com.android.tools.r8.R8.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:79)
		at com.android.tools.r8.R8.run(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:3)
		at com.android.tools.r8.utils.CompileDumpCompatR8.main(CompileDumpCompatR8.java:266)
	Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError
		at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006)
		at com.android.tools.r8.internal.eh0.awaitFutures(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:4)
		at com.android.tools.r8.internal.Qg0.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:2)
		at com.android.tools.r8.internal.Qg0.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:13)
		at com.android.tools.r8.internal.Qg0.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:14)
		at com.android.tools.r8.internal.dh0.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:15)
		at com.android.tools.r8.internal.dh0.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:5)
		at com.android.tools.r8.internal.dh0.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:1)
		at com.android.tools.r8.internal.jY.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:27)
		at com.android.tools.r8.internal.kY.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:50)
		at com.android.tools.r8.internal.kY.b(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:4)
		at com.android.tools.r8.R8.a(R8_8.5.33_ad460409bfce2fe329c672ceed1e015e505f16037727fb982221121c7cd2a1ee:380)
		... 5 more
	Caused by: java.lang.AssertionError
		at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
		at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
		at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
		at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
		at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:603)
		... 17 more
	[CIRCULAR REFERENCE:java.lang.AssertionError: During NARROWING, @NotNull java.lang.Object {com.google.android.gms.internal.ads.zzmb} < @NotNull java.lang.Object {} at Assume               v1206 <- v51; not null (context: void com.google.android.gms.internal.ads.zzkb.<init>(com.google.android.gms.internal.ads.zziv, com.google.android.gms.internal.ads.zzco))]

com.google.android.gms.internal.ads.zzmb is from https://maven.google.com/web/index.html#com.google.android.gms:play-services-ads:22.5.0

Would it be possible for you to share the dump you captured privately with christofferqa@google.com and sgjesse@google.com?

How did you get the error in comment #3? You must have enabled Java assertions to hit this. Was this from using ./tools/compiledump.py -d mydump.zip as mentioned in compiledump.md? If so, you may still be able to reproduce the StackOverflowError from the dump file using ./tools/compiledump.py -d mydump.zip --disable-assertions.

Re #4, would it be possible to get an NDA signed so I could share a compiler dump? Alternatively, I could help to find the culprit using some extra logging.

I added logging before the call to analyzeInstructionsInBlock on line 185. I can see that the following block gets processed more than 3.4k times until R8 crashes with StackOverflowError:

analyzeInstructionsInBlock currentDepth: 0, block: block 2, pred-counts: 1, succ-count: 1, filled: true, sealed: false
predecessors: 0 
successors: 3  (no try/catch successors)
no phis
 #0:hashCode;0:m: -1: ConstNumber          v9(1237) <-  1237 (INT)
                : -1: Goto                 block 3

Does this give any clues?

Since you've already cloned the repo and modified the source, it should be rather straight-forward to diagnose the issue. Did you checkout main or the 8.5 branch?

Does the method contain any sensitive code, or would it be possible for you to share the full IR of the method that is causing the issue (by calling code.toString())?

If you can't share the code in full, could you elide the instructions you can't share, but leave the block structure? Having the complete block structure might be enough to get an idea of what is happening.

One odd thing from looking at the stack trace in comment #1 and the fact that you are hitting the same block thousands of times in analyzeInstructionsInBlock is that this method has the following prelude, where there is logic in place to backtrack if the block has already been seen.

private SimpleInliningConstraintWithDepth analyzeInstructionsInBlock(
    BasicBlock block, int instructionDepth, InstructionIterator instructionIterator) {
  // If we reach a block that has already been seen, or one that has catch handlers, then give up.
  if (!seen.add(block) || block.hasCatchHandlers()) {
    return SimpleInliningConstraintWithDepth.getNever();
  }
  ...

Can you confirm whether, after hitting analyzeInstructionsInBlock with the block above in comment #5 for the second time, the method always returns at the third line return SimpleInliningConstraintWithDepth.getNever()?

I'm on the main branch of R8. Looks like R8 is running out of stack when analysing a long method which looks as follows:

fun mapDrawableId(id: Int): Int? =
    when (id) {
        R.drawable.my_drawable_1 -> R.drawable.my_new_drawable_1
        R.drawable.my_drawable_2 -> R.drawable.my_new_drawable_2
        R.drawable.my_drawable_3 -> R.drawable.my_new_drawable_3
        R.drawable.my_drawable_4 -> R.drawable.my_new_drawable_4
        R.drawable.my_drawable_5 -> R.drawable.my_new_drawable_5
        // 400+ branches
        else -> null
    }

The default stack size is 1MB on Linux. I doubled it to 2MB, and StackOverflowError no longer occurs. I wasn't able to reproduce the issue on a trivial program with a long when expression, it seems that the program has to be more complex.

Here's the code I tried to reproduce the issue with.

R8 rules:

-keep,allowobfuscation,allowoptimization class com.example.MainKt {
    *** main();
}

Thank you for your patience while our engineering team worked to resolve this issue. A fix for this issue is now available in:

• Android Studio Koala Feature Drop | 2024.1.2 Patch 1
• Android Gradle Plugin 8.6.1

We encourage you to try the latest update.

If you notice further issues or have questions, please file a new bug report.

Thank you for taking the time to submit feedback — we really appreciate it!