NullPointerException in R8 Following Control Flow Obfuscation [132897042]

Fixed

Bug

NeedsInfo

Status Update

No update yet.

Description

pe...@gmail.com

created issue #1

May 16, 2019 09:02PM

I am encountering a NullPointerException within R8 when trying to build a libgdx application, after running it through DashO's block splitting transformation (i.e., a part of its control flow obfuscation):

Current version is: 1.4.94 (build 390954928f0db9c3b888a367f7f128ce3bbfb160 from go/r8bot (luci-r8-ci-archive-0-5g74)).

...
Caused by: java.lang.NullPointerException
at

com.android.tools.r8.ir.analysis.constant.ConstLatticeElement.meet(ConstLatticeElement.java:23)
at com.android.tools.r8.ir.analysis.constant.SparseConditionalConstantPropagation.visitPhi(SparseConditionalConstantPropagation.java:137)
at com.android.tools.r8.ir.analysis.constant.SparseConditionalConstantPropagation.run(SparseConditionalConstantPropagation.java:55)
at com.android.tools.r8.ir.conversion.IRConverter.rewriteCode(IRConverter.java:961)
at com.android.tools.r8.ir.conversion.IRConverter.processMethod(IRConverter.java:805)
at com.android.tools.r8.ir.conversion.IRConverter.lambda$optimize$5(IRConverter.java:551)
at com.android.tools.r8.ir.conversion.CallGraph.lambda$forEachMethod$6(CallGraph.java:468)

This occurs only when optimization is applied. It appears to be encountering this specifically if block splitting is applied to com.badlogic.gdx.backends.android.AndroidInputFactory.newAndroidInput.

Since the Java bytecode that causes this is produced by a proprietary transform that is nontrivial to set up, I've attached a zip file containing two jars and some rules. You can reproduce the issue by invoking R8 in the following manner:

java -jar [path to r8 jar] --pg-conf dontshrink.txt --pg-conf proguard-android-optimize.txt-3.4.0 --lib 3.jar --lib [path to android.jar] 0.jar

I most recently used a local build of r8 using the master branch, commit 559e5b3b1e112e12e31e7f46f48d2a3d004176a9.

r8-nullpo.zip

3.2 MB

Download

Comments

js...@google.com <js...@google.com> May 16, 2019 09:15PM

Assigned to js...@google.com.

js...@google.com <js...@google.com> #2May 16, 2019 09:58PM

Are there any updates here?

js...@google.com <js...@google.com> #3May 16, 2019 10:23PM

I looked at it and I'm planning to remove the deduplicating lookup the callstack refers to. It should be no longer needed. How was the 80ms in comment#1 calculated? If it's straightforward I'll calculate it again before my change goes in.

r8.jar

26 MB

Download

js...@google.com <js...@google.com> #4May 17, 2019 04:55PM

Marked as fixed.

I couldn't move the logging off thread but I was able to address an area of concern in the callstack:
com.google.protobuf.Descriptors$FileDescriptor.internalBuildGeneratedFileFrom(Descriptors.java)
com.android.build.gradle.internal.cxx.logging.CxxLogging.<clinit>(CxxLogging.java:82)
com.android.build.gradle.internal.cxx.logging.LoggingMessage.getDescriptor(LoggingMessage.java:110)
com.android.build.gradle.internal.cxx.logging.LoggingMessage.hashCode(LoggingMessage.java:489)
java.util.HashMap.hash(HashMap.java:340)
java.util.HashMap.containsKey(HashMap.java:592)
java.util.HashSet.contains(HashSet.java:204)

This logic was deduplicating messages (including chatty 'info' messages). This deduplication shouldn't be needed anymore so I removed it.

I don't know how the 80ms from

comment#1

was gathered so I can't say how much it has improved logging, but it should help quite a bit. Let me know if you can still measure a performance issue. The next step would be to reduce the 'info' that's logged.

ze...@google.com <ze...@google.com> #5Jul 10, 2019 03:22PM

Status: Assigned (reopened)

The change mentioned in

comment#4

(internal Id I4e51a10d9ee2a81ca56c8e81c7890dfb0bf239d7) should be available in Canary 8 of Flamingo.

ze...@google.com <ze...@google.com> #6Jul 10, 2019 04:11PM

Marked as fixed.

Closing again with cherry pick in progress here:

https://r8-review.googlesource.com/c/r8/+/40470

ap...@google.com <ap...@google.com> #7Jul 10, 2019 08:07PM

Project: r8
Branch: d8-1.5

commit ee9b886184c753ca53c1cbb0766871272e5541ba
Author: Ian Zerny <zerny@google.com>
Date: Wed Jul 10 17:45:23 2019

Version 1.5.60

Cherry pick: Check other lattice element type before casting.

CL:

https://r8-review.googlesource.com/c/r8/+/38520
Bug: 132897042
Change-Id: If7b87ac08f33a64e7de556e22c38fc560779fce4

M src/main/java/com/android/tools/r8/Version.java
M src/main/java/com/android/tools/r8/ir/analysis/constant/ConstLatticeElement.java
A src/test/java/com/android/tools/r8/ir/analysis/constant/B132897042.java

https://r8-review.googlesource.com/40470