Minifying Renderscript code: huge performance drop when upgrading AGP from 7.3.1 to 7.4.0 [267990059]

Verified

Bug

Status Update

No update yet.

Description

bv...@gmail.com

created issue #1

Feb 6, 2023 09:26AM

We're using Renderscript in one of our projects and noticed that our build times for a minified release build went up by a factor of 3 after upgrading the Android Gradle Plugin from 7.3.1 to 7.4.0: from 17 minutes to 53 minutes.

For each Renderscript file, two files are generated: a ScriptC_$name file which contains bridging functions, and a $nameBitCode file which contains a few huge byte arrays that are compiled into GPU code on the user's phone. R8 takes a very long time to process this last file.

I made a sample project here: https://github.com/bvschaik/r8-renderscript-minify-issue
This project contains a copy of the BitCode class that's generated by Renderscript here.

Using AGP 7.4.0 (uses R8 4.0.48), running assembleRelease on this project takes almost 4 minutes, with 3m40s taken up by the minifyReleaseWithR8 task.

Downgrading AGP to 7.3.1 (uses R8 3.3.83), the minifyReleaseWithR8 "only" takes 1m10s, which is still quite long for processing some static byte arrays.

For now we're forcing R8 to the last available 3.3 version to get back to acceptable build times.

What could cause this increase in build times?
Why is R8 spending so much time on relatively simple code?
What can we do to tell R8 "don't change anything in this file"? I tried various -keep rules but it didn't make a difference.

Comments

ch...@google.com <ch...@google.com> Feb 6, 2023 09:35AM

Reassigned to ch...@google.com.

ch...@google.com <ch...@google.com> #2Feb 7, 2023 07:57AM

[Comment deleted]

ap...@google.com <ap...@google.com> #3Feb 8, 2023 11:31AM

[Comment deleted]

ap...@google.com <ap...@google.com> #4Feb 8, 2023 02:53PM

Also of note is the adb error when trying to install bad APK: INSTALL_FAILED_DEXOPT

ap...@google.com <ap...@google.com> #5Feb 8, 2023 03:16PM

load dex files over 5Gb. -> load dex files over 5Mb.

ap...@google.com <ap...@google.com> #6Feb 9, 2023 10:06AM

Same here! Looking forward to a solution :)

Android Studio version: 0.8.12
buildToolsVersion 21.0.1
Gradle 1.11

ap...@google.com <ap...@google.com> #7Feb 9, 2023 10:27AM

There is already an option in dx allowing to force generation of smaller dex files:
--set-max-idx-number=<value>
Unfortunately changing the default is not a solution since the linearAlloc limit can be reached at very different levels depending on the classes hierarchy and other criteria.

In addition for most applications, moving to multidex will only help to workaround the linearalloc limit for the installation. But the application will still crash against the same limit at execution. The only working use case where I know multidex can help with linearalloc is when the apk does not contains one application but distinct pieces running in separate process.

ap...@google.com <ap...@google.com> #8Feb 9, 2023 10:27AM

Thanks for your quick response.

It's nice to know about that command line option. I do not see it in the output of 'dx --help', might be good to add that.

I'm not very familiar with the 'linearAlloc limit' issue outside of the context of the dexopt step. My sample app is able to run once the lower idx value is set, although I do not actually call into any of the library code that is bundled with the app. I assume it's undefined when/if the 'linearAlloc limit' will be hit in a large application on gb.

I'm a bit confused as to the platform compatibility of multidex given the 'linearAlloc limit' bug. What specific versions of Android are supported? The multidex code implies back to v4 (

https://android.googlesource.com/platform/frameworks/multidex/+/master/library/src/android/support/multidex/MultiDex.java) but it would seem that ICS is the earliest supported platform. Is this correct?

ap...@google.com <ap...@google.com> #9Feb 9, 2023 10:27AM

The option is not documented in --help because it was designed for testing and we're not capable of documenting a reliable way to use it as a workaround of the linearalloc limit.

The linearalloc limit is reached when loading classes. At install time dexopt is loading all classes contained in the dex so it's facing the limit immediately. At execution the limit may be reached after some delay dependending of the usage you have of the packaged classes. If you face it at install time but not at execution, this means you never trigger the loading of some classes. In a real application those never loaded classes should have been shrinked away manually or by Proguard. The exception is when there are different groups of classes in the dex files used in separate process.

About multidex library supported versions I've merged recently a change to try to be clearer

https://android-review.googlesource.com/#/c/108023/
The summary is that the library should work down to API 4 (Donut), but below ICS applications will probably be hit by the linearalloc limit

ap...@google.com <ap...@google.com> #10Feb 9, 2023 10:27AM

for Android studio use:
dexOptions {
additionalParameters = ['--multi-dex', '--set-max-idx-number=40000']
}

ap...@google.com <ap...@google.com> #11Feb 9, 2023 10:27AM

I still have this issue and it's driving me nuts

ap...@google.com <ap...@google.com> #12Feb 9, 2023 10:30AM

smgshantokhan55698@gmail.com

ap...@google.com <ap...@google.com> #13Feb 9, 2023 10:30AM

https://twitter.com/Shanto59597264?t=AcggXWk2PPhi4bU9UC3YHA&s=09 @shantokhan59597264

ap...@google.com <ap...@google.com> #14Feb 9, 2023 10:30AM

Project: r8
Branch: 8.0

commit 33930d7b7ff37a746a304bb0230b34123b1e0e6f
Author: Christoffer Quist Adamsen <christofferqa@google.com>
Date: Thu Feb 09 11:22:50 2023

Fix repeated analysis of values in dead code removal

Bug:

b/267990059

Change-Id: I9deebf700e936ad81a8fb25d067acc04637d980d

M src/main/java/com/android/tools/r8/ir/code/ArrayPut.java
M src/main/java/com/android/tools/r8/ir/code/ValueIsDeadAnalysis.java
M src/main/java/com/android/tools/r8/ir/optimize/DeadCodeRemover.java
M src/main/java/com/android/tools/r8/utils/MapUtils.java

https://r8-review.googlesource.com/75772

ap...@google.com <ap...@google.com> #15Feb 9, 2023 10:30AM

Project: r8
Branch: 8.0

commit 923cf259db1b2f35578a486960afcd28369c520f
Author: Christoffer Quist Adamsen <christofferqa@google.com>
Date: Thu Feb 09 11:22:16 2023

Rewrite value-is-dead analysis to a worklist algorithm

Bug:

b/267990059

Change-Id: I699f43eba8774083a4b5f2aa2edf7c5d80df4c8e

M src/main/java/com/android/tools/r8/ir/code/ValueIsDeadAnalysis.java

https://r8-review.googlesource.com/75771

ap...@google.com <ap...@google.com> #16Feb 9, 2023 10:30AM

Project: r8
Branch: 8.0

commit 736dd75501862e23764d265887a99ef8d8af0c9d
Author: Christoffer Quist Adamsen <christofferqa@google.com>
Date: Thu Feb 09 11:21:02 2023

Refactor value-is-dead analysis to separate analysis class

Bug:

b/267990059

Change-Id: I8b881203ebe21395f48c8eeb6434faa03f7f9d46

M src/main/java/com/android/tools/r8/ir/code/BasicBlock.java
M src/main/java/com/android/tools/r8/ir/code/Value.java
A src/main/java/com/android/tools/r8/ir/code/ValueIsDeadAnalysis.java
M src/main/java/com/android/tools/r8/ir/optimize/DeadCodeRemover.java

https://r8-review.googlesource.com/75770

ch...@google.com <ch...@google.com> #17Feb 9, 2023 10:34AM

Verified by ch...@google.com.

This should now be fixed in R8 versions 4.0.53 and 8.0.29 - and next week in 8.1.19-dev.

On my machine the build time for compiling the reproduction with R8 is down from ~190s to ~3s.

bv...@gmail.com <bv...@gmail.com> #18Feb 9, 2023 12:15PM

I just tested 4.0.53: on the sample project it also runs in ~3s in my machine. The larger project now finishes within a few minutes, and that previously took almost an hour, wow. Thanks!

Any idea when this fix will become available in the AGP? Next release of Android Studio, I assume?

ch...@google.com <ch...@google.com> #19Feb 9, 2023 02:24PM

Thanks for the confirmation. This will hopefully make it into Flamingo (AGP 8.0).

Issue 267990059

Description

Issue summary

Comments

ch...@google.com <ch...@google.com> Feb 6, 2023 09:35AM

ch...@google.com <ch...@google.com> #2Feb 7, 2023 07:57AM

ap...@google.com <ap...@google.com> #3Feb 8, 2023 11:31AM

ap...@google.com <ap...@google.com> #4Feb 8, 2023 02:53PM

ap...@google.com <ap...@google.com> #5Feb 8, 2023 03:16PM

ap...@google.com <ap...@google.com> #6Feb 9, 2023 10:06AM

ap...@google.com <ap...@google.com> #7Feb 9, 2023 10:27AM

ap...@google.com <ap...@google.com> #8Feb 9, 2023 10:27AM

ap...@google.com <ap...@google.com> #9Feb 9, 2023 10:27AM

ap...@google.com <ap...@google.com> #10Feb 9, 2023 10:27AM

ap...@google.com <ap...@google.com> #11Feb 9, 2023 10:27AM

ap...@google.com <ap...@google.com> #12Feb 9, 2023 10:30AM

ap...@google.com <ap...@google.com> #13Feb 9, 2023 10:30AM

ap...@google.com <ap...@google.com> #14Feb 9, 2023 10:30AM

ap...@google.com <ap...@google.com> #15Feb 9, 2023 10:30AM

ap...@google.com <ap...@google.com> #16Feb 9, 2023 10:30AM

ch...@google.com <ch...@google.com> #17Feb 9, 2023 10:34AM

bv...@gmail.com <bv...@gmail.com> #18Feb 9, 2023 12:15PM

ch...@google.com <ch...@google.com> #19Feb 9, 2023 02:24PM

Add comment

Issue metadata