Skip full compilation if the app is already full aot compiled [230453509]

Fixed

Bug

Status Update

No update yet.

Description

py...@squareup.com

created issue #1

Apr 26, 2022 07:34PM

We're running several macrobenchmark tests with full AOT compilation. Our app is large so full AOT compilation takes a long time. It would be nice if the macrobenchmark code that triggers full AOT compilation had a check for "is this app already full AOT compiled? If yes, nothing to do"

This would be harder to do with partial compilation unless you were able to cache the previous profile inputs and compare it against the new ones.

Comments

cc...@google.com <cc...@google.com> #2May 11, 2022 06:33PM

This was partially addressed by

https://android-review.googlesource.com/c/platform/frameworks/support/+/2076902, available now in 1.1 RC02, see

b/230665435

. As compilation on user builds requires a full target reinstall (which is a behavior change), we offer an opt out, which can be used to accomplish this feature request: 1) configure every macrobench to use `CompilationMode.Full()` 1) manually issuing the compile command `cmd package compile -f -m speed <packagename>` for your target 1) pass the instrumentation arg `androidx.benchmark.compilation.enable` = `false` to skip compilation/reinstall for each macrobenchmark. This should still give you the numbers you've been seeing, while avoiding the cost of a large AOT each test. Leaving this bug open, since in general we should be able to do this more automatically for everything without warmup driven profiles. (Somewhat related bug - there have been excess compilations issued specifically for `Compilation.None`, `StartupMode.COLD` benchmarks, which has been fixed, but not shipped publicly yet:

b/231976084

)

py...@squareup.com <py...@squareup.com> #3May 11, 2022 09:41PM

Note: ideally there's be a way to turn off all install related behavior without having to pass in an arg flag. If I want to handle installation & optimization without benchmark getting in the way, there should be an API for that. Arguably that's what Compilation.None should do, or there should be a Compilation.Interpreted and a Compilation.NoOp.

py...@squareup.com <py...@squareup.com> #4May 11, 2022 09:43PM

Also, generally clearing profiles doesn't make much sense for Full compilation mode. I understand this makes sense for profile based benchmarks, but not really for full where the output shouldn't vary.

cc...@google.com <cc...@google.com> #5May 11, 2022 09:51PM

> ideally there's be a way to turn off all install related behavior without having to pass in an arg flag. We can't change APIs in an RC, but we'll consider it for 1.2 > Arguably that's what Compilation.None should do, or there should be a Compilation.Interpreted and a Compilation.NoOp. `None` means "none of my app should be compiled", so at least a reset (if compilation has happened before) should occur. `Compilation.NoOp` or equivalent sounds reasonable. > Also, generally clearing profiles doesn't make much sense for Full compilation mode. I assume so too, I would want to validate that with ART and test it, but either way, we opted for full reinstallation for every test in 1.1 since it's predictable and consistent.

ma...@google.com <ma...@google.com> Jul 11, 2022 04:24PM

Reassigned to ma...@google.com.

ap...@google.com <ap...@google.com> #6Jul 14, 2022 03:58AM

Project: platform/frameworks/support
Branch: androidx-main

commit 78a3847fafbb5a810b60b27b1361400bf8dcfa58
Author: Marcello Albano <maralb@google.com>
Date: Mon Jul 11 14:07:07 2022

Added a new compilation mode `CompilationMode.Ignore` to skip profile reset and compilation.

Bug: 230453509
Test: ./gradlew :benchmark:integration-tests:macrobenchmark:cC -Pandroid.testInstrumentationRunnerArguments.class=androidx.benchmark.integration.macrobenchmark.CompilationModeTest

This change adds a new compilation mode `CompilationMode.Ignore` that will skip profile reset and compilation. This can be used to customize the benchmark loop.

Change-Id: Ibbcf8e8c627d227e0f290bddf464b67d9b6653bc
Relnote: Added a new compilation mode `CompilationMode.Ignore` to skip profile reset and compilation.

A benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/ExperimentalMacrobenchmarkApi.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/CompilationMode.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/MacrobenchmarkTest.kt
A benchmark/integration-tests/macrobenchmark/src/androidTest/java/androidx/benchmark/integration/macrobenchmark/CompilationModeTest.kt
M benchmark/benchmark-macro/api/public_plus_experimental_current.txt

https://android-review.googlesource.com/2150519

ma...@google.com <ma...@google.com> Jul 14, 2022 04:50PM

Marked as fixed.