Fixed
Status Update
Comments
ap...@google.com <ap...@google.com> #2
Project: platform/frameworks/support
Branch: androidx-main
Author: Chris Craik <
Link:
Reimplement BenchmarkRule / BenchmarkState on top of coroutines
Expand for full commit details
Reimplement BenchmarkRule / BenchmarkState on top of coroutines
Test: benchmark-common, benchmark-junit4
Bug: 389149423
Fixes: 311242861
This refactor achieves several important goals:
- Move from the extremely complex stateful nature of BenchmarkState to layered suspending functions (the old state deferral system was very obtuse and brittle)
- Simplify the existing measureRepeatedOnMainThread yield behavior to use actual yield()
- Rewrite the thermal throttling sleep to yield() instead of sleep() (which works around ANRs when using measureRepeatedOnMainThread)
- Provide a clear config -> execution -> output control flow, including a clear top level measureRepeated function (which declares all its inputs, and almost wires up the outputs)
- Move as much logic as possible from the JUnit4 layer to benchmark-common
These will indirectly make it more possible to:
- Run trace post-processing within microbenchmark (e.g. interference detection and retry)
- More cleanly add configuration in MicrobenchmarkConfig (similar to what warmupCount / repeatCount BenchmarkState constructor params were previously trying to offer)
- Move benchmark-common to KMP if desired
- Write suspending variants of the benchmark APIs
The downsides are:
- Benchmarks with the `BenchmarkRule.getState().keepRunning()` pattern will no longer capture Perfetto traces
- In a follow up, a replacement API will be offered for Java that more closely matches kotlin
- BenchmarkState is implemented on top of a complex and custom coroutine logic mimicking a suspending sequence
- Ideally this implementation will be deprecated and not have to change much in the future
- Several experimental BenchmarkState APIs were removed, to make way for the new way of solving these problems (configuring benchmarks)
- The most useful of these moved to MicrobenchmarkConfig
The old BenchmarkRule/BenchmarkState are temporarily moved to internal
BenchmarkRuleLegacy/BenchmarkStateLegacy to enable easy internal
comparisons in the short term. Eventually these will be removed.
Relnote: "Refactored BenchmarkRule to be built on top of coroutines, and support better yield() behavior. This refactor removed several experimental BenchmarkState APIs, but will be followed by replacements as needed. Additionally, added runWithMeasurementDisabled to clarify behavior (all measurement is paused). In the future, runWithTimingDisabled will be deprecated."
Change-Id: I19837fc604b9e308957fc01ed009c0e921b8fe28
Files:
- M
benchmark/benchmark-common/api/current.txt
- M
benchmark/benchmark-common/api/restricted_current.txt
- M
benchmark/benchmark-common/build.gradle
- M
benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/BenchmarkStateLegacyConfigTest.kt
- M
benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/BenchmarkStateLegacyTest.kt
- M
benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/MicrobenchmarkPhaseConfigTest.kt
- M
benchmark/benchmark-common/src/main/java/androidx/benchmark/Arguments.kt
- M
benchmark/benchmark-common/src/main/java/androidx/benchmark/BenchmarkState.kt
- A
benchmark/benchmark-common/src/main/java/androidx/benchmark/BenchmarkStateLegacy.kt
- A
benchmark/benchmark-common/src/main/java/androidx/benchmark/Microbenchmark.kt
- M
benchmark/benchmark-common/src/main/java/androidx/benchmark/MicrobenchmarkConfig.kt
- A
benchmark/benchmark-common/src/main/java/androidx/benchmark/MicrobenchmarkOutput.kt
- M
benchmark/benchmark-common/src/main/java/androidx/benchmark/MicrobenchmarkPhase.kt
- M
benchmark/benchmark-common/src/main/java/androidx/benchmark/Profiler.kt
- A
benchmark/benchmark-common/src/main/java/androidx/benchmark/TestDefinition.kt
- M
benchmark/benchmark-common/src/main/java/androidx/benchmark/ThrottleDetector.kt
- M
benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoCaptureWrapper.kt
- M
benchmark/benchmark-junit4/api/current.txt
- M
benchmark/benchmark-junit4/api/restricted_current.txt
- M
benchmark/benchmark-junit4/src/androidTest/java/androidx/benchmark/junit4/BenchmarkRuleAnnotationTest.kt
- A
benchmark/benchmark-junit4/src/androidTest/java/androidx/benchmark/junit4/BenchmarkRuleLegacyTest.kt
- M
benchmark/benchmark-junit4/src/androidTest/java/androidx/benchmark/junit4/BenchmarkRuleTest.kt
- M
benchmark/benchmark-junit4/src/main/java/androidx/benchmark/junit4/BenchmarkRule.kt
- A
benchmark/benchmark-junit4/src/main/java/androidx/benchmark/junit4/BenchmarkRuleLegacy.kt
- M
benchmark/benchmark/src/androidTest/java/androidx/benchmark/benchmark/BenchmarkStateBenchmark.kt
- A
benchmark/benchmark/src/androidTest/java/androidx/benchmark/benchmark/BenchmarkStateLegacyBenchmark.kt
- A
benchmark/benchmark/src/androidTest/java/androidx/benchmark/benchmark/MeasureRepeatedSampleBenchmark.kt
- A
benchmark/benchmark/src/androidTest/java/androidx/benchmark/benchmark/TrivialKotlinBenchmarkLegacy.kt
- M
compose/runtime/runtime/compose-runtime-benchmark/src/androidTest/java/androidx/compose/runtime/benchmark/ComposeBenchmarkBase.kt
Hash: 329bd0d25247f1eb6c683b921b8cd06f60bf3ad2
Date: Fri Jul 12 15:09:22 2024
na...@google.com <na...@google.com> #3
The following release(s) address this bug.It is possible this bug has only been partially addressed:
androidx.benchmark:benchmark-common:1.4.0-alpha07
androidx.benchmark:benchmark-junit4:1.4.0-alpha07
Description
See b/300472956 , https://android-review.git.corp.google.com/c/platform/frameworks/support/+/2783737
BenchmarkRule.measureRepeatedOnMainThread() has several remaining issues which were punted past the initial implementation to unblock fixing ANRs: