Status Update
No update yet.
Comments
ap...@google.com
Branch: androidx-main
commit bfdd111df98ae51af5266573ad915bfc006bb7bf
Author: Rahul Ravikumar <rahulrav@google.com>
Date: Tue Jun 06 15:17:45 2023
Cleanup method tracing implementation in Macrobenchmark.
* No longer requires the use of `:benchmark:benchmark-internal`.
* Relies on `androidx.benchmark.profiling.mode` set to `MethodTracing`.
* Correctly handles process cold starts, given we use `am start --start-profiler <FILE> -W <INTENT>`.
Relnote: "
Provides the ability to generate method traces as a result of running a Macrobenchmark
using the following test instrumentation arguments.
```groovy
android {
defaultConfig {
testInstrumentationRunnerArguments [
'androidx.benchmark.profiling.mode': 'MethodTracing',
// Other arguments
]
}
}
```
"
Test: Ran TrivialStartupBenchmark + added unit tests.
Bug:
Change-Id: I7ad37bee17646cfef1564c9f0c84d20923d63ed9
M benchmark/benchmark-common/src/main/java/androidx/benchmark/Arguments.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/Shell.kt
D benchmark/benchmark-internal/build.gradle
D benchmark/benchmark-internal/src/main/AndroidManifest.xml
D benchmark/benchmark-internal/src/main/java/androidx/benchmark/internal/MethodTracingReceiver.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/MacrobenchmarkScopeTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/StartupTimingMetricTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoSdkHandshakeTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/BaselineProfiles.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/MacrobenchmarkScope.kt
M benchmark/integration-tests/macrobenchmark-target/build.gradle
M settings.gradle
ap...@google.com
Remaining things to do here, copied from comment on CL:
- disable metric collection and reporting during method tracing - is this part done already, via the warning prefix?
- link the trace in the output (we may need a good general improvement to Outputs/Instrumentation stuff to make this easier, maybe we need to generalize ProfilerResult somehow, I can get to this eventually)
- run method tracing as a single iteration, in place of measurement (I think it's better to replace for macro, since runs are so long)
How big are the method traces for a startup vs scrolling macrobench, out of curiousity?
ap...@google.com
Method traces are < 10 M for activity startup cases (~10s CUJ).
disable metric collection and reporting during method tracing - is this part done already, via the warning prefix?
We don't disable collection, but the results include a warning. We should add a prefix to our metrics like we do for microbenchmark.
Run method tracing as a single iteration, in place of measurement (I think it's better to replace for macro, since runs are so long)
Rather than do this, I wanted to introduce a phase where all we do is just method tracing after metrics are captured. This way, we don't affect metrics at all; but include a usable method trace in addition to the metrics.
cc...@google.com
pr...@google.com
Method traces are < 10 M for activity startup cases (~10s CUJ).
I expect startup cases we have are pretty trivial. What about scrolling cases?
introduce a phase
I don't know if I like that as the default, for method tracing or for sampling in macrobench, since tests are usually so much longer than micro. Consider a 5 min scrolling test, where you're just not collecting profiling results for minutes on end while the developer waits.
Could always eventually make it an option like androidx.benchmark.profiling.primaryMeasurePhase.enable
Description
Two stacked features requests here: