Status Update
Comments
mo...@google.com <mo...@google.com> #2
This would really help with performance testing of Compose
cc...@google.com <cc...@google.com>
ap...@google.com <ap...@google.com> #3
Branch: androidx-master-dev
commit a42f3e66b63a9303bdc7350302bbd83fc9eb77b0
Author: Chris Craik <ccraik@google.com>
Date: Fri Jun 19 14:25:22 2020
Enable profileable for all benchmarks, and benchmark tests
Test: ./gradlew benchmark:b-b:cC
Bug: 158303822
This is a prerequisite for simpleperf on user builds.
Change-Id: Idcb0024aad12d56edcf36427fa65235b3eab74ec
M ads/ads-identifier-benchmark/src/androidTest/AndroidManifest.xml
M appcompat/appcompat-benchmark/src/androidTest/AndroidManifest.xml
M benchmark/benchmark/src/androidTest/AndroidManifest.xml
M benchmark/integration-tests/startup-benchmark/src/androidTest/AndroidManifest.xml
M collection/collection-benchmark/src/androidTest/AndroidManifest.xml
M compose/compose-runtime/compose-runtime-benchmark/src/androidTest/AndroidManifest.xml
M navigation/benchmark/src/androidTest/AndroidManifest.xml
M recyclerview/recyclerview-benchmark/src/androidTest/AndroidManifest.xml
M room/benchmark/src/androidTest/AndroidManifest.xml
M slices/benchmark/src/androidTest/AndroidManifest.xml
M ui/integration-tests/benchmark/src/androidTest/AndroidManifest.xml
M work/workmanager-benchmark/src/androidTest/AndroidManifest.xml
ap...@google.com <ap...@google.com> #4
Branch: androidx-master-dev
commit dfae3689d8dd651c25cbd2bd42393fc402503cb9
Author: Chris Craik <ccraik@google.com>
Date: Wed Jun 17 11:32:08 2020
Create Profiler abstraction
Bug: 158303822
Test: ProfilerTest
Test: ./gradlew benchmark:b-c:cC benchmark:b-j:cC
Allows us to factor significant functionality out of BenchmarkState,
and will let us support additional profiling modes (simpleperf!).
Additionally, renames all tracing params for clarity, while
maintaining backwards compatibility.
Change-Id: Id5b3fb034638077a97c9d30c7f9f12f0ef31102c
M benchmark/common/api/1.1.0-alpha02.txt
M benchmark/common/api/current.txt
M benchmark/common/api/public_plus_experimental_1.1.0-alpha02.txt
M benchmark/common/api/public_plus_experimental_current.txt
M benchmark/common/api/restricted_1.1.0-alpha02.txt
M benchmark/common/api/restricted_current.txt
M benchmark/common/src/androidTest/java/androidx/benchmark/ArgumentInjectingApplication.kt
A benchmark/common/src/androidTest/java/androidx/benchmark/ProfilerTest.kt
D benchmark/common/src/androidTest/java/androidx/benchmark/ProfilingModeTest.kt
M benchmark/common/src/main/java/androidx/benchmark/Arguments.kt
M benchmark/common/src/main/java/androidx/benchmark/BenchmarkState.kt
M benchmark/common/src/main/java/androidx/benchmark/Errors.kt
A benchmark/common/src/main/java/androidx/benchmark/Profiler.kt
D benchmark/common/src/main/java/androidx/benchmark/ProfilingMode.kt
ap...@google.com <ap...@google.com> #5
Branch: androidx-master-dev
commit 42d424831e41ad853d8d4ee3e0c7e83302a181f6
Author: Chris Craik <ccraik@google.com>
Date: Wed Jun 17 11:32:08 2020
Simpleperf method sampling - initial support
Bug: 158303822
Test: ./gradlew benchmark:b-c:cC benchmark:b-j:cC
Test: ProfilerTest
Remaining simpleperf work to do:
- Additionally spit out .trace file for opening in Studio
- Use additionalOutputDir
- Enable this mode by default on supported devices (when methodSampling is selected)
- Avoid need for python prepare script (can just call ADB directly)
Additionally adds args for sample freq, and sampling duration.
The 'simpleperf' package is copied from the simpleperf app_api sample:
Change-Id: I5ccef4b434710ae6561c63c5ae1cd5d87ed77b9f
M benchmark/common/src/androidTest/java/androidx/benchmark/ProfilerTest.kt
M benchmark/common/src/main/java/androidx/benchmark/Arguments.kt
M benchmark/common/src/main/java/androidx/benchmark/BenchmarkState.kt
M benchmark/common/src/main/java/androidx/benchmark/Errors.kt
M benchmark/common/src/main/java/androidx/benchmark/Profiler.kt
A benchmark/common/src/main/java/androidx/benchmark/simpleperf/ProfileSession.java
A benchmark/common/src/main/java/androidx/benchmark/simpleperf/RecordOptions.java
au...@google.com <au...@google.com> #6
Note that androidx.benchmark.ProfilerTest#methodSamplingSimpleperf has been 100% failing in CI
java.lang.Error: linux perf events aren't enabled on the device. Please run api_profiler.py.
at androidx.benchmark.simpleperf.ProfileSession.checkIfPerfEnabled(ProfileSession.java:296)
at androidx.benchmark.simpleperf.ProfileSession.startRecording(ProfileSession.java:142)
at androidx.benchmark.simpleperf.ProfileSession.startRecording(ProfileSession.java:125)
at androidx.benchmark.MethodSamplingSimpleperf.start(Profiler.kt:174)
at androidx.benchmark.ProfilerTest.verifyProfiler(ProfilerTest.kt:56)
at androidx.benchmark.ProfilerTest.methodSamplingSimpleperf(ProfilerTest.kt:80)
at java.lang.reflect.Method.invoke(Native Method)
cc...@google.com <cc...@google.com> #7
With the patch in #5, simpleperf is supported, but it's cumbersome, and doesn't support outputting a file that Studio can read. A few things remain to make this easier to use, so we can ship externally:
1) Create Studio .trace file
Simpleperf docs explain how to convert the output <name>.data file output by simpeperf on device to a .trace file Studio can open. See
Currently, the trace file is output to /data/data/<package>/simpleperf_data/***.data
. We can leave that file there for now, so that it can be copied by simpleperf/scripts/api_profiler.py collect
, but we should put a copy of it in additionalTestOutputDir/
.
2) Replicate api_profiler.py prepare functionality
See
3) Switch MethodTracing to use simpleperf by default on supported devices
Once the above 2 are resolved, rename MethodTracing -> MethodTracingLegacy, and MethodTracingSimpleperf -> MethodTracing.
Use simpleperf by default on API 28+, but let people opt into the old version by passing "MethodTracingLegacy".
cc...@google.com <cc...@google.com> #8
@Aurimas - that failure is because this functionality requires running a host-side script before it will work, but fixing 2) above will fix that.
I marked the test @FlakyTest, but if there's a better way to make it not run at all in pre/postsubmit, happy to make that change.
ap...@google.com <ap...@google.com> #9
Branch: androidx-master-dev
commit 04e4b033916342deb8dc343f5dacc41f12597d0d
Author: Louis Pullen-Freilich <lpf@google.com>
Date: Thu Aug 20 00:39:26 2020
Changes ProfilerTest#methodSamplingSimpleperf to be @Ignore instead of @FlakyTest
Since this test fails consistently, there's no value to running it in postsubmit, as it just pollutes the testing dashboard.
Bug: 158303822
Test: ProfilerTest
Change-Id: I0c9f1ffd57d0058816b838e75502807f66fef7ed
M benchmark/common/src/androidTest/java/androidx/benchmark/ProfilerTest.kt
cc...@google.com <cc...@google.com>
ap...@google.com <ap...@google.com> #10
Branch: androidx-main
commit c28da961a6bffa438a957c07cc6cccb347765fb4
Author: Chris Craik <ccraik@google.com>
Date: Thu Aug 05 14:55:18 2021
Remove need for setup/post-process scripts for simpleperf, and make it default
Bug: 158303822
Test: ./gradlew benchmark:benchmark-common:cC benchmark:benchmark-macro:cC
Test: ./gradlew compose:material:material-benchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.profiling.mode=StackSampling -P android.testInstrumentationRunnerArguments.class=androidx.compose.material.benchmark.CheckboxesInRowsBenchmark#draw
Relnote: Switched to Simpleperf as default sampling profiler on API 29+
Simpleperf is now the 'default' stack sampling profiler, as it doesn't
need the simpleperf setup scripts to be used on API 29+.
This CL also renames the profilers for clarity going forward.
Lowered default sampling rate to reduce liklihood of interference and
overwhelming the measure thread.
Unified logic for temporarily overriding a setprop with new
PropOverride class.
Change-Id: Ic4b346221bb91dea1e7078e369f106f9e48be1e3
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/ProfilerTest.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/Arguments.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/Errors.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/Profiler.kt
A benchmark/benchmark-common/src/main/java/androidx/benchmark/PropOverride.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/Shell.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/simpleperf/ProfileSession.java
M benchmark/benchmark-common/src/main/java/androidx/benchmark/simpleperf/RecordOptions.java
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoCaptureWrapper.kt
cc...@google.com <cc...@google.com> #11
Landed documentation updates with cl/406002135. Marking this as fixed, and filed
Description
Because the sampling profiler has trouble giving accurate results, the recommendation is to use simpleperf. Documentation on how to set it up would be helpful.
Making it automatic from gradle would be even more helpful, if it is possible.