Status Update
Comments
cc...@google.com <cc...@google.com> #2
See also
ap...@google.com <ap...@google.com> #3
Branch: androidx-main
commit 1a48f33d2d01d135a0d647f83b1baaac10499ef0
Author: Chris Craik <ccraik@google.com>
Date: Tue Jan 25 12:35:23 2022
Clarify FrameTimingMetric names
Fixes: 216337830
Relnote: "Renamed FrameCpuTime -> FrameDurationCpu, FrameUiTime -> FrameDurationUi to clarify these are durations, not timestamps, and to match prefixes."
Test: ./gradlew benchmark:benchmark-common:cC
Test: TrivialListScrollBenchmark
Change-Id: I0eba35542431905ab926f5dc7db4ab6e292fde69
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Metric.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/FrameTimingQuery.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/FrameTimingQueryTest.kt
ap...@google.com <ap...@google.com> #4
Branch: androidx-main
commit 723cb6a05a830fd8215773f43b00861179e53a2c
Author: Chris Craik <ccraik@google.com>
Date: Fri Jan 26 10:59:21 2024
Add back GfxInfo frame timing metric as experimental
Fixes: 250725959
Bug: 322232828
Test: FrameExperimentBenchmark
Relnote: "Added FrameTimingGfxInfoMetric, an experimental alternate
implementation of FrameTimingMetric with measurements coming directly
from the platform, rather than extracted from the Perfetto trace."
Modified from previous version:
- Actually stop with the stop command, to more closely match standard
metric behavior
- Prefix each output with 'gfx' to clarify different metric sources
Change-Id: I457cbf351ee86141130d1667b6f352cd2ade453b
M benchmark/benchmark-macro/api/current.txt
M benchmark/benchmark-macro/api/restricted_current.txt
A benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/JankCollectionHelper.java
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Metric.kt
M compose/integration-tests/macrobenchmark/src/main/java/androidx/compose/integration/macrobenchmark/FrameExperimentBenchmark.kt
ap...@google.com <ap...@google.com> #5
Branch: androidx-main
commit 74638b3f1fd47614a3655a7a911914c0d33d7ea7
Author: Chris Craik <ccraik@google.com>
Date: Mon Jan 29 15:26:05 2024
Create alternate frame timing metric, and fix unterminated frames
Bug: 322232828
Test: FrameTimingQueryTest
Relnote: "Fixes issue where unterminated frames at the beginning and
end of the trace could be paired together, which would incorrectly
report as a single extremely long frame."
Creates alternative frameDurationFullMs metric implementation, which
isn't yet used in FrameTimingMetric but can be used to account for
vsync intended delays on UI thread.
Also fixes an issue that became more visible when validating its
measurements in traces.
Change-Id: I393531f8cf983b2700c419c00a9c7c47ec382e67
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/FrameTimingQueryTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/FrameTimingQuery.kt
cc...@google.com <cc...@google.com> #6
Bumping in priority after some discussions with +Andrew and +Leland.
We found that in CI, we saw fluctuations in both frame timing and frame overrun due to buffer stuffing periodically hitting Sargo API 31.
Periodically we'd see very large spikes in P90 frame timing because randomly more than 10% of interactions would hit triple buffering, and thus have an additional 16+ms of latency. I expect gfxinfo is less significantly affected since that's median of 90th percentile frame time across each iteration.
There's more to investigate though in instability in the other scenarios, however, and this noise from buffer stuffing is likely not the only issue.
I'd like to move us to something that accounts for buffer stuffing more generally, will file blocker bugs for improvements to perfetto frame timing metrics.
jj...@google.com <jj...@google.com> #7
Would disabling triple buffering be an option here?
cc...@google.com <cc...@google.com> #8
For app developer macrobenchmarks, that would be great. How do we do that? (ideally on user builds too)
Description
FrameTimingMetric
has a few issues that make it hard to align with user impact, from internal feedback/discussions:frameDurationCpuMs
accounts for the UI slice, but surprisingly doesn't account for any main thread delays from runnables that delay Choreographer#doFrame from starting. (This was an old decision to keep behavior consistent both pre 31 and post 31, when expected/actual frames were available, should consider revisiting.)frameOverrunMs
does account for delays, but still treats runnables different from in-frame work, due to how out-of-frame runnable main thread delays will delay the expected frame in the frame timeline.