Provide FrameOverrun/FrameCpuDuration/FrameUiDuration metrics [202072375]

Fixed

Bug

Status Update

No update yet.

Description

cc...@google.com

created issue #1

Oct 5, 2021 12:43AM

See per-frame metrics proposal here:

https://docs.google.com/document/d/1msD4bpnqhBp4VdYY0-F5XR3EjqnZnspvhQKSmCvUYkY/edit?hl=en&forcehl=1&resourcekey=0-HIt8BjDLqrfT6V0cbpCGRg#heading=h.b5v32z3z9r3w

These metrics have already landed as part of macrobenchmark with https://android-review.googlesource.com/c/platform/frameworks/support/+/1828135 (though FrameUiTime isn't used, as Renderthread time is always queryable from trace data)

So, on FrameData, instead of just frameDurationNanos, it would be:

@RequiresApi(31)
frameNegativeSlackNanos

@RequiresApi(24)
frameCpuTimeNanos

frameUiTimeNanos

As an example, implementation of frameCpuTimeNanos would be effectively:

listOf(
    FrameMetrics.UNKNOWN_DELAY_DURATION,
    FrameMetrics.INPUT_HANDLING_DURATION,
    FrameMetrics.ANIMATION_DURATION,
    FrameMetrics.LAYOUT_MEASURE_DURATION,
    FrameMetrics.DRAW_DURATION,
    FrameMetrics.SYNC_DURATION,
    FrameMetrics.COMMAND_ISSUE_DURATION,
    FrameMetrics.SWAP_BUFFERS_DURATION
).sumOf { frameMetrics.getMetric(it) }

(need to optimize that, though)

Comments

ch...@google.com <ch...@google.com> #2Jan 6, 2022 10:50PM

ideally for alpha, to make JankStats more consistent with benchmark

ch...@google.com <ch...@google.com> #3Jan 14, 2022 12:10AM

Chris:
Are you suggesting we replace frameDurationNanos with negativeSlack, Cpu, and Ui metrics? Or that we keep duration and add these other three?
Also, would be good to be able to point to some reference doc about this, especially for "negativeSlack," which seems like a difficult term for people to inherently understand. If the advantage of adding that is one of consistency, I'd like to point to language in other APIs/guides/whatever where it's being used

cc...@google.com <cc...@google.com> #4Jan 14, 2022 12:45AM

Replace - frameDurationNanos is an ambiguous amalgamation of FrameCpuTime and FrameUiTime metrics, which changes behavior across API levels. Note that NegativeSlack has been renamed to FrameOverrun, and is already the primary output metric of macrobenchmarks on newer API levels.

You can reference Macrobench api docs: FrameTimingMetric (which doesn't list UI, because we can always see RT times from it's data sources)

In general though, agree we need more docs for these metrics. Some of it can be in the JankStats guide docs, and we can coordinate with +Kate to pull some of that to a general location on DAC, like we have with startup metrics.

ch...@google.com <ch...@google.com> #5Jan 19, 2022 07:27PM

'Overrun' sounds better, will use that.
Wouldn't 'Duration' be better than 'Time'? Time is less obvious what it refers to, but Duration is pretty clear.
So that would be:
frameOverrunMs (to be consistent with macrobenchmark)
frameCpuDurationNanos
frameUiDurationNanos

For the @RequiresAPI(...) suggestions in #1, are you saying we add that to the API I expose (so nobody can query/get those values from JankStats below those API levels), or are you saying we use that internally to retrieve those values (or not), then JankStats returns 0 below the appropriate API levels?

cc...@google.com <cc...@google.com> #6Jan 19, 2022 11:42PM

Duration SGTM, updated the metrics glossary, and I'll update macrobenchmark metrics to be consistent, since we're still in beta.

I'd prefer nanos in all cases, so we're not throwing away data. You can add ***Millis equivalents, but suggest using a getter to avoid extra storage/work when it's not needed:

val frameCpuDurationMillis: Float get() = frameCpuDurationNanos / 1_000_000.0

ch...@google.com <ch...@google.com> #7Jan 24, 2022 05:20PM

I'm fine sticking with nanos, was only mentioning ms because that's what I see in macrobenchmark and I thought we were aiming for consistency.
I'll just expose three Nanos metrics for now, we can always add millis later if we think it's useful.
Still wondering about the @RequiresApi request.

ch...@google.com <ch...@google.com> #8Jan 24, 2022 06:12PM

Also, wondering whether it should be "frame[Cpu|Ui]Duration" or "frameDuration[Cpu|Ui]"
The latter feels better to me, like cpu/ui are qualifiers (at the end) of a similar core concept (frameDuration)

ch...@google.com <ch...@google.com> #9Jan 24, 2022 06:56PM

btw, the overrun docs (

https://developer.android.com/reference/kotlin/androidx/benchmark/macro/FrameTimingMetric) say that it requires API 29, vs the 31 stated in #1 above. I assume we need the new 'deadline' metric, so 31 is correct and the doc is wrong?

cc...@google.com <cc...@google.com> #10Jan 24, 2022 07:28PM

I'll just expose three Nanos metrics for now

SGTM. I prioritize keeping thins in ns for API consistency (time is always a long, in ns) in the app, whereas macrobench prioritizes user reading a string.

@RequiresAPI is about making it clear which is available where.

Duration cpu/ui SGTM.

That doc is correct - we can implement it in API 29 in traces, since deadlines are there, but it's not in FrameMetrics until API 31.

Message last modified on Jan 24, 2022 07:28PM

ap...@google.com <ap...@google.com> #11Jan 28, 2022 11:38PM

Project: platform/frameworks/support
Branch: androidx-main

commit 7173e42e85d2a209555a6e45251d50dc5fc1744b
Author: Chet Haase <chet@google.com>
Date: Mon Jan 24 15:19:14 2022

Align timing metrics with other performance libraries

Previously, JankStats used a single "actualDuration" in reporting
and determining jank frames. Now, JankStats uses UI duration (equivalent
to previous actualDuration) on releases prior to API 24. As of API 24,
a new Cpu duration is added (with additional timing info provided by
FrameMetrics). And in API 31, a final Overrun value is added to
provide additional information on how much a jank frame missed its
deadline.

Relnote: Added enhanced timing metric properties
Test: existing JankStatsTest automated tests
Bug: 202072375
Change-Id: I05ec5afc5d3e4ef7ea27ab70dc7fc4e8e372a9ee

M metrics/metrics-performance/src/main/java/androidx/metrics/performance/JankStatsApi31Impl.kt
M metrics/metrics-performance/api/public_plus_experimental_current.txt
M metrics/metrics-performance/src/main/java/androidx/metrics/performance/FrameData.kt
M metrics/metrics-performance/api/restricted_current.txt
M metrics/metrics-performance/src/main/java/androidx/metrics/performance/JankStatsApi24Impl.kt
M metrics/metrics-performance/src/main/java/androidx/metrics/performance/JankStats.kt
M metrics/metrics-performance/src/main/java/androidx/metrics/performance/JankStatsApi16Impl.kt
M metrics/metrics-performance/src/androidTest/java/androidx/metrics/performance/test/JankStatsTest.kt
M metrics/integration-tests/janktest/src/main/java/androidx/metrics/performance/janktest/JankAggregatorActivity.kt
M metrics/integration-tests/janktest/src/main/java/androidx/metrics/performance/janktest/JankLoggingActivity.kt
A metrics/metrics-performance/src/main/java/androidx/metrics/performance/FrameDataApi31.kt
M metrics/metrics-performance/src/main/java/androidx/metrics/performance/JankStatsApi26Impl.kt
A metrics/metrics-performance/src/main/java/androidx/metrics/performance/FrameDataApi24.kt
M metrics/metrics-performance/api/current.txt

https://android-review.googlesource.com/1959648

ch...@google.com <ch...@google.com> Mar 15, 2022 04:03PM

Marked as fixed.