Status Update
Comments
ys...@google.com <ys...@google.com> #2
on bundled trace_processor - current scripts downloads and caches a platform specific prebuilt
Would this pin each androidx release to some bundled version for stability?
On new androidx artifact - I think there are a bunch of batch processing cases that will require this as a separate lib (androidx.benchmark.traceprocessor). I'm assuming the flow is generally to upload a trace from each daily/commit build in CI, then if you add new queries to a dashboard, you need to run against all those historical traces as quickly as possible.
cc...@google.com <cc...@google.com> #3
Would this pin each androidx release to some bundled version for stability?
Initially, we'd just open up the querying APIs, and require you to wire up your own trace_processor_shell on host.
After that, we'd start bundling in host versions of trace_processor_shell
, so it would be versioned together with benchmark, though I imagine we'd keep the ability to use your own version of trace_processor_shell
if you prefer.
As to portability / version numbers - we considered matching Perfetto version numbers in a new group, but decided against it. We want to be able to depend on new traceprocessor versions at arbitrary stages in the benchmark release cycle.
On new androidx artifact - I think there are a bunch of batch processing cases that will require this as a separate lib (androidx.benchmark.traceprocessor).
Yes, it would be a separate artifact. (KMP for JVM + android, so that host/device wouldn't get a bunch of trace_processor_shell
copies they can't use)
I'm assuming the flow is generally to upload a trace from each daily/commit build in CI, then if you add new queries to a dashboard, you need to run against all those historical traces as quickly as possible.
On host JVM, it's up to you - you can use this to run against historical traces with adding metrics to a dashboard, just write your tests on host, build custom insights that operate on traces in isolation (completely decoupled from Macrobenchmark / Android devices). (the last assumes we'd open up a standard api for Insights in the traceprocessor artifact).
ap...@google.com <ap...@google.com> #4
Project: platform/frameworks/support
Branch: androidx-main
Author: Chris Craik <
Link:
Move PerfettoTraceProcessor -> TraceProcessor in benchmark-traceprocessor
Expand for full commit details
Move PerfettoTraceProcessor -> TraceProcessor in benchmark-traceprocessor
Bug: 381134564
Test: Tests in benchmark-macro
Test: TraceProcessorBenchmark
Relnote: """Move PerfettoTraceProcessor to TraceProcessor in a new
androidx.benchmark:benchmark-traceprocessor artifact, and make most of
its API non-experimental. Any custom TraceMetric or anything reading
from traces will need to update to the new TraceProcessor import. The
new TraceProcessor API works exactly like the old one, but is a
standalone interface library (somewhat analogous to the androidx.sql
layer from Room) with an Android-specific implementation built into
macrobenchmark. The new artifact can be used on JVM as well, but
currently you'll need to start your own copy of the TraceProcessor
binary and offer a port to connect to it on."""
This is a first step toward writing Metric / Insights code that runs
(and can be tested) on host
Change-Id: I6256342f25d3f00ef864cf14b75a7159c6a40573
Files:
- M
benchmark/benchmark-common/api/current.txt
- M
benchmark/benchmark-common/api/restricted_current.txt
- M
benchmark/benchmark-common/build.gradle
- M
benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/PerfettoTraceTest.kt
- D
benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoTrace.kt
- A
benchmark/benchmark-common/src/main/java/androidx/benchmark/traceprocessor/PerfettoTrace.kt
- M
benchmark/benchmark-junit4/api/current.txt
- M
benchmark/benchmark-junit4/api/restricted_current.txt
- M
benchmark/benchmark-junit4/src/main/java/androidx/benchmark/junit4/PerfettoTraceRule.kt
- M
benchmark/benchmark-macro-junit4/build.gradle
- A
benchmark/benchmark-macro/api/current.ignore
- M
benchmark/benchmark-macro/api/current.txt
- A
benchmark/benchmark-macro/api/restricted_current.ignore
- M
benchmark/benchmark-macro/api/restricted_current.txt
- M
benchmark/benchmark-macro/build.gradle
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/ArtMetricTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/InsightTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/PerfettoTraceRuleTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/PowerMetricTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/StartupTimingMetricTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/TraceMetricTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/TraceSectionMetricTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/AndroidxTracingTraceTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/BatteryDischargeQueryTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/FrameTimingQueryTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/MemoryCountersQueryTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/MemoryUsageQueryTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoCaptureSweepTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoSdkTraceTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PowerQueryTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/StartupTimingQueryTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/traceprocessor/RowTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/traceprocessor/SliceTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/traceprocessor/TraceProcessorTest.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/MacrobenchmarkPhase.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Metric.kt
- A
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/TraceProcessorExtensions.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/BatteryDischargeQuery.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/FrameTimingQuery.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/MemoryCountersQuery.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/MemoryUsageQuery.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/PowerQuery.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/StartupInsights.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/StartupTimingQuery.kt
- D
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/server/PerfettoHttpServer.kt
- M
benchmark/benchmark-traceprocessor/api/current.txt
- M
benchmark/benchmark-traceprocessor/api/restricted_current.txt
- M
benchmark/benchmark-traceprocessor/build.gradle
- A
benchmark/benchmark-traceprocessor/src/androidMain/kotlin/androidx/benchmark/traceprocessor/Log.android.kt
- M
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/ExperimentalTraceProcessorApi.kt
- A
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/Log.kt
- A
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/PerfettoTrace.kt
- M
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/QueryResultIterator.kt
- M
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/Row.kt
- A
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/ServerLifecycleManager.kt
- M
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/Slice.kt
- M
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/TraceProcessor.kt
- A
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/TraceProcessorHttpServer.kt
- A
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/perfetto/protos/package-info.java
- A
benchmark/benchmark-traceprocessor/src/jvmMain/kotlin/androidx/benchmark/traceprocessor/Log.jvm.kt
- M
benchmark/integration-tests/macrobenchmark/src/main/java/androidx/benchmark/integration/macrobenchmark/TraceProcessorBenchmark.kt
- M
wear/compose/compose-material3/macrobenchmark/src/main/java/androidx/wear/compose/material3/macrobenchmark/metric/FrameCostQuery.kt
- M
wear/compose/compose-material3/macrobenchmark/src/main/java/androidx/wear/compose/material3/macrobenchmark/metric/Metric.kt
- M
wear/compose/integration-tests/macrobenchmark/src/main/java/androidx/wear/compose/integration/macrobenchmark/CompositionMetric.kt
Hash: 3ecf191783372810d8cbe9dededeea09dde6ae07
Date: Mon Nov 25 16:20:10 2024
ap...@google.com <ap...@google.com> #5
Project: platform/frameworks/support
Branch: androidx-main
Author: Chris Craik <
Link:
Add initial, empty benchmark-traceprocessor
Expand for full commit details
Add initial, empty benchmark-traceprocessor
Bug: 381134564
Test: N/A
Relnote: N/A
Change-Id: If3e85be06a29ad105f3eb1d21090531772204e6f
Files:
- A
benchmark/benchmark-traceprocessor/api/current.txt
- A
benchmark/benchmark-traceprocessor/api/res-current.txt
- A
benchmark/benchmark-traceprocessor/api/restricted_current.txt
- A
benchmark/benchmark-traceprocessor/build.gradle
- M
docs-tip-of-tree/build.gradle
- M
settings.gradle
Hash: 6c7159fd997ef48fa0f3c5b12c85c2938e47d64a
Date: Tue Dec 03 16:07:21 2024
cc...@google.com <cc...@google.com> #6
Reverted due to build breakage.
ap...@google.com <ap...@google.com> #7
Project: platform/frameworks/support
Branch: androidx-main
Author: Chris Craik <
Link:
Reland "Move PerfettoTraceProcessor -> TraceProcessor in benchmark-traceprocessor"
Expand for full commit details
Reland "Move PerfettoTraceProcessor -> TraceProcessor in benchmark-traceprocessor"
Bug: 381134564
Bug: 382741383
Test: Tests in benchmark-macro
Test: TraceProcessorBenchmark
Test: ./gradlew :docs-tip-of-tree:docs
Revert submission 3407319-revert-3376991-QEQTPRWJDK
Reason for revert: dependent change aosp/3394905 was reverted, but has since relanded as aosp/3406159
Reverted changes: /q/submissionid:3407319-revert-3376991-QEQTPRWJDK
Relnote: """Move PerfettoTraceProcessor to TraceProcessor in a new
androidx.benchmark:benchmark-traceprocessor artifact, and make most of
its API non-experimental. Any custom TraceMetric or anything reading
from traces will need to update to the new TraceProcessor import. The
new TraceProcessor API works exactly like the old one, but is a
standalone interface library (somewhat analogous to the androidx.sql
layer from Room) with an Android-specific implementation built into
macrobenchmark. The new artifact can be used on JVM as well, but
currently you'll need to start your own copy of the TraceProcessor
binary and offer a port to connect to it on."""
This works around a crash in the docs build by moving
package-info.java out of commonMain to both jvmMain/androidMain.
Change-Id: I3a7675f25f4425745b6f255dcd1a0cec81a068ba
Files:
- M
benchmark/benchmark-common/api/current.txt
- M
benchmark/benchmark-common/api/restricted_current.txt
- M
benchmark/benchmark-common/build.gradle
- M
benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/PerfettoTraceTest.kt
- D
benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoTrace.kt
- A
benchmark/benchmark-common/src/main/java/androidx/benchmark/traceprocessor/PerfettoTrace.kt
- M
benchmark/benchmark-junit4/api/current.txt
- M
benchmark/benchmark-junit4/api/restricted_current.txt
- M
benchmark/benchmark-junit4/src/main/java/androidx/benchmark/junit4/PerfettoTraceRule.kt
- M
benchmark/benchmark-macro-junit4/build.gradle
- A
benchmark/benchmark-macro/api/current.ignore
- M
benchmark/benchmark-macro/api/current.txt
- A
benchmark/benchmark-macro/api/restricted_current.ignore
- M
benchmark/benchmark-macro/api/restricted_current.txt
- M
benchmark/benchmark-macro/build.gradle
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/ArtMetricTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/InsightTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/PerfettoTraceRuleTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/PowerMetricTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/StartupTimingMetricTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/TraceMetricTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/TraceSectionMetricTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/AndroidxTracingTraceTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/BatteryDischargeQueryTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/FrameTimingQueryTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/MemoryCountersQueryTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/MemoryUsageQueryTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoCaptureSweepTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoSdkTraceTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PowerQueryTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/StartupTimingQueryTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/traceprocessor/RowTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/traceprocessor/SliceTest.kt
- M
benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/traceprocessor/TraceProcessorTest.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/MacrobenchmarkPhase.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Metric.kt
- A
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/TraceProcessorExtensions.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/BatteryDischargeQuery.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/FrameTimingQuery.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/MemoryCountersQuery.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/MemoryUsageQuery.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/PowerQuery.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/StartupInsights.kt
- M
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/StartupTimingQuery.kt
- D
benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/server/PerfettoHttpServer.kt
- M
benchmark/benchmark-traceprocessor/api/current.txt
- M
benchmark/benchmark-traceprocessor/api/restricted_current.txt
- M
benchmark/benchmark-traceprocessor/build.gradle
- A
benchmark/benchmark-traceprocessor/src/androidMain/kotlin/androidx/benchmark/traceprocessor/Log.android.kt
- A
benchmark/benchmark-traceprocessor/src/androidMain/kotlin/perfetto/protos/package-info.java
- M
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/ExperimentalTraceProcessorApi.kt
- A
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/Log.kt
- A
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/PerfettoTrace.kt
- M
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/QueryResultIterator.kt
- M
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/Row.kt
- A
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/ServerLifecycleManager.kt
- M
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/Slice.kt
- M
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/TraceProcessor.kt
- A
benchmark/benchmark-traceprocessor/src/commonMain/kotlin/androidx/benchmark/traceprocessor/TraceProcessorHttpServer.kt
- A
benchmark/benchmark-traceprocessor/src/jvmMain/kotlin/androidx/benchmark/traceprocessor/Log.jvm.kt
- A
benchmark/benchmark-traceprocessor/src/jvmMain/kotlin/perfetto/protos/package-info.java
- M
benchmark/integration-tests/macrobenchmark/src/main/java/androidx/benchmark/integration/macrobenchmark/TraceProcessorBenchmark.kt
- M
wear/compose/compose-material3/macrobenchmark/src/main/java/androidx/wear/compose/material3/macrobenchmark/metric/FrameCostQuery.kt
- M
wear/compose/compose-material3/macrobenchmark/src/main/java/androidx/wear/compose/material3/macrobenchmark/metric/Metric.kt
- M
wear/compose/integration-tests/macrobenchmark/src/main/java/androidx/wear/compose/integration/macrobenchmark/CompositionMetric.kt
Hash: a13168a07a6545e9614541c28bf4ab19a2f6523e
Date: Thu Dec 12 09:48:11 2024
cc...@google.com <cc...@google.com> #8
Over to Rahul, who was looking at landing the host artifacts to make this work trivially on JVM
gr...@google.com <gr...@google.com> #9
na...@google.com <na...@google.com> #10
Is this the same issue as
Description
Metric queries and insights would benefit from iterating on host, while currently PerfettoTraceProcessor can only run on Android.
First step here would be decoupling PerfettoTraceProcessor API from our trace_processor_shell android binary.
Follow ups could improve portability: