Clarify failures on devices misconfigured for tracing/compilation [282191686]

Fixed

Bug

Status Update

No update yet.

Description

cc...@google.com

created issue #1

May 12, 2023 05:48PM

Since fixes are slow to roll out, we should have benchmark clearly tell people they're using unsupported emulators if they run into the following bugs:

b/272339637 b/244619933

For the tracing issue, if perfetto capture start fails, do something like:

if (!File("/sys/kernel/debug/tracing/trace_marker").exists() && !File("/sys/kernel/tracing/trace_marker").exists()) {
    val deviceType = DeviceInfo.isEmulator ? "emulator" : "device"
    throw IllegalStateException("This $deviceType system image is misconfigured, and doesn't support tracing. Try a different $deviceType (or updating, if possible).")
}

And add something similar for the compilation failure.

Comments

cc...@google.com <cc...@google.com> #2May 12, 2023 05:50PM

Also consider adding an application subclass that does these checks in benchmark-macro, so that we get far fewer breakages showing up in the http://go/androidx-perf-tests, and they show up as module errors instead.

ma...@google.com <ma...@google.com> Jun 6, 2023 07:19PM

Accepted by ma...@google.com.

cc...@google.com <cc...@google.com> #3Jun 6, 2023 08:13PM

Thought about this a bit more, and suggest the following parts for tracing:

check for file presence in process init (e.g. DeviceInfo initializer) without throwing
In macro, add a runtime check with a clear error about unsupported/noncomplient device build
In PerfettoHelper.startCollecting(), check(!DeviceInfo.supportsTracing) {...}
In micro, we're ok with tracing failing, so suggest changing enableTracing = !Arguments.dryRunMode here to enableTracing = !Arguments.dryRunMode && DeviceInfo.supportsTracing

This way, macrobench and PerfettoTrace.capture will error clearly, without it affecting baseline profile capture or microbench.

ma...@google.com <ma...@google.com> Jun 13, 2023 05:44PM

Assigned to ma...@google.com.

cc...@google.com <cc...@google.com> Jun 13, 2023 05:45PM

Reassigned to cc...@google.com.

ap...@google.com <ap...@google.com> #4Jul 15, 2023 12:00AM

Project: platform/frameworks/support
Branch: androidx-main

commit 24d0d7a6f7e7100d0e3d05131de1cd872a7e840f
Author: Chris Craik <ccraik@google.com>
Date: Fri Jul 14 13:52:36 2023

Add warning and test for device with misconfigured tracing

Bug: 282191686
Test: DeviceInfoTest # on user build
Test: TrivialStartupBenchmark
Test: TrivialKotlinBenchmark
Relnote: "Report error in macrobench when OS is misconfigured for
tracing, as was recently fixed in API 26/28 ARM64 emulators."

Change-Id: I0a328b08a0ef4b356d3d7b593dbb89e47d00815e

A benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/DeviceInfoTest.kt
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/PerfettoHelperTest.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/DeviceInfo.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoHelper.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoTrace.kt
M benchmark/benchmark-junit4/src/main/java/androidx/benchmark/junit4/BenchmarkRule.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoCaptureSweepTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoCaptureTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt

https://android-review.googlesource.com/2656748

ap...@google.com <ap...@google.com> #5Jul 17, 2023 05:45PM

Project: platform/frameworks/support
Branch: androidx-main

commit 2189e12ba790617267b4ee81923553ffcc71e90d
Author: Chris Craik <ccraik@google.com>
Date: Fri Jul 14 16:21:41 2023

Add error suggesting emulator update if compile reset fails

Bug: 282191686
Test: CompilationModeTest
Relnote: "Added detail to compilation reset failure to suggest
updating emulator, as some emulators have recently fixed this issue."

Change-Id: I8c81570e0b2e03b9f6453e13f0706b10f59f0455

M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/CompilationMode.kt
M benchmark/integration-tests/macrobenchmark/src/main/java/androidx/benchmark/integration/macrobenchmark/CompilationModeTest.kt

https://android-review.googlesource.com/2657262

cc...@google.com <cc...@google.com> Jul 17, 2023 08:07PM

Marked as fixed.

na...@google.com <na...@google.com> #6Jul 26, 2023 06:38PM

The following release(s) address this bug.It is possible this bug has only been partially addressed:

androidx.benchmark:benchmark-common:1.2.0-beta02
androidx.benchmark:benchmark-junit4:1.2.0-beta02
androidx.benchmark:benchmark-macro:1.2.0-beta02