Startup Metrics not always captured via emulator [329145809]

Fixed

Bug

Status Update

No update yet.

Description

en...@meta.com

created issue #1

Mar 12, 2024 09:30AM

Component used: Version used: Benchmark Macro v.1.2.3 Devices/Android versions reproduced on: Android emulator (API 30)

I am often seeing the error: "Skipping results from iter $iteration, it didn't capture all metrics" from emulator runs of cold start tests. I currently have my test set to 10 iterations and get any where from 2-9 metrics in the output json file although I see 10 successful startups in the UI and. in the Benchmark logs.

If this is a bug in the library, we would appreciate if you could attach:

Sample project to trigger the issue.
A screenrecord or screenshots showing the issue (if UI related).

Comments

cc...@google.com <cc...@google.com> #2Mar 25, 2024 08:29PM

Any suggestions on where such an interface might live? I don't think it makes sense for the interface to be in the drawerlayout artifact if you're specifically going to use the interface to *not* use DrawerLayout.

en...@meta.com <en...@meta.com> #3Apr 1, 2024 12:24PM

androidx.customview would be a good candidate. This is used by all of the widget libraries.

Uncaptured_Metrics_Logs.txt

8.6 KB

View

Download

cc...@google.com <cc...@google.com> #4Apr 1, 2024 03:57PM

Project: platform/frameworks/support
Branch: androidx-master-dev

commit 3ed3fb003fa6c1244f923202859a616225b5b2fa
Author: Ian Lake <ilake@google.com>
Date: Fri Feb 14 11:17:46 2020

Create an interface for layouts that can be opened

Create a common interface that represents layouts
that have two states: open and closed. This allows
higher level libraries to rely on the interface,
rather than concrete implementations such as
DrawerLayout, making them more resilient to
changes in the current recommended implementation.

Fixes: 129979320
Test: ./gradlew checkApi
Change-Id: I0f2a1414977825aa053c6555261f2b7d4417bd19

M customview/customview/api/1.1.0-alpha02.txt
M customview/customview/api/current.txt
M customview/customview/api/public_plus_experimental_1.1.0-alpha02.txt
M customview/customview/api/public_plus_experimental_current.txt
M customview/customview/api/restricted_1.1.0-alpha02.txt
M customview/customview/api/restricted_current.txt
A customview/customview/src/main/java/androidx/customview/widget/Openable.java
M drawerlayout/drawerlayout/api/1.1.0-alpha03.txt
M drawerlayout/drawerlayout/api/current.txt
M drawerlayout/drawerlayout/api/public_plus_experimental_1.1.0-alpha03.txt
M drawerlayout/drawerlayout/api/public_plus_experimental_current.txt
M drawerlayout/drawerlayout/api/restricted_1.1.0-alpha03.txt
M drawerlayout/drawerlayout/api/restricted_current.txt
M drawerlayout/drawerlayout/build.gradle
M drawerlayout/drawerlayout/src/main/java/androidx/drawerlayout/widget/DrawerLayout.java
M jetifier/jetifier/migration.config
M slidingpanelayout/slidingpanelayout/api/1.1.0-alpha01.txt
M slidingpanelayout/slidingpanelayout/api/current.txt
M slidingpanelayout/slidingpanelayout/api/public_plus_experimental_1.1.0-alpha01.txt
M slidingpanelayout/slidingpanelayout/api/public_plus_experimental_current.txt
M slidingpanelayout/slidingpanelayout/api/restricted_1.1.0-alpha01.txt
M slidingpanelayout/slidingpanelayout/api/restricted_current.txt
M slidingpanelayout/slidingpanelayout/src/main/java/androidx/slidingpanelayout/widget/SlidingPaneLayout.java

https://android-review.googlesource.com/940787

en...@meta.com <en...@meta.com> #5Apr 1, 2024 05:35PM

Ah right, I'm blocked on using 1.2.3 in a specific use case due to

https://issuetracker.google.com/issues/329145808. I have a local workaround to use 1.2.3 but didn't use it for today's repro. Let me try to repro with 1.2.3 today

en...@meta.com <en...@meta.com> #6Apr 1, 2024 06:11PM

Okay, looks like I'm unable to repro this on 1.2.3 anymore. I'm getting an entirely different error consistently with 1.2.3 which is "java.util.NoSuchElementException: Collection contains no element matching the predicate." I'll create a separate issue for this if one doesn't exist already.

Feel free to close this issue out. Thanks for all your help!

ap...@google.com <ap...@google.com> #7Apr 1, 2024 07:22PM

Project: platform/frameworks/support
Branch: androidx-main

commit d55b483632c5662ae95d19479ebbc14f8b23b0ff
Author: Chris Craik <ccraik@google.com>
Date: Mon Apr 01 11:27:57 2024

Add logs / error messages to all startup detection failures

Test: StartupTimingQueryTest
Bug: 329145809
Bug: 305733267

Relnote: "Added log.w / exception labels to all startup detection
failures. This does not change current behavior (so some errors throw,
and others silently fail to detect the startup), just makes it more
understandable. Generally the ones that Log.w() and fail to report
startup metrics are those where non-frame events are missing,
exceptions are thrown when startup is detected except for frame timing
information (from UI/RT slices)."

Likely will want to revisit this in the future to make detection
failure conditions more consistent, this change just makes errors more
comprehensible without needing to dig into the trace.

Change-Id: Id240f7698dfb977457362a137f0070e73dc2495c

M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/StartupTimingQuery.kt

https://android-review.googlesource.com/3021184

cc...@google.com <cc...@google.com> #8Apr 1, 2024 08:20PM

Above CL should make problems like these a lot easier to diagnose (both missing metric, and collection contains no element matching predicate).

cc...@google.com <cc...@google.com> #9Apr 1, 2024 08:21PM

Marked as fixed.

Actual missing content issue was tracked/fixed in b/245414235

Treating this bug as about the discoverability/logging side, so closing this as fixed.

na...@google.com <na...@google.com> #10Apr 17, 2024 05:17PM

The following release(s) address this bug.It is possible this bug has only been partially addressed:

androidx.benchmark:benchmark-macro:1.3.0-alpha03