IssueTracker

Chris, could you take a look at this benchmark? It seems weird that it is showing significant improvement without any related changes in the vicinity.

This is a super weird improvement - during this period of improvement, androidx.room.benchmark.InvalidationTrackerBenchmark and RelationBenchmark were consistently failing. After the fix, the number went back to where it was.

Seems that either:

1. androidx.room.benchmark crashing was affecting these other benchmarks in an entirely separate process
2. the room change itself was affecting the other benchmark
3. this is a weird coincidence

Double checking it's not #3 with some ABTD runs to try and repro.

Nothing in the traces jump out - most of the secondary metrics are the same or extremely similar but for some reason time is very different.

Could still be a unlucky layout issue.

Looked into this with help of Yigit's new ABTD comparison workflow.

The thing that appears to have caused the improved performance is that the test running right before the wear benchmarks was crashing, which caused a full reset just before wear runs. See screenshot #1, this is from the invocation perfetto trace which is a top level artifact in the ATI page.

These wear benchmarks are likely very sensitive to device resets as they have a very large number of binder transactions, which are running faster in the faster configurations. My guess is hammering on binder slightly overwhelms system server.

When running faster, the average binder transaction takes 0.86ms (2nd screenshot) When running slower, the average binder transaction takes 1.05ms (3rd screenshot)

As the loop seems to have at least 2 transactions, that probably accounts for the gap with and without the reset.

This is a broader problem though - neither of these look trivial to remove from the benchmarks.

One is: AndroidComposeViewAccessibilityDelegateCompat.<init>: (Landroidx/compose/ui/platform/AndroidComposeView;)V -> android.view.accessibility.AccessibilityManager.getEnabledAccessibilityServiceList: (I)Ljava/util/List;

And another is: androidx.wear.compose.material.TimeBroadcastReceiver.register: (Landroid/content/Context;)V -> android.content.ContextWrapper.registerReceiver: (Landroid/content/BroadcastReceiver;Landroid/content/IntentFilter;)Landroid/content/Intent;

I got both of these by looking at a trace (on an API 35 device so it gets method traces), and selecting the binder transactions that happen within the method trace. Trace here: https://ui.perfetto.dev/#!/?s=046b40a01c217a9b33ce09ef0c0a48aafa8ff82f

Leland, what would you recommend we do here? Avoid the binder transactions with microbenchmark-specific caching? (Only in cases where we've reviewed it to be unavoidable?)

With the recent microbench refactor, it's much more possible for us to create trace based metrics in microbenchmark, so counting transactions may be a good long term tool to help with cases like this, but in the short term, how can we avoid cases like these in the benchmarks?

Going to try and add some sleeps when binder is detected.