Status Update
Comments
rp...@google.com <rp...@google.com>
sh...@google.com <sh...@google.com> #2
I am not sure I understand the use case. how can the benchmark be code to real world scenario when it's not possible to do right now ? which scenario is it ?
In any case, since this would be for benchmarking, this would clearly not be available through the public DSL. We should find a semi-private way of doing this (maybe the private variant API object could offer that functionality for instance or a property).
la...@gmail.com <la...@gmail.com> #3
We want benchmarks to measure code after Progaurd / R8, but it's not possible to turn that on for androidTests in library modules at the moment (to my knowledge?)
Benchmarks are also a public facing thing, but we have a plugin to help configure gradle builds for our users, so if support for this ends up in a private API, we could try to keep those usages localized to our code perhaps.
sh...@google.com <sh...@google.com> #4
Any update on the status of this request and when it can be supported?
Thanks,
Amanda
la...@gmail.com <la...@gmail.com> #5
la...@gmail.com <la...@gmail.com> #6
This is a high priority request for Compose, to enable their benchmarks to measure release accurate performance. (Micro) Benchmarks are library modules, as they don't need the complexity of multi-apk tests - they're self measuring APKs that depend on libraries. (d.android.com/benchmark)
there are not a lot of library module that have android tests, most only rely on unit-tests.
To clarify, this is for com.android.library
modules, not jars - I'd expect most of those to use android tests (all of the libraries in jetpack for example do).
we would need to simulate usage patterns to minify against and such, this seems substantial amount of work
Simulate usage patterns? I don't understand - the dev can themselves provide a keep rule for test infra / classes if necessary. Long term, keep rules should be provided by test libraries.
sh...@google.com <sh...@google.com> #7
We've been experimenting with ways to work around this for Compose. Performance results from R8 seem significantly different, and would enable us to measure much more accurately. I've tried to come up with a workaround using a com.android.app module, and while it almost works (and we can get measurements), it's extremely hacky and doesn't let us run tests anymore via Studio:
sh...@google.com <sh...@google.com> #8
Bumping this request, as Compose has recently had more interest in the ability to benchmark with and without R8 enabled.
We're fine if the default implementation doesn't work with minification (tree shaking) - we're happy to supply those rules ourselves, or simply evaluating with minification off to take advantage of other optimizations.
Description
Scenario
Performance issue
We are using a Samsung S21 Ultra 5G. The phone is set up to range with two UWB sensors. Also, the phone acts as a controlee. We are able to get ranging results from both sensors with decent update rate. However, when one sensor is blocked, the blocked sensor no longer produce any ranging results which is expected, but the unblocked sensor also experience slower update rate (e.g., previously 4Hz, now 1Hz).
Tentative solution
We tried to time out the blocked sensor, and allow the unblocked sensor to produce ranging results with normal update rate (e.g., about 4 Hz). This turns out to be quite problematic due to a potential bug(or missing feature) described below. We tried to put ranging logic in one thread, separate threads, and even in multiple processes (by extending the Service class), none of them worked. At some point, restarted ranging sessions will stop responding or produce ranging results with much lower update rates. When we try our multiple process set up, if we only ranging with one sensor on a first process, we get ranging results from the sole sensor, but if we start ranging with a second sensor on a second process, the second process starts to report ranging results, but the first process stops reporting ranging results. We also noticed a ranging already started exception. We figured multi-process may not be supported.
Tests and Findings
When core uwb creates SessionScopes in
core.uwb.impl.UwbManagerImpl.kt
using methodcreateGmsClientSessionScope
, it will creates a uwbClient variable from gms.Nearby. When we use the phone to range with two UWB sensors, we will need to call this method twice, which causes two UwbClient instances get created.If we follow normal workflow, we get ranging results through the callback in
core.uwb.impl.UwbManagerImpl.kt
, more specifically theonRangingResult
result callback.However, when we timeout one of the UWB sensors in testing because it is blocked (and hence stop producing ranging results and causes the other sensor to produce results with slower update rate), we invoked the
uwbClient.stopRanging(callback: RangingSessionCallback)
function. Things started to get messy from this point on. It seems like current core.uwb implementation can only reliably work with a single UwbClient. There are several scenarios that may happen:onRangingSuspended
incore.uwb.impl.UwbClientSessionScopeImpl.kt
) and this is what we want!!!. In this situation, if we restart the ranging session for sensor 1 with a new uwbClient, we get ranging results with normal date rate (e.g., 4hz)One additional finding we noticed is that the ranging suspended reason, though normally will be 4, but sometimes can be 1. Four means ranging stopped by peer, while one means wrong parameters. This may or may not be helpful.
Code changes we made in our testing
We disabled RxJava and Kotlin's Flow in core.uwb implementation, and tried with both release Alpha 07 and Alpha 05. The reason we disabled RxJava and Kotlin Flow is that we can call stopRanging on uwbClient instances when we want to and make sure RxJava and Kotlin Flow play no part in the issue we observed.
I hope the detailed writing provides enough details for you to debug.