IssueTracker

I am not sure I understand the use case. how can the benchmark be code to real world scenario when it's not possible to do right now ? which scenario is it ?

In any case, since this would be for benchmarking, this would clearly not be available through the public DSL. We should find a semi-private way of doing this (maybe the private variant API object could offer that functionality for instance or a property).

We want benchmarks to measure code after Progaurd / R8, but it's not possible to turn that on for androidTests in library modules at the moment (to my knowledge?)

Benchmarks are also a public facing thing, but we have a plugin to help configure gradle builds for our users, so if support for this ends up in a private API, we could try to keep those usages localized to our code perhaps.

this is not part of our OKR at this point so we are not talking soon. at first glance, we would need to simulate usage patterns to minify against and such, this seems substantial amount of work. there are not a lot of library module that have android tests, most only rely on unit-tests.

how important is this ? we are out of PM right now but I suspect the next step will be to negotiate with J. Eason and xav@ to scale a priority level.

This is a high priority request for Compose, to enable their benchmarks to measure release accurate performance. (Micro) Benchmarks are library modules, as they don't need the complexity of multi-apk tests - they're self measuring APKs that depend on libraries. (d.android.com/benchmark)

there are not a lot of library module that have android tests, most only rely on unit-tests.

To clarify, this is for com.android.library modules, not jars - I'd expect most of those to use android tests (all of the libraries in jetpack for example do).

we would need to simulate usage patterns to minify against and such, this seems substantial amount of work

Simulate usage patterns? I don't understand - the dev can themselves provide a keep rule for test infra / classes if necessary. Long term, keep rules should be provided by test libraries.

We've been experimenting with ways to work around this for Compose. Performance results from R8 seem significantly different, and would enable us to measure much more accurately. I've tried to come up with a workaround using a com.android.app module, and while it almost works (and we can get measurements), it's extremely hacky and doesn't let us run tests anymore via Studio: b/184378053#comment3

Bumping this request, as Compose has recently had more interest in the ability to benchmark with and without R8 enabled.

We're fine if the default implementation doesn't work with minification (tree shaking) - we're happy to supply those rules ourselves, or simply evaluating with minification off to take advantage of other optimizations.