IssueTracker

I have a feeling there's a lot of implicit behavior behind the startup/compilation modes, which is not easy for developers to discover or understand.

For example:

• StartupMode.COLD --> process dies between setupBlock and measureBlock while not in other modes
• reinstalling the app on (older) user device builds
• StartupMode.WARM --> there's a additional warmup iteration

And now we'll build additional behavior that is different for StartupMode.COLD.

I understand your concern, but with the current shape of the API: we don't really have too many options to control this behavior.

Once the lower level APIs are available to write your own Macrobenchmark loop, you can do a lot more and define the bits exactly like you want. That being said, I also want to extensively document the lifecycle of a method trace better in Macrobenchmark.

Ok, sounds good. Thanks for explaining!

I have a feeling there's a lot of implicit behavior behind the startup/compilation modes, which is not easy for developers to discover or understand.

Agree, this is tracked in b/278214396

And now we'll build additional behavior that is different for StartupMode.COLD.

This is a good point, and I agree we shouldn't be making StartupMode.COLD even more complex to reason about. I'd prefer we say method tracing is always on during the measurement in macrobenchmarks (assuming you use macrobench to start/kill the process), and we should make that work automatically.

I'd prefer we say method tracing is always on during the measurement in macrobenchmarks (assuming you use macrobench to start/kill the process)

When a process is being started is not entirely obvious to an app developer. Additionally, method traces collected during startup are actually very important; so we need to make that part easy. Which is why startActivityAndWait(...) signals a method trace start (when StartupMode.COLD is being used), and the end of the measurement is when a method trace ends.

What scenario do you have in mind that you'd want the method tracing for startup, but it wouldn't be covered by the startActivityAndWait or measureBlock?

If I have

startupMode = COLD
setupBlock= {
 // prepare something
}
measureBlock = {
 startActivityAndWait()
}

I would say I still want to only see the method trace of what I'm measuring.

If I have

startupMode = COLD
setupBlock= {
 startActivityAndWait()
}
measureBlock = {
  // do something in the app
}

This doesn't work, because the process is killed between setupBlock and measureBlock.

I can have something like

startupMode = COLD
setupBlock= {
 sendABroadcastWhichWakesTheProcess()
}
measureBlock = {
 startActivityAndWait()
}

But I would still expect the method trace to run in the measureBlock? The process would be killed after setupBlock anyway, so the method trace can't really survive, can it?

Maybe I'm missing some scenario though.

Summarizing an offline convo about how we plan to do this:

• at beginning of measure block:
  • start method tracing if process alive
• (in measure block) process started
  • start process with method tracing
• (in measure block) process killed
  • flush method trace before kill
• at end of measure block:
  • flush method trace

No special handling for startup modes, or tracing of the setup block.

This requires a lot of checking process state, but that should only happen while method tracing, so it's worth the cost to get predictable tracing.

Note that you'll potentially get arbitrary numbers of traces depending on how many instances of the process run sequentially.

I think that SG. I think the arbitraty numbers of traces is fine, because these probably matter only for further analysis.