IssueTracker

It seems the print is benign so I'll remove it; Could be worth it to find out how the message carrying information about "cb 0" (0 is not a valid color buffer handle) ever arrives to that line though

I agree with Sergey -- while I think it's fine to remove some logging messages that aren't necessary, the root problem here is that Studio shows everything emitted as if they are errors, with one notification bubble per message. We don't do that for other tools that Studio launches, and in particular, this isn't specific to the "cb 0" message I showed here; there were earlier (different) error messages during the boot as well.

This behavior was requested specifically by the emulator team (vharron@ iirc, so at least 3-4 years ago). Let me see if I have any more details on why this was requested.

I couldn't find any relevant threads. IIRC, the main concern at that time was that warnings at startup (e.g. missing acceleration) really needed to be bubbled up to the users. If that is not an issue anymore, and the emulator team only wants errors to be visible, then the proposed change SGTM.

Yes, I vaguely remember that in some cases, you'd hit run, and then the emulator would fail to start (for example aborting because of some missing thing it needed), so we needed a way to surface these reasons. I don't know if there are any fatal startup errors like this. It seems like there are several things we could do here:

(1) Simplest: just reroute the errors into the log, and if emulator fails to start, that's where people can look. (I think that's what Sergey has already implemented?)

(2) If emulator could update itself to prefix truly fatal error messages in some way (such as error:) and use that for only fatal/exit(-1) reasons, then we could multiplex the output -- errors are shown in notifications as today, everything else is silently dumped to the log

(3) Hold on to the emulator output, and if there's a problem starting the emulator (say if it exits within a few seconds), then show all the emulator output.

The pending fix in #8 changes AGP's behavior to include native libraries from the app's external native build in the android test APK, but that will make it so that users would need to manually exclude the test .so files from their application APK.

Is the desired behavior to support having a src/androidTest/cpp source directory and package the resulting .so files only in the android test APK?

Is the desired behavior to support having a src/androidTest/cpp source directory and package the resulting .so files only in the android test APK?

It'd likely be quite cumbersome to do so. There are extra hoops that would need to be jumped through for the test in src/androidTest/cpp to refer to the implementation in src/main/cpp. I don't think there's a good way to simplify that workflow either.

The intent behind externalNativeBuild is that users should be able to reuse existing CMake projects in AGP without needing the dramatically reorganize the project to fit AGP. Some porting work is usually expected, but it shouldn't require changing the layout of a project's source, so we sort of need to keep test sources in src/main/cpp.

We do need some way to communicate to AGP that some libraries defined by externalNativeBuild should be in the test APK but not in the main APK. I'd be content with an explicit packagingOptions DSL for that. CMake does have add_test, but that's for executables, which isn't how junit-gtest works, so that won't work. I'm not sure if there's another way to identify which CMake (or ndk-build, for that matter) outputs are tests and which are implementation, so it likely does need to be explicitly managed by the user?

+jomof in case he has better ideas.

+xav

We do need some way to communicate to AGP that some libraries defined by externalNativeBuild should be in the test APK but not in the main APK.

This is possible with the snippet from #2 after Change-Id: I2caf6a4832a2781790b5c7ee9a66d11e4835bd36 (I added this to the test in the latest patch).

I'd be content with an explicit packagingOptions DSL for that. CMake does have add_test, but that's for executables, which isn't how junit-gtest works, so that won't work. I'm not sure if there's another way to identify which CMake (or ndk-build, for that matter) outputs are tests and which are implementation, so it likely does need to be explicitly managed by the user?

If there's no way for AGP to detect this implicitly, then we can consider adding a DSL for this special case (for ease of use, compared to #2).

I think there are 2 separate problems here:

• AGP currently skips all the native libs from test APKs (which Scott's CL attempts to fix). We need to be careful there and understand what happens if the same lib is packaged in both prod and test APKs. It does make sense that native libs coming from test-only dependencies are packaged in the test APKs but we should filter out libs that are coming via the prod dependencies.

• Building native code via externalNativeBuild does not go in the test APK. The issue is that there's no way to target what it's for (prod and/or test) and because the external build can build several .so files, we need granularity in pointing each native libs to their target APK (or AAR). We need to figure this one out.

did I forget something?

Thanks, Xav.

I think I'll need to write a doc for this, but I'm thinking we can add something like val testOnly: MutableSet<String> to the JniLibsPackaging DSL.

Example usage:

android {
    packaging {
        jniLibs {
            testOnly += '**/testStuff.so'
        }
    }
}

If nothing is added to testOnly, the behavior will be the same as today, but if patterns are added to testOnly, matching native libs (from externalNativeBuild or otherwise) will be excluded from the prod APK/AAR and included in the android test APK.

Yes, something like this could work.

+1 from me as well. That'd work great!

Hi Scott, did you get a chance to write a doc for the proposal you have in comment#14?

Dan, this is something we still need, right? Is it a blocker for b/37081137 (Unit test support)?

Hi Emre, sorry for the delay... I just got back from leave this week.

I haven't written a doc yet, but I'll prioritize it now that I'm back.

Yes, still needed.

Sorry again for the delay. I'm looking into this now, but I want to clarify what's needed.

The proposal in comment#14 would provide a mechanism to include particular .so files in the androidTest APK and exclude them from the main APK. For your use case, would you need this to work for .so files from dependencies and/or the jniLibs source directory, or do you only need it to work for .so files from the external native build?

In a comment in ag/20833989, you suggest that it might be sufficient for your use case to simply include the .so files in both the androidTest APK and the main APK. I don't think we'd want to do that by default in AGP, but it might make sense to add a flag to enable that behavior instead of adding the DSL if that solution seems equally good to you. WDYT?

We probably do need it for both dependencies and jniLibs. My first instinct was that a dependency that made test-only libraries available in an AAR should do so in a separate AAR, but that's actually what the problem is here. That separate test AAR would need to be pulled in as part of the same gradle configuration, since for externalNativeBuild, there isn't a distinct test configuration. Tests are built by the same configuration as the implementation. That's actually partly the case in the motivating example for this bug. The dependency we first discovered this with was junit-gtest. We don't want those libraries in the real APK, but we do need them at test time.

In a comment in ag/20833989, you suggest that it might be sufficient for your use case to simply include the .so files in both the androidTest APK and the main APK.

Sorry. Not sure what I was thinking when I left that comment, but that's just wrong. The problem we have is that when we build a library that contains native tests, e.g. libapp_test.so, libapp_test.so will end up in the prod APK. That's useless bloat that just increases on-disk and download side for users. If we try to exclude the library from the APK, the tests no longer work because the test library is not available, because it's not in the test APK either.

This is a problem both for app APKs and for library AARs. For example, junit-gtest originally shiped its own tests that consumers needed to filter out of their own APKs: https://github.com/android/ndk-samples/blob/be0ebf254566cffecde52e81691dd8dc4bf92c07/unit-test/app/build.gradle#L42-L48. Until this bug is fixed, junit-gtest only had two options:

1. Remove its own tests
2. Move its tests to a separate gradle project

It went with option 2, but it's cumbersome and isn't going to be a viable option for every project.

Internal design doc here. PTAL :)

As mentioned in the doc, there is a workaround to exclude .so files from release variants that can be used in cases where devs are not testing against the release variant.

For example, adding this block to Junit-Gtest's build.gradle file will exclude the unnecessary test .so files from the published AAR:

androidComponents {
    onVariants(selector().withBuildType("release"), {
        packaging.jniLibs.excludes.add("**/libtestartifact.so")
        packaging.jniLibs.excludes.add("**/libtest.so")
    })
}

Question after reviewing the above doc with AGP team:

Would it be possible for the external native build to instead detect and output test-only .so files to a separate external native build output directory?

This has been fixed here. The fix will be in AGP 8.4.0-alpha12 8.4.0-alpha13.

Usage is the same as suggested in #14:

android {
    packaging {
        jniLibs {
            testOnly += '**/testStuff.so'
        }
    }
}

Thank you for your patience while our engineering team worked to resolve this issue. A fix for this issue is now available in:

• Android Studio Jellyfish | 2023.3.1 Canary 12
• Android Gradle Plugin 8.4.0-alpha12

We encourage you to try the latest update.

If you notice further issues or have questions, please file a new bug report.

Thank you for taking the time to submit feedback — we really appreciate it!

Re: #24 and #25 - sorry, my mistake, it's AGP 8.4.0-alpha13 (not alpha12).

https://github.com/android/ndk-samples/pull/1085 updates the NDK samples. Sorry that took so long. Thanks for getting this implemented!