IssueTracker

any equivalent/alternative approach for test fixtures / sharedTest best practice on android?

I guess a "common pattern" is to create a test-fixtures Gradle module and create the test fixtures in this module. This module can depend on other modules via the implementation configuration while the other modules can be used this test-fixtures via the testImplementation configuration. And no, this will not lead to a circular dependency :). Gradle is smart enough to handle this for you.

Just want to add a visualization of the idea. See the attachment or here - modules-for-test-fixtures

What I meant is not to create a new module for each but rather create one module for all of your test fixtures.

We've been doing this but it is truly a workaround. The problem is that the classes will be seen as production code in the IDE. These are test code and they will also be seen as untested code in the test coverage report.

We use this workaround, and while there have been some "fun" moments during AGP upgrades, it's quite stable and non-invasive solution:

// gradle/export-test-helpers.gradle

/*
 * Creates a new configuration and publishes a test utility JAR so you can depend on it from other modules.
 * For example:
 * testImplementation project(path: ':feature:other-module', configuration: 'test-helpers')
 * 
 * To export test-helpers JAR from any com.android.library module, add to its build.gradle:
 * apply from: rootProject.file('gradle/export-test-helpers.gradle')
 */

task jarTestHelpers(type: Jar, dependsOn: 'assembleDebugUnitTest') {
    from "$buildDir/intermediates/javac/debugUnitTest/classes"
    from "$buildDir/tmp/kotlin-classes/debugUnitTest"

    // exclude classes ending in 'Test' or 'TestKt' and any inner or anonymous classes
    exclude '**/*Test.class'
    exclude '**/*TestKt.class'
    exclude '**/*Test_*.class'
    exclude '**/*Test$*.class'
    exclude '**/*TestKt$*.class'

    // as of AGP 3.4+ R classes are generated within intermediates/javac/debugUnitTest/compileDebugUnitTestJavaWithJavac/classes
    // when using android.testOptions.unitTests.includeAndroidResources = true, this causes conflicts with merging of resources
    // the ids in these R files are different to those in the app/androidTest apk
    exclude '**/R.class'
    exclude '**/R$*.class'

    includeEmptyDirs false
    classifier = 'test-helpers'
}

configurations {
    maybeCreate 'test-helpers'
}

artifacts {
    add 'test-helpers', jarTestHelpers
}

One downside: it's not aware of flavors/build types/variants, so it'll always use the debug classes. This could probably be alleviated with some loops on libraryVariants and a utility function for adding all the right dependencies to the right configurations.

Thanks for this samples: https://issuetracker.google.com/issues/139438142#comment15

The workaround for build variants support, might a little bit tedious to store it on environment variable, thus the above code modified into this:

def buildVariant = System.getenv("BUILD_VARIANT_ENV") // beware of null env variable, might need to fallback to default env instead

debugUnitTest -> ${buildVariant}UnitTest

it's good when running on CI, but when development, the hardest part is changing env variable every time we want test another build variant

#17 You could try looking at using one of these https://docs.gradle.org/current/javadoc/org/gradle/StartParameter.html#getTaskRequests-- (mind you that these could be abbreviated, like :mod:asDeb instead of :module:assembleDebug) https://docs.gradle.org/current/javadoc/org/gradle/api/execution/TaskExecutionGraph.html#whenReady-groovy.lang.Closure- (not sure if delaying it this far works with configurations/dependencies)

Hate to be that guy, but is there an update after having this ticket on P1 for over a year now? Testing story on Android is far from ideal as it is, and this feature would certainly help.

My understanding is that Kotlin support is lacking for this feature as well

https://youtrack.jetbrains.com/issue/KT-34901

Is this issue tracking the work for supporting testFixtures for emulator tests, JVM tests, or both?

Would love to have support for both JVM and instrumentation tests.

Both unit tests and instrumentation tests should be able to consume the test fixtures, so yes this issue is for both

The feature is currently incubating for Android projects, the IDE support for both java-test-fixtures's and the android testFixtures source sets is under development.

If you want to try it out in AGP 7.1 without studio's support, you can add the following to your project's buildscript

android {
    testFixtures {
        enable true

        // enable testFixtures's android resources
        // androidResources true
    }
}

Please, let us know when Android Studio support is ready to be tried in a Canary, I'll be happy to start testing and using it.

Alpha versions of 7.1 are available, the latest one is 7.1.0-alpha12

Currently the kotlin plugin doesn't support compiling android testFixtures kotlin sources

Starting Chipmunk alpha 5 which will be released next week, Android Studio supports both android and java testFixtures.

To enable testFixtures in your android library module, add the following to your build.gradle file

android {
    testFixtures {
        enable true

        // enable testFixtures's android resources
        // androidResources true
    }
}

You can then add your testFixtures sources to $projectDir/src/testFixtures/java and $projectDir/src/testFixtures/res. By default, the testFixtures will have a dependency on the main sources, while both test and androidTest will have a dependency on the testFixtures sources. To add a specific dependency for the testFixtures you can use the testFixtures prefix + the configuration type, same as what you would do for test and androidTest

dependencies {
    testFixturesApi 'com.google.truth:truth:0.44'
    testFixturesImplementation 'com.google.code.gson:gson:2.2.4'
}

To consume the local library's testFixtures in another module, you can use Gradle's helper method testFixtures same as what you would do for a java library

dependencies {
    testImplementation testFixtures(project(":lib"))
}

By default, publishing your library also publishes the testFixtures aar with the main library, Gradle metadata file will contain information for Gradle to be able to consume the right artifact when requesting the testFixtures component.

android {
    publishing {
        singleVariant("release")
    }
}

To disable publishing the testFixtures aar of a library, add the following to your build.gradle

afterEvaluate {
    components.release.withVariantsFromConfiguration(
        configurations.releaseTestFixturesVariantReleaseApiPublication) { skip() }
    components.release.withVariantsFromConfiguration(
        configurations.releaseTestFixturesVariantReleaseRuntimePublication) { skip() }
}

Consuming the testFixtures aar of a published android library is the same as java libraries

dependencies {
    testImplementation testFixtures('com.example.company:publishedLib:1.0')
}

By default, lint will analyze test fixtures sources. You can configure lint to ignore test fixtures sources as follows:

android {
    lint {
        ignoreTestFixturesSources true
    }
}

Note that currently the kotlin plugin doesn't support compiling the testFixtures kotlin sources. You can follow the updates on kotlin support on the youtrack issue KT-50667.

Please try out the feature if you can and file a new bug if you encounter any problem with it.

#36 You'd rather want them to rush experimental features in stable builds?

Not the original poster, but I'd rather have just 2 AS channels, IDEA style, where the EAP gets becomes a beta then a release 4 times in a year rather than once a year with never ending compromises between 3 channels.

That way, we'd get new features like testFixtures stabilized within 3 months, even if behind a flag.

BTW, the feature was requested over 2 years ago, waiting another year is quite demoralizing as to when the community can fully take advantage of something that's been in the JVM world for a while.

Priorities... we're all devs, know how it works. At least this one got to the top, good to see investment in better ways of testing. We should be saying thanks, but it's never enough, isn't it? 😅

So, Thank You! for making it work, whenever it gets released.

Also pretty good quick-start at #35, will that get into release notes of alpha05 and Chipmunk final too?

To prevent flaming: I'm in full support of smaller, more focused releases with a faster cadence as described in #38.

(I'm sorry this is slightly off-topic)

AGP is developed entirely in open source, if anyone doesn't like the speed of execution they're welcome to submit a CL, I imagine.

Also, I've been on the Chipmunk canary since Alpha 1, it's remarkably stable (for my workflow), and for the bugs I have found (with previous AS+AGP alpha/betas) the team has been incredibly responsive.

Either join the canaries or accept that the features won't land in prod for ~6-9mo. You can't have it both ways :)

That setup doesn't work for me.

I have enabled test fixtures - https://github.com/emartynov/MultimoduleExample/blob/test/database/build.gradle

I run ./gradlew :dat:bundleDebugTestFixturesAar

I see database-debug-testFixtures-testFixtures.aar output however, it is empty aar file - no classes.

#43, can you try with .java class?

Thank you for your patience while our engineering team worked to resolve this issue. A fix for this issue is now available in:

• Android Studio Chipmunk Canary 6 (2021.2.1.6)
• Android Gradle Plugin 7.2.0-alpha06

We encourage you to try the latest update.

If you notice further issues or have questions, please file a new bug report.

Thank you for taking the time to submit feedback — we really appreciate it!

Given that Kotlin and lint are not supported in testFixtures are there separate issues tracking their development? Or should we expect updates here?

Lint support is already implemented as published on Android Studio Chipmunk preview features page.

By default, lint will analyze test fixtures sources. You can configure lint to ignore test fixtures sources as follows:

android {
    lint {
        ignoreTestFixturesSources true
    }
}

You can follow the updates on kotlin support on the youtrack issue KT-50667.

I have added ticket that is probably duplicate to this https://issuetracker.google.com/issues/219687418

Do you have a testFixturesApi dependency on these libraries?

I wasn't able to reproduce the problem locally with a similar setup. Could you file a new bug with a small project that reproduces the issue?

I found that it only works in limited cases. If I define a java library with fixtures I can resolve those to an app. However, if I define an android library with fixtures enabled that depends on that same library it breaks. The IDE resolves imports so that there are no apparent errors but when I try to compile those same fixtures it breaks. The same trying to use fixtures from that Android library in app tests. The IDE resolves the class so there is no apparent error until compile time.

You can see the failures by trying to run any of the included tests in the attached zip.

comment#58 It seems that the unresolved class in your project is a kotlin class, writing it in java fixes the problem.

As mentioned before, kotlin support for android testFixtures is still a WIP, you can track the progress on the youtrack issue KT-50667.

Given that Kotlin is a suggested language that many Google features are now written in this is obviously not an acceptable answer. You should not have listed testFixtures as a supported feature in the 7.2.0 AGP release and it should be listed as a caveat in the release notes.

Furthermore, as I pointed out the testFixture written in Kotlin in the java library works just fine in a test using the Android plugin with an application module but not a library module. That inconsistency seems strange to blame on JetBrains.

Consuming testFixtures of a java library should work whether it's written in kotlin or java (in both android app and library modules). I've verified that it's working with a dummy project. Can you file a bug with a project reproducing the issue?

#62 I think you're bugging the wrong ticket. Kotlin support is in Kotlin Gradle Plugin: https://youtrack.jetbrains.com/issue/KT-50667

@comment#64 We all understand that it doesn't work. comment#63 is telling you that there is nothing that Google/Android can do to resolve this, because it is a Kotlin bug, not an Android bug.

Therefore the only way to fix it is to resolve the Kotlin ticket: https://youtrack.jetbrains.com/issue/KT-50667

Now it's on Google to solve this for Kotlin too. They have plans to make it part of AGP.

Please vote on https://issuetracker.google.com/issues/259523353 to track your interest!

@google dupe this?

JetBrains has rejected https://youtrack.jetbrains.com/issue/KT-50667 as a third-party issue with the resolution that the problem should be solved in AGP. This issue has been unresolved for over three years. Please fix it!

We've found an issue with the IDE support for consuming testFixtures sourcesets in an android project if the version property is set. A fix is already submitted and will land in AGP 8.0. If you are running into that issue, you can workaround it by adding the following snippet to your build.gradle file where you have testFixtures enabled

afterEvaluate {
    project.configurations.configureEach {
        def testFixturesCapability =
                it.getOutgoing().capabilities.find { it.name.endsWith("-test-fixtures") }
        if (testFixturesCapability != null) {
            it.getOutgoing().capability(
                    new org.gradle.internal.component.external.model.ImmutableCapability(
                            testFixturesCapability.group, testFixturesCapability.name, null))
        }
    }
}

I've tried the above but it doesn't seem to be enough. The IDE does not recognize the src/testFixtures directory and it does not produce compiled test fixture outputs.

for kts you might want to replace find with outgoing.capabilities.firstOrNull { it.name.endsWith("-test-fixtures") }

There's no difference. That function is equivalent to find()

You're right sorry, you can't apply java-test-fixtures plugin in an android library as it depends on the java plugin being applied. To enable testFixtures in an android library you need to set android.testFixtures.enable = true. Note that kotlin plugin support for android testFixtures sources is not available yet.

Got it. Is that support what's coming in 8.0?

Unfortunately not, the fix mentioned in comment#68 is to a bug for versioned android libraries that have testFixtures enabled.

The support for kotlin sources requires non-trivial migration for the whole kotlin gradle plugin away from the Android gradle plugin old variant and dsl APIs. The process of the migration is tracked in https://issuetracker.google.com/259523353

Maybe we could also share raw assets from testFixtures?

Not sure what setting androidResources = true should do.

For me it currently breaks the build with a "Must provide a folder to export generated class list" message.

The cause for this seems to be a clash with databinding through AGP 7.2.2 (issues reproduces with never 7.* versions also, cannot yet update to 8.+) when I try building the project. Note that there are no layouts in the testFixtures sourceset.

> Task :library:bundleReleaseAar
> Task :library:extractReleaseTestFixturesAnnotations

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':library:javaPreCompileReleaseTestFixtures'.
> Error while evaluating property 'annotationProcessorClassNames' of task ':library:javaPreCompileReleaseTestFixtures'.
   > Failed to calculate the value of task ':library:javaPreCompileReleaseTestFixtures' property 'annotationProcessorClassNames'.
      > failure, see logs for details.
        Must provide a folder to export generated class list
...
Must provide a folder to export generated class list
	at android.databinding.tool.util.L.printMessage(L.java:134)
	at android.databinding.tool.util.L.e(L.java:107)
	at android.databinding.tool.util.Preconditions.check(Preconditions.java:25)
	at android.databinding.tool.CompilerArguments.<init>(CompilerArguments.kt:93)
	at android.databinding.tool.CompilerArguments.<init>(CompilerArguments.kt:31)
	at com.android.build.gradle.internal.tasks.databinding.DataBindingCompilerArguments.asArguments(DataBindingCompilerArguments.kt:163)
	at com.android.build.api.component.impl.AnnotationProcessorImpl$finalListOfClassNames$1.transform(AnnotationProcessorImpl.kt:87)

Assets should be supported when you enable androidResources in testFixtures

Please file a new bug with the issue you're facing, it'd be helpful if you can provide a stacktrace or a reproducing project.

It seems that the original issues described by this bug have been resolved. If there are still other issues, please file a separate bug. If there are still any issues with the java-test-fixtures plugin, please let me know.

What AGP/studio version?

All the fixes mentioned previously should be in AGP 8.0 and above. For your previous comment, I believe it is still pending on the issue mentioned in comment#74.

Re # 80 - Kotlin support for test fixtures is blocked by Issue 259523353... working on support in AGP 8.5

AGP 8.5 is now released as stable 🙌

Can someone affirm that AGP 8.5 fixed this?

Yes, it works with AGP 8.5, we sill see issues with IDE (IDE warning about usage of internal members in testFixture code, but compilation works fine)

So, the IDE showing warning about internal entities access from testFixtures - is it an issue or by design?

Re: #83 - Yes, there is basic support for Kotlin in testFixtures with AGP 8.5. See b/259523353#comment21.

Re: #84 and #85 - That's a bug in the IDE; I've created b/352353030 to track it.

I was able to get testFixtures working, please check this comment: https://issuetracker.google.com/issues/219687418#comment9