IssueTracker

If it helps, I have a screencap attached that shows the build/.transforms dir of the module that has the mixed source set. You can see that the contents differ dramatically based on whether or not the ObserverFix.java file is present. I will note that the actual jar produced by normal javac/kotlinc has all the expected class files.

At this point, I don't know if the bug is in:

1. AGP
2. Gradle
3. D8

I think I know more or less where the issue is, but I haven't precisely pinpointed the problem just yet. Sorry of this comment is a bit messy, but I haven't delved this deep into AGP code in a while and I wanted to do a brain dump before the weekend.

In AGP 4.2.0, in SubtractingArtifactCollection, line 89, there is a function initArtifactResult(), which I believe to be buggy. Here's why, to summarize:

1. mainArtifacts contains our artifact at build/.transforms/51c03e... This is correct.
2. removedArtifacts does NOT contain our artifact. This is correct.
3. artifactResults does NOT contain our artifact. This is incorrect.

So the logic in initArtifactResult() is wrong: it is producing an incorrect artifactResults object, which is then used to filter mainArtifacts.

This buggy function is using the ComponentArtifactIdentifer (it.id) rather than the path, so maybe there's an ID collision?

On line 102, there is a lambda

mainArtifacts.artifactFiles.filter { f -> artifactFileSet.value.contains(f) }

when f = /Users/trobalik/Development/android-register/common/rx2-utilities-jvm/testing/build/.transforms/51c03e5924694fbadf87d9d7444d7bb0/transformed/main, that filter should be true, but is actually false. This f points to a file hierarchy that ultimately contains com/squareup/rx2/utilities/DeprecatedRxKillSwitchRule.dex. That class was written in Kotlin.

artifactFileSet does not contain the artifact it should, because artifactResults does not contain the artifact it should, because initArtifactResult() has a bug (I assume!).

I did a build comparison with Gradle Enterprise. Recall that the issue description pointed to a problem with mixed Java/Kotlin source.

Changing inputs for dexDirs
With Java and Kotlin:
  common/rx2-utilities-jvm/testing/build/.transforms/cce801b89890a0d5783032b8034a8667/transformed/main
With only Kotlin:
  common/rx2-utilities-jvm/testing/build/.transforms/51c03e5924694fbadf87d9d7444d7bb0/transformed/main
  common/rx2-utilities-jvm/testing/build/.transforms/d7b5653982cffdfe5eb48bf61ce9dd32/transformed/empty

Note those hashes. When I look in artifactFileSet, I see a hit for cce801b89890a0d5783032b8034a8667, but nothing for 51c03e5924694fbadf87d9d7444d7bb0. These transform artifacts come from different builds. I have been building with clean --no-build-cache, but it doesn't seem to matter, these transforms stay behind. I haven't manually deleted them yet because I'm still debugging.

Another round of debugging, and some deeper results:

  initArtifactResult()
    mainArtifacts.artifacts.filter { it.id.displayName.contains("rx2") }
      - main (project :common:rx2-utilities-jvm:testing)
        - This has a single file that points to ObserverFix.dex alone.
      - 5 other elements
    mainArtifacts.artifactFiles.files.filter { it.absolutePath.contains("common/rx2") }
      - [0] points to a dir that contains only ObserverFix.dex (the sole file from Java source).
        build/.transforms/cce801b89890a0d5783032b8034a8667/transformed/main
      - [1] points to a dir that contains all the Kotlin files.
        build/.transforms/51c03e5924694fbadf87d9d7444d7bb0/transformed/main
      - 3 other elements

The code snippets are what I ran in the AS debugger.

When I look at mainArtifacts.artifacts, I see only a single matching artifact, and it has a single file (based on Java source). When I instead look at mainArtifacts.artifactFiles.files, I see two files of interest, both directories. One points to the Java-based dex files, and the other to the Kotlin-based dex files.

In other words, artifacts and artifactFiles seems to be pointing to different sets of objects? That's weird?

I'm not sure if I mentioned this elsewhere, but the task I'm executing is :path:to:an:app:mergeLibDexDebugAndroidTest

I repeated the above, but this time after removing the Java source and having only Kotlin in the module in question.

  initArtifactResult()
    mainArtifacts.artifacts.filter { it.id.displayName.contains("rx2") }
      - main (project :common:rx2-utilities-jvm:testing)
        This has a single file that points to all the classes we expect.
        build/.transforms/51c03e5924694fbadf87d9d7444d7bb0/transformed/main
        This transform directory is identical to the one from the bad build with the Kotlin files.
      - 6 other elements
    mainArtifacts.artifactFiles.files.filter { it.absolutePath.contains("common/rx2") }
      - [1] points to a dir that contains all the classes we expect.
        build/.transforms/51c03e5924694fbadf87d9d7444d7bb0/transformed/main
        This transform directory is identical to the one from the bad build with the Kotlin files.
      - 4 other elements
    ImmutableSet.copyOf(mainArtifacts.artifacts.filter { !removed.contains(it!!.id) }).filter { it.file.absolutePath.contains("common/rx2") }
      - [1] points to a dir that contains all the classes we expect.
        build/.transforms/51c03e5924694fbadf87d9d7444d7bb0/transformed/main
        This transform directory is identical to the one from the bad build with the Kotlin files.
      - 1 other element

This seems to confirm that the transforms are stable, as they always produce the same hash, and produce separate outputs for each type of source (two outputs, one each for Kotlin and for Java). What's weird is that the final ArtifactCollection looks different depending on whether you look at it via mainArtifacts.artifacts vs mainArtifacts.artifactFiles. I'm not sure if this is an AGP bug or a Gradle bug.

I do note that the AGP source seems to assume they are basically identical. Consider these two functions:

// Uses `artifact.file
private fun initArtifactFileSet() = ImmutableSet.builder<File>().apply {
    for (artifact in artifactResults.value) {
        add(artifact.file)
    }
}.build()

// uses `ArtifactCollection.artifactFiles`
private fun initFileCollection() = objectFactory.fileCollection().from(
    mainArtifacts.artifactFiles.filter { f -> artifactFileSet.value.contains(f) }
).builtBy(mainArtifacts.artifactFiles, removedArtifacts.artifactFiles)

I executed :path:to:an:app:mergeLibDexDebugAndroidTest again, this time with AGP 4.1.3, and saw just a single classes.dex file in the build/.transforms directory. Inspecting this with dexdump showed me all the classes, both Java and Kotlin, were in this one dex file.

Thanks for filing this Tony. Did you manage to reduce this to a small sample project that you could share?

Update: I managed to reproduce. Setup requires:

• app with minSdkVersion 24+ (so that desugaring classpath is not needed)
• lib project without Android with 2 classes: 1 Kotlin and 1 Java
• app with androidTestImplementation project(":lib")
• android test apk will not have all classes from lib

Here is project that reproduces the issue. Running ./gradlew aDAT creates android test APK with missing classes.