IssueTracker

I took a quick look at MergeFileTask and think what it's doing is quite deterministic: it's preserving the order of the given input files.

Sorting sometimes introduces a bug rather than fixing it, because with sorting, the original order of the input files is not preserved (especially if these files come from the Gradle cache with SHA1 hashes in their names: if the SHA1 in a file's path has changed, sorting will reorder it among the other files, which is unexpected).

I'm trying to confirm that this is due to inconsistent file ordering when merging via monkeypatch in our internal plugin.

Please let us know when you get a result from this experiment. It would be great if you could figure out and attach the exact file contents of the two files which Gradle reports to be different.

I'm renaming the bug title to reflect the issue you're having, pls correct me otherwise.

Sorry I wasn't able to access this before. I was able to confirm it was due to ordering changes, and confirmed monkeypatching it to sort first resolved it. It also took our cached CI lint time down from 16min to ~5min

Sorry we made this issue internal by mistake.

Could you share your patch?

Also, would you be able to share with us the before and after contents of the file? As mentioned at comment #2, I think sorting is not entirely safe, so it's better for us to understand the issue first before fixing it.

The fix is: https://github.com/slackhq/slack-gradle-plugin/pull/227/files.

Nice, thx!

@Zac: Could you add some more debugging to figure out the root cause? inputFiles is expected to have stable order, so sorting it will just hide the real cause. Can you find out in what cases the order of inputFiles has changed?

(Alternatively, you can send me a reproducible project and I will do all the debugging, but I suspect that might be a bit difficult for you.)

I couldn’t find anything that ensured stable order, and in our case we saw differences on CI and local machines (Linux and macOS), likely due to different ordering due to different filesystem casing.

I’ll see if I still have the files compared before

(I reached out to you on Slack, but I'll add it here too so it's easier to keep track of the status of this bug:)

The core of the issue is that :libraries:core-lib:api:mergeReleaseGeneratedProguardFiles (with type MergeFileTask) produces a non-determnistic proguard.txt file at libraries/core-lib/api/build/intermediates/generated_proguard_file/release/proguard.txt.

So, can you show me (1) the input files of that task, and (2) the contents of the proguard.txt file?

To get (1), you can use the following script:

// Add this to libraries/core-lib/api/build.gradle
afterEvaluate {
    tasks.register("printTaskDetails") {
        def task = tasks.named("mergeReleaseGeneratedProguardFiles")
        dependsOn(task)
        doLast {
            println "MergeFileTask.inputFiles:"
            (task.get() as com.android.build.gradle.internal.tasks.MergeFileTask).inputFiles.files.each {
                println(it.path)
            }
        }
    }
}

Then run :libraries:core-lib:api:printTaskDetails.

(Got the data from Zac on Slack.)

This is caused by the use of FileTree in our code, and Gradle's FileTree.files are currently non-deterministic (just like java.io.File.listFiles() or java.nio.file.Files.walkFileTree()).

I've asked Gradle to fix it here: https://github.com/gradle/gradle/issues/21379#issuecomment-1452019038.

In the mean time, we might need to fix our use of FileTree at MergeGeneratedProguardFilesCreationAction (and also in general).

We have fixed this issue with https://android.googlesource.com/platform/tools/base/+/d238000026b21703dba648a261d3ecddb6421529. It will be included in AGP 8.1.0-beta01. Thank you for this report!