IssueTracker

Ideally there would be two thresholds we could customize:

1. Pixel difference threshold, i.e how much any given pixel can change before it's considered different from the target pixel.
2. Total difference threshold, i.e how much of the total screenshot can be different from the target

In my situation, the ideal settings (and what I currently use on other tools/platforms) is ~1% pixel difference threshold to ignore text rendering and anti-aliasing differences that occur across hardware and a 0% total difference threshold so that any change more significant than that is flagged immediately.

Using a single total difference threshold to ignore aliasing is unideal, because on screenshots with text this value can end up being very large to account for every possible pixel that could receive different aliasing. So large that other actually important differences may sneak by undetected, e.g. the height or color of a divider line.

I like the idea of having multiple thresholds. Could you please explain option 2 (total diff) in more detail?

Option 2 (total difference) is the percentage of the image that is allowed to be 'different' from the reference image. For example, if the reference image is 10x10 pixels, a 5% threshold would allow the test to pass even if up to 5 pixels do not match the reference. In my opinion, this is the least useful threshold option, even though most other screenshot testing tools have it, as it's the most straightforward threshold to implement. I avoid using this kind of threshold because it doesn't scale well with large screenshots. If I'm taking a screenshot of an entire screen of my app, say 1080x1920, even a small threshold of 0.5% allows for over 10,000 pixels to not match the reference. This means that it's possible the test won't catch important changes like the color of a border, a change of font, or potentially even a small icon disappearing.

What I find to be a more useful option is configuring when any given pixel is considered different from the reference pixel. For example, if I have a screenshot of an black circle on a white background, my laptop (an Apple Silicon MacBook) will anti-alias the image slightly differently than a colleague's Windows desktop or the Linux instance running my tests for CI. Even though to the human eye the image is identical, it will fail due to the minor differences in the gray pixels used to anti-alias the edge of the circle. In this situation, I don't care if a pixel that was #9A9A9A is now #9B9B9B; they're essentially identical, and I'd want the test to ignore that difference.

Trying to work around this kind of issue with option 2 isn't ideal. Larger screenshots have more anti-aliasing, which necessitates a larger total difference threshold to ignore the larger number of imperceptible differences, which in turn just increases the potential for a meaningful change to not be detected.

seems someone is working on this check out this commit. Tried it in alpha-05 but didn't work so I suspect it will be available in alpha-06.

For configuring check the ScreenShotTest.kt file

Diff threshold feature is enabled since alpha06 version. This does not solve the platform related rendering differences. Tracking that as a separate issue and closing this

Hi, I was looking into this issue again and it appears to have been fixed after updating Kotlin Coroutines to version 1.3.5-native-mt. Note that this coroutines version is based on this WIP pull request (a preview release): https://github.com/Kotlin/kotlinx.coroutines/pull/1648

The normal "stable" coroutines version 1.3.5 still crashes. That means that something changed in the native-mt branch which fixes this issue.

The native-mt branch (see PR) has some changes to how the classes/functions CoroutineStart, startCoroutineUndispatched, etc are called in the library. Those changes appear to have fixed/workaround this ClassCastException issue.

We have a similar crash but with a different class (kotlinx.coroutines.channels.ProducerCoroutine being cast to kotlin.jvm.functions.Function2)

I have updated to Coroutines 1.3.6 and will release to production soon. I will report back if this issue is solved then.

Android Version: 100% Android 7
Devices:

• TCL 5049S
• Asus ZenFone 3 Max (ZC520TL)
• Hisense F24
• TECNO i5-Pro
• Sony Xperia XA
• STF DUO
• OPPO CPH1717

Stack Trace:

Fatal Exception: java.lang.ClassCastException: k.a.l2.u cannot be cast to kotlin.jvm.functions.Function2
       at kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt$createCoroutineUnintercepted$$inlined$createCoroutineFromSuspendFunction$IntrinsicsKt__IntrinsicsJvmKt$4.invokeSuspend(IntrinsicsKt__IntrinsicsJvmKt.java:199)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(BaseContinuationImpl.java:33)
       at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.java:56)
       at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.java:571)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.java:738)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.java:678)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.java:665)

I still see the issue in 1.3.6 unfortunately.

The Coroutines native-mt branch has not been merged into stable yet (in your case version 1.3.6). That will probably not happen anytime soon, unfortunately. See: https://github.com/Kotlin/kotlinx.coroutines/pull/1648

Does AGP 4.0 stable fix this? https://issuetracker.google.com/issues/154531810

I saw lots of this type of crash in OPPO Android 7.

Unsure if this will help with discovering the VM issue, but I started getting a new crash from the manufacturer (Tecno) on Android 7:

Fatal Exception: java.lang.AbstractMethodError: abstract method "int kotlin.coroutines.jvm.internal.SuspendLambda.getArity()"
       at kotlin.jvm.internal.TypeIntrinsics.getFunctionArity(TypeIntrinsics.java:259)
       at kotlin.jvm.internal.TypeIntrinsics.isFunctionOfArity(TypeIntrinsics.java:336)
       at kotlin.jvm.internal.TypeIntrinsics.beforeCheckcastToFunctionOfArity(TypeIntrinsics.java:341)
       at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(UndispatchedKt.java:91)
       at kotlinx.coroutines.flow.internal.FlowCoroutineKt.flowScope(FlowCoroutineKt.java:33)
       at kotlinx.coroutines.flow.internal.ChannelFlowTransformLatest.flowCollect(ChannelFlowTransformLatest.java:24)
       at kotlinx.coroutines.flow.internal.ChannelFlowOperator.collectTo$suspendImpl(ChannelFlowOperator.java:128)
       at kotlinx.coroutines.flow.internal.ChannelFlowOperator.collectTo(ChannelFlowOperator.java:1)
       at kotlinx.coroutines.flow.internal.ChannelFlow$collectToFun$1.invokeSuspend(ChannelFlow.java:53)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(BaseContinuationImpl.java:33)
       at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.java:56)
       at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.java:571)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.java:738)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.java:678)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.java:665)

eygraber, the crash reported in comment #35 looks like a different issue. Can you open a separate issue on that? If possible provide the APK which cause the crash. Also mention if this is only happening on some devices together with specific device information on the devices where this crash has been seen.

I've seen a similar issue affect this device:

Brand: Blackview

Model: BV8000 Pro

Android version: 7.0

Here's the stacktrace that I got:

Fatal Exception: java.lang.ClassCastException: a0.a.t2.s cannot be cast to kotlin.jvm.functions.Function2
       at kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt$createCoroutineUnintercepted$$inlined$createCoroutineFromSuspendFunction$IntrinsicsKt__IntrinsicsJvmKt$4.invokeSuspend(IntrinsicsKt__IntrinsicsJvmKt.java:205)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(BaseContinuationImpl.java:33)
       at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.java:56)
       at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.java:571)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.java:738)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.java:678)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.java:665)

I think if you get that device (can be found for cheap on Ali Express I guess), and you run the apk provided in comment#1, you'll be able to reproduce the issue @comment#36.

Just wanted to give an update on this old issue. For the past few years I have been using the native-mt versions of coroutines, where the issue was not reproducible, so that worked fine for me. However, recently support for the native-mt versions has ended and everyone is supposed to use the normal coroutines releases now. So I have changed my coroutines version to "1.6.1".

Tried reproducing the crash with "1.6.1" and was not able to get my app to crash. I used the steps in comment #16. Of course, a lot of my code and coroutines code has changed in the meantime.

I am no longer able to reproduce the issue in my current project, but I am not sure if it is because of my code changes or coroutines code changes. Or maybe even because I am using a newer R8 version now (3.3.26-dev).

In case the crash would appear again in the future, I will keep you updated and hope to find some keep rules that work.