Assigned
Status Update
Comments
ak...@google.com <ak...@google.com>
ru...@gmail.com <ru...@gmail.com> #2
It looks like it is because of anti-aliasing differences between your host and Github action runner, we are seeing the same between MacOS dev machines and Bitrise runner causing all screenshot tests to fail. This could be resolved by this Feature Request: https://issuetracker.google.com/issues/342461560 allowing for some slight pixel variation due to different platform implementations
hu...@google.com <hu...@google.com>
ma...@gmail.com <ma...@gmail.com> #3
Allowing for slight pixel variation kind of breaks the whole intent of screenshot testing. This would be better resolved by ensuring the anti-aliasing is the same across all platforms.
as...@gmail.com <as...@gmail.com> #4
I understand the concern about maintaining the integrity of screenshot testing. However, it's important to note that anti-aliasing like this is typically managed at the OS graphics layer. I think achieving consistent anti-aliasing across all environments and platforms might not be feasible. Instead, allowing for slight pixel variations in screenshot testing might be a more practical approach given these inherent differences. One that other screenshot testing platforms have also used. This way, we can accommodate the natural variations while still effectively validating the visual aspects of our applications. If it's configurable, we could make the default threshold no differences and for people who do all their testing on a single platform that would likely be sufficient. This just opens up options for everyone else.
tr...@liip.ch <tr...@liip.ch> #5
We've used the imageDifferenceThreshold settings in our build.gradle file and it works fine. But sometimes there are bigger differences between platform results, as you can see in the screenshot above.
I'm not sure that these settings will fix the anti-aliasing issues :/
I'm not sure that these settings will fix the anti-aliasing issues :/
da...@itau-unibanco.com.br <da...@itau-unibanco.com.br> #6
Comment has been deleted.
da...@itau-unibanco.com.br <da...@itau-unibanco.com.br> #7
Do we have any news? Is the compose team working on any definitive solution?
ge...@paymentsense.com <ge...@paymentsense.com> #8
Same issue with Bitrise. Would love to see an update on this one :D
ni...@gmail.com <ni...@gmail.com> #9
Do we have any news?
Description
I've been testing Compose screenshot test feature.
It's working well locally but, when testing with a github action the test fail and in the report the error is very minimal.
STEPS TO REPRODUCE:
1. Create a screenshot test
2. Ask github action to run the test
3. Get a failure report cause of minus error in report from github action
Put the studio project as a zip and the reports as a zip