My favorite testing approach in Flutter

[@FlutterBountyHunters]

Ignoring the variance will definitely bite you. I've had goldens that fail with a 0.0% difference where the mismatch was actually relevant. In Super Editor we've taken two steps to deal with this. First, we created a golden runner that uses Docker to get as close to matching platforms no matter who runs the tests. Second, for tests that still don't match, we created a golden matcher that lets you specify the number of mismatched pixels instead of mismatched percent, because in practice, percent diffs are unpredictable. If your golden mismatch is 103 pixels due to platform differences, then you can allow up to exactly 103 pixels difference. Otherwise it fails.

[@RobertBrunhage]

Pinning this. interesting, do you have some links? Would love to learn more here and make sure I share a better approach next time. Seen this approach in multiple large codebases as well so for sure those small differences could be valid, but they might also be so small that they haven't mattered.

[@ant_code]

@@RobertBrunhage there is the opened issue on flutter github #111739 related to M1 machines golden tests mismatch.

[@imamhatipoglu4040]

Welcome again, awesome contents

[@atreeon]

Yes to Golden tests! was waaay too slow to adopt them. Unit tests for UI stuff - yes. Also integration tests for UI stuff - better. All three? perfect!

[@jayKrollin]

I struggle with the exact same issue but I would highly recommend to not use a tolerance. E.g. if any dev changes a padding by 1px the related test results could be false positives.

[@tayabfarooq2062]

Your followers eagerly look for your content that you post! Please take care of them by uploading something for them as much as possible!

[@RobertBrunhage]

Going to do my best haha

[@tbm98]

the pain is the result still different between OS

[@RobertBrunhage]

Yes, the best scenario will always be generating the tests with a single OS. I am for sure hoping people share some insights of potential better solutions. Mainly wanted to share my experience!

[@artofnessa]

Golden tests ftw

[@aouadicharaf1038]

❤❤❤

[@krtirtho]

Wait a sec...u guys test in Flutter?

[@coffee-is-power]

you should've just changed the implementaton to instead of comparing the bytes, just open the two images and compare the *pixels* not the bytes which is what flutter should do by default but it doesn't