Flutter Test Automation with widget and golden tests

Widget tests

Widgets tests despite their name can help testing the whole user flow, not only separate widgets. For example, Burger King Suomi app has many widget tests to test that the most important parts of what the user does in the app work as expected: signing in, finding a restaurant, using coupons, customizing a meal, placing an order. Some tests check the whole flow end to end: from app launch to successfully placed order.

Since widgets tests can be run headlessly (without a device or emulator) it’s impossible to make network calls, use actual user accounts and real backend responses. In order to overcome this we’re using a lot of mock data.

Most of the mocks are happening at the very bottom of the app’s layered architecture. For example, HTTP client returns mock API responses in order to return a certain menu structure, coupon or error code. Mocking API calls makes the app work architecturally the same way it does when using actual API responses. These mocks don’t change the way app’s business logic works, there’s no special “test app behavior”.

The reason we’re using widgets tests to test user flows is that they’re easier to write and faster to run than Flutter’s integration tests. We run widget tests before every pull request can be merged to main.

Here’s an example of a simple widget test for ShoppingCardWidget’s functionality of adding cart items:

testWidgets(
  'ShoppingCartWidget can add items to the cart',
  (WidgetTester tester) async {
    // Build the widget
    await tester.pumpWidget(ShoppingCartWidget());
    await tester.pumpAndSettle();

    // Verify that the cart is initially empty
    expect(find.text('Cart is empty'), findsOneWidget);

    // Tap on the "Add Item" button
    await tester.tap(find.text('Add to cart'));
    await tester.pumpAndSettle();

    // Verify that the cart now contains the added item
    expect(find.text('Item added to cart'), findsOneWidget);
    expect(find.text('Cart: 1 item'), findsOneWidget);
  },
);

Golden tests

Running tests headlessly is the same as running the app on a device but instead of an app or a window for drawing it uses a virtual canvas. This canvas is an array of pixels and can be saved as a picture.

Golden tests compare the current state of the canvas with the previously saved state. In practice it works as pixel by pixel comparison of screenshots. In a way it’s the same old “expect” but instead of checking an object it compares two images.

As with other test types, it’s important to test what actually matters. Golden tests easily fail if there’s a tiny UI change. It’s easy to update golden files but it’s harder to validate these changes. For example, if there’s a one-liner color change in the app’s theme it’s quite possible that a lot of PNGs would need to be regenerated and thoroughly checked by a human at the pull request review step. This could lead to issues when reviewer didn’t notice some unwanted changes.

Here’s an addition to the previous code example where golden test is used to check how widget looks before and after new card item is added:

// Build the widget
// ...

// Initial state: cart is empty
await screenMatchesGolden(tester, 'shopping_cart_widget_initial');

// Tap on the "Add Item" button
// ...

// One cart item state
await screenMatchesGolden(tester, 'shopping_cart_widget_one_item');

Testing is an important part of the app development process. Deciding what and how to test is crucial for ensuring the reliability and functionality of the app. Framework and community offer many options for testing. To learn more about testing Flutter apps explore official documentation and golden_toolkit package documentation.

Wrap up

We hope you have enjoyed reading some of our thoughts on Flutter test automation and perhaps will take a few ideas that can promote active discussion before you jump in. Follow our Rebel App Studio X account if you haven’t already. We share timely topics and provide updates on Flutter related events we are arranging or participating in.