4 posts tagged with "testing"

Ladle is a tool designed for building and testing React components through stories. It serves as a seamless alternative to Storybook and is built using Vite and SWC. Our main goal is to ensure it's as swift and user-friendly as conceivable.

Introduced 18 months ago, Ladle is now utilized in 335 different Uber projects with a total of 15,896 stories. The community response has been positive as well:

• 🎯 60,000 weekly downloads.
• 🖊️ 40 contributors.
• 💬 300 folks joined our Discord community.

Having taken our break from Storybook, Webpack, and Babel, today I'm eager to share insights from our journey and introduce the upcoming major release of Ladle. This version emphasizes the open-source community, addresses existing issues, enhances and outlines standard configurations with libraries like Next, Emotion, and styled-components, and establishes future priorities.

What's new in v3?​

• SWC (Speedy Web Compiler) is now the default compiler, replacing Babel for transforming React and Typescript in Ladle projects. This shift leads to 2x faster production builds, swift initial startup, and incremenetal builds. We've exclusively adopted it for our internal projects. Furthermore, we've crafted our own set of plugins to replace certain Babel plugins. We're confident in its scalability; however, if preferred, you can revert to the Babel-based @vite/plugin-react.
• Official support for Node v18 and v20, while Node v16 is now deprecated.
• Added MDX stories and markdown compatibility.
• A streamlined method to mock dates within stories.
• CSS-in-JS libraries function without any extra tweaks. Documented configurations are available for Emotion, Styled-components, BaseWeb + Styletron, Tailwind.
• An outlined setup for Next.js.
• Exported types like StoryDefault and Meta are now interfaces, allowing for extensions.

Recent additions include:

• Global hotkeys for accelerated navigation.
• Prebundled dependencies to prevent later reloads.
• Global args and argTypes.
• A specialized control for adjusting backgrounds.
• Enhanced Typescript types via satisfies.
• HTTP/2 support.
• The storyOrder feature to customize story sequences.
• Additional features and bug resolutions.

Lessons Learned​

The primary catalyst behind Ladle was performance enhancement. Incremental builds transitioned from several seconds or even minutes to mere milliseconds. Startup durations reduced dramatically. And bellow a chart showcases our progression to a 100% Ladle adoption by 08/2023 and its impact on CI build times.

Technically, we've been doing three different migrations at the same time:

• From Storybook to Ladle.
• From Webpack to Vite.
• From Babel to SWC.

Storybook to Ladle​

This phase was relatively smooth. Ladle was architectured to be compatible with the majority of Storybook's core APIs and features. As we integrated individual setups, we incorporated missing elements into Ladle. The transition was seamless for about 99% of our stories, requiring minimal changes.

However, Ladle is more strict in some cases. For instance, story titles must be string literals to support automated code-splitting.

Webpack to Vite​

Transitioning from Webpack to Vite posed significant challenges, primarily because they operate differently. While Webpack automatically adjusts any JS code for browser compatibility, Vite interacts minimally with the code, leveraging native browser functionalities, primarily ES modules.

Importing Types​

It was common for developers to overlook the subtle distinctions between:

import type { Foo } from "baz";

vs

import { Foo } from "baz";

The issue is that Vite will pass the second import into the browser without any changes and and browser will complain about non-existing export Foo since types get stripped from the code before being sent to the browser.

Commonjs Modules​

Vite isn't designed for CommonJS modules. We transitioned our codebase to ES modules, replacing occasional require calls with alternatives like await import or leveraging SWC's dead code elimination.

import window​

import window from "global/window";

will result in

ReferenceError: Cannot access 'window' before initialization

The solution is to rename window to something else.

Startup time​

It turns out that Vite can be quite slow if you need to import thousands of modules. To be fair, it's often a self-inflicted issue. There is a popular pattern where developers create these feature/index.js god modules that re-export every single module from the feature folder. Then, they import from feature/index.js instead of importing individual modules.

This is not a big issue with Webpack since it always bundles code together and it can eliminate code that is not used. Vite, on the other hand, needs to import all these modules individually and each import results in a separate HTTP request. This can be make browser quite unhappy when it reaches thousands of HTTP requests.

It's hard to refactor large codebases like that. We partially alleviate this issue by adapting HTTP/2 so browsers can download multiple modules in a single HTTP request. SWC also makes individual module transformations much faster. The Vite team is aware of this issue and they made a lot of changes to make Vite faster over time. This persistent cache plugin should also help.

Babel to SWC​

This was the easiest part. SWC is mostly a drop-in replacement for Babel. We had some custom babel plugins we had to migrate into Rust and SWC plugins. I had no previous experience with Rust and the SWC plugin API is still marked as experimental and not that well documented. It also seems less flexible than Babel's plugin API that provides more information about the AST and allows you to modify it more easily (I'm still not sure how to properly track scopes with SWC). However, ChatGPT was extremely helpful and I was able to migrate all our custom Babel plugins into SWC in a few days. There are also a few existing SWC plugins for some popular libraries.

There were some pretty rare (incorrect) syntaxes that Babel happily parses and SWC fails on. Those are pretty easy to find and fix. We also ran into a syntax being compiled differently by Babel (not adhering to the spec) and SWC (adhering to the spec) that caused a subtle runtime error related to class fields. This one took a few days to debug but it was a good lesson to not rely on Babel's non-standard behavior.

Overall, the migration was pretty straightforward. It took a while since we didn't really force it and did it mostly in a spare time. Also, the amount of setups and stories we had to migrate was abnormally large.

What's Next​

The secondary reason why Ladle was created was to make our test automation easier. That's why Ladle has statically analyzable meta parameter and first-class programatic API. We might open-source/release more of it, there are some testing concepts described here.

We want to make easier to generate permutations of stories for automated testing. This could mean utilizing typescript types and making some changes to controls API to make it statically analyzable. Ladle also doesn't support component story format 3.0. All these things are somewhat related and we are looking into how to best combine them to improve our internal testing framework.

On the other hand, we have no plans to add support for other frameworks than React since we are React only shop. If you are looking for Vue/Svelte Vite based alternatives, you should try https://histoire.dev/. It is also not our goal to support all the features of Storybook or trying to duplicate its addon ecosystem. Storybook is a great tool and we are not trying to replace it, it's also trying to address some performance issues by introducing vite builder.

Last Words​

I extend my gratitude to all the OSS maintainers who've contributed to Ladle's existence. A special thanks to Vite for their continuous support (including Ladle into their ecosystem test suite). Additionally, SWC's emergence as a solid alternative to Babel, courtesy of its recent plugin API, has been instrumental.

If you use stories to develop your React components, you might be also interested in some sort of test automation. Ladle and Playwright makes it easy to take screenshots of your stories and compare them against the previous version before you changed your code. We call this visual snapshot testing. Let's take a look at how you can automate it with Ladle. This solution is quick, free and self-hosted.

It might surprise you how easy it is - less than 10 lines of code! Ladle exports a static meta.json file that lists all your stories and their parameters. We can use this file to generate our tests. Note: Terms snapshots and screenshots are used interchangeably.

You can jump into the working example right away. Or you can follow the steps below.

The Workflow​

1. Build Ladle using your stories
2. Start an HTTP server for Ladle
3. Fetch the meta.json file
4. Dynamically generate a test for each story
5. Take a screenshot & compare it with the baseline
6. Commit changes if there are any
7. Profit

Playwright​

Playwright is a great headless browser testing framework. It works cross-platform and cross-browser and has a nice TypeScript API. It also comes with a test runner. We use it to navigate through Ladle and capture screenshots. It can be installed as:

pnpm install @playwright/test

We also need some other dependencies:

pnpm install sync-fetch

Stories​

Let's create a story file src/abc.stories.tsx so we have something to test. Presumably, you already have stories with your own React components.

export const First = () => {
  return <h1>First</h1>;
};

export const Second = () => {
  return <h1>Second</h1>;
};
Second.meta = {
  skip: true,
};

Test Setup​

Now we need to tell Playwright what & how to test. We can create a single test file that dynamically creates subtests for each individual story. The following code in tests/snapshot.spec.ts is the secret sauce of this setup:

import { test, expect } from "@playwright/test";
// we can't create tests asynchronously, thus using the sync-fetch lib
import fetch from "sync-fetch";

// URL where Ladle is served
const url = "http://127.0.0.1:61000";

// fetch Ladle's meta file
// https://ladle.dev/docs/meta
const stories = fetch(`${url}/meta.json`).json().stories;

// iterate through stories
Object.keys(stories).forEach((storyKey) => {
  // create a test for each story
  test(`${storyKey} - compare snapshots`, async ({ page }) => {
    // skip stories with `meta.skip` set to true
    test.skip(stories[storyKey].meta.skip, "meta.skip is true");
    // navigate to the story
    await page.goto(`${url}/?story=${storyKey}&mode=preview`);
    // stories are code-splitted, wait for them to be loaded
    await page.waitForSelector("[data-storyloaded]");
    // take a screenshot and compare it with the baseline
    await expect(page).toHaveScreenshot(`${storyKey}.png`);
  });
});

Run It​

Our setup is ready, we just need to run it. Let's add some package.json scripts that we can use as shortcuts:

{
  "scripts": {
    "serve": "ladle serve",
    "build": "ladle build && ladle preview -p 61000",
    "test:dev": "TYPE=dev pnpm exec playwright test",
    "test": "pnpm exec playwright test",
    "test:update": "pnpm exec playwright test -u"
  }
}

We also need to setup playwright so it starts a web server before running our snapshots:

export default {
  webServer: {
    command: process.env.TYPE === "dev" ? "pnpm serve" : "pnpm build",
    url: `http://127.0.0.1:61000`,
  },
};

If you use yarn or npm you might want to use npx instead.

The first time you run the test script it will error out since it needs to create the baseline screenshots in tests/snapshot.spec.ts-snapshots folder. The second run will succeed:

Running 2 tests using 1 worker

  ✓  tests/snapshot.spec.ts:23:3 › abc--first - compare snapshots (259ms)
  -  tests/snapshot.spec.ts:23:3 › abc--second - compare snapshots


  1 skipped
  1 passed (952ms)

You can keep adding more stories and they get automatically covered by visual snapshots. If you change an existing story, the test will fail. The playwright also outputs a diff image that highlights the differences between the baseline and actual screenshot. If everything seems ok, you need to run test:update to update existing snapshots. When your code matches the desirable looks you should commit both the code and snapshot changes. This updates the baseline for the future code changes.

Possible Improvements​

The first problem you might run into is that your CI (tried Github Actions yet?) might render fonts differently than your local machine. The easiest way to make sure you use the same environment locally and remotely is to use docker.

Our example setup uses a made up meta.skip parameter to opt-out of snapshots for selected stories. You could add more customization like viewport size or browser by adding arbitrary meta parameters to the story and updating the tests/snapshot.spec.ts logic accordingly.

Playwright can do a lot more than just screenshot taking. You can also use it to write all other sorts of assertion/integration tests since you already have the environment ready.

You might reach some point when storing screenshots in your repository is not working anymore due to the size - imagine hundreds of developers using a single monorepo churning thousands of images. At Uber, we've built an internal service that uses S3 to store screenshots, Aurora DB to store test metadata and provides a custom UI to compare & approve changes (GitHub's PR review UI is pretty good for images too!). The goal was to move the snapshot creation and updates into a remote environment so developers are not slowed down by it. It also helps to prevent the issue of local vs remote environment.

Conclusion​

We used Playwright and wrote 10 lines of code to automate visual snapshots for our stories and gained great coverage for our React components! Ladle was built for this - to be small, fast and integrate well with other tools. This should give you a good starting point for your own automated and end-to-end testing.

Ladle has been out for 3 months and the community feedback was overwhelming and amazing. Thank you! Some numbers:

• 🎯 20,000 unique visitors on this website.
• ⭐ 1,300+ GitHub stars.
• 🖊️ 10 contributors.
• 💬 100+ folks joined our Discord community.

Today, it's time to graduate Ladle to its first major stable version 1. We've fixed many bugs discovered by early adopters and added some big features as well:

• Addon to test accessibility
• Addon to show story source

V1 brings the most requested feature - the full access to Vite configuration. When Ladle was released, it was intended as a tool that could replace Storybook. The bundler (Vite) was just an implementation detail and completely hidden away. However, for more advanced setups, you need to customize various aspects of compilation (like Storybook's Webpack). It makes sense to fully expose vite.config.js and embrace Vite first applications as well. Ladle's configuration has been rewritten from the ground up.

Check the new config documentation.

Breaking Changes​

• Ladle now loads top-level vite.config.js/ts/mjs and uses all its options.
• All Vite related options from config.mjs are removed and an error will be thrown, use vite.config.js instead.
• enableFlow option removed, you can create your own plugin (check our e2e/flow test).
• Programmatic API imports changed to @ladle/react/serve and @ladle/react/build.
• --out renamed to --outDir to mimic Vite configuration, added -o alias, outDir moved to the top-level in config.mjs.
• --port has an alias -p, port moved to the top-level in config.mjs.
• vite.config.js can be customized through viteConfig and --viteConfig.
• --base-url removed, use base in vite.config.js.
• --open removed, use server.open in vite.config.js.
• --sourcemap removed, use build.sourcemap in vite.config.js.

This makes our internal setup much simpler since we don't need to pass through individual Vite settings - you can now customize all of them without Ladle being in your way.

Next big features?​

We are adding MDX support and making it easier to use Ladle for documentation.

Ladle is a tool for developing and testing your React components in an environment that's isolated and faster than most real-world applications. It supports Component Story Format – a concept widely popular thanks to Storybook. In fact, Ladle has been developed as a drop-in replacement of Storybook – it should already work with your existing stories.

Storybook ❤️​

At Uber, we are big fans of Storybook. We have over 100 instances of Storybook in our web monorepo. Our teams use it to develop, showcase, document and test React components. We have developed a CI based system that automatically deploys each Storybook with every change and runs automated visual snapshot tests. This happens a thousand times each day. It's a critical tool for our web workflows. The performance is extremely important.

Unfortunately, there are some areas where Storybook is not doing as good as we would like to:

• production build time - often times it's the slowest part of our CI
• dev mode start up time - not always faster than the related prod app
• bundle output - hosting storage + slow initial time to interactive
• maintenance - we repackaged Storybook, its dependencies and configuration to provide a seamless setup for our developers; however, the addon versioning and monorepo setup makes maintenance difficult
• testing - for our automated visual testing, we need to crawl and parse stories in a separate process since Storybook doesn't export a static list of stories (and other metadata)

Next Generation Frontend Tooling ⚡​

A shift is happening. ES modules are now natively supported by all browsers and Node. We don't need to bundle our components anymore in order to serve them. This is a major slowdown for Storybook and other Webpack based environments.

Ladle is built around Vite, which serves modules directly to the browser and uses fast esbuild when bundling is required for dependencies. It provides the performance leap we were looking for.

Performance Numbers ⏱️​

We used Base Web to benchmark Ladle and latest Storybook v6.4.19. Base Web is a complex component library and has about 350 stories. The Storybook uses the default bootstrapped settings. The test is made on a 2018 Macbook Pro, i7 2.7 GHz. The time is measured in seconds and less is better.

Both Ladle and Storybook utilize cache. The initial dev startup takes 6s vs 58s. Once the cache is built, Ladle starts almost instantly (3s is just the browser tab being opened + time to interactive). Storybook always takes about 25s to start. The production build is about 4x faster with Ladle.

There is another improvement - hot reload takes less than 100ms with Ladle and preserves the state. Storybook takes about 2.5s and doesn't preserve the state.

We also care about the bundle size and the amount of resources that browser has to initially download:

• Ladle build folder is 4.6 MB vs Storybook's 16.1 MB
• Ladle downloads 388 kB of resources to open the default story, Storybook 14.3 MB

How is this even possible? Ladle code-splits each story by default, so it doesn't matter how many stories you have. It always keeps the initial bundle reasonable.

Not Just Fast 📝​

Ladle is a single package and command. It's easy to install and setup - no configuration required. Some other features:

• Supports controls, links, dark theme, RTL and preview mode
• Built-in addons always preserve the state through the URL - useful for testing
• A11y and keyboard friendly
• Responsive, no iframes
• Code-splitting and React Fast Refresh enabled by default
• Exports meta.json file with list of stories and additional metadata
• Programmatic API so it can be easily repackaged

Conclusion 🔮​

The new set of web tools is coming and brings huge performance wins. Ladle is using them to provide a significantly faster environment for developing, sharing and testing your React components. Are you ready to give it a try? Also, check our GitHub and give us ⭐.

mkdir my-ladle
cd my-ladle
pnpm init
pnpm add @ladle/react react react-dom
mkdir src
echo "export const World = () => <p>Hey</p>;" > src/hello.stories.tsx
pnpm ladle serve