github

7 Month Update

  • E

It's been 7 months since the initial release of Schema Benchmarks! We now have a wide range of benchmarks (12 different types), and measure the vast majority of notable schema validation libraries (currently 24). Schema Benchmarks has helped library authors identify performance regressions, and provides a useful resource for developers to compare libraries.

Updates to Schema Benchmarks

  1. Initial Release

    Schema Benchmarks was released to the public, with initial benchmarks:

  2. Standard Schema

    Measuring the runtime performance of libraries that implement the Standard Schema interface, which allows many other libraries to accept them without needing to specialise for each library.

    import { personSchema } from "./schemas";

const person = await upfetch(url, { schema: personSchema });

// internally

const result = await personSchema["~standard"].validate(data);

  3. String formats

    Measuring the runtime performance of libraries that validate specific string formats, such as email addresses, URLs, and UUIDs.

    import * as v from "valibot";

const emailSchema = v.pipe(v.string(), v.email());

  4. Stack traces

    Inspired by a tweet from @trav, these benchmarks capture the error stack traces produced by libraries when validation fails, and measure how many frames it takes to find the current caller.

    This provides a useful comparison of the error messages produced by libraries, and how easy it is to trace back to the source of the error.

  5. Codecs

    Measuring the runtime performance of "codecs", two-way schemas that support both encoding and decoding of data.

    import * as z from "zod";

const bigIntFromString = z.codec(z.string(), z.bigint(), {
  encode: (bigInt) => bigInt.toString(),
  decode: (str) => BigInt(str),
});

bigIntFromString.encode(123n); // "123"
bigIntFromString.decode("123"); // 123n

  6. –

    Upkeep

    We continue to maintain and improve Schema Benchmarks, adding new libraries and updating existing ones.

    New libraries added include:

    For a full list of libraries, see the Libraries page.

  7. JSON Schema

    Based on suggestions from @sinclairzx81 (creator of TypeBox) and @DZakh (creator of Sury), we added multiple benchmarks for libraries that support JSON Schema, including:

  8. TypeScript

    Contributed by @DZakh, this benchmark measures the type inference performance and editor experience of TypeScript schema libraries, including compiler instantiations, inferred types, and hover display.

Ecosystem Impact

  1. TypeBox

    On initial release of our benchmarks, TypeBox was consistently underperforming in runtime performance.

    After discussion with sinclairzx81 (author of TypeBox), it emerged that the v1.x release had introduced a change in behaviour, specifically that schemas would try to coerce invalid values into valid ones. Understandably, this made the library slower than most other libraries that don't do this.

    A v1.1.0 release made this behaviour opt-in, bringing TypeBox's performance back in line with other libraries.

    // v1.0.x
// A = 12345
const A = Value.Parse(Type.Number(), "12345");

// v1.1.x
// throw! - Expected Number
const A = Value.Parse(Type.Number(), "12345");

  2. Decoders

    Based on our download benchmarking, we were able to identify a tree shaking issue with decoders, and collaborated with the author to get it fixed.

    Despite already having "sideEffects": false, we found that manually annotating functions with #__NO_SIDE_EFFECTS__ and function calls with #__PURE__ allowed bundlers to tree shake the library more effectively, reducing the overall bundle size for users.

    // #__NO_SIDE_EFFECTS__
function exampleFunction() {
  // function implementation
}

const result = /* #__PURE__ */ anotherFunction();

  3. Valibot

    Using our initialization benchmarking, the valibot team were able to identify a performance regression in their v1.x release.

    This has been fixed as of v1.5.0, resulting in a significant speed up of schema initialisation.

New Benchmark Ideas

If you have any suggestions for new benchmarks, or would like to see a library added, we'd love to hear from you! Please open an issue on GitHub.

Some current ones that have been suggested include:

If you think you could contribute to any of these, or offer opinions, please do! Be aware of our contributing guidelines.

Preferences

Style
Theme
NPM browser
Code ligatures