
It's been 7 months since the initial release of Schema Benchmarks! We now have a wide range of benchmarks (12 different types), and measure the vast majority of notable schema validation libraries (currently 24). Schema Benchmarks has helped library authors identify performance regressions, and provides a useful resource for developers to compare libraries.
Measuring the runtime performance of libraries that implement the Standard Schema interface, which allows many other libraries to accept them without needing to specialise for each library.
Copy to clipboardimport { personSchema } from "./schemas"; const person = await upfetch(url, { schema: personSchema }); // internally const result = await personSchema["~standard"].validate(data);
Measuring the runtime performance of libraries that validate specific string formats, such as email addresses, URLs, and UUIDs.
Copy to clipboardimport * as v from "valibot"; const emailSchema = v.pipe(v.string(), v.email());
Inspired by a tweet from @trav, these benchmarks capture the error stack traces produced by libraries when validation fails, and measure how many frames it takes to find the current caller.
This provides a useful comparison of the error messages produced by libraries, and how easy it is to trace back to the source of the error.
Measuring the runtime performance of "codecs", two-way schemas that support both encoding and decoding of data.
Copy to clipboardimport * as z from "zod"; const bigIntFromString = z.codec(z.string(), z.bigint(), { encode: (bigInt) => bigInt.toString(), decode: (str) => BigInt(str), }); bigIntFromString.encode(123n); // "123" bigIntFromString.decode("123"); // 123n
–
We continue to maintain and improve Schema Benchmarks, adding new libraries and updating existing ones.
New libraries added include:
For a full list of libraries, see the Libraries page.
Based on suggestions from @sinclairzx81 (creator of TypeBox) and @DZakh (creator of Sury), we added multiple benchmarks for libraries that support JSON Schema, including:
Contributed by @DZakh, this benchmark measures the type inference performance and editor experience of TypeScript schema libraries, including compiler instantiations, inferred types, and hover display.
On initial release of our benchmarks, TypeBox was consistently underperforming in runtime performance.
After discussion with sinclairzx81 (author of TypeBox), it emerged that the v1.x release had introduced a change in behaviour, specifically that schemas would try to coerce invalid values into valid ones. Understandably, this made the library slower than most other libraries that don't do this.
A v1.1.0 release made this behaviour opt-in, bringing TypeBox's performance back in line with other libraries.
Copy to clipboard// v1.0.x // A = 12345 const A = Value.Parse(Type.Number(), "12345"); // v1.1.x // throw! - Expected Number const A = Value.Parse(Type.Number(), "12345");
Based on our download benchmarking, we were able to identify a tree shaking issue with decoders, and collaborated with the author to get it fixed.
Despite already having "sideEffects": false, we found that manually annotating functions with #__NO_SIDE_EFFECTS__ and function calls with #__PURE__ allowed bundlers to tree shake the library more effectively, reducing the overall bundle size for users.
Copy to clipboard// #__NO_SIDE_EFFECTS__ function exampleFunction() { // function implementation } const result = /* #__PURE__ */ anotherFunction();
Using our initialization benchmarking, the valibot team were able to identify a performance regression in their v1.x release.
This has been fixed as of v1.5.0, resulting in a significant speed up of schema initialisation.
If you have any suggestions for new benchmarks, or would like to see a library added, we'd love to hear from you! Please open an issue on GitHub.
Some current ones that have been suggested include:
If you think you could contribute to any of these, or offer opinions, please do! Be aware of our contributing guidelines.