github

Welcome to the Schema Benchmarks project. This aims to compare the performance of different schema validation libraries in detail, including separating each step of the process.

Note

Some libraries have multiple benchmarks for the same thing (e.g. zod has both a safeParse and a parse method). Where that's the case, every variant is plotted in the same row - hover over a point for details about that specific benchmark.

Download

We first test the bundle size of each library. This is important for browser usage, where this will affect download time.

We do this by compiling example usage files with Rolldown, and measuring the size of the output, both minified and unminified.

With minification and tree shaking:

Without minification (but still treeshaken):

Benchmarks

Runtime benchmarks are run in sequence, on a GitHub runner.

Steps benchmarked include:

Initialization

Creating the schema itself. This is usually a one time cost.

schemas.ts
import * as v from "valibot";

export const personSchema = v.object({
  name: v.string(),
  age: v.number(),
});
export type Person = v.InferOutput<
  typeof personSchema
>;


schemas.ts
import * as v from "valibot";

export const personSchema = v.object({
  name: v.string(),
  age: v.number(),
});
export type Person = v.InferOutput<typeof personSchema>;

Validation

Checking if a given value matches the schema. Crucially, this is different to parsing because it doesn't return a new value.

import * as v from "valibot";

import { personSchema } from "./schemas";

if (v.is(personSchema, data)) {
  // data is narrowed to Person
}

Validating valid data:

Validating invalid data:

Note

Some libraries only support validation or parsing. In these cases, we categorise them accordingly.

Parsing

Checking if a given value matches the schema, and returning a new value. This will include any transformations.

import * as v from "valibot";

import { personSchema } from "./schemas";

const person = v.parse(
  personSchema,
  data,
);
// person is of type Person


import * as v from "valibot";

import { personSchema } from "./schemas";

const person = v.parse(personSchema, data);
// person is of type Person
Info

Some libraries were benchmarked with a method that throws an error when parsing invalid data, so the benchmark includes a try/catch - which may have an unknown performance impact.

Some libraries abort early when parsing invalid data, so will tend to be faster than those that collect every error.

Some libraries return the same object when parsing, instead of creating a new value, so will tend to be faster.

Parsing valid data:

Parsing invalid data:

Tags

Optimizations

Some libraries utilise specific optimizations to improve performance. We specifically track:

  • JIT: Libraries that use Just-In-Time compilation (usually via new Function) to generate optimized code at runtime, e.g. arktype
  • Precompiled: Libraries that generate optimized code at build time, e.g. typia
Error handling

Some libraries support different error handling strategies. We specifically track:

  • All errors: Parse the entire value before returning/throwing an error.
  • Abort early: Return/throw an error as soon as an issue is found.

Standard Schema

Many libraries implement the Standard Schema interface, which allows many other libraries to accept them without needing to specialise for each library.

import { personSchema } from "./schemas";

const person = await upfetch(url, {
  schema: personSchema,
});


import { personSchema } from "./schemas";

const person = await upfetch(url, { schema: personSchema });

We benchmark the time taken to parse using a standard schema.

Info

Some libraries require an adapter before they can be used as a standard schema. The time to convert the schema is not measured, only the time to parse using it.

Parsing valid data:

Parsing invalid data:

Codec

Some libraries support two way conversion of data, often referred to as "encoding" and "decoding".

We benchmark the time taken to encode and decode a BigInt to and from a string.

import * as z from "zod";

const bigIntFromString = z.codec(
  z.string(),
  z.bigint(),
  {
    decode: (str) => BigInt(str),
    encode: (bigInt) =>
      bigInt.toString(),
  },
);

bigIntFromString.encode(123n); // "123"
bigIntFromString.decode("123"); // 123n


import * as z from "zod";

const bigIntFromString = z.codec(z.string(), z.bigint(), {
  decode: (str) => BigInt(str),
  encode: (bigInt) => bigInt.toString(),
});

bigIntFromString.encode(123n); // "123"
bigIntFromString.decode("123"); // 123n
Invalid data

We don't benchmark codecs with invalid data, as many libraries require the input to be correctly typed before passing it to the codec.

Codecs that do accept unknown input may be slower - hover over a point in the chart below for details, including whether the library was benchmarked in more than one way.

Encode () and decode ():

JSON Schema

Schema to JSON

Some libraries can convert a schema into a JSON Schema, which lets documentation generators, OpenAPI builders and LLM structured outputs consume it.

import { personSchema } from "./schemas";

const jsonSchema = z.toJSONSchema(
  personSchema,
  { target: "draft-2020-12" },
);


import { personSchema } from "./schemas";

const jsonSchema = z.toJSONSchema(personSchema, {
  target: "draft-2020-12",
});


import { personSchema } from "./schemas";

const jsonSchema = z.toJSONSchema(personSchema, { target: "draft-2020-12" });

We collect data on which targets and type (input/output) each library supports, then benchmark the time taken to generate a JSON schema from a small schema with a codec, for supported combinations. Libraries that implement the standard JSON schema interface are marked in the results.

With a target of draft-2020-12 and input type:

JSON to Schema

Some libraries can also go the other way, converting a JSON Schema into a library schema at runtime.

const schema =
  z.fromJSONSchema(jsonSchema);


const schema = z.fromJSONSchema(jsonSchema);

Compliance

We test JSON Schema implementations against the JSON Schema Test Suite.

Validation
assert(
  Schema.Check(
    remotes,
    schema,
    data,
  ) === expected,
);


Validation
assert(Schema.Check(remotes, schema, data) === expected);

The graph shows each library's validation results for Draft 2020-12.

We also test libraries that allow schemas to be created from JSON Schema, both using their native schema validation ("semantics"), and after converting back into JSON Schema ("roundtrip").

Semantics
assert(
  z
    .fromJSONSchema(schema)
    .safeParse(data).success ===
    expected,
);


Semantics
assert(
  z.fromJSONSchema(schema).safeParse(data).success ===
    expected,
);


Semantics
assert(z.fromJSONSchema(schema).safeParse(data).success === expected);

For semantics with Draft 2020-12:

Roundtrip
assert(
  Schema.Check(
    remotes,
    z.toJSONSchema(
      z.fromJSONSchema(schema),
    ),
    data,
  ) === expected,
);


Roundtrip
assert(
  Schema.Check(
    remotes,
    z.toJSONSchema(z.fromJSONSchema(schema)),
    data,
  ) === expected,
);


Roundtrip
assert(
  Schema.Check(remotes, z.toJSONSchema(z.fromJSONSchema(schema)), data) ===
    expected,
);


Roundtrip
assert(Schema.Check(remotes, z.toJSONSchema(z.fromJSONSchema(schema)), data) === expected);

For roundtrip with Draft 2020-12:

Preferences

Style
Theme
NPM browser
Code ligatures