Tooling for running the Test262 conformance suite against JavaScriptCore and for importing new versions of the suite into the tree.
To run the suite, invoke the runner:
./Tools/Scripts/test262-runner --releaseThe build configuration defaults to --release; pass --debug to use a debug build
instead (the two are mutually exclusive). The runner discovers the matching jsc binary
under WebKitBuild by default; pass --jsc <path> to use a specific binary instead.
By default, every test is run, results are compared against the checked-in
expectations file and the script exits non-zero if any new failures appear
(failures already listed in the expectations file are ignored).
Reports are written to test262-results/ in the current directory.
If you need to customize the execution, check out ./Tools/Scripts/test262-runner --help for extra commands.
| Option | Description |
|---|---|
--release / --debug |
Which build of jsc to run (defaults to --release). |
-j, --jsc <path> |
Use a specific jsc binary. |
--gtk / --wpe / --jsc-only |
Find jsc under the given port's build directory. |
-o, --test-only <path> |
Run only a subdirectory or single test file (repeatable). |
--filter <regex> |
Run only tests whose path matches the regex. |
-f, --features <feature> |
Run only tests with the given feature (repeatable). |
-p, --child-processes <n> |
Number of parallel worker processes. |
--timeout <ms> |
Per-test timeout in milliseconds. |
-v, --verbose |
Print each test's result and a new failure/pass summary. |
--save |
Save current failures to the expectations file. |
-x, --ignore-expectations |
Report all failures, not just new ones. |
-i, --ignore-config |
Ignore config.yaml. |
-l, --latest-import |
Run only the files changed by the last import. |
-F, --failing-files |
Re-run only the tests that failed in a previous results file. |
-S, --skipped-files |
Run only the tests that the config skips. |
--stats |
Recompute summaries from an existing results file without running tests. |
--sanitizer <name> |
Declare the sanitizer in use (e.g. asan) for conditional skips. |
This file controls which tests are skipped and which JSC runtime flags to enable for particular features. Example:
---
# Enable a JSC feature flag for every test that lists the given feature.
# Adds `--<flag>=1` to the jsc command line.
flags:
SharedArrayBuffer: useSharedArrayBuffer
Temporal: useTemporal
skip:
# Skip any test whose metadata lists one of these features.
features:
- decorators
- source-phase-imports
# Skip tests whose path matches one of these regexes.
paths:
- test/staging/Intl402
# Skip these individual test files.
files:
- test/built-ins/Array/prototype/reverse/length-exceeding-integer-limit-with-object.js
# Skip only under matching build/platform conditions. Each entry's `if`
# clause may constrain os, os_version, build (release/debug) and sanitizer;
# files/paths/features are skipped only when the condition matches.
conditions:
- if:
os: macOS
os_version: 26.4
features:
- Atomics
- if:
build: debug
paths:
- test/built-ins/RegExp/property-escapesThis file records every expected failure. When JSC or Test262 changes, it should be updated so that developers see only the failures they introduce.
Each entry maps a test file to the modes it's expected to fail in (default,
strict mode, module or raw), and each mode to the exit code jsc is
expected to terminate with:
3: the test ran to completion and failed (a failed assertion or an uncaught exception; JSC'sEXIT_EXCEPTION).128 + N:jscwas killed by signalN, i.e. it crashed (e.g.134SIGABRT,139SIGSEGV).1:jsccould not run the test at all, or the runner timed out.
Because the recorded exit code has to match, a test that starts crashing where it previously failed an assertion is still reported as a new failure.
On Linux, the runner uses expectations-linux.yaml instead.
To update the expectations file, run:
./Tools/Scripts/test262-runner --release --save--save updates entries incrementally: if you run a subset of the suite (for
example with --test-only or --filter), only the tests that actually ran are
rewritten and the rest of the file is left untouched.
Written to the current directory on every run:
results.yaml: results for every test that ran.summary.txt,summary.yaml,summary.html: pass/fail/skip statistics by feature and by folder.index.html: an expandable report of the failures (styled byreport.css).
To import a new version of Test262, run the importer. With no arguments,
it shallow-clones the canonical upstream repository and updates the
JSTests/test262 folder:
./Tools/Scripts/test262-importIt records the imported commit in JSTests/test262/test262-Revision.txt
and a list of the changed files in JSTests/test262/latest-changes-summary.txt
(which the runner's --latest-import mode reads). If the revision has not
changed since the last import, it exits without doing anything.
Run ./Tools/Scripts/test262-import --help for details.
| Option | Description |
|---|---|
-s, --src <path> |
Import from a local Test262 checkout instead of cloning. |
-r, --remote <url> |
Clone from a specific remote (default https://github.com/tc39/test262.git). |
-b, --branch <name> |
Branch to clone from a remote (default main). |
--src and --remote are mutually exclusive. A local source must be a clean
Git checkout containing test/ and harness/ directories.
Both tools require Python 3 and PyYAML
(pip3 install pyyaml). They rely on webkitcorepy and webkitpy, which
live in the tree under Tools/Scripts, so no other setup is needed.
The runner and importer live in Tools/Scripts/test262/ (runner.py and
importer.py); the top-level test262-runner and test262-import scripts are
thin wrappers around them.
The test262 tree is registered with test-webkitpy:
./Tools/Scripts/test-webkitpy test262Or run a single module directly:
cd Tools/Scripts/test262
python3 -m unittest runner_unittest importer_unittest