Skip to content

Commit bea74e1

Browse files
jasnelladuh95
jasnell
authored and
aduh95
committed
lib: put node:bench behind an --experimental-bench flag
Signed-off-by: James M Snell <[email protected]> Assisted-by: Opencode PR-URL: #65920 Reviewed-By: Jordan Harband <[email protected]> Reviewed-By: Rafael Gonzaga <[email protected]> Reviewed-By: Vinícius Lourenço Claro Cardoso <[email protected]> Reviewed-By: Xuguang Mei <[email protected]> Reviewed-By: Marco Ippolito <[email protected]> Reviewed-By: Filip Skokan <[email protected]>
1 parent 900fae0 commit bea74e1

43 files changed

Lines changed: 304 additions & 51 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎benchmark/_node-bench.js‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ function runBenchmark(binary, file, options) {
5353
const args = [
5454
...options.nodeArgs,
5555
'--no-warnings',
56+
'--experimental-bench',
5657
'--bench',
5758
'--bench-reporter=json',
5859
'--bench-samples=1',

‎doc/api/bench.md‎

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,8 @@ added: REPLACEME
1212

1313
The `node:bench` module supports defining and running JavaScript benchmarks in
1414
the current process, and running one benchmark file in a fresh child process.
15-
To access it:
15+
The module is only available when Node.js is started with the
16+
`--experimental-bench` flag and can only be imported with the `node:` scheme:
1617

1718
```mjs
1819
import { bench, suite } from 'node:bench';
@@ -22,8 +23,6 @@ import { bench, suite } from 'node:bench';
2223
const { bench, suite } = require('node:bench');
2324
```
2425

25-
This module is only available under the `node:` scheme.
26-
2726
## Example benchmark
2827

2928
Save the following as `benchmark.mjs`:
@@ -57,7 +56,7 @@ suite('URL', () => {
5756
Run the benchmark from the command line:
5857

5958
```console
60-
node --bench benchmark.mjs
59+
node --experimental-bench --bench benchmark.mjs
6160
```
6261

6362
Benchmarks are executed serially in declaration order. Declared benchmarks are
@@ -183,8 +182,8 @@ declarations and a second call to `run()` is an error.
183182
The `--bench` flag runs one or more explicit benchmark files or glob patterns:
184183

185184
```console
186-
node --bench benchmark.mjs
187-
node --bench --bench-reporter=json 'benchmarks/**/*.js'
185+
node --experimental-bench --bench benchmark.mjs
186+
node --experimental-bench --bench --bench-reporter=json 'benchmarks/**/*.js'
188187
```
189188

190189
Files are sorted and executed serially. The default

‎doc/api/cli.md‎

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -482,10 +482,13 @@ Starts the Node.js command-line benchmark runner. At least one explicit file or
482482
glob pattern is required:
483483

484484
```console
485-
node --bench benchmark.mjs
486-
node --bench 'benchmarks/**/*.js'
485+
node --experimental-bench --bench benchmark.mjs
486+
node --experimental-bench --bench 'benchmarks/**/*.js'
487487
```
488488

489+
The `--experimental-bench` flag is required to use this flag or any other
490+
`--bench-*` option.
491+
489492
Quote glob patterns to prevent expansion by the shell. Matching files are
490493
sorted and executed serially. By default, each file runs in a separate child
491494
process. Benchmark files declare benchmarks using `node:bench`; they must not
@@ -1210,6 +1213,16 @@ changes:
12101213
12111214
Enable experimental import support for `.node` addons.
12121215

1216+
### `--experimental-bench`
1217+
1218+
<!-- YAML
1219+
added: REPLACEME
1220+
-->
1221+
1222+
> Stability: 1 - Experimental
1223+
1224+
Enable the experimental `node:bench` module and command-line benchmark runner.
1225+
12131226
### `--experimental-config-file=path`, `--experimental-config-file`
12141227

12151228
<!-- YAML
@@ -4062,6 +4075,7 @@ one is included in the list below.
40624075
* `--entry-url`
40634076
* `--experimental-abortcontroller`
40644077
* `--experimental-addon-modules`
4078+
* `--experimental-bench`
40654079
* `--experimental-detect-module`
40664080
* `--experimental-dtls`
40674081
* `--experimental-eventsource`

‎doc/node-config-schema.json‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,63 @@
4545
"$schema": {
4646
"type": "string"
4747
},
48+
"bench": {
49+
"type": "object",
50+
"additionalProperties": false,
51+
"required": [],
52+
"properties": {
53+
"bench": {
54+
"type": "boolean",
55+
"description": "launch benchmark runner on startup"
56+
},
57+
"bench-isolation": {
58+
"type": "string",
59+
"description": "configures the type of benchmark isolation used in the benchmark runner"
60+
},
61+
"bench-name-pattern": {
62+
"type": "string",
63+
"description": "run benchmarks whose name matches this regular expression"
64+
},
65+
"bench-reporter": {
66+
"oneOf": [
67+
{
68+
"type": "string"
69+
},
70+
{
71+
"type": "array",
72+
"minItems": 1,
73+
"items": {
74+
"type": "string"
75+
}
76+
}
77+
],
78+
"description": "report benchmark output using the given reporter"
79+
},
80+
"bench-reporter-destination": {
81+
"oneOf": [
82+
{
83+
"type": "string"
84+
},
85+
{
86+
"type": "array",
87+
"minItems": 1,
88+
"items": {
89+
"type": "string"
90+
}
91+
}
92+
],
93+
"description": "report the given benchmark reporter to the given destination"
94+
},
95+
"bench-samples": {
96+
"type": "number",
97+
"description": "specify the number of measured benchmark samples"
98+
},
99+
"bench-warmup": {
100+
"type": "number",
101+
"description": "specify the number of unreported benchmark warmup samples"
102+
}
103+
}
104+
},
48105
"nodeOptions": {
49106
"additionalProperties": false,
50107
"required": [],
@@ -119,6 +176,52 @@
119176
"type": "boolean",
120177
"description": "Improve AsyncLocalStorage performance with AsyncContextFrame"
121178
},
179+
"bench-isolation": {
180+
"type": "string",
181+
"description": "configures the type of benchmark isolation used in the benchmark runner"
182+
},
183+
"bench-name-pattern": {
184+
"type": "string",
185+
"description": "run benchmarks whose name matches this regular expression"
186+
},
187+
"bench-reporter": {
188+
"oneOf": [
189+
{
190+
"type": "string"
191+
},
192+
{
193+
"type": "array",
194+
"minItems": 1,
195+
"items": {
196+
"type": "string"
197+
}
198+
}
199+
],
200+
"description": "report benchmark output using the given reporter"
201+
},
202+
"bench-reporter-destination": {
203+
"oneOf": [
204+
{
205+
"type": "string"
206+
},
207+
{
208+
"type": "array",
209+
"minItems": 1,
210+
"items": {
211+
"type": "string"
212+
}
213+
}
214+
],
215+
"description": "report the given benchmark reporter to the given destination"
216+
},
217+
"bench-samples": {
218+
"type": "number",
219+
"description": "specify the number of measured benchmark samples"
220+
},
221+
"bench-warmup": {
222+
"type": "number",
223+
"description": "specify the number of unreported benchmark warmup samples"
224+
},
122225
"conditions": {
123226
"oneOf": [
124227
{
@@ -209,6 +312,10 @@
209312
"type": "boolean",
210313
"description": "experimental import support for addons"
211314
},
315+
"experimental-bench": {
316+
"type": "boolean",
317+
"description": "experimental node:bench module and benchmark runner"
318+
},
212319
"experimental-detect-module": {
213320
"type": "boolean",
214321
"description": "when ambiguous modules fail to evaluate because they contain ES module syntax, try again to evaluate them as ES modules"

‎doc/node.1‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -296,9 +296,11 @@ Error: Access to this API has been restricted
296296
Starts the Node.js command-line benchmark runner. At least one explicit file or
297297
glob pattern is required:
298298
.Bd -literal
299-
node --bench benchmark.mjs
300-
node --bench 'benchmarks/**/*.js'
299+
node --experimental-bench --bench benchmark.mjs
300+
node --experimental-bench --bench 'benchmarks/**/*.js'
301301
.Ed
302+
The \fB--experimental-bench\fR flag is required to use this flag or any other
303+
\fB--bench-*\fR option.
302304
Quote glob patterns to prevent expansion by the shell. Matching files are
303305
sorted and executed serially. By default, each file runs in a separate child
304306
process. Benchmark files declare benchmarks using \fBnode:bench\fR; they must not
@@ -677,6 +679,9 @@ It is possible to run code containing inline types unless the
677679
.It Fl -experimental-addon-modules
678680
Enable experimental import support for \fB.node\fR addons.
679681
.
682+
.It Fl -experimental-bench
683+
Enable the experimental \fBnode:bench\fR module and command-line benchmark runner.
684+
.
680685
.It Fl -experimental-config-file Ns = Ns Ar path , Fl -experimental-config-file
681686
If present, Node.js will look for a configuration file at the specified path.
682687
If the path is not specified, Node.js will look for a \fBnode.config.json\fR file
@@ -2098,6 +2103,8 @@ one is included in the list below.
20982103
.It
20992104
\fB--experimental-addon-modules\fR
21002105
.It
2106+
\fB--experimental-bench\fR
2107+
.It
21012108
\fB--experimental-detect-module\fR
21022109
.It
21032110
\fB--experimental-dtls\fR

‎lib/internal/bench_runner/cli.js‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -106,7 +106,11 @@ const kEventTypes = new SafeSet([
106106
'bench:diagnostic',
107107
'bench:summary',
108108
]);
109-
const kFilterArgs = ['--bench', '--experimental-default-config-file'];
109+
const kFilterArgs = [
110+
'--bench',
111+
'--experimental-bench',
112+
'--experimental-default-config-file',
113+
];
110114
const kFilterArgValues = [
111115
'--bench-isolation',
112116
'--bench-name-pattern',
@@ -741,7 +745,8 @@ function getInheritedChildArgs() {
741745
function getChildArgs(path, options) {
742746
const args = options.execArgv === undefined ?
743747
getInheritedChildArgs() : ArrayPrototypeSlice(options.execArgv);
744-
ArrayPrototypePush(args, '--bench', '--bench-isolation=none');
748+
ArrayPrototypePush(
749+
args, '--experimental-bench', '--bench', '--bench-isolation=none');
745750
if (options.namePatternSource.length > 0) {
746751
ArrayPrototypePush(
747752
args, `--bench-name-pattern=${options.namePatternSource}`);

‎lib/internal/bootstrap/realm.js‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,8 @@ const schemelessBlockList = new SafeSet([
137137
]);
138138
// Modules that will only be enabled at run time.
139139
const experimentalModuleList = new SafeSet([
140+
'bench',
141+
'bench/reporters',
140142
'dtls',
141143
'ffi',
142144
'quic',

‎lib/internal/process/pre_execution.js‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,7 @@ function prepareExecution(options) {
114114
setupNetworkInspection();
115115
setupNavigator();
116116
setupWarningHandler();
117+
setupBench();
117118
setupFFI();
118119
setupSQLite();
119120
setupStreamIter();
@@ -391,6 +392,16 @@ function setupNavigator() {
391392
defineReplaceableLazyAttribute(globalThis, 'internal/navigator', ['navigator'], false);
392393
}
393394

395+
function setupBench() {
396+
if (!getOptionValue('--experimental-bench')) {
397+
return;
398+
}
399+
400+
const { BuiltinModule } = require('internal/bootstrap/realm');
401+
BuiltinModule.allowRequireByUsers('bench');
402+
BuiltinModule.allowRequireByUsers('bench/reporters');
403+
}
404+
394405
function setupFFI() {
395406
if (!getOptionValue('--experimental-ffi')) {
396407
return;

‎src/node.cc‎

Lines changed: 23 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -725,6 +725,14 @@ void ResetStdio() {
725725
#endif // __POSIX__
726726
}
727727

728+
// Validates the benchmark runner options of the global (per-process) options
729+
// once every option source has been parsed. See
730+
// EnvironmentOptions::CheckBenchOptions().
731+
static void CheckGlobalBenchOptions(std::vector<std::string>* errors) {
732+
Mutex::ScopedLock lock(per_process::cli_options_mutex);
733+
per_process::cli_options->per_isolate->per_env->CheckBenchOptions(errors);
734+
}
735+
728736
static ExitCode ProcessGlobalArgsInternal(std::vector<std::string>* args,
729737
std::vector<std::string>* exec_args,
730738
std::vector<std::string>* errors,
@@ -834,8 +842,16 @@ int ProcessGlobalArgs(std::vector<std::string>* args,
834842
std::vector<std::string>* exec_args,
835843
std::vector<std::string>* errors,
836844
OptionEnvvarSettings settings) {
837-
return static_cast<int>(
838-
ProcessGlobalArgsInternal(args, exec_args, errors, settings));
845+
const ExitCode exit_code =
846+
ProcessGlobalArgsInternal(args, exec_args, errors, settings);
847+
if (exit_code != ExitCode::kNoFailure) return static_cast<int>(exit_code);
848+
// Embedders parse every option source in a single pass, so the benchmark
849+
// options can be validated right away.
850+
CheckGlobalBenchOptions(errors);
851+
if (!errors->empty()) {
852+
return static_cast<int>(ExitCode::kInvalidCommandLineArgument);
853+
}
854+
return static_cast<int>(ExitCode::kNoFailure);
839855
}
840856

841857
static std::atomic_bool init_called{false};
@@ -1016,6 +1032,11 @@ static ExitCode InitializeNodeWithArgsInternal(
10161032
if (exit_code != ExitCode::kNoFailure) return exit_code;
10171033
}
10181034

1035+
// Every option source has now been parsed, so cross-source option
1036+
// constraints can finally be validated.
1037+
CheckGlobalBenchOptions(errors);
1038+
if (!errors->empty()) return ExitCode::kInvalidCommandLineArgument;
1039+
10191040
// Set the process.title immediately after processing argv if --title is set.
10201041
if (!per_process::cli_options->title.empty())
10211042
uv_set_process_title(per_process::cli_options->title.c_str());

‎src/node_builtins.cc‎

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -158,15 +158,17 @@ BuiltinLoader::BuiltinCategories BuiltinLoader::GetBuiltinCategories() const {
158158
#if !HAVE_FFI
159159
"internal/ffi-shared-buffer", "internal/ffi/fast-api",
160160
#endif // !HAVE_FFI
161-
"dtls", // Experimental.
162-
"ffi", // Experimental.
163-
"quic", // Experimental.
164-
"sqlite", // Experimental.
165-
"stream/iter", // Experimental.
166-
"sys", // Deprecated.
167-
"vfs", // Experimental.
168-
"wasi", // Experimental.
169-
"zlib/iter", // Experimental.
161+
"bench", // Experimental.
162+
"bench/reporters", // Experimental.
163+
"dtls", // Experimental.
164+
"ffi", // Experimental.
165+
"quic", // Experimental.
166+
"sqlite", // Experimental.
167+
"stream/iter", // Experimental.
168+
"sys", // Deprecated.
169+
"vfs", // Experimental.
170+
"wasi", // Experimental.
171+
"zlib/iter", // Experimental.
170172
#if !HAVE_SQLITE
171173
"internal/webstorage", // Experimental.
172174
"internal/inspector/webstorage",

0 commit comments

Comments
 (0)