uutils

Pointing at the error: compiler-style diagnostics in uutils coreutils

2026-08-31T00:00:00+00:00

For 50 years, Coreutils have never stopped evolving. Now, we're pushing that innovation further by rethinking how they report errors.

Unix tools report errors as a single line on stderr. That line says what went wrong, not where. For most commands there is nowhere else to point anyway, but a few take arguments that are small languages: a test expression, a chmod mode, a sort key, a tr set. When one of those fails to parse, what you actually want to know is which argument, or which character of it, the parser tripped over.

rustc has been answering that question with a caret for years, and ariadne puts the same rendering one dependency away. The idea of bringing it to a command-line tool comes from uutils awk, which already reports errors in an awk program that way; coreutils arguments are smaller languages, but they parse just the same. Starting with 0.11.0, coreutils uses it. When stderr is a terminal, a parse error is printed as a report: the arguments are echoed back as a source line, a caret marks the culprit, and a help line explains the syntax when we have something useful to say about it.

What it looks like

Start with tr, whose GNU message assumes you already know what a collating sequence is.

Before:

$ tr 'qw[y-b]' x

tr: range-endpoints of 'y-b' are in reverse collating sequence order

After:

$ tr 'qw[y-b]' x

tr: range-endpoints of 'y-b' are in reverse collating sequence order
   ╭─[ tr:1:7 ]
   │
 1 │ tr qw[y-b] x
   │       ─┬─
   │        ╰─── did you mean 'b-y'?
   │
   │ Help: a range goes from the lower character to the higher one, as in a-z
───╯

Try it in the playground.

A cut list can be long, with a single bad item in it.

Before:

cut

$ cut -f 1,4-2,9-12 notes.txt

cut: invalid decreasing range
Try 'cut --help' for more information.

After:

cut

$ cut -f 1,4-2,9-12 notes.txt

cut: invalid decreasing range
   ╭─[ cut:1:10 ]
   │
 1 │ cut -f 1,4-2,9-12 notes.txt
   │          ─┬─
   │           ╰─── this range ends before it starts
   │
   │ Help: a list is N, N-M, N- or -M, separated by commas, as in -f1,4-6,9-
───╯
Try 'cut --help' for more information.

Try it in the playground.

The caret does not have to cover a whole argument. It can land on one character.

Before:

chmod

$ chmod 'g+rw?x' notes.txt

chmod: invalid operator (expected +, -, or =, but found ?)

After:

chmod

$ chmod 'g+rw?x' notes.txt

chmod: invalid operator (expected +, -, or =, but found ?)
   ╭─[ chmod:1:5 ]
   │
 1 │ g+rw?x notes.txt
   │     ─
   │
   │ Help: a mode is either octal, as in 644, or clauses such as u+rwx,go-w
───╯

sort keys are short enough that a stray character is easy to miss.

Before:

sort

$ sort -k2.3x notes.txt

sort: stray character in field spec: invalid field specification '2.3x'

After:

sort

$ sort -k2.3x notes.txt

sort: stray character in field spec: invalid field specification '2.3x'
   ╭─[ sort:1:11 ]
   │
 1 │ sort -k2.3x notes.txt
   │           ─
   │
   │ Help: a key is FIELD[.CHAR][OPTS][,FIELD[.CHAR][OPTS]], as in -k2.3,4nr
───╯

Try it in the playground.

env -S takes a whole command line and splits it the way a shell would. The old message could only quote the offending fragment back at you. Note that the string contains spaces, so it is echoed back quoted, and the caret still lands inside the quotes.

Before:

env

$ env -S 'echo ${1FOO}'

env: only ${VARNAME} expansion is supported, error at: ${1FOO}

After:

env

$ env -S 'echo ${1FOO}'

env: only ${VARNAME} expansion is supported, error at: ${1FOO}
   ╭─[ env:1:14 ]
   │
 1 │ env -S 'echo ${1FOO}'
   │              ─┬─
   │               ╰─── a variable name cannot start with a digit
   │
   │ Help: only $NAME and ${NAME} are expanded; the other shell forms are not
───╯

test builds its expression out of separate arguments. The report echoes the expression on its own, without the test in front, and marks the argument that broke it.

Before:

test

$ test 7 -eq zap

test: invalid integer 'zap'

After:

test

$ test 7 -eq zap

test: invalid integer 'zap'
   ╭─[ test:1:7 ]
   │
 1 │ 7 -eq zap
   │       ───
   │
   │ Help: -eq, -ne, -lt, -le, -gt and -ge compare integers; use =, !=, < or > to compare strings
   │       -eq equal, -ne not equal, -lt less than, -le less than or equal, -gt greater than, -ge greater than or equal
───╯

Try it in the playground.

A SIZE is a number followed by a unit. The report says which half was rejected.

Before:

head

$ head -c 1fb notes.txt

head: invalid number of bytes: '1fb'

After:

head

$ head -c 1fb notes.txt

head: invalid number of bytes: '1fb'
   ╭─[ head:1:10 ]
   │
 1 │ head -c 1fb notes.txt
   │          ─┬
   │           ╰── not a known unit
   │
   │ Help: a size is a number and an optional unit: K, M, G and so on for 1024, KB, MB, GB for 1000
───╯

Try it in the playground.

One parser handles every SIZE in the suite, so the same report shows up for tail -c, truncate -s, split -b, shred -s, od -N, sort -S, the block sizes of du -B, df -B and ls --block-size, and the threshold of du -t.

numfmt --format is a printf-style format that allows exactly one conversion. The old message just restated the rule. The annotation names the conversion you actually wrote.

Before:

numfmt

$ numfmt --format=%q 1000

numfmt: invalid format '%q', directive must be %[0]['][-][N][.][N]f

After:

numfmt

$ numfmt --format=%q 1000

numfmt: invalid format '%q', directive must be %[0]['][-][N][.][N]f
   ╭─[ numfmt:1:18 ]
   │
 1 │ numfmt --format=%q 1000
   │                  ┬
   │                  ╰── f is the only conversion numfmt has; %d, %e, %g and the other C conversions are not accepted
   │
   │ Help: a format is [PREFIX]%[0]['][-][WIDTH][.PRECISION]f[SUFFIX], as in "%'-10.2f"
───╯

Try it in the playground.

csplit patterns contain regexes, and the regex engine already knows which character it choked on. We were simply throwing that position away.

Before:

csplit

$ csplit notes.txt '/a{2,1}/'

csplit: '/a{2,1}/': invalid pattern

After:

csplit

$ csplit notes.txt '/a{2,1}/'

csplit: '/a{2,1}/': invalid pattern
   ╭─[ csplit:1:20 ]
   │
 1 │ csplit notes.txt /a{2,1}/
   │                    ──┬──
   │                      ╰──── invalid repetition count range, the start must be <= the end
   │
   │ Help: a pattern is a line number N, /REGEXP/[OFFSET] or %REGEXP%[OFFSET], each optionally followed by {N} or {*}
───╯

Try it in the playground.

That label comes straight from the regex engine and is not translated, since it is the only place the wording exists.

Where it applies

28 utilities use it in 0.11.0. The linked examples run in the playground; the others are for utilities the WebAssembly build does not ship, so try those locally:

Utility	What the caret points at	Try it
`test`	the argument that made the expression fail	`test 7 -eq zap`
`expr`	the argument that made the expression fail	`expr 9 + foo`
`chmod`	the failing clause (or character) of an invalid symbolic or octal mode	`chmod 'g+rw?x' fruits.txt`
`mkdir`	the failing part of the mode given to `-m`/`--mode`	`mkdir -m u+q mydir`
`mkfifo`	the failing part of the mode given to `-m`/`--mode`	`mkfifo -m u+q mypipe`
`mknod`	the failing part of the mode given to `-m`/`--mode`	`mknod -m u+q mydev c 1 3`
`install`	the failing part of the mode given to `-m`/`--mode`	`install -m u+q fruits.txt dest`
`tr`	the part of a set that is at fault (bad class, backwards range, bad repeat count, …)	`tr 'qw[y-b]' x`
`sort`	the failing part of a `-k`/`--key` or field specification, or of the SIZE given to `-S`	`sort -k2.3x fruits.txt`
`numfmt`	the failing part of a `--format` or `--field` specification, the value given to `--from`, `--to`, `--from-unit`, `--to-unit`, `--padding` or `--header`, or the input number itself	`numfmt --format=%q 1000`
`printf`	the failing conversion or escape in the format string	`printf %5.2c q`
`seq`	the failing conversion in the format given to `-f`/`--format`	`seq -f %5.2c 1 3`
`stat`	the failing directive of a `-c`/`--format` or `--printf` format	`stat -c %d%.3 fruits.txt`
`env`	the failing part of a `-S`/`--split-string` string	`env -S 'echo ${1FOO}'`
`dd`	the failing key, value or flag of a `KEY=VALUE` operand	`dd conv=ucase,zap`
`join`	the failing field of the output format given to `-o`	`join -o 1.2,2.x fruits.txt fruits.txt`
`cut`	the failing range in the list given to `-b`, `-c`, `-f` or `-F`	`cut -f 1,4-2 fruits.txt`
`csplit`	the failing pattern operand, the character of its regex that broke, or the format given to `-b`/`-n`	`csplit fruits.txt '/a(b/'`
`split`	the failing part of the SIZE given to `-b`, `-C` or `-l`	`split -b 7zq fruits.txt`
`shred`	the failing part of the SIZE given to `-s`/`--size`	`shred -s 4vv fruits.txt`
`head`	the failing part of the SIZE given to `-c` or `-n`	`head -c 1fb fruits.txt`
`tail`	the failing part of the SIZE given to `-c` or `-n`	`tail -c 1fb fruits.txt`
`truncate`	the failing part of the SIZE given to `-s`/`--size`	`truncate -s 10fb fruits.txt`
`od`	the failing part of the SIZE given to `-j`, `-N`, `-S` or `-w`	`od -N 3zz fruits.txt`
`du`	the failing part of the SIZE given to `-B`/`--block-size` or `-t`/`--threshold`	`du -B 1fb`
`df`	the failing part of the SIZE given to `-B`/`--block-size`	`df -B 1fb`
`ls`	the failing part of the SIZE given to `--block-size` (also `dir` and `vdir`)	`ls --block-size=1fb`
`stdbuf`	the failing part of the buffering mode given to `-i`, `-o` or `-e`	`stdbuf -o 6pq head`

Compatibility first

Being a drop-in replacement for GNU coreutils comes first, so this is strictly an interactive nicety:

Reports are only rendered when stderr is a terminal. In a script, a pipe or a test suite, each utility keeps printing exactly the plain one-line message shown as "Before" in the examples above, so anything that greps stderr keeps working.
Exit codes are unchanged.
Colors are only used on a terminal and respect NO_COLOR.
Like the rest of uutils, the messages, labels and help lines are localized; translations are managed on Weblate.
It can be compiled out to save a little space: the rendering sits behind the feat_diagnostics cargo feature (on by default), and building without it drops the ariadne dependency while every utility keeps its plain messages.

Turning it on and off

By default the rendering keys off stderr being a terminal and nothing else, which is usually but not always what you want. UUTILS_DIAG overrides it: always draws the report even into a file or a pipe, never keeps the plain line even at a terminal, and auto, or an unset variable, decides from stderr as before. An unrecognized value is deliberately not an error. This is the kind of variable people export from a shell profile once and forget about, and a typo in it should not be able to make a utility fail.

There is no command-line flag to go with it. The utilities that would need one most cannot have it: in test, printf and expr, a new option would be either illegal or ambiguous with the operands themselves.

To get a report out of a script or a CI log, to paste into a bug report for instance:

$ UUTILS_DIAG=always sort -k2.3x notes.txt 2> parse.log
$ cat parse.log
sort: stray character in field spec: invalid field specification '2.3x'
   ╭─[ sort:1:11 ]
   │
 1 │ sort -k2.3x notes.txt
   │           ─
   │
   │ Help: a key is FIELD[.CHAR][OPTS][,FIELD[.CHAR][OPTS]], as in -k2.3,4nr
───╯

Colors are decided separately, and still by the terminal: a report forced into a file is written without them, so there are no escape sequences to strip back out. NO_COLOR sits in between at a terminal, where the report is still drawn, just in plain text.

The two tricks people used before the variable existed still work. Sending stderr somewhere that is not a terminal gets the plain line:

$ sort -k2.3x notes.txt 2>&1 | cat
sort: stray character in field spec: invalid field specification '2.3x'

And giving a command a pty (script -qec "sort -k2.3x notes.txt" /dev/null, or unbuffer from expect) gets the report back, which is handy when the command has to run under a terminal for other reasons.

What comes next

coreutils is where this starts, not where it stops. The rendering lives in uucore::diagnostics, which the other uutils projects already depend on, so picking it up is mostly a matter of handing the parser's error a span. findutils and sed are being wired up now; a find expression and a sed script are exactly the kind of small languages a caret helps with, and grep, awk and the rest have the same regexes and format strings to point at.

If a utility you use still prints an unhelpful one-liner, patches are welcome!

Try Uutils Coreutils directly in your browser

2025-04-04T00:00:00+00:00

We are happy to announce the Uutils playground - an interactive terminal that lets you run real Rust Uutils Coreutils directly in your browser, with no installation, no server, and no network round-trips after the initial page load.

Just open the page and start typing commands like ls, sort, cat, head, factor, or any of the 60+ utilities available. You can pipe commands together (sort fruits.txt | uniq -c | sort -rn), use redirections (echo hello > file.txt, cat < file.txt), explore the virtual filesystem, and even switch between 30+ languages using the locale dropdown.

How it works

The Uutils Coreutils are compiled to WebAssembly (targeting wasm32-wasip1) as a single multicall binary - similar to BusyBox. When you type a command, a minimal JavaScript shell parses your input, sets up pipes between stages, and executes each one by instantiating the WASM binary with the right arguments. A virtual in-memory filesystem backed by browser_wasi_shim provides file I/O. Everything runs client-side - once loaded, the playground works entirely offline.

An important distinction worth noting: coreutils only provides individual commands (sort, cat, ls, etc.). Features like if/then/else, while loops, variable expansion ($VAR), and globbing (*.txt) are shell features provided by Bash, Zsh, or similar - not by coreutils. The playground's JavaScript shell is intentionally minimal, keeping the focus on the coreutils themselves.

For a deeper look at the architecture - including diagrams, the multicall dispatch mechanism, the feat_wasm feature gate, and how localization is handled - check out the full technical deep-dive.

Try it, break it, improve it

Give the playground a spin and let us know what you think! Contributions are welcome - whether it is improving WASI compatibility to enable more commands, adding translations via Weblate, or enhancing the playground shell itself.

Extending the Coreutils project - Rewriting base tools in Rust

2025-02-01T00:00:00+00:00

As I just presented at FOSDEM, we are moving into the next phase of the uutils project.

Over the last five years, we have been working on reimplementing some of the key Linux tools in Rust. We started with the Coreutils and findutils.

As we approach feature parity with the GNU Coreutils implementation, and as its adoption in production environments continues to expand, we have been thinking about what is next.

Given the very positive feedback around this initiative, we are going to extend our efforts to rewrite other parts of the modern Linux/Unix/Mac stack in Rust (still with Windows support in mind when relevant). These efforts will be managed under the uutils umbrella, which will allow us to maintain a cohesive and unified approach across various utilities.

We also noticed a lot of contributions on these projects coming from a diverse group of hackers (530 different contributors on Coreutils alone!). With the growing enthusiasm for Rust and the eagerness to learn it, now is the best time to push this project forward. Rewriting in Rust will help with the long-term maintenance of the ecosystem, ensuring it stays robust, safe, and welcoming for new generations of contributors.

Vision for the Future

As we expand the scope of the project, we are working on a future where Rust becomes the backbone of essential Linux, Unix, and potentially macOS system tools. Our focus will be on improving security, ensuring long-term maintainability, and optimizing performance for both modern and legacy systems. Through the uutils umbrella, we aim to create a more secure, efficient, and scalable alternative to the traditional tools, replacing software developed in older, less secure programming languages like C.

This project is also an investment for the future. C is becoming less popular with the next generation of developers. Just like how the Linux kernel continues to evolve and includes Rust, it is our responsibility to upgrade the core utilities of the system to a modern, safer programming language. By doing so, we ensure that the essential tools used in Linux/Unix systems remain relevant, maintainable, and accessible to future contributors.

Specific Benefits of Rust

Rust offers several advantages over C/C++, particularly in terms of memory safety, concurrency, and performance. By eliminating common security issues such as null pointer dereferences and buffer overflows, Rust allows us to build tools that are inherently safer. Its modern compiler tooling also enables us to optimize performance without sacrificing code safety, ensuring that our utilities can handle the demands of modern computing environments while remaining easy to maintain.

Challenges and Opportunities

As with any large-scale reimplementation, there are challenges to overcome. Porting complex utilities from C to Rust requires careful consideration of edge cases and platform-specific behaviors. Additionally, ensuring full compatibility across Linux, Unix, and macOS environments presents unique challenges. However, these challenges also offer opportunities for growth and innovation, allowing us to extend and refine core system utilities for modern needs. Rust's modern design gives us the chance to improve both security and performance while maintaining cross-platform compatibility.

Next Steps

For the next phase of the project, we are adopting the same approach: a drop-in replacement of the C implementations. Here's what's coming next:

Name	Description	Status
acl	Access control list utilities
bsdutils	Basic utilities for BSD compatibility
coreutils	Basic utilities for the system	Production level
diffutils	File comparison utilities	Almost ready
findutils	Utilities for finding files	Getting close to completion
hostname	Utility to show or set system hostname
login	User login management utilities
procps	Utilities for monitoring and controlling processes
util-linux	Utilities essential for Linux systems

These packages are part of the essential list for Debian and Ubuntu, and we're excited to push their Rust reimplementation further.

GSoC 2024 Participation

In 2024, we had the pleasure of mentoring three students during Google Summer of Code (GSoC):

Sreehari Prasad worked on improving the support of Rust CCoreutils. His focus was on making uutils compatible with the GNU coreutils test suite. Sreehari resolved most of the failing tests for the cp, mv, and ls utilities and significantly enhanced compatibility.
Hanbings tackled the implementation of key GNU findutils utilities like xargs and find in Rust. His work focused on improving compatibility with the GNU suite while enhancing performance, resulting in major progress on missing features and test code.
Krysztal Huang worked on implementing the procps suite, which includes utilities like slabtop, top, pgrep, pidof, ps, pidwait, and snice. This project involved implementing some of these commands.

Call to Action for Contributors

Contributors who are passionate about system-level programming and Rust are always welcome. Whether you're experienced with GNU utilities or just learning Rust, your contributions will be invaluable to this project. You can get involved by picking up good-first issues, reviewing code, or even helping us test new features across various platforms. The uutils GitHub organization has all the information you need to get started. You can also sponsor the project through GitHub Sponsors.

FAQ

How long is it going to take?

Some programs, like diff, acl, or hostname, could be completed quickly, while others will take years to finish.

Do you hate the GNU project?

Not at all. We often contribute to the upstream implementations when we identify issues, add missing tests, and so on.

Why the MIT License?

For consistency purposes. We're not interested in a license debate and will continue to use the MIT license, as we did with Coreutils.

The binaries are too big. Does it really matter?

Yes, Rust binaries are generally bigger than GNU's, but we don't think it's a blocker, even for embedded devices.

Is it really safer?

While Rust is better than C/C++ for security, these programs are often already very safe. Security is not the key argument for us in this rewrite even if Rust provides some automatic improvements.

What about performance?

Although we haven't started optimizing yet, Rust's design facilitates performance improvements naturally. We are confident that, in time, these tools will match or exceed the performance of their GNU counterparts.