Fix CFF2 stem and hint mask stripping - #339

Background

For node mode PDF embedding in LuaTeX, luaotfload's fontloader converts CFF and CFF2 charstrings into Type 2 streams. For CFF2 variable fonts, the converter is meant to strip stem hints, so the output is unhinted. CFF2 support in the fontloader has been fixed several times before: commit 9ced93d fixed CFF2 INDEX and FDArray reading, commit 18e81f3 handled multiple ItemVariationData subtables, and commit 9ab4495 fixed hint mask handling in the separate HarfBuzz variable CFF2 path.

This PR fixes two bugs in the justpass converter in the default and reference fontloaders.

Problem

With Source Serif 4 Variable Italic in node mode under luaotfload 3.29, some CFF2 glyphs produce empty or corrupt PDF glyph streams. Greek capital Beta (U+0392) is one example.

The converter should produce a complete unhinted Type 2 stream. Instead, it mishandles the vstemhm operator, and it keeps hint masks after removing the stems they refer to. The first bug derails parsing of the input charstring, and the second leaves the output stream inconsistent.

Evaluation

justpass checks the current operator t for stem operators 1, 3, and 18, but for operator 23 (vstemhm) it checks operation, which is undefined. So vstemhm falls through to generic operator handling, its operands and opcode are copied to the output, and the operand stack is cleared. Since this skips p_getstem(), those stems are not counted, and a following mask can be computed as too short. When that happens, the parser reads too few mask bytes and treats the rest as charstring instructions. For CFF2, it also leaves in a stem operator that should have been stripped.

Even after fixing the above, a second bug remains. The converter drops CFF2 stem declarations but still emits each hintmask or cntrmask along with its mask bytes. Those masks refer to stems that are not in the output, so the stream can be misinterpreted. To produce unhinted output, the converter has to drop both stems and masks, while still counting input stems and skipping over all input mask bytes.

The unconditional mask emission goes back to the ConTeXt fontloader import in commit c98b44c (December 2018). That import added the justpass stem and mask branches behind if true, with unused else branches that already discard the CFF2 data.

The February 2022 import in commit 36750eb switched the stem branch to if version == "cff", so CFF2 stems going through that branch are now stripped. In the mask branch right next to it, the same import added the matching condition but commented it out:

-- if version == "cff" then
if true then

The commit doesn't say why the two branches differ. Both are annotated cff 1: (when cff2 strip them), and both have a discard path. Using the same condition in both matches the stated intention to strip CFF2 hints.

Solution

The fix is split into two commits:

1. Change operation == 23 to t == 23, so vstemhm goes through stem counting and version-dependent emission.
2. In the mask branch, replace if true with if version == "cff". For CFF2, the existing else branch skips the mask bytes and clears the operand stack.

With these changes, CFF1 keeps its hints, vstemhm stems are counted correctly, and CFF2 output has both stems and masks stripped. The mask change does not affect CFF1. The dispatch fix does, in a good way: CFF1 mask parsing no longer undercounts stems after a vstemhm.

Both fixes are applied to src/fontloader/misc/fontloader-font-cff.lua, the generated src/fontloader/runtime/fontloader-reference.lua, and the bundled src/auto/fontloader-2023-12-28.lua. The default configuration selects the bundled loader through luaotfload-status.lua, so updating it makes the fixes available without a configuration override.

Testing Done

In my project, the following test passes with both fixes applied, using LuaHBTeX 1.24.0 from TeX Live 2026:

• Select Source Serif 4 Variable Italic in node mode.
• Typeset Greek capital Beta.
• Get the glyph stream from fonts.hashes.streams.
• Assert that the stream exists and is 141 bytes long.
• Check that the glyph renders correctly (visually).

I haven't added an integration test, because Source Serif 4 Variable Italic isn't in luaotfload's test support files.

Notes to Reviewers

Start with src/fontloader/misc/fontloader-font-cff.lua. The changes in src/fontloader/runtime/fontloader-reference.lua and src/auto/fontloader-2023-12-28.lua apply the same fixes to the reference and default runtime files.

Since fontloader-font-cff.lua comes from ConTeXt, these fixes should also go upstream to ConTeXt so the next fontloader import doesn't undo them. I'd be happy to work on that once this PR is merged.