Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ from the root; changing directory loses assistant context.
| **Quick** | every non-draft PR push | fmt, lint, check, host tests, dependency license policy (NOTICE required; C/C++ graphs wait for Full scancode). Target under 10 minutes (hal 15). |
| **Full** | `ci:full` or `ci:hardware` label, `workflow_dispatch`, or merge queue | platform matrix, boards, coverage, scancode. Once per PR, not once per push. |
| **Nightly** | schedule, only if `main` moved | Full plus slow suites. Quiet days cost nothing. |
| **Advisories** | every schedule, gated or not | `cargo audit`. See the exception below. |

Labels:

Expand All @@ -57,6 +58,7 @@ The tier decides, not the job. What is being optimised differs per tier:
| **Full** | speed, cost accepted | `larger` |
| **Release** | speed, cost accepted | `larger` |
| **Nightly** | speed, cost accepted | `larger` (nothing blocks on it; see below) |
| **Advisories** | cost, absolutely | `hosted`, hard-coded — see the exception below |

1. **hosted** — `ubuntu-24.04`, `ubuntu-24.04-arm`, `macos-latest`,
`windows-latest`. Free and unmetered on public repositories. This is the
Expand All @@ -79,6 +81,33 @@ such a lane names the label directly rather than going through
`runner-class-*`, put `# runner-class: larger` in the file with the reason so
the audit can tell the two apart.

### The one exception: advisory scanning

`advisories.yml` breaks both rules above on purpose, and the reasons are
narrow enough that nothing else should copy it.

**It is not gated.** Every other Nightly lane asks a question about the code,
so skipping an unchanged commit loses nothing. `cargo audit` asks a question
about the RustSec database, which changes daily whether the code does or not.
Gated, it goes quiet exactly when a new advisory lands against a frozen
`main` -- which is not hypothetical: the 2026-09-15 nightly failed on
RUSTSEC-2026-0204 against a `Cargo.lock` nobody had touched.

**It is `hosted`, not `larger`, and hard-codes the label** rather than
honouring the caller's `runner-class-*`. "Cost accepted" is affordable for
Nightly because the gate bounds it to days when `main` moved. An ungated lane
has no such bound, so it only stays affordable on a free class. Hard-coding
means a caller cannot put an every-night lane on a billed runner by setting
`runner-class-linux: larger` for unrelated reasons.

The exception is affordable only because the job is trivial: `cargo audit`
parses `Cargo.lock` and never compiles, so it skips `setup-rust` and is
seconds of a free runner. A lane that builds anything does not qualify --
gate it, or leave it out of the nightly.

`runner-audit` does not flag `advisories.yml`, because `ubuntu-24.04` is not a
billed label; it needs no `# runner-class: larger` marker.

## How to call the shared workflows

Pin the **commit SHA** of `EdgeFirstAI/.github` and record the tag in a comment:
Expand Down
52 changes: 52 additions & 0 deletions .github/workflows/advisories.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
name: advisories

# Advisory scanning, deliberately outside the nightly gate.
#
# Every other nightly lane answers a question about the code, so skipping it
# on an unchanged commit loses nothing. This one answers a question about the
# RustSec database, which changes daily whether or not the code does. Gated,
# it would go quiet exactly when a new advisory lands against a frozen main --
# which is not hypothetical: the 2026-09-15 nightly failed on RUSTSEC-2026-0204
# against a Cargo.lock nobody had touched.
#
# Callers run this ungated, so it must cost nothing. It reads Cargo.lock and
# does not build, so it is seconds of a free standard runner.
#
# To downgrade an advisory to a warning, commit `.cargo/audit.toml` in the
# calling repository; cargo-audit reads it from the working directory. That is
# a reviewable file in the repo it applies to, which a workflow input would
# not be.

on:
workflow_call:
inputs:
timeout-minutes:
type: number
default: 10

jobs:
audit:
name: cargo audit
# Hard-coded, not the caller's runner-class. This lane runs every night
# regardless of the gate, so it must never land on a `larger` runner --
# those bill by the minute even on public repositories, where the standard
# classes are free. A caller cannot opt into a cost here by accident.
runs-on: ubuntu-24.04
timeout-minutes: ${{ inputs.timeout-minutes }}
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

# No setup-rust: cargo-audit parses Cargo.lock and never compiles, so a
# toolchain install and a cargo cache would be the only slow parts of an
# otherwise instant job. The runner image's cargo is enough to dispatch
# the subcommand, and if that ever stopped being true this fails loudly
# rather than skipping the scan.
- uses: taiki-e/install-action@d438492cf8a250514fa2d34b30bc3c0dc37c65ff # v2.87.8
with:
tool: cargo-audit

- run: cargo audit
27 changes: 27 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,33 @@ jobs:
sys.exit(1)
print("all scanned uses: refs are SHA-pinned")
PY
- name: Template self-pin placeholder check
run: |
python3 - <<'EOP'
import pathlib, re, sys
# templates/ pins EdgeFirstAI/.github to all zeros on purpose. A
# real SHA there is stale the moment this repository moves, and a
# stale one still resolves: a copied skeleton then silently runs
# old CI, or forwards an input the pinned revision does not
# declare. Both have happened. The shape check above cannot catch
# it, because a stale SHA is a valid SHA, so assert the
# placeholder instead. Third-party actions in templates/ keep
# real pins; only self-references are placeholders.
placeholder = "0000000000000000000000000000000000000000"
ref = re.compile(r"uses:\s*EdgeFirstAI/\.github/\S+@([0-9a-fA-F]{40})")
bad = []
for path in sorted(pathlib.Path("templates").rglob("*.yml")):
for lineno, line in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
match = ref.search(line)
if match and match.group(1) != placeholder:
bad.append(f"{path}:{lineno}: {line.strip()}")
if bad:
print("templates/ must pin EdgeFirstAI/.github to the placeholder:")
for item in bad:
print(item)
sys.exit(1)
print("all template self-pins use the placeholder")
EOP
- name: timeout-minutes check
run: |
python3 - <<'PY'
Expand Down
41 changes: 38 additions & 3 deletions .github/workflows/nightly-gate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@ name: nightly-gate

on:
workflow_call:
inputs:
force:
description: >-
Run even when main has not moved since the last nightly. This is the
only way to re-run on an unchanged commit, so it is also how you
re-try after fixing a lane rather than the code.
type: boolean
required: false
default: false
outputs:
run:
description: true when nightly Full should run
Expand All @@ -21,13 +30,39 @@ jobs:
- id: c
env:
GH_TOKEN: ${{ github.token }}
FORCE: ${{ inputs.force }}
run: |
set -euo pipefail
wf="$(basename "${GITHUB_WORKFLOW_REF%@*}")"
last="$(gh api "repos/${GITHUB_REPOSITORY}/actions/workflows/${wf}/runs?status=success&branch=main&per_page=1" --jq '.workflow_runs[0].head_sha // ""')"
if [[ "${GITHUB_EVENT_NAME}" == schedule && "$last" == "$GITHUB_SHA" ]]; then
# The question is "has main moved since we last looked at it", so
# what counts is a run that reached a verdict on a commit.
#
# success and failure both did: the lanes ran and reported. Filtering
# on success alone makes the gate useless in exactly the repository
# that needs it -- hal's last green nightly on main was 2025-12-08,
# so every comparison since would be against a nine-month-old commit
# and the gate would never skip. Re-running an unchanged commit after
# fixing a lane is what `force` is for.
#
# Every other conclusion is discarded, cancelled above all. A run can
# be cancelled before a single lane starts, so it proves nothing
# about its commit, yet it is still "completed" at that sha and would
# suppress the next night. Same for timed_out, stale, neutral,
# action_required and startup_failure. Discarding them looks further
# back instead, which at worst re-runs a nightly.
#
# status=completed also excludes this run, which is in_progress.
last="$(gh api "repos/${GITHUB_REPOSITORY}/actions/workflows/${wf}/runs?status=completed&branch=main&per_page=30" --jq '[.workflow_runs[] | select(.conclusion == "success" or .conclusion == "failure")][0].head_sha // ""')"
if [[ "$FORCE" == "true" ]]; then
echo "run=true" >> "$GITHUB_OUTPUT"
echo "forced (last=$last sha=$GITHUB_SHA)"
elif [[ "$last" == "$GITHUB_SHA" ]]; then
# Applies to every trigger, not just schedule. A manual dispatch
# used to bypass this silently, so re-dispatching a nightly
# rebuilt the same commit and re-ran the board -- the slowest and
# only hardware-bound lane -- for no new information.
echo "run=false" >> "$GITHUB_OUTPUT"
echo "main unchanged since last successful nightly ($last)"
echo "main unchanged since the last nightly ($last); pass force to override"
else
echo "run=true" >> "$GITHUB_OUTPUT"
echo "nightly will run (last=$last sha=$GITHUB_SHA event=$GITHUB_EVENT_NAME)"
Expand Down
12 changes: 7 additions & 5 deletions .github/workflows/rust-full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -760,9 +760,13 @@ jobs:
timeout-minutes: 20

nightly-extra:
name: Nightly extras
name: Feature combinations
needs: setup
if: ${{ inputs.nightly }}
# cargo audit used to live here too, which is why this was called "extras".
# It moved to advisories.yml so it runs outside the nightly gate. What is
# left is the hack lane, so skip the job entirely when the caller has no
# hack-args rather than booting a runner to install tools and run nothing.
if: ${{ inputs.nightly && inputs.hack-args != '' }}
runs-on: ${{ fromJSON(needs.setup.outputs.linux) }}
timeout-minutes: 45
permissions:
Expand All @@ -789,17 +793,15 @@ jobs:
bash --noprofile --norc -euo pipefail -c "$PRE_COMMAND"
- uses: ./.ef-ci/.github/actions/setup-rust
with:
tools: cargo-hack,cargo-audit
tools: cargo-hack
cache-key: nightly-extras
- name: Feature combinations
if: inputs.hack-args != ''
env:
HACK_ARGS: ${{ inputs.hack-args }}
run: |
set -euo pipefail
# shellcheck disable=SC2086 # HACK_ARGS is an argument list and must split
cargo hack check $HACK_ARGS --locked
- run: cargo audit

verify-version:
name: verify-version
Expand Down
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- `advisories.yml`, a reusable workflow running `cargo audit`, meant to be called **without** `needs: changed` so the nightly gate does not skip it. Every other nightly lane asks a question about the code, so skipping an unchanged commit loses nothing; this one asks a question about the RustSec database, which changes daily whether the code does or not. Gated, it goes quiet exactly when a new advisory lands against a frozen `main` -- not hypothetical, since the 2026-09-15 nightly failed on RUSTSEC-2026-0204 against a `Cargo.lock` nobody had touched. It parses `Cargo.lock` without building and pins `ubuntu-24.04` rather than honouring the caller's `runner-class`, so a caller cannot accidentally put an every-night lane on a `larger` runner, which bills by the minute even on public repositories. Downgrade an advisory by committing `.cargo/audit.toml` in the calling repository.

- `hack-args` on `rust-full`, for the nightly `cargo hack check` feature-combination lane, defaulting to the previous hardcoded `--feature-powerset --depth 2`. `--locked` is still always added, and an empty string skips the lane. A crate with mutually exclusive features needs `--exclude-no-default-features`: the powerset otherwise builds a combination the crate deliberately rejects with `compile_error!`, which is the guard working rather than a defect, and the invocation was previously hardcoded with no way for a caller to say so.

- `board-binaries` and `board-min-tests` on `rust-full`, and `binaries` / `min-tests` on the `board-run` action. `board-binaries` selects test binaries to run **directly, one process per binary**, given one per line as `<name-prefix>[=<filter>[,<filter>...]]` — a bare prefix runs the whole binary, filters run it once per filter as a libtest substring match. This is an alternative to `board-extra-args`, which routes the archive through nextest; callers that do not set `board-binaries` are unaffected. Use it for any board with a GPU. nextest runs one process per test, so each test pays a full driver init and teardown, and coverage instrumentation writes one profraw per process. Measured on an i.MX 8M Plus: 6 to 22 seconds per GL test and 307 profraw files totalling 2.8 GB, against 4.1 GB free, where whole-binary execution of the same selection is ~20 files and ~190 MB. Two behaviours come with it, both of which nextest's model cannot express: a non-zero exit is tolerated when the log shows a clean libtest summary with no failures, because the Vivante driver aborts while unloading its library after tests have passed; and `board-min-tests` fails the lane when fewer than that many tests ran, since a selection matching nothing otherwise reports success.
Expand Down Expand Up @@ -41,6 +43,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- `templates/` now pins every `EdgeFirstAI/.github` `uses:` to an all-zero placeholder instead of a real commit, and a new lint step enforces it. The pins had drifted to a commit from nine months ago, which is worse than it sounds: a stale SHA still *resolves*, so a copied skeleton silently ran old CI. This change made the failure mode concrete -- the skeleton referenced `advisories.yml`, which does not exist at the old pin, and forwarded a `force` input the old `nightly-gate` does not declare. An unresolvable ref fails immediately and says what to fix. Third-party actions in `templates/` keep real pins and can still be copied as-is.

- `rust-full`'s `nightly-extra` job no longer runs `cargo audit`; that moved to `advisories.yml` so it is not gated. The job is now named **Feature combinations**, which is all it still does, and it skips entirely when `hack-args` is empty rather than booting a runner to install tools and run nothing. Callers wanting advisory scanning must add the `advisories.yml` job, as `templates/nightly.yml` now does.

- `nightly-gate` now actually skips. Two things stopped it. It only applied its "has main moved" check on `schedule`, so a `workflow_dispatch` bypassed it silently and re-dispatching a nightly rebuilt the same commit, re-running the board -- the slowest and only hardware-bound lane -- for no new information. And it compared against the last **successful** run, which in a repository whose nightly is not consistently green means comparing against an arbitrarily old commit: hal's last green nightly on main was 2025-12-08, so the gate had not skipped once in nine months. The check now applies to every trigger and compares against the last run that reached a **verdict** -- `success` or `failure`, since both mean the lanes ran and reported. Other conclusions are discarded, `cancelled` above all: a run can be cancelled before a single lane starts, so it proves nothing about its commit, yet it is still "completed" at that sha and would otherwise suppress the next night.

A new `force` input runs anyway. It is the only way to re-run an unchanged commit, which makes it the way to re-try after fixing a lane rather than the code. Callers that do not pass it get `false`. `templates/nightly.yml` forwards it as `${{ inputs.force || false }}` -- the fallback is required, because `inputs` is null on a schedule trigger.

- Runner policy is now per tier rather than a single cost-first ordering.
Quick stays on free standard runners and never bills; Full and Release
default to `larger`, which is what they were before this migration. Moving
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ The organisation profile README lives in [`profile/`](profile/README.md).
| `.github/workflows/python-quick.yml` | ruff + pytest |
| `.github/workflows/cmake-quick.yml` | ccache + ctest |
| `.github/workflows/nightly-gate.yml` | skip nightly when `main` is unchanged |
| `.github/workflows/advisories.yml` | `cargo audit`, run ungated so a new advisory is still reported |
| `.github/workflows/sbom.yml` | `dependency` or `full` scancode |
| `.github/workflows/tag-release.yml` | `release/X.Y.Z` merge → annotated `vX.Y.Z` |
| `.github/workflows/release-rust.yml` | crates OIDC, wheels as artifacts, GitHub Release |
Expand Down
14 changes: 11 additions & 3 deletions templates/ci.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Caller skeleton — copy into the product repository as .github/workflows/ci.yml
# Pin every uses: to a 40-char EdgeFirstAI/.github commit. Dependabot bumps them.
# Replace the all-zero placeholder in every EdgeFirstAI/.github `uses:` with
# the commit you intend to pin, then let Dependabot bump it. Third-party
# actions below are pinned for real and can be copied as-is.
#
# The placeholder is deliberate. A real SHA here goes stale the moment the
# shared repository moves, and a stale one is worse than an obviously invalid
# one: it resolves, so a copied skeleton silently runs old CI, or forwards an
# input the pinned revision does not declare. An unresolvable ref fails
# immediately and says what to fix.

name: CI

Expand Down Expand Up @@ -73,7 +81,7 @@ jobs:
quick:
needs: changes
if: needs.changes.outputs.code == 'true'
uses: EdgeFirstAI/.github/.github/workflows/rust-quick.yml@eec0cb31b6576a47735099b91e39a9bdb5fbde3a
uses: EdgeFirstAI/.github/.github/workflows/rust-quick.yml@0000000000000000000000000000000000000000
with:
python: false
# Optional: caller setup after checkout (OpenCV, ANGLE, testdata).
Expand All @@ -83,7 +91,7 @@ jobs:
full:
needs: changes
if: needs.changes.outputs.full == 'true' || needs.changes.outputs.hardware == 'true'
uses: EdgeFirstAI/.github/.github/workflows/rust-full.yml@eec0cb31b6576a47735099b91e39a9bdb5fbde3a
uses: EdgeFirstAI/.github/.github/workflows/rust-full.yml@0000000000000000000000000000000000000000
with:
lanes: ${{ needs.changes.outputs.full == 'true' && 'all' || 'hardware' }}
boards: nxp-imx8mp-latest
Expand Down
Loading