CI CHECK · PYTHON STDLIB · ONE FILE

My portfolio published tests that did not exist.

Seven of my own landing pages described test suites nobody had written. They were live for nine days — while my CV linked to those pages inviting the reader to go and audit them. The fix that day was manual. This is the part that should not have been: a check that reads a published page, pulls out every claim that can be verified, and fails the build when the repository does not back it.

7 pages
describing a suite that was never written
9 days
live, while the CV invited an audit
5 checks
run against the repo as it is on the remote
0 deps
standard library, one file, no install
The incident

Generated in a batch. Nobody read it line by line.

On 16 July 2026 I audited my own portfolio. What I found was not a typo.

01

A suite that never existed

The same invented test_robustness file appeared in six repositories, with coverage figures and latencies nobody had ever run. All seven pages carried the same date. One of them went as far as insisting in writing that the figures were measured rather than estimated.

02

A promise that broke in one click

Three landings had a "View the code" button pointing at repositories that publish only a licence and a readme. An architecture write-up is real work and sells as such — what breaks trust is letting a reader infer an implementation, then losing them on the next click.

03

A rule enforced on two surfaces out of three

A hardware detail leaked into seven public readmes while the CV and the landings used the careful phrasing. The rule existed and was being applied — everywhere people look. A rule applied only where people look is a coat of paint.

04

A clone that was behind

Documenting from a stale local checkout describes software that no longer exists. And raw.githubusercontent.com cheerfully returns HTTP 200 for branches that were never there, so "I checked the raw file" proves nothing at all.

The rule was already written down

"No source in the repo, it does not get written" had been my rule for months. It is a good rule. It failed because enforcing it depended on a person reading every line of every page every time — and the failure mode is not malice, it is completion. Text flows, one more sentence sounds right, and the sentence is furniture. Writing the rule down is what failed. So it became code.

What it checks

Five checks, four of them fatal.

CheckFails whenDefault
denylistA term that must never reach a public surface appears on oneFAIL
path-claimThe page names a file path that is not in the repositoryFAIL
test-claimThe page names a test that appears in no source fileFAIL
code-promiseThe page says "view the code" and the repo publishes almost noneFAIL
measured-claimA number is presented as measured with no source markerWARN

To mark a number as sourced, put the source beside it. The gate does not re-derive the figure — it insists that somewhere, someone can.

landing/index.html
<!-- source: runs/2026-08-04-rtx.json -->
<p>Measured throughput: 10.5 tok/s.</p>
Where it looks

It verifies against the remote, not your disk.

Every target is fetched fresh from origin before anything is checked. That is not caution, it is the second half of the same incident: the checkout on disk was behind, and a page validated against it would have passed while lying. Whichever branch the remote hands back is the one that counts — so a portfolio split across main and master never has to be configured.

Pages can be local files, checked before they ship, or URLs, checked as actually served. The second one is what a reader really clicks.

shallow clone per run default branch from the remote local file or live URL exit 1 on any FAIL
Calibration

Two decisions that keep it honest.

A gate that cries wolf gets switched off, and a gate switched off is worse than none — it leaves the belief that something is watching. Both calibrations below are rules, not conveniences.

AMBIGUITY, NOT EXCEPTIONS

A bare filename warns; a path fails

tests/test_robustness with a directory in it can only be a file in this repository — if it is missing, the page lies. A bare name might be a third-party library. The dividing line is how ambiguous the token is, never a hand-maintained list of exceptions, because that list is the thing that rots.

DO NOT PUNISH HONESTY

A sentence in the negative is not a claim

"There is no coverage measurement in this repository, so this page will not quote you one" is exactly the writing the rule wants. A gate that flags it teaches you to stop declaring your ceilings — which is how you get back to the original problem by a different road.

Building this surfaced a third one, against itself: the ruleset is a list of the things you do not publish, so committing the ruleset publishes them. The real one stays out of the repository; the example shipped here is generic.
Tested in red

Both fixtures, or neither proves anything.

Two pages ship with the repository. The red one reproduces each real mistake and must trip all five checks and exit 1. The green one is the honest version of the same page — correct phrasing, a file that exists, a sourced number, a ceiling declared out loud — and must pass in silence.

R

The page that lies

Every block is a mistake this portfolio actually made. If any check stops tripping, the gate has quietly stopped working.

exit 1
G

The honest page

The same content, written correctly. If anything trips here, the gate is punishing the writing it exists to reward.

exit 0
The second fixture is the one that matters. A check only ever seen passing tells you nothing. A check only ever seen failing tells you nothing either. What you need to know is that it can tell the two apart — and the only way to know that is to show it both.
The ceiling

What this does not prove.

This verifies verifiable claims. It reads a page, extracts the things that can be checked against a working tree, and checks them. It cannot tell you whether a sentence is true.

"Designed for high availability" passes, because there is nothing in it to check. A number with a source marker passes because the marker is there — the gate does not open the file and recompute it. Those are real holes and I am not going to pretend otherwise on the page whose entire subject is not doing that.

What it does close is the confident, checkable, fabricated detail: the test name, the file path, the measured figure, the promise of code. That is the hole I actually fell into. It is also the one a language model writing your copy will find again, for the same reason I did — not dishonesty, just the pull to finish the paragraph. Everything else still needs a human willing to be wrong.
no truth detection no recomputation of sourced numbers vague prose passes a human still reads it