The conformance suite

The invariants on this site are not a description of intent. Each one has a test that tries to break it against a running control plane, and those tests are a required CI job.

The suite talks to the instance through its public surface only — no access to internals, no references to the server’s own projects. It is maintained by the maintainers rather than generated alongside the code it checks, so it cannot be adjusted by the thing it tests. Remove a check from the server and the corresponding test fails.

Every case also checks that the refusal reached the audit ledger. A rule that is enforced but not recorded only half exists.

The twelve

#	Test	The check it guards
1	The subject is always the human	sub is the human and the agent only ever in act; a subject token that names an agent is invalid_grant
2	Scope at exchange is the intersection	user ∩ agent ∩ request, nothing outside it; an empty intersection is invalid_scope
3	Scope only narrows on refresh	a scope the task was not granted is invalid_scope even when the user and the agent allow it
4	The audience must be allowed, and a task is bound to it	invalid_target for an audience outside allowed_audiences, and for a refresh to another audience
5	A token never outlives its task	exp is bounded by the task’s expiry at issue and at refresh, not only by max_token_ttl
6	A client assertion is accepted once, from the registered key	a replay, a forged signature and an unknown agent are all invalid_client
7	Revocation and disabling take effect at once	the kill switch, agent-side revocation and disabling refuse refresh, and introspect as inactive with the reason
8	A subject token is trusted only from the identity provider	expired, foreign-signed, wrong audience and wrong issuer are all invalid_grant
9	A task dies with the human it acts for	a sponsor disabled or deleted at the identity provider fails the task’s next refresh, as access_denied
10	The audit ledger is a chain	every record’s prev_hash is the previous record’s hash, so a record removed or inserted shows
11	A high-risk audience says so in the token	introspect_required is true for an audience in high_risk_audiences and absent for any other
12	No refusal hands back the credential it refused	no subject token, client assertion or task grant appears in an error body, whole or in part

Read against the five invariants: tests 1, 2 and 3 hold the first two rules, test 5 holds the third, tests 9 and 10 and every test’s ledger assertion hold the fourth, and test 12 holds the fifth.

What it stands up

The suite stands in for everything the instance calls out to: the upstream identity provider — its discovery document, its JWKS, the admin users API and the token endpoint the sponsor check uses — and every agent’s JWKS. The instance is pointed at one HTTPS base URL for all of it, and the suite listens there with a certificate the instance trusts.

The URL must be HTTPS, because agent JWKS are only ever fetched over HTTPS.

Variable	Meaning
ONBE_CONFORMANCE_URL	Base URL of the instance under test
ONBE_CONFORMANCE_ADMIN_KEY	Its Onbe:Admin:ApiKey
ONBE_CONFORMANCE_STUB_URL	The HTTPS base URL the instance was given for the identity provider and agent JWKS
ONBE_CONFORMANCE_STUB_PFX	A password-less PKCS#12 file for that host; the stub keeps the identity provider’s signing key beside it, so repeated runs against one instance present the same key

What it needs of the instance

The suite drives the instance through its public surface, so a few settings have to be within reach of what the checks assume. These are the shipped defaults, but a conformance job sets them rather than relying on that:

Setting	Required	Why
Onbe:Audit:Aggregation:Window	no longer than a minute	A denial that names nobody is written once per window; the suite looks one minute back for it
Onbe:Agents:MinTaskTtl	no longer than PT1M	Case 5 registers a one-minute task to watch a token be cut to what is left of it
Onbe:Tokens:DefaultTaskTtl	at least PT1M	Case 5 refreshes twenty seconds into that task
Onbe:RateLimit:Burst	at least 120	The suite makes around seventy requests from one address in a few seconds, counted as one source
Onbe:UpstreamIdp:SponsorCheck:CacheTtl	anything	Case 9 uses a different person for each outcome, so no answer is ever served from the cache

Running it against your own instance

The suite is not only for this project’s CI. Pointing it at an instance you operate is the way to find out whether your deployment still keeps the rules — after an upgrade, after a configuration change, after somebody adds a feature.

ONBE_CONFORMANCE_URL=https://onbe.internal.example.com \
ONBE_CONFORMANCE_ADMIN_KEY=$ONBE_ADMIN_KEY \
ONBE_CONFORMANCE_STUB_URL=https://localhost:5199 \
ONBE_CONFORMANCE_STUB_PFX=./stub.pfx \
dotnet test tests/Onbe.Conformance

It registers its own agents and leaves records in the ledger, so run it against an instance you are willing to write to.

Missing configuration or an unreachable instance fails every test with the reason, and the CI job fails if the suite runs no tests at all. A green conformance run that ran nothing would be worse than a red one.