The conformance suite

A copy, not the original

This page is docs/conformance.md from the Onbe control plane repository, taken at commit d516e99 on 2026-09-16. The contract lives next to the code that implements it; this copy is generated, never edited here, and the build fails if the two drift apart.

tests/Onbe.Conformance is the spec’s teeth: twelve adversarial cases, run against a live instance through its public surface only, each guarding one server check and each also checking that the refusal reached the audit ledger. It is maintained by the maintainers, references no server project, and is a required CI job; if a check is removed from the server, the corresponding test fails.

#	Test	The check it guards
1	The subject is always the human	sub is the human, the agent only ever in act; a subject that names an agent is invalid_grant
2	Scope at exchange is the intersection	user ∩ agent ∩ request, nothing outside it; an empty intersection is invalid_scope
3	Scope only narrows on refresh	a scope the task was not granted is invalid_scope even when user and agent allow it
4	The audience must be allowed and a task is bound to it	invalid_target for an audience outside allowed_audiences, and for a refresh to another audience
5	A token never outlives its task	exp is bounded by the task’s expiry at issue and at refresh, not only by max_token_ttl
6	A client assertion is accepted once, from the registered key	replay, a forged signature and an unknown agent are invalid_client
7	Revocation and disabling take effect at once	kill switch, agent-side revocation and disabling refuse refresh and introspect as inactive with the reason
8	A subject token is trusted only from the identity provider	expired, foreign-signed, wrong audience and wrong issuer are invalid_grant
9	A task dies with the human it acts for	a sponsor disabled or deleted at the identity provider fails the task’s next refresh, as access_denied
10	The audit ledger is a chain	every record’s prev_hash is the previous record’s hash, so a record removed or inserted shows
11	A high-risk audience says so in the token	introspect_required is true for an audience in high_risk_audiences and absent for any other
12	No refusal hands back the credential it refused	no subject token, client assertion or task grant appears in an error body, whole or in part

Running it

The suite stands in for everything the instance calls out to: the upstream identity provider (discovery, JWKS, the admin users API and token endpoint the sponsor check uses) and every agent’s JWKS. The instance is configured to find all of that at one https base URL, and the suite listens there with a certificate the instance trusts; agent JWKS are only ever fetched over https, so the URL must be https.

Variable	Meaning
ONBE_CONFORMANCE_URL	Base URL of the instance under test
ONBE_CONFORMANCE_ADMIN_KEY	Its Onbe:Admin:ApiKey
ONBE_CONFORMANCE_STUB_URL	The https base URL the instance was given for the upstream identity provider and agent JWKS, for example https://localhost:5199
ONBE_CONFORMANCE_STUB_PFX	A password-less PKCS#12 file for that URL’s host; the stub keeps the identity provider’s signing key beside it, so repeated runs against one instance present the same key

What the suite needs of the instance

The suite drives the instance through its public surface, so a few of its settings have to be within reach of what the checks assume. These are the shipped defaults, but the conformance job sets them rather than relying on that:

Setting	Required	Why
Onbe:Audit:Aggregation:Window	no longer than one minute	A denial that names nobody is written through once per window and counted for the rest of it. The suite looks one minute back for it (LiveInstance.DenialLookback), so against a longer window a second run against the same instance finds nothing and fails a check the server still performs.
Onbe:Agents:MinTaskTtl	no longer than PT1M	Case 5 registers a one-minute task to watch a token be cut to what is left of it.
Onbe:Tokens:DefaultTaskTtl	at least PT1M	Case 5 refreshes twenty seconds into that task.
Onbe:RateLimit:Burst	at least 120	The suite makes around seventy requests from one address in a few seconds, and admission control counts them as one source.
Onbe:UpstreamIdp:SponsorCheck:CacheTtl	anything	Case 9 uses a different person for each outcome, so no answer is ever served from the cache.

The instance is configured with, for a stub at https://localhost:5199:

Onbe__UpstreamIdp__MetadataUrl=https://localhost:5199/.well-known/openid-configuration
Onbe__UpstreamIdp__Audience=onbe
Onbe__UpstreamIdp__SponsorCheck__UsersUrl=https://localhost:5199/admin/realms/main/users
Onbe__UpstreamIdp__SponsorCheck__TokenUrl=https://localhost:5199/realms/main/protocol/openid-connect/token
Onbe__UpstreamIdp__SponsorCheck__ClientId=onbe

The conformance job in .github/workflows/ci.yml is the reference: it makes a CA and a localhost certificate, trusts the CA, migrates and starts an instance against a Postgres service, and runs the suite with dotnet test tests/Onbe.Conformance. Missing configuration or an unreachable instance fails every test with the reason, and the job fails if the suite runs no tests, so it can never pass by not running.