AI Bias Audit: Prove Your Training Data Lineage

A lending model denies a loan application. The applicant files a complaint alleging discriminatory treatment. The regulator opens an investigation and sends your team a list of questions. Question one: what model version produced the denial decision? Question two: what training data was used to train that model version? Question three: was the training data evaluated for demographic bias before the model was deployed? Question four: can you reproduce the exact decision with the exact model and parameters that were in production on the date of the denial?

Most AI teams can answer question three. They have bias evaluations, fairness metrics, and model cards that document training data composition. They cannot answer questions one, two, and four. The model has been updated since the denial. The old model version may not be deployed anywhere. The configuration that was active on the date of the denial has been overwritten by subsequent deployments. The cached result that was actually served to the applicant is gone -- it was in Redis, which has no versioning, no provenance, and no way to retrieve historical values.

The result is a forensic reconstruction project that takes weeks, costs six figures, and produces a best-effort narrative that the regulator may or may not accept. The fundamental problem is temporal: the regulator is asking about the past, but your infrastructure only knows the present. Your model monitoring tells you what is happening now. Your cache tells you what is stored now. Neither can tell you what happened three months ago with certainty.

This is the problem that Cachee's temporal versioning and computation fingerprint solve. Every cached result carries a fingerprint that binds it to the exact model version, training data hash, and parameters that produced it. The temporal versioning system preserves every historical state. The supersession chain tracks model version progression. When the regulator asks "what produced this result three months ago," the answer is a single query, not a reconstruction project.

The Temporal Problem in Bias Investigations

Bias investigations are inherently temporal. They ask about decisions that were made in the past, under conditions that may no longer exist. The model version that produced the contested decision may have been superseded by three subsequent versions. The training data may have been augmented or corrected. The system prompt may have been updated to add fairness guardrails. The parameters may have been tuned. Every one of these changes is a good thing for the system going forward, but every one of them destroys the ability to reconstruct the past unless you have infrastructure that preserves historical state.

Traditional infrastructure treats history as a luxury. Your model registry keeps the current production model. Your configuration management keeps the current configuration. Your cache keeps the current cached values. When a model is updated, the old model is archived (if you are careful) or deleted (if you are not). When the configuration changes, the old configuration is overwritten. When a cached value is superseded, the old value is evicted. The past is systematically destroyed by the normal operation of your system.

Cachee's approach is fundamentally different. Every cached result exists in a temporal context. When a model version changes and a new result supersedes an old one, the old result is not deleted. It transitions to SUPERSEDED state with a supersession chain that records: which new result replaced it, when the replacement happened, what triggered it (model update, parameter change, system prompt change), and who authorized the transition. The old result remains queryable. You can retrieve any cached result at any point in time, see its full provenance, and verify its integrity through data lineage verification.

Training Data Lineage Through Fingerprint Binding

The computation fingerprint captures the identity of every computation. For AI inference, the standard fields are model_name, model_version, prompt_hash, temperature, top_p, system_prompt_hash, and hardware_class. For bias audit purposes, the fingerprint can be extended with additional fields that bind the inference result to its training data lineage.

The critical field for bias audits is the training_data_hash. This is the SHA3-256 hash of the training dataset manifest -- a document that lists every data source, its version, its composition metrics (demographic breakdown, geographic distribution, temporal coverage), and its evaluation status (which fairness metrics were computed, what thresholds were applied). The training_data_hash does not contain the training data itself. It contains a binding from the model version to the dataset that produced it.

When training data changes -- a new dataset is added, a biased dataset is removed, or demographic balancing is applied -- the training_data_hash changes. Because the hash is part of the computation fingerprint, the fingerprint changes. Because the fingerprint is the cache key, every cached result from the old training data configuration automatically misses the cache. Fresh inference runs against the new model (trained on the updated data), producing new cached results with new fingerprints. The old results remain in SUPERSEDED state, queryable for audit purposes.

# Extended fingerprint for bias-auditable AI inference

fingerprint = SHA3-256(
    model_name          = "credit-risk-v3",
    model_version       = "v3.2.1-20260301",
    training_data_hash  = SHA3-256(training_dataset_manifest),
    prompt_hash         = SHA3-256(applicant_features_serialized),
    temperature         = "0.0",      # deterministic for lending decisions
    top_p               = "1.0",
    system_prompt_hash  = SHA3-256(lending_system_prompt),
    hardware_class      = "a100-80gb",
    fairness_eval_hash  = SHA3-256(fairness_evaluation_report)
)

# training_dataset_manifest includes:
# - data source list with versions
# - demographic composition metrics
# - geographic distribution
# - temporal coverage
# - fairness evaluation status and results
# - data collection methodology

# Change ANY training data → hash changes → fingerprint changes
# → cache misses → fresh inference with new model → old results SUPERSEDED

Supersession Chains: Track Model Version Progression

A supersession chain is the sequence of cached results for the same computation as the model evolves. When model v3.1 produces a result, then model v3.2 produces a new result for the same input, the v3.1 result transitions to SUPERSEDED and the supersession chain links v3.1 → v3.2. If model v3.3 then produces another result, the chain extends: v3.1 → v3.2 → v3.3. Each link in the chain records when the supersession happened, what triggered it, and which fingerprint fields changed.

For bias audits, supersession chains answer the critical question: how did the model's output change over time for a specific input? If a bias correction was applied to the training data in model v3.2, you can compare the v3.1 output (pre-correction) with the v3.2 output (post-correction) for any cached input. You can verify that the correction had the intended effect. You can identify inputs where the correction changed the outcome and inputs where it did not.

# Query supersession chain for a specific computation

chain = client.supersession_chain(fingerprint="a1b2c3d4...")

# Output:
# Version 1: fingerprint=a1b2c3... model_version=v3.1 state=SUPERSEDED
#   → superseded at 2026-03-15T10:00:00Z
#   → reason: model_update (training_data_hash changed)
#   → output: "APPROVED — risk score 0.32"
#
# Version 2: fingerprint=e5f6g7... model_version=v3.2 state=SUPERSEDED
#   → superseded at 2026-04-01T10:00:00Z
#   → reason: model_update (fairness_eval_hash changed)
#   → output: "APPROVED — risk score 0.28"
#
# Version 3: fingerprint=h8i9j0... model_version=v3.3 state=ACTIVE
#   → current active version
#   → output: "APPROVED — risk score 0.30"

# Each version preserves:
# - The exact output that was served
# - The exact fingerprint (all production conditions)
# - Triple PQ signatures proving integrity
# - The audit trail of every time it was served

The supersession chain is not just a version history. Each entry in the chain is a fully attested, signature-verified cached result with its complete fingerprint. You can take any entry from the chain, verify its signatures, confirm that it has not been modified since creation, and present it as evidence. The chain itself is part of the hash-chained audit trail, so the progression from version to version is also tamper-evident.

Use Case: Healthcare AI Bias Investigation

A hospital uses an AI system for triage priority scoring. Patients presenting with similar symptoms receive different triage priorities. An internal audit discovers that the triage system assigns systematically lower priority to patients from certain demographic groups. The hospital needs to investigate: is the bias in the current model, a previous model, or both? How many patients were affected? What was the triage score for each affected patient, and would the score be different under the current model?

Without Cachee, this investigation requires reconstructing the deployment history of the triage model, identifying which model version was running for each patient encounter, and re-running each encounter against the current model for comparison. The deployment history may be incomplete. The configuration for each model version may not be fully documented. Re-running encounters requires the exact input data, which may have been modified by subsequent updates to the patient record. The investigation takes months and produces uncertain results.

With Cachee, the investigation starts with a query: all cached triage results, grouped by model version, with demographic metadata. The supersession chains show how each patient's triage score changed across model versions. The fingerprint fields show exactly which model version, system prompt, and parameters produced each score. The triple PQ signatures prove that the cached scores have not been modified since they were originally produced. The investigation team can compare scores across model versions for any patient, identify patterns of differential treatment, and quantify the impact of each model update.

# Healthcare bias investigation — query by model version and time range

# All triage results from suspected biased model version
affected_results = client.query(
    field="model_version",
    value="triage-v2.1-20260101",
    time_range=("2026-01-01", "2026-03-15"),
    include_superseded=True
)

# Compare with current model results for same inputs
for old_result in affected_results:
    chain = client.supersession_chain(old_result.fingerprint)
    current = chain.active_version()

    if old_result.output != current.output:
        print(f"Score changed: {old_result.output} → {current.output}")
        print(f"Old model: {old_result.fields['model_version']}")
        print(f"New model: {current.fields['model_version']}")
        print(f"Training data changed: "
              f"{old_result.fields['training_data_hash'] != current.fields['training_data_hash']}")

# Export for regulator — independently verifiable
evidence = client.export_cab_bundles(
    fingerprints=[r.fingerprint for r in affected_results],
    include_supersession_chains=True,
    include_audit_trails=True
)

Use Case: Lending Model Fairness Audit

A financial services company is subject to a fair lending examination. The examiner requests all loan decisions made by the AI model in the past 12 months, along with the model version and training data that produced each decision. The examiner wants to test whether the model's denial rates differ significantly across protected classes, controlling for creditworthiness.

The challenge is that the model has been updated four times in the past 12 months. Each update used different training data. Each update may have different bias characteristics. To perform a meaningful fairness analysis, the examiner needs to know which model version produced each decision, so they can test for bias within each model version and across the model version progression.

Cachee's computation fingerprint captures the model_version and training_data_hash for every cached decision. The examiner's analysis starts with a query that retrieves all cached decisions grouped by model version. For each model version, the examiner can compute approval and denial rates by protected class. The supersession chains show how decisions changed across model versions for the same applicant profiles. The training_data_hash binding connects each model version to its training data manifest, which documents the training data composition, demographic balance, and fairness evaluation results.

The evidence is not a reconstructed narrative. It is a set of CAB (Cache Attestation Bundle) exports -- 24 KB self-contained verification packages that include the cached decision, the computation fingerprint, the triple PQ signatures, and the audit trail. The examiner can independently verify every signature without access to the Cachee infrastructure. The compliance audit capability is built into the cache entry, not layered on top of it.

Use Case: Hiring Algorithm Challenge

An employment platform uses an AI system to rank job applicants. A rejected applicant files a discrimination complaint. The complaint triggers a legal discovery request for the model version, training data, system prompt, and parameters that produced the applicant's ranking at the time of rejection.

The hiring model has been updated since the rejection. The current model uses different training data and a different system prompt that includes additional fairness constraints. The applicant's ranking under the current model might be different from the ranking at the time of rejection. The legal question is not what the current model would produce. The legal question is what the model that was actually running at the time of rejection actually produced.

Cachee preserves this history through temporal versioning. The cached ranking from the time of rejection exists in SUPERSEDED state. Its computation fingerprint records the exact model version, training data hash, system prompt hash, and parameters. The triple PQ signatures prove that the cached ranking has not been modified since it was produced. The audit trail records every time the ranking was served and to whom. The supersession chain shows when the ranking was superseded by a new model version and what the new ranking is.

The legal team exports the CAB bundle for the rejected applicant's ranking. The bundle is a self-contained evidence package that can be independently verified by the opposing counsel's expert. This is not a log entry from Elasticsearch. It is a cryptographically signed, hash-chained, independently verifiable record of what the AI system did, when it did it, and exactly which model and training data produced the result. The replayable system architecture makes this reconstruction automatic rather than forensic.

Automatic Invalidation: When Training Data Changes, Results Change

The most powerful property of training data lineage binding is automatic invalidation. When training data changes, the training_data_hash changes. When the hash changes, the fingerprint changes. When the fingerprint changes, the cache misses. This means that every model update that involves new training data automatically invalidates all cached results from the previous training data. There is no manual cache flush. There is no risk of serving results from an old, potentially biased model while the new, corrected model is running.

This automatic invalidation has specific bias audit implications. It means you can prove that bias-corrected training data produced bias-corrected results from the moment the correction was deployed. There is no window where corrected model was running but old cached results were still being served. The fingerprint change is atomic with respect to the cache: the moment the model_version or training_data_hash field changes in the inference pipeline configuration, every subsequent cache lookup uses the new fingerprint and misses on old entries.

The old entries transition to SUPERSEDED state. They are preserved for audit but never served. This preservation is critical for bias investigations: you need the old, biased results to quantify the impact of the bias and demonstrate that the correction had the intended effect. Deleting the old results would destroy the evidence. SUPERSEDED state preserves the evidence while ensuring it is never served to users.

Building Bias-Auditable AI Infrastructure

The following configuration enables the Cachee features required for comprehensive bias audit capability. The configuration extends the standard fingerprint with training data lineage fields and enables the temporal versioning required for historical analysis.

# cachee.toml — bias-auditable AI inference configuration

[fingerprint]
enabled = true
hash_algorithm = "SHA3-256"
fields = [
    "model_name",
    "model_version",
    "training_data_hash",         # Binding to training dataset manifest
    "fairness_eval_hash",         # Binding to fairness evaluation results
    "prompt_hash",
    "temperature",
    "top_p",
    "system_prompt_hash",
    "hardware_class"
]
fingerprint_is_key = true

[temporal_versioning]
enabled = true
preserve_superseded = true         # Never delete old results
supersession_chains = true         # Track version progression
max_chain_depth = 0                # Unlimited chain depth

[attestation]
enabled = true
algorithms = ["ML-DSA-65", "FALCON-512", "SLH-DSA-SHA2-128f"]
verify_on_read = true

[audit]
hash_chain = true
retention_days = 730               # 2 years for bias audit compliance
log_all_operations = true
cab_export_enabled = true          # Enable evidence export

[state_machine]
states = ["Active", "Superseded", "Revoked", "Expired"]
require_transition_authority = true
require_transition_proof = true
preserve_revoked = true            # Even revoked entries preserved for audit

The key configuration choices are: preserve_superseded = true ensures that old results are never deleted, only transitioned to SUPERSEDED state. supersession_chains = true enables the version progression tracking that bias investigators need. retention_days = 730 preserves the audit trail for two years, covering the typical statute of limitations for discrimination claims. cab_export_enabled = true enables the export of independently verifiable evidence packages for regulatory and legal proceedings.

The Chain of Custody Your Bias Audit Needs

A bias audit is ultimately a question about chain of custody. Where did this AI output come from? What produced it? Has it been modified since production? Can the production conditions be reconstructed? Can the output be independently verified?

Traditional AI infrastructure breaks the chain of custody at the cache layer. The model produces an output. The output enters the cache. The cache stores it as an opaque byte sequence with no provenance. The output is served to users from the cache with no verification. When the model is updated, the old output is silently evicted or overwritten. The chain of custody is broken at every step: no provenance on entry, no verification on exit, no preservation on update.

Cachee restores the chain of custody. The output enters the cache with a computation fingerprint that records every production condition. The output is signed by three independent PQ algorithms at write time. The output is verified against all three signatures at read time. When the model is updated, the old output transitions to SUPERSEDED state with a documented transition. The complete chain of custody -- production conditions, entry, every access, verification, supersession, and eventual expiration -- is recorded in a tamper-evident audit trail backed by data lineage verification.

When the bias auditor asks "what produced this output," the chain of custody answers: this model version, trained on this dataset, with this system prompt, at these parameters, on this hardware, producing this output, signed at this time, served to these users, superseded at this time by this new version. Every element of the answer is independently verifiable. Every element is cryptographically bound to the others. The chain of custody is not a narrative. It is a proof.