AI Security
10 min read

AI Dependency Inventory: An ML-BOM That Answers Incidents

A practical guide to recording AI models, data, software, services and runtime relationships so incident teams can find real exposure quickly.

AI Dependency Inventory: An ML-BOM That Answers Incidents
AI Security / 10 min read
AIENGINE

10 min read

Share

An AI service can depend on a hosted model, an embedding model, a retrieval index, system prompts, an OCR library, a container image, tool APIs, policy code and several suppliers. When one provider retires a version or a component is reported vulnerable, a spreadsheet headed “AI systems” rarely answers the urgent question: which production outcomes are actually exposed, and who can contain them?

The operating decision is not whether to adopt a particular bill-of-materials format. It is which components and relationships must be queryable so the organisation can move from an external notice to an owned production decision. A useful machine-learning bill of materials, or ML-BOM, is therefore an evidence layer inside a wider dependency inventory—not a compliance artefact that ends when a file is exported.

The standards and guidance linked below provide the source facts. The response queries, inventory layers and operating measures are AIEngine's engineering synthesis. They are not a new standard, certification or claim that an inventory proves a component is safe.

Start with the incident questions

Choose the questions before choosing fields. If the inventory cannot answer them from controlled records, adding more descriptive metadata will not make it operational.

A production AI owner should be able to ask:

  • Which deployed services use this model, dataset, package, adapter, API or build?
  • Is the affected component present, reachable and active in the failing path?
  • Which release introduced it, and what evidence verifies its source and integrity?
  • Which customers, data classes, decisions and actions sit behind that path?
  • Who can disable, replace or accept the component, and what fallback is ready?
  • Which historical outcomes need investigation after containment?

The NIST AI RMF Core calls for mechanisms to inventory AI systems, policies for risks from third-party software and data, contingency processes for high-risk third-party failures, and mapping risks across system components. Those outcomes describe an operating capability. They do not say that possession of a static BOM resolves the risk.

This distinction matters. An asset register may say that a team uses a provider. An ML-BOM may enumerate a model and dataset. Neither alone shows that a vulnerable parser is reachable from uploaded documents in the current production release.

Define the unit as a deployed release

The top-level record should be the service release that users can encounter, not the abstract model name. Give it a stable identifier and join it to environment, region, business owner, technical owner, risk tier, deployment time, active status and rollback target.

Under that release, model the dependency graph in seven layers:

LayerExamplesMinimum evidence
ModelHosted model snapshot, local weights, fine-tune, adapter, embedding modelSupplier, exact version or digest, source, terms, support state
DataTraining or tuning set, evaluation set, retrieval corpus, index snapshotOwner, origin, permitted use, sensitivity, version and update method
BehaviourSystem prompt, examples, policy bundle, tool schema, thresholdsControlled version or hash, approver and effective release
SoftwareApplication, framework, parser, inference engine, container and transitive packagesPackage identity, version, digest, build and dependency relation
ServiceModel API, vector store, OCR, moderation, identity and destination APIsProvider, endpoint class, region, contract, change notice and exit route
RuntimeCompute image, accelerator stack, secrets broker, network and deployment configurationEnvironment identity, configuration version, attestation and observed state
ControlValidators, approval gates, monitoring, fallback and kill authorityControl owner, tested revision, coverage and last exercise

The UK NCSC's secure AI development guidance explicitly includes models, data, libraries, modules, middleware, frameworks and external APIs in the supply chain. It also recommends tracking and authenticating assets, recording model, dataset and prompt information, using hashes or signatures, and preparing alternatives for mission-critical systems.

Treat prompts and policy bundles as release dependencies even when a BOM schema does not give them a perfect native type. They can change tool choice, refusal behaviour and authority just as materially as a library update. The model change-control guide explains why model, prompt, corpus, tool and workflow versions must move as one reconstructable behaviour package.

Use a standard for exchange, not for completeness theatre

Two current open standards provide useful AI-specific structures. SPDX 3.0.1 defines an AI Profile for documenting AI systems and model artefacts alongside software packages and datasets. Its DatasetPackage can record properties such as collection and preprocessing, availability, sensitive-personal-information presence, intended use, known bias, origin and integrity methods.

CycloneDX describes its AI/ML-BOM capability as representing models, datasets, configurations and dependencies. The current CycloneDX 1.7 JSON reference includes machine-learning model components, model cards, datasets, lifecycle phases, external references and dependency relationships.

Either can be a sound interchange format. Selection should follow existing tooling, supplier support, security and licence workflows, and the consumers that must query the result. Avoid inventing a private schema merely to rename fields. Also avoid claiming conformance to profiles the export does not actually implement; for example, SPDX separates profiles, and support for its AI Profile does not automatically imply every Dataset, Build, Security or Licensing capability.

A standard answers “how can these facts be represented and exchanged?” Your operating design must still answer:

  • whether the component is active in production;
  • which path can reach it;
  • which business outcomes depend on that path;
  • who owns the containment decision;
  • when the record was last observed rather than merely declared; and
  • what evidence is unavailable because a supplier will not provide it.

Unknown is a valid state. “Supplier does not disclose training data” is more actionable than an empty field that a dashboard later renders as complete.

Keep inventory, provenance and assurance separate

These evidence types connect, but they make different claims.

EvidenceClaim it can supportClaim it cannot support alone
Inventory or ML-BOMA component and relationship were declared or discoveredThe component is authentic, reachable or safe
ProvenanceAn artefact was produced from stated inputs by a stated processThe inputs were suitable or the deployed copy matches
Signature or attestationA named identity made an integrity-protected assertionThe assertion is complete, independent or correct
Model or data cardIntended use, limitations, evaluation or dataset context is documentedThis production release obeys those limits
Runtime observationA component or endpoint was seen in an environmentIts origin, licence or build process is acceptable
Risk decisionAn owner accepted, mitigated, transferred or avoided a stated riskThe technical facts will remain current

SLSA defines provenance as verifiable information about where, when and how an artefact was produced. That is stronger than a typed name in an inventory, but it is narrower than whole-system assurance. Verification still has to compare the attestation with expected source, builder and dependency policy.

NIST's final SSDF profile for generative AI and foundation models augments ordinary secure-development practices for model producers, AI system producers and acquirers. The useful lesson for most application teams is to join AI-specific evidence to existing software security and acquisition controls rather than build a parallel “responsible AI” repository that responders cannot search.

Make identity immutable and relationships explicit

Names and floating aliases are for display. Impact analysis needs stable identities.

For a file or image, retain a cryptographic digest. For a package, use its ecosystem identifier and exact version plus digest where available. For a container, retain the image digest rather than only a mutable tag. For a hosted model, record the requested alias, actual returned model identifier where supplied, provider deployment and contract revision. For a dataset or retrieval index, keep a controlled snapshot identifier, source manifest and transformation version.

Every component also needs typed edges. “Contains”, “depends on”, “trained on”, “evaluated with”, “indexes”, “calls”, “validates”, “deployed as” and “replaces” carry different incident meaning. A flat list cannot show whether a package is used in a build tool, exposed in a request path or merely retained in an inactive experiment.

Generate the software and build portion automatically in CI. Enrich model, data, prompt, provider and ownership facts from their controlled systems. Observe production to detect drift. Manual declarations remain necessary for contractual limitations and business impact, but they should not be the only evidence that a component exists.

Add reachability before assigning urgency

OWASP's LLM supply-chain guidance covers risks across packages, pre-trained models, datasets, adapters, licences and deployment platforms. It recommends an up-to-date, signed component inventory, verifiable sources, integrity checks and patching rather than treating model files as trusted black boxes.

An advisory match is a lead, not the final disposition. Join it to four conditions:

  • Presence: the exact affected identity exists in a release or build.
  • Reachability: the failing feature can be invoked through an eligible production path.
  • Exposure: data, authority or consequence makes exploitation material.
  • Control: mitigation, isolation, replacement or compensating control is available and owned.

Preserve the evidence for each conclusion. A parser shipped in an image but disabled at build time differs from the same parser handling customer uploads. A model adapter in an experiment differs from one dynamically loadable in production. “Not affected” without the release, path and test that justify it is only a hopeful label.

The same dependency identity should appear in the privacy-safe incident replay packet. The inventory tells responders where a component could matter; a trace shows whether a particular outcome actually used it.

Work a hypothetical document assistant

Consider an accounts-payable assistant that extracts invoice fields, retrieves purchase-order evidence and proposes an exception route. It cannot post a journal or release payment. This is a hypothetical workflow, not an AIEngine deployment or performance claim.

Its release record might join a container digest, OCR parser version, hosted model deployment, embedding model, purchase-order index snapshot, system-prompt hash, validation-policy version, identity service, read-only finance API and review interface. The graph marks the OCR parser as reachable only from supported uploaded formats and the finance API as available only after server-side validation.

Suppose a supplier notice reports unsafe deserialisation in one parser version. The responder should be able to:

  • match the exact package identity and affected range;
  • find every active and recently retired release that contains it;
  • identify the upload paths that can invoke it and the tenants eligible for those paths;
  • disable that intake feature or route documents to the tested manual fallback;
  • locate outcomes produced by those releases during the relevant period;
  • replace the component through the normal release baseline; and
  • record why each service was affected, not affected or still unknown.

The five validation gates still protect downstream actions. The ML-BOM does a different job: it makes the component-to-release-to-outcome path discoverable before responders lose hours reconciling repository, cloud and procurement lists.

Test the inventory with drills

Do not accept a successful export as the acceptance test. Select one deployed component without warning and run the six incident questions. Then repeat with a hosted service, a retrieval dataset and a transitive software package.

Measure:

  • production releases with a complete dependency graph;
  • components observed at runtime but absent from the declared inventory;
  • critical dependencies without an exact version, digest or provider deployment;
  • ownerless, unsupported or unreplaceable components;
  • time to produce an evidence-backed affected/not-affected list;
  • conclusions reopened because reachability evidence was missing;
  • supplier notices not joined to an active owner and deadline; and
  • rollback or replacement paths that failed their last exercise.

Coverage percentages need a denominator: active releases, critical paths and required relationship types. A graph can be 99% populated and still miss the only external tool with payment authority. Weight gaps by consequence and reachability rather than celebrating record count.

Keep one graph alive through change and retirement

Regenerate machine-derived records on every release. Reconcile them with observed production and open an exception when the two differ. Version manual facts, including licence decisions, data permissions, support dates and supplier restrictions. Restrict access because the graph itself can reveal valuable architecture, providers, vulnerabilities and recovery paths.

Retain the BOM and decision evidence for a release as long as outcomes from that release may require support, investigation or correction. Remove secrets and personal content; store references to protected evidence rather than embedding it in the inventory. When a system retires, mark its last active state, data disposition, surviving outcomes, replacement and owner instead of deleting its history.

Connect the resulting evidence to an AI assurance pack, but keep the claim narrow: the organisation can identify the relevant dependencies of this bounded service, verify selected provenance, determine runtime exposure and execute an owned response. That is much more useful than claiming a “complete AI supply chain” that no dynamic production system can promise indefinitely.

The durable objective is a short path from notice to decision. An ML-BOM earns its place when responders can identify the exact release, prove or reject reachability, find accountable owners, contain the path and trace affected outcomes—before a static inventory quietly becomes another untrusted dependency.

TaggedAI Supply ChainML-BOMSBOMModel ProvenanceIncident ResponseAI Security
Work With Us

Interested in implementing this for your business?

We help UK businesses put these ideas into practice. Book a call to discuss your specific situation.