Assess · Healthcheck
GenAIOps maturity assessment
GenAIOps maturity assessment across the six VOF pillars.
Scope and timing agreed after enquiry
Discuss this starting pointOperations
GenAIOps
Your prototype works on your laptop. It demos well. Then it goes live, the traffic shape changes, the model provider tweaks something upstream, a tool call times out at 3am — and someone has to know what to do about it.
GenAIOps is the run-time operations practice for AI applications: observability tuned to model behaviour and tool-use chains, on-call design for failure modes that don't look like normal outages, runbooks that distinguish a degradation from an incident.
If Coherence is the build-time discipline that keeps intent and implementation aligned, GenAIOps is the run-time practice that keeps the live system honest. See /coherence for the other half.
How it works
The operating model
The operating model has four parts: observability, on-call design, runbooks, and the maturity ladder that ties them together. Treat them as a stack — each layer assumes the one below it works.
Observability
Model-aware signals — token shape, latency-per-step, tool-call success/failure, judge disagreement, retrieval hit rates. The instrumentation that lets you tell what the system is doing, not just that it's running.
On-call design
Failure modes that don't look like normal outages — silent quality degradation, prompt-injection in the wild, a provider rate-limit dropped without warning. Paging rules and severity definitions adapted for "the model got worse without going down."
Runbooks
Decision trees for the failure modes above. A runbook is a one-page document an on-call engineer can read at 3am and act from. Templates for the most common shapes; you adapt them to your stack.
The maturity ladder
Five levels from deployed but not run through continuously evolved. A self-assessment grades your current state; the GenAIOps Maturity Model worksheet maps each level to the practices it needs.
Steve is an Ambassador at The Centre for GenAIOps. The operating model on this page is the practice he helps shape there.
Buy a starting point
Three ways to start
These are possible starting points within Surfaces and Inference. Scope, outputs and timing are agreed after an enquiry; they are not pre-booked packages.
Plan · Plan
Operating model + on-call + runbooks
Operating model + on-call design + runbook templates.
Scope and timing agreed after enquiry
Discuss this starting pointExecute · Execute
Stand up observability, run the playbook
Stand up observability, run the playbook, train on-call.
Scope and timing agreed after enquiry
Discuss this starting pointTake it with you
Resources
Resource
GenAIOps Maturity Model
Related writing
More writing on Operations coming.