In the room, I spent most of my time on the Azure side of this: Content Safety, API Management, model routing. Under those controls, a team still has to prove an AI feature is safe to ship without Security or Infrastructure reviewing it by hand.
I think you were testing how to stop Security and Infrastructure from finding out what a product team shipped only after something's already broken. A team has to meet a fixed set of conditions before it's allowed to ship.
Product Team A
Product Team B
Product Team C
AI Gateway
APIM · Content Safety
only path to a model
Azure OpenAI
Audit · cost · safety log
The acceptance contract
On a clean slate, every AI feature has to satisfy six conditions before it reaches production. The platform checks all six. Nobody reviews them by hand, so a feature can't ship with one missing.
DATA Purview classifies every input source before that source can be used in a prompt.
SAFETY Every call has to pass Azure AI Content Safety thresholds at the gateway. No team gets a bypass.
COST Every feature declares its quota and spend ceiling before the first call. API Management enforces both directly, so a spike doesn't wait on someone noticing a dashboard alert.
AUDIT The gateway logs every call: who, which model, what data, what came back.
ROLLBACK One switch at the gateway disables a feature. No code deploy required to pull the plug.
THREAT MODEL One page that names what the feature can touch, and what happens if the output is wrong.
Quality gates that keep the pager quiet
The acceptance contract stops a bad feature before it ships. Something will still get through. These gates are how I keep that from turning into a 2am call for Security or Infrastructure.
ROLLOUT Every change ships behind a flag to a small slice of traffic first. It doesn't reach everyone until the health checks on that slice pass.
AUTO-ROLLBACK Health checks watch error rate and latency continuously. A breach reverts the change on its own, before anyone has to notice and send a page.
BLAST RADIUS Every downstream call has a timeout and a circuit breaker. When a dependency fails, the caller fails fast instead of hanging and taking the next system down with it.
ERROR BUDGET Each service has an explicit uptime and latency target. Once a team has spent that budget, it stops shipping features and fixes reliability.
ON-CALL Platform carries the pager for what Platform ships. Security and Infrastructure get read-only dashboards into Sentinel, not 2am texts.
POSTMORTEM Every incident gets a blameless write-up within 48 hours. Whatever it finds gets added to the acceptance contract, so the next ship has to account for that failure.
Who owns what
Platform
Owns the gateway, enforces the acceptance contract there, and carries the pager for what it ships.
Product
Owns business logic and UX inside the paved road. The gateway, the safety checks, and the on-call coverage are already there, so the team doesn't rebuild them for each feature.
Security
Defines the policy: classification rules and safety thresholds. Stops reviewing every deployment by hand.
Infrastructure
Owns capacity and network boundaries for one gateway, not N point solutions, and watches it in Sentinel.
How this plays out
SCENARIOA tax associate wants to build a tool that summarizes engagement letters using Opus 5.
1
They build against the AI Gateway directly, with no infrastructure ticket and no security meeting first.
2
The gateway requires three declarations before it'll route a single call: what data the feature touches (engagement letters, classified through Purview), a cost ceiling, and a one-page threat model.
3
Once that's declared, every call runs through Azure AI Content Safety automatically. If the associate ever points it at data outside what they declared, the gateway blocks the call before it reaches the model.
4
It ships behind a flag to a handful of users first. Health checks watch error rate and latency on that slice.
5
Once those health checks pass, it rolls out to everyone. Security never reviews the code. The calls, the data, and the model are in the audit log if they want to look.
02 / INTEGRATION
Integration Architecture
This one didn't change much from what I said live. Systems should talk to a central bus, not directly to each other. Every point-to-point connection you wire up today is one more thing somebody has to understand, secure, and eventually untangle when it breaks.
Salesforce
M-Files
Legacy systems
iPaaS
MuleSoft / Workato
Event Bus
Warehouse
dbt · data marts
reverse ETL to source systems
Buy vs. build
For connectors into Salesforce and M-Files, I'd buy rather than build. The iPaaS vendors already sell those, and there's no reason to hand-roll them. KR's legacy systems are different. Nobody sells an adapter for those, so that's where I'd spend engineering time.
How this ships without breaking what's live
You kept coming back to how to build the new integration layer without breaking what's already running. This is the order I'd use.
01 Existing point-to-point integrations keep running, untouched, on day one.
02 New traffic goes through the bus behind a flag, one integration at a time.
03 The new path runs in production beside the old one. Cut over only after that shadow run. The old path comes out last, never first.
04 Every hop is encrypted in transit. Credentials are scoped per connector and logged through the same audit trail as the AI gateway from the first question.
How this plays out
SCENARIOA new client signs an engagement in Salesforce.
1
Salesforce fires an event the moment the engagement record is created. Nobody writes custom code in Salesforce to make that happen.
2
The iPaaS connector picks the event up and drops it on the event bus.
3
The event lands in the warehouse, where dbt transforms it into the engagement data mart other systems read from.
4
Reverse ETL pushes the client's status into M-Files, which creates the engagement folder automatically.
5
If M-Files is down, the event stays on the bus and retries. The call doesn't time out, and nobody gets paged.
03 / INTELLIGENCE
Practice Automation Intelligence
Who checks the model's work before a person ever sees it matters more than which model you pick. I'd put a dedicated agent on that check. Its only job is to find what the primary agent got wrong before the output reaches Tax, Audit, or Risk.
Four layers, in order
I'd build these four in this order. The classifier and the agents assume the model is already running on a private endpoint inside KR's tenant.
FOUNDATIONPrivate, VPC-isolated model endpoints inside KR's own tenant. Data never crosses to a public endpoint, and every call still runs through the same gateway from the first question.
DURABILITYSmall, fine-tuned local models handle document classification. Telling a W-2 from a K-1 doesn't need a frontier model, and that step keeps working if the vendor changes.
TRUSTMulti-model consensus, plus a QA agent that checks the primary agent's output against the source document before any person sees it.
ORCHESTRATIONLangGraph runs the agents in sequence, and LangSmith records every step, for the same reason the gateway logs every model call.
How this plays out
SCENARIOA K-1 comes in for a tax engagement.
Tax / Audit / Risk docs
OCR
doc intelligence
Local classifier
small, fine-tuned
Primary agent
private LLM endpoint
QA agent
adversarial check
Human review
only after consensus
1
The document lands in the ingestion pipeline. OCR and document intelligence pull the raw text and layout.
2
The local classifier identifies it as a K-1. Sorting the mail doesn't need a frontier model.
3
The primary agent, running on a private endpoint inside KR's tenant, extracts the structured fields: partner name, share of income, deductions.
4
The QA agent checks that extraction against the source document independently, looking for numbers that don't reconcile.
5
If both agree, it's queued for the preparer with the extraction pre-filled. If they don't, it's flagged and routed to a person before anyone downstream sees a number.
All three of these answers run on the same platform. The gateway from the first question enforces the same audit trail, cost controls, and rollback switch, whether the caller is a product team, an integration, or an agent.