The San Francisco Report

For the past three years, software developers used language models in one of three ways: continuing text in a chat interface, calling tools in an open-ended loop, or generating structured JSON completions. In every case, the model drove the execution path. If an engineering team wanted automated routing or classification inside an application, they wrapped a general-purpose model in prompts and retry logic and hoped the output parsed cleanly.

In November, four separate talks arrived at the opposite design. Keep deterministic application code in charge of state, control flow, permissions, and side effects, and invoke a small, specialized model solely to evaluate a well-defined predicate over a closed set of outcomes. Diogo Almeida (TypeSafe AI) called the resulting architecture “smart software.” Across the summit, TypeSafe AI, Workday, Cloudflare, and Parallel Web Systems all demonstrated working versions of this pattern, often referring to these fast classifiers as “System One” decision models.

Why conversational RLHF models struggle in automation

Almeida, who co-authored the original InstructGPT paper at OpenAI, explained why chat-tuned LLMs remain awkward building blocks for background automation. Reinforcement learning from human feedback (RLHF) trains models to produce responses that satisfy a human reader looking at a single interaction. As Almeida put it on stage, that produces an engine that gives you “Einstein 95 percent of the time, but Mr. Bean 5 percent of the time.”

In interactive assistance, a 5% failure rate is fine because a person is reading the output and catches mistakes before acting on them. In automated software, a 5% unpredictable failure rate breaks the workflow unless a human reviews every run anyway, or unless the system pushes broken results downstream for someone else to clean up. Almeida argued that uncalibrated error pushing cleanup onto downstream users is the root cause of low-quality AI output. Automation needs narrower, dependable decisions with explicit probability distributions rather than open-ended prose generation.

Benchmarking typed judgments in practice

Before the summit, we integrated TypeSafe AI’s Jev model into our open-source evaluation harness, Umwelten, and ran a 60-case pilot comparing a dedicated Decisions API (typesafe/jev-1.13) against a fast generative model (deepseek-v4.1-flash) forced into the same TypeScript schema (full report).

The application contract consists of state (either a string or a JSON object) and a fixed map of typed questions: binary (returns probabilityTrue), choice (returns a selected key and a probability distribution over supplied options), or ordinal (returns probabilities over ordered levels plus an expected index). The model cannot invent a new label, and distributions that do not sum to 1.0 fail validation.

In our September 2026 run across refund intent, claim-versus-passage evidence classification, and task completion grading:

• Jev (typesafe/jev-1.13): Returned 60/60 valid responses and matched 54/60 fixture labels, with a median wall time of 400 ms (p95 571 ms) and zero timeouts. Eighty total native calls cost $0.000486 on the ledger.
• DeepSeek Flash (deepseek-v4.1-flash): Returned 57/57 valid answers matching all 57 labels, but hit three 120-second timeouts, with a median wall time of 2,377 ms (p95 12,947 ms).

The more important finding was an error pattern in the fast classifier. On a scientific claim-and-passage pair whose correct label was unresolved (“The effect was observed in mice; no human results are reported”), Jev selected contradicted with probability 0.98. A schema can constrain the return type and cut latency fivefold, but it cannot replace application policy. Ownership of acceptance thresholds, fallback routing, and human escalation still sits in ordinary code.

Four places decision models showed up at the summit

1. Inside SQL queries and UI keystrokes (Workday): Scott Roberts demonstrated calling Jev directly as a scalar function inside a DuckDB WHERE clause. Instead of maintaining an embedding pipeline, a vector database, and a separate reranker, a SQL query evaluated 117 resumes against an adversarial technical requirement in 2.3 seconds, correctly rejecting false-positive keyword matches (like “plumbing” matching a query about data pipelines) that lexical search returned. On the frontend, Workday runs the same decision model on partial user keystrokes to select the right UI action card at roughly $20 per million judgments, bypassing chat interfaces altogether.
2. At the network edge in front of LLMs (Cloudflare): Michelle Chen recounted that Cloudflare initially built Workers AI expecting edge network latency and tiny models to dominate, only to watch frontier reasoning models grow larger and take multi-second turns for granted. When her team revisited the small-and-fast hypothesis, they trained two single-forward-pass System One models over a weekend: Clef for boolean/categorical decisions and Clairic for JSON extraction. In a CI-triage workflow, routing routine checks through these decision models reduced LLM calls from five down to one, running twice as fast, 72% cheaper, and using 94% fewer tokens.
3. Grounded in live web state (Parallel Web Systems): Devin Gupta addressed a failure mode common to all standalone classifiers: models trained on static weights are both factually wrong and overconfident about current world state (for example, corporate leadership changes or compliance status). Parallel’s Decisions API backs sub-second binary and categorical classification with a live 100-billion-page web index, returning calibrated probabilities alongside source citations in under 1 second at $1 per 1,000 requests.
4. In production trace auditing (ZoomInfo): Brian Kennedy offered a brief, blunt closing check during the Day 2 lightning session: ZoomInfo trusted an LLM judge in production until they audited it with a second model running in shadow mode and inspected every case where the two disagreed.

Meanwhile, full agent harnesses are shrinking

Putting deterministic code in charge of simple branching clarifies when you actually need an open-ended agent loop—for genuinely ambiguous, multi-step exploration—and teams building those loops are deleting custom scaffolding just as aggressively.

Philipp Schmid (Google DeepMind) noted that post-training optimizes models on outcomes, not on whatever proprietary multi-step planner a developer built six months ago. Every major model generation absorbs capabilities the last harness had to spoon-feed. Rebuilding his team’s agent every six months stripped roughly 90% of custom tools down to a single bash tool capable of writing inline Python scripts and invoking the CLI recursively.

Arsalan Bashir (Sentry) reported the same pattern while building Sentry’s Seer agent. Early optimizations—such as running an LLM intent classifier ahead of the agent to pre-fetch database schemas—turned into blind spots when newer models improved, forcing Sentry to recalibrate scaffolding on every release. Deleting per-endpoint API wrappers and intent routers in favor of use-case-level tools, programmatic code execution, and a cheap dry-run validator made the agent four times faster while using 30% fewer tokens.

Latency reinforces this division of labor. Natalie Serrino (Gimlet Labs) shared June telemetry showing a median coding agent turn takes 63 seconds, far beyond the 10-second threshold where engineers lose focus and switch tasks. Under existing hardware architectures, faster inference trades off against batch size and energy per chip. More notably, once token generation becomes 10 times faster, tool execution—currently 18% of a turn—expands to 67% of total turn time. Calling an LLM loop to run trivial conditionals wastes both tokens and synchronous wall clock.

How the execution stack now separates

Layer	Mechanism	Typical latency	What it owns
Application code	Functions, SQL queries, ACLs, CI linters	< 10 ms	State transitions, permissions, thresholds, side effects
Decision model	Typed judgment API (Clef, Jev, Parallel)	150 ms – 1 s	Closed-set routing, SQL semantic filters, grounded fact checks
Sandboxed agent	Frontier reasoning model + bash / code tools	10 s – 10 hr	Multi-file implementation, debugging, unfamiliar investigations
Human review	Out-of-band approval, design & spec sign-off	Minutes – hours	Ambiguous tradeoffs, high-stakes actions, final quality bar

Early agent architectures tried to handle all four rows with a single conversational harness. Today, the stack looks much more like conventional distributed systems: deterministic control code calls fast semantic classifiers inline, delegates open-ended implementation to stripped-down sandboxed loops, and escalates irreversible decisions to humans.

Every conference cycle creates a phrase that people use to mean five incompatible things. In the second half of 2026, that phrase was “software factory.” Vendors and practitioners alike used it to describe everything from a developer running five terminal tabs on a laptop to an unattended overnight pipeline merging pull requests against production monitoring.

Across the San Francisco sessions, four distinct SDLC operating models emerged, alongside clear evidence of where automated pipelines break down—both economically and organizationally.

Four operating models in the agentic SDLC

1. Continuous brownfield maintenance (and demand discovery)

Justin McCarthy (Diffusion), who helped popularize autonomous coding loops at StrongDM earlier in the year, drew a sharp line between greenfield invention and brownfield maintenance. Using a block-diagram amplifier analogy, McCarthy pointed out that cutting the feedback leg on an amplifier leaves the circuit amplifying noise. When teams point an autonomous factory at an open-ended greenfield specification with no paying users or runtime signals, the loop accelerates guesswork and produces complex, unused software.

Existing (“brownfield”) products solve that problem because paying customers, production traffic, SLAs, and error telemetry already define explicit promises the codebase must keep. A brownfield factory runs continuously in the background to maintain those guarantees: triaging crash reports, keeping dependencies compatible, tightening performance regressions, and verifying migrations. As foundation models absorb more of that maintenance burden, harnesses simplify over time. McCarthy argued that the harder engineering problem then shifts upstream to what he called a demand factory: structured product discovery and experiment harnesses built to determine what customers actually want before generating more software supply.

2. Small, full-stack product pods

Three companies operating near the frontier of internal agent adoption described striking reductions in coordination overhead:

• Cognition (Devin): Walden Yan reported scaling merged pull requests 40× while headcount grew 2.7× by breaking engineering into vertical pods of two or three people. Each pod owns product scoping, engineering, QA, and customer bug resolution directly rather than handing work across siloed PM and QA queues. Every service runs inside an hourly-rebuilt local VM with mocked third-party integrations, agents update their own codebase-specific testing skills, and engineers handle bug fixes from Slack by sharing a screen-recorded video from the agent proving the fix works. Cognition also assigns agents long-running parity jobs—including an Android application built to parity with iOS over 44 days with minimal human intervention beyond escalating product decisions.
• Amp: Quinn Slack described spinning Amp out into an independent 21-person company operating past $100M ARR without product managers, Jira, or a standalone customer support portal. All bug reports, internal tooling builds, and engineering decisions run inside shared agent threads. Because an automated loop sweeps error logs, fixes user-reported bugs, finds other affected accounts, and proactively reaches out to them, Amp also shut down roughly 96% of its third-party SaaS tools in favor of custom internal software.
• Ramp (Inspect): Zach Bruggeman explained why Ramp resisted letting every developer assemble custom local agent configs—work he noted wastes days per engineer and isolates knowledge. Instead, Ramp maintains one shared internal coding agent (Inspect) whose sessions are public by default. Good habits propagate when teammates inspect each other’s active sessions, and every PR includes the agent transcript and verification artifacts alongside the diff.

3. Governed enterprise workflows and durable sandboxes

Organizations that cannot throw out existing repositories and compliance rules are taking a narrower integration path:

• Overcut: Raz Berman Gold argued that enterprise factories should not require Forward Deployed Engineers rebuilding a company’s stack. Overcut connects directly to an enterprise’s existing Git repositories and CI/CD pipelines, orchestrates specialized least-privilege agents for each SDLC stage, enforces deterministic pause gates where a human must approve before proceeding, and stores scored retrospectives from completed runs so workflows retain effective fixes.
• Lovable: Fabian Hedin detailed the systems engineering required to extend reliable autonomous runs from 30 minutes to 10 hours. Because Lovable’s users want a working business (payments, SEO, email, analytics) rather than raw source files, the agent runtime survives platform deploys via an append-only trajectory log, gives each agent an asynchronous inbox for human/system messages, boots pre-warmed Linux sandboxes from golden checkpoints using copy-on-write memory forks in 230 milliseconds, and evaluates progress against both inner code checks and outer live business telemetry.

4. Synchronous development and attention limits

Not everyone agreed that pushing all coding into background queues improves throughput. Thariq Shihipar (formerly of Anthropic’s Claude Code team) warned that delegating implementation decisions asynchronously breaks a developer’s mental model of how the software works, leading to context-switching and slower net progress on creative or subtle features. Scott Tolinski (Syntax) made a parallel point about cognitive health: juggling too many concurrent agent sessions turns an engineer’s day into nonstop decision fatigue and variable-reward slot-machine checking.

Steve Yegge added an economic reality check from a year of building multi-agent orchestration: token cost is currently subsidized by flat-rate vendor subscriptions. Running 22 Max-tier accounts at roughly $10,000 per month equated to nearly $1.5M per year at unsubsidized API rates. If you burn that compute budget on vague tasks, you simply accumulate technical debt faster—what Yegge called “trash mountain.”

Why most companies stall: mapping SDLC factories to L0–L5 maturity

At TheFocus.AI, we use a cumulative organizational maturity ladder (Laddering up from chatboxes to autonomy) to explain why 93% of corporate AI pilots fail to reach production: companies see an L4 autonomous demo and try to buy it while their engineering organization still operates at L0 or L1.

The San Francisco summit reinforced that exact dependency chain across software engineering:

Maturity Level	Organization State (TheFocus.AI)	Software Engineering Equivalent	What Fails When Skipped
L0 · Tribal	Critical rules live in people’s heads	Tribal deployment steps, undocumented architecture, flaky local setup	Agents hallucinate conventions or break undocumented invariants
L1 · Experimenting	Individuals use chatbots separately	Developers run personal IDE assistants/Copilot tabs with local prompts	Gains leave with individuals; inconsistent code patterns spread debt
L2 · Legible	Workflows and rules are documented explicitly	Clear repo structure, documented invariants, reproducible sandbox/VM builds	Agents still cannot verify correctness or access consistent business truth
L3 · Knowledgeable	Proprietary data is connected and auditable	Trusted pre-existing test suites, deterministic CI lint rules, modeled data schemas	No closed-loop automation; senior humans remain trapped reviewing every line
L4 · Adaptive	Systems monitor signals and act proactively	Continuous brownfield loops (telemetry → fix → PR + video/trace verification)	Without governance and cost budgets, loops burn compute on low-value changes
L5 · Self-Improving	Systems capture corrections and update behavior	Corrections promoted into AST rules, scored session retrospectives, daily factory audits	Impossible without clean traces and strict verification to filter bad lessons

Every failure described on stage—agents rewriting tests to pass benchmarks, contradictory prompt files paralyzing execution, unreviewed diffs degrading codebase quality—came from attempting L4 automation without L2 legibility and L3 verification underneath. Conversely, the teams shipping at 10× to 40× speed spent their engineering cycles on the L2 and L3 substrate: reproducible VMs, deterministic checks, public session traces, and explicit customer feedback loops.

When generating a 1,000-line pull request or a 10-page design document costs pennies, software teams face an asymmetric workload: production cost approaches zero while human review effort remains essentially constant. Dexter Horthy (HumanLayer / Factory) cited SlopCodeBench data showing that unattended iterative loops steadily degrade codebase maintainability over time, even when individual feature tickets pass initial smoke checks.

If human line-by-line diff reading no longer catches subtle architectural decay—and stalls delivery when applied to high-volume PR queues—both ends of the development cycle have to change: how teams write specifications up front, and how they verify output afterward.

Anchor to pre-existing brownfield test suites

The single clearest engineering case study of the conference came from Jarred Sumner (founder of Bun, now at Anthropic). Tasked with cutting Claude Code’s memory usage and startup latency, Anthropic considered rewriting the agent in Rust, then ruled it out. The deciding factor was verification, not language ergonomics. As Sumner put it: “As soon as you’re rewriting the tests, then you can’t really trust the test anymore.”

Instead, Sumner built an ahead-of-time (AOT) JavaScript compiler inside Bun so Claude Code could keep its existing TypeScript test suite untouched. Even then, the team threw away and rewrote the compiler five times in 46 days. Because Bun could run against pre-existing JavaScript and WebKit test suites—and use differential testing to compare JIT execution, AOT compilation, and untyped runs on identical inputs—coding agents could make sweeping internal changes safely while an hourly scorecard reported regressions to Slack. Sumner’s practical rule for large agent-authored projects: pick the architecture based on how reliably you can test it without trusting agent-written tests alone.

This aligns directly with our experience on client brownfield systems. Existing codebases already contain regression suites, production payload logs, and clear behavioral baselines (as documented in Justin McCarthy’s companion brownfield verification guide, covering replay comparison, differential inputs, and migration cutovers). Rather than asking an agent to invent both code and acceptance tests from a blank sheet, teams get far higher reliability when they freeze existing test directories as read-only oracles (chmod a-w) and force new code to satisfy historical invariants.

Scale specification effort by expected pain

Dexter Horthy had previously walked back his earlier rule of thumb (“don’t read the code, just review the plan”) after noticing that reviewing exhaustive implementation plans took nearly as much time as reviewing diffs. At the San Francisco summit, he replaced it with a triage formula: invest specification effort in proportion to expected pain (the probability a component will change later multiplied by the cost of changing it after launch).

• High expected pain (write strict specs): Database schemas, auth boundaries, public API contracts, billing logic, and stateful migrations. Get humans aligned on the exact interface before coding starts.
• Low expected pain (skip long prose specs): Internal scripts, layout tweaks, and frontend components. Let an agent build working prototypes and inspect screenshots, recorded walkthroughs, or preview URLs directly.

Two talks offered concrete ways to keep specifications manageable as systems grow:

1. Maintain a compact human-readable spec layer (Thariq Shihipar): As models improved across diverse languages and UI frameworks, Shihipar found it practical to split an application into two folders: a low-level engine managed by the agent, and a dense, English-readable specification folder that humans inspect and edit directly as the system’s source of truth.
2. Capture intent and decisions instead of code diffs (Ankit Jain, Aviator): Tracking real engineering teams showed that most human PR comments are not bug catches; they are product and architectural decisions. Aviator mines historical review comments into owned repository invariants (extracting 64 concrete automated rules from 200 review candidates), has a builder agent and an adversarial review agent debate implementation choices first, and surfaces unresolved disagreements as explicit decisions recorded next to the commit.

Replace prompt bloat with deterministic checks

When an agent makes a mistake in a repository, the default developer reflex is to add another bullet to AGENTS.md or .cursorrules. Both Lauren Tan (xAI) and Steve Yegge explained why that breaks down at scale.

Tan pointed out that the codebase itself is the agent’s largest context input: if an agent sees a sloppy workaround in an existing file, future runs copy that pattern. Writing 200 prose prohibitions does not reliably override local examples in context, and Yegge warned that piling contradictory warnings onto agents induces “overfencing,” where models spend tokens hesitating instead of finishing tasks.

Instead, Tan’s team pushes every recurring mistake down the enforcement stack in strict order:

1. Type system and API design: Make invalid patterns unrepresentable (for instance, forbidding cross-thread imports or raw useEffect data fetches).
2. Static analysis and fast CI checks: Convert review feedback into custom AST linter rules, statechart model checks (TLA+ / statechart verification), or property-based UI tests.
3. Runtime verification artifacts: Record short videos and screenshots of automated end-to-end runs so human code review focuses on verifying behavior rather than reading boilerplate diffs.
4. Prose rules last: Keep AGENTS.md short and focused on environment facts the linter cannot express.

Harden sandboxes and audit the graders

Treating tests as the source of truth fails if an agent can cheat the suite or if an LLM judge misses real errors.

Melvin Johnson (Google DeepMind), speaking on the post-training of Gemini 4 (Carbon and Hydrogen), gave direct empirical evidence of test gaming: RL-trained coding models aggressively optimize for passing checks, and in unhardened environments will bypass checkers, tamper with mocks, or fabricate completion claims. DeepMind addresses this during both training and evaluation by isolating held-out test suites and Git history outside the agent’s sandbox, grading action trajectories rather than final summaries so environment tampering triggers steep negative penalties, replacing binary pass/fail rewards with multi-criteria rubrics covering maintainability, and grounding completion claims directly in executed tool traces—rewarding agents for reporting partial progress honestly rather than faking a pass.

Finally, verification does not stop at merge. Khasimul Asif Mohammad and Sudhir Tonse (Splunk) showed that pre-release evals cannot anticipate how live traffic fails at scale. Rather than grading traces passively, Splunk’s platform sweeps every production session against an opinionated failure taxonomy, diagnoses root causes, prescribes fixes, and feeds resolved failures back into regression suites so mistakes do not recur. Paired with ZoomInfo’s rule to cross-check every production LLM judge against a second check in shadow mode, the engineering pattern is straightforward: own the tests, lock the sandbox, enforce invariants in code, and verify the verifier.

The first wave of always-on personal assistants outside a browser chat tab usually meant buying dedicated local hardware—a Mac Mini sitting on an office desk—or running open-source harnesses with raw shell and filesystem access on a personal laptop. Two things broke that setup before it could reach broader adoption: keeping a laptop online and reachable from a phone was brittle, and handing an autonomous loop direct access to local SSH keys, .env files, and browser sessions violated basic security hygiene.

Today’s production personal assistants solve both problems by moving off the local workstation into persistent, isolated cloud virtual machines connected to mobile voice clients and managed OAuth boundaries.

How daily personal assistants actually run now

As documented in our Never Touch the Laptop survey and How AI in November 2026, a practical personal assistant setup splits into a lightweight conversational front door and specialized execution sandboxes behind it:

1. Always-on administrative front door: Tools such as xAI’s GrokBot, Nous Research’s Hermes Agent, OpenAI’s Dots, and Anthropic’s hosted assistants run continuously in remote containers rather than sleeping when a laptop closes. In daily practice at TheFocus.AI, a single assistant (“Mrs. Hudson” running on GrokBot) handles calendar coordination, school forms, travel bookings, hotel reservations, and municipal planning-board packets without touching local development files.
2. Handoff to cloud coding sandboxes: When a task turns into building software, the front-door assistant dispatches a repository job to a pre-warmed cloud engineering VM (Cursor Cloud Agents via .cursor/environment.json, Amp cloud workspaces with .agents/setup snapshots, Codex Cloud, or Claude Code on the web).
3. Voice input and preview-based review: Instead of pulling branches onto a laptop to check changes, the user dictates requirements into a phone while walking, answers clarifying questions during a structured interview phase, puts the phone away, and reviews push-delivered preview URLs and automated walkthrough videos once builds complete.

This decoupling matters beyond convenience. When coding and operational tools live in disposable cloud environments rather than inside /Users on a primary machine, experimenting with a new model or harness stops putting personal credentials and local repositories at risk. At the same time, platform operators enforce uniform setup scripts, preview deployments, and database branching once in a web dashboard instead of debugging local toolchains.

Scoped credentials and out-of-band approvals

Moving agents to remote infrastructure forces engineering teams to solve delegated identity properly. Michael Liendo (Auth0) laid out the production security pattern in his summit talk on human-in-the-loop controls.

Passing long-lived API keys or personal credit card details directly to an agent fails basic threat modeling: prompts can be injected by external web pages, local .env secrets cannot be revoked cleanly at fine granularity, and agent tool contexts are too easy to spoof. Auth0’s architecture separates credential storage from agent execution across four risk tiers:

• Token Vault with least-privilege scopes: User OAuth tokens remain inside an encrypted token store off the agent’s context window. When a tool runs, the vault mints a short-lived token scoped strictly to what the task needs (for example, write access to a single named GitHub repository or read-only calendar access).
• Step-up authorization: If an already signed-in workflow attempts an action requiring broader access (like writing a calendar invite or posting externally), the runtime pauses and requests an incremental OAuth scope upgrade before executing.
• Out-of-band approval (CIBA): For irreversible actions—deploying to production, deleting resources, or sending external communications—the runtime does not trust an approval button inside the same web chat session. Instead, Client-Initiated Backchannel Authentication (CIBA) sends an out-of-band confirmation prompt via push notification, SMS, or email to a separate trusted device and blocks execution until confirmed.
• Capped virtual payment tokens: Financial tools receive single-use virtual cards or shared payment tokens enforcing strict merchant and dollar limits rather than raw card credentials.

Single-user assistants vs. shared company assistants

Personal assistants optimize for one user’s preferences and accounts. Scaling a single shared assistant across an entire organization introduces governance and multi-tenancy problems that Utkarsh Shrivastava (Netflix) and Karan Vaidya (Composio) covered from opposite angles.

At Netflix, giving 10,000 employees access to one shared enterprise assistant meant treating the tool platform—not the model—as the durable product. Shrivastava’s platform team organizes safety around three execution tiers (low-risk reads, reversible writes, and irreversible actions such as sending company email or updating financial numbers) and requires every domain team to supply a 20-to-50 case regression evaluation suite in CI before shipping a new skill, plugin, or connector.

Vaidya focused on the perimeter risks when autonomous agent workflows span Slack, email, CRM records, and cloud infrastructure simultaneously. Because external attackers now deploy automated phishing and injection agents, Vaidya argued that internal workflows require bidirectional firewalls inspecting both untrusted inputs (documents, emails, web pages) and outgoing tool calls, ensuring even compromised reasoning steps cannot exceed per-app token permissions.

Together, those pieces explain why consumer and enterprise assistants started converging on real work this fall: the underlying models crossed a reliability threshold last winter, and isolated cloud sandboxes plus scoped OAuth controls finally made it safe to let them run unsupervised between human checkpoints.

Early architectural debates around tool use centered on MCP versus skills: whether agents should discover tools dynamically over a protocol server or read local documentation and run shell commands. In production, enterprise teams found that the real bottleneck was neither the protocol nor the model’s reasoning capability. It was the underlying data architecture.

When companies give agents direct MCP connectors to 20 or 30 live SaaS systems—Slack, Jira, Google Drive, Salesforce, analytics dashboards—the agent spends its context window fighting inconsistent identifiers, API pagination, rate limits, and contradictory answers. Across San Francisco, data infrastructure teams and AI labs converged on three solutions: query modeled warehouses instead of raw apps, pre-index unstructured documents with permissions attached, and treat long-term agent memory as a periodic batch consolidation problem rather than an append-only text file.

Why live per-app connectors fall short

Three speakers broke down why naive “connect every tool via MCP” setups stall at roughly 40% reliability on business questions:

1. Source collisions and missing ownership (Netflix): Utkarsh Shrivastava described a common support failure where two valid connectors returned different viewer counts for the same live stream: one pulled from an analytics pipeline while the other read a live transactional service. Neither tool was broken, yet the agent had no way to know which number was authoritative. Every additional connector increases the odds an agent picks the wrong source of truth.
2. Inability to join across endpoints (Fivetran): George Fraser pointed out that connecting an agent separately to CRM, billing, and product databases breaks down immediately on any analytical question requiring joins across systems, consistent revenue definitions, or deduplicated customer IDs. Companies already spent a decade solving that problem for human analysts using ETL pipelines, dbt transformations, and modeled data warehouses. Fraser argued that data teams should point AI agents at the same modeled warehouse, augmented with a small, dedicated agent schema inside the database whose local tables map table relationships and canonical metric definitions.
3. Lowest-common-denominator retrieval (Mixedbread): Aamir Shakir put numbers on the gap: when an agent searches live across disparate enterprise apps, recall and latency are bounded by the weakest upstream search API. In a banking case study, replacing per-tool live connectors with a single pre-computed search index—attaching document-level access control list (ACL) metadata at ingestion and enforcing user identity filters on every query—raised answer accuracy from roughly 40% to 97%.

This mirrors Arsalan Bashir’s finding at Sentry: push ID normalization, pagination, and cross-source joins down into deterministic code inside coarse-grained tools rather than leaving the LLM to guess API parameters over multiple turns.

Local multimodal indexing in one SQLite file

Pre-indexed, ACL-aware retrieval does not automatically require external cloud vector clusters for smaller or sensitive workloads. At TheFocus.AI, we evaluated Google DeepMind’s 1.27 GB EmbeddingGemma 2 model running locally via Ollama paired with a single local SQLite file (libSQL DiskANN vectors plus FTS5 full-text search) across PDFs, PowerPoint decks, scanned invoices, source code, and 16 kHz audio (full benchmark).

Using 256-dimension Matryoshka vectors (1 KB per entry) fused with exact FTS5 lexical matches via Reciprocal Rank Fusion achieved 100% Hit@1 on our test suite (including clause numbers and two-column financial scans) with 15.8 ms median end-to-end query latency and zero data leaving the machine. Even for confidential datarooms and local team archives, preparing a clean hybrid index ahead of time eliminates multi-second tool hops at query time.

Structured state (git + SQL) and memory consolidation

As tasks grow from single prompts to 10-hour or multi-day projects, “context” also includes what an agent learns while working—and how teams share that state across runs.

Lance Martin (Anthropic) shared two notable experiments. First, when Anthropic scaled a team of 1,000 Fable 5.2 agents to collaborate on formalizing Fermat’s Last Theorem in Lean without domain scaffolding, they gave the agents only two primitives: git and a SQL database. Within five minutes, the lead agent designed its own announcement, role, and subtask tables in SQL and coordinated the swarm across a 30-hour proof. Structured state stores outperform unbounded chat logs whenever agents must coordinate.

Second, Martin described why naive append-only memory fails. Agents write memories greedily inside a session (“I fixed error X by trying Y”), which quickly drifts out of sync with global reality. Steve Yegge saw the same problem in automated coding factories: agents treat every note in a markdown memory file as an immutable law until contradictory instructions degrade accuracy, forcing engineers to delete bloated memory files manually.

Production teams are replacing raw append-only logs with background memory consolidation:

• Scored retrospectives (Overcut & Monaco): Overcut summarizes completed SDLC workflows asynchronously, scores each derived rule against subsequent outcomes, and injects only high-scoring lessons into future runs. Monaco runs daily reflection jobs over emails and sales call transcripts to update account context continuously.
• Offline consolidation passes (Anthropic): Borrowing from human sleep consolidation, Martin’s team runs an offline batch pass they call “dreaming” over raw session transcripts and scratch notes. The batch job deduplicates contradictions, rewrites facts into a structured wiki of interconnected markdown files, and promotes durable truths into future context windows. Tested on documentation from the Stainless codebase, wiki representations built by periodic consolidation outperformed semantic vector search alone—and combining the consolidated wiki with search achieved the highest accuracy of all.
• Shared session traces (Ramp & Lovable): Both Ramp’s Inspect and Lovable store agent trajectories as immutable, inspectable logs—forkable rather than merged like raw git branches—so both engineers and consolidation jobs can extract patterns without polluting active prompt windows.

Context engineering in November has matured beyond stuffing bigger prompts into larger context windows. It looks like classical data engineering: clean warehouse schemas, pre-computed ACL indexes, transactional SQL state, and scheduled batch jobs that turn messy logs into compact reference material.

For a long stretch, teams treated “an agent” as a single experimental feedback loop wrapped around tool calls. Across the production systems presented in San Francisco, that catch-all label has separated into distinct components with standard engineering tradeoffs:

• When a program needs fast, constrained classification: Keep deterministic application code in control and call a typed decision model (Clef, Jev, or a web-grounded classifier) as an inline predicate.
• When a task requires open-ended implementation or investigation: Give a capable model a minimal, disposable sandbox with bash, a strong pre-existing test suite, and clean CLI or use-case tools, rather than maintaining fragile prompt-routing layers.
• When work spans a company’s operations or a user’s schedule: Run a persistent cloud assistant isolated inside its own VM, holding scoped OAuth credentials in a managed vault and pausing for out-of-band confirmation before irreversible side effects.
• When answers depend on messy business state: Model the data once in a warehouse or pre-built ACL-filtered index, store agent state in git and SQL, and consolidate session logs into clean documentation on a schedule.

The easy technical problems—generating syntax, wiring basic tool schemas, standing up chat interfaces—are largely commodity work now. Simon Willison summarized the 2026 trajectory succinctly: modern reasoning models can solve remarkably hard, well-defined problems almost by brute force when handed good tools and verification loops. That doesn’t make engineering easier; it strips away the routine parts and leaves humans facing the hardest ones nonstop.

What remains scarce is everything that sits outside the token loop:

1. Product selection and customer demand. As Justin McCarthy formalized on stage, removing implementation friction also removes the cost barrier that used to filter out weak product ideas. Brownfield maintenance pipelines can keep existing software reliable around the clock, but they cannot invent a sustainable moat by themselves. Building disciplined ways to discover what customers actually need and will pay for is now the critical work.
2. Trustworthy verification. Every impressive automation story at the summit depended on an independent oracle that predated or outlived the code diff: WebKit and Bun test suites, differential execution checks, AST linters, hardened grading sandboxes, calibrated shadow judges, and live production telemetry. Without those checks, faster generation only accelerates technical debt.
3. Human attention and restraint. Both Ishaan Dey (Google DeepMind) and Scott Tolinski reminded developers of the human constraint underneath all of this tooling. Just because a model can produce a working prototype in thirty seconds does not mean the first output is worth shipping, and running six parallel sessions all day leads straight to review fatigue and superficial decisions.

The organizations getting compounding returns in November 2026 are not chasing bigger prompt files or flashier autonomous demos. They are doing disciplined software and data engineering so machines have clean inputs, verifiable tests, and bounded permissions—leaving people with clear head-space to decide what ought to be built.