Episode Details

When law firms evaluate private AI deployments, most frame the choice as a binary: a public model behind a business agreement, or a hardened in-house server room. This episode of Law.co argues the real decision has three doors — and that which door you walk through shapes your audit exposure, your annual budget, and your malpractice risk long before it shows up in a vendor demo. The team's deep-dive on private LLM deployment for law firms forms the basis for the conversation.

The episode works through all three architecture options — single-tenant cloud VPC, on-premises GPU cluster, and hybrid burst — examining what each genuinely costs, where each fails, and what confidentiality trade-offs partners and general counsel actually inherit when they sign the contract. Here's what's covered:

• The ethics frame comes first. The architecture choice is a confidentiality decision, not an IT one. ABA Formal Opinion 512 implicates Model Rules 1.6, 5.3, and 3.3 — and a federal court's $5,000 sanction against attorneys who filed AI-hallucinated citations illustrates exactly what's at stake under Rule 3.3.
• Single-tenant cloud VPC is the default for good reason: logically isolated networks, firm-controlled model weights and vector stores, and no shared endpoints — but the hyperscaler still runs the hypervisor, meaning a subpoena served on the provider is one the firm must be ready to answer.
• On-premises deployment offers the strongest confidentiality ceiling — weights and prompts never leave the data center — but a realistic five-year TCO turns $3M in GPUs into closer to $15M once power, cooling, staffing, and maintenance are factored in. Break-even favors cloud below roughly 60–70% sustained utilization.
• Hybrid burst routes sensitive matters (privileged communications, sealed filings, NDA deal materials) to a private on-prem or VPC island, while lower-sensitivity workloads burst to a larger reserved pool. The engineering discipline is the routing policy layer — per-prompt decisions based on client, matter, jurisdiction, and document classification. For firms considering how this intersects with legal AI cybersecurity posture, the routing logic is often the highest-risk component.
• Staffing is the honest number. A mid-size firm running a serious VPC deployment realistically needs one MLOps engineer, one security engineer with cloud posture experience, and shared data engineering time — costs that rarely appear in vendor demos but dominate the actual budget.
• Hybrid economics aren't automatically cheaper. When less than 60–70% of inference volume is safe to burst, the on-prem footprint dominates, and firms end up paying on-prem costs with an unnecessary cloud dependency on top. Pairing a hybrid model with well-governed audit trails for legal AI is what makes the architecture defensible in practice.

More from the show: if this episode's governance themes resonate, the earlier episode Spellbook vs Law.co: Why AI Contract Drafting Is a Governance Decision covers how deployment architecture intersects with the contract review workflow specifically — a natural companion listen.

Law.co