Episode Details

A smooth demo is not due diligence. This episode of Phony.ai unpacks what it actually takes to vet a voice AI vendor before committing — drawing on the show's fourteen-question vendor evaluation guide to go deep on the layers that marketing decks rarely reach: telephony infrastructure, consent enforcement, model transparency, handoff logic, and pricing legibility. If a vendor can answer these questions in specifics, they've run real calls. If they answer in adjectives, they haven't.

The episode covers five critical evaluation areas every buyer should probe:

• Telephony ownership and number portability — Whose carrier account does your traffic run on, and can you port your numbers out within 48 hours if you leave? STIR/SHAKEN A-level attestation isn't optional; without it, outbound calls can be blocked before a phone ever rings.
• Consent enforcement as a hard control — Following the FCC's February 2024 ruling, AI-generated voice calls fall under the TCPA's robocall consent standard. The right platform blocks the call when a call recording consent record is missing — not a warning, a hard stop — and propagates mid-call opt-outs to every downstream campaign automatically.
• Model transparency and hallucination rates — A vendor who can't name the model powering your calls can't explain your risk. Stanford HAI's 2024 data puts hallucination rates for smaller models as high as 27.8% on specialised benchmarks, versus 1.5% for frontier models. Knowing which model runs your calls — and whether the agent is grounded in a structured knowledge base — is non-negotiable.
• Human handoff triggers and warm transfer quality — Production agents break not in demos but in real calls with frustrated callers. A warm transfer with a context summary is the baseline; specific, codified triggers (explicit request, detected distress, three failed intents, out-of-scope topic) signal that a vendor has actually learned from real-world failures.
• Pricing legibility and bring-your-own flexibility — A bundled per-minute rate hides cost structure. Buyers should ask for a layer-by-layer breakdown and ask explicitly what happens to pricing if they bring their own model API key or carrier account. A platform that treats bring-your-own as first-class will discount; one that doesn't will deflect.

The episode closes with what the source article calls the best question to save for last: ask the vendor for the last three failure modes they fixed in production. Real bugs, real customers, real fixes. A vendor with production experience will have a specific list. One without will change the subject. With Gartner projecting that over 40% of agentic AI projects will be cancelled by 2027 due to inadequate risk controls, the buyers who survive will be the ones who asked the boring questions first. For more on how platform architecture shapes these trade-offs, check out the companion episode All-In-One vs. Build-Your-Own: The Real Cost of a Voice Agent Stack.

Phony.ai