People ask us all the time what the hardest part of building a voice agent is. They expect the answer to be speech recognition, or latency, or tone. It is none of those. The hardest part is booking the appointment.
Booking sounds trivial. Check the calendar, pick a slot, confirm. It is not trivial. It is where most voice agents quietly fail and nobody notices because the agent says "I booked you for Tuesday" and hangs up, and only later does the business realise there is no event on the calendar, or there is an event but it conflicts with another one, or the customer wrote down Tuesday but the agent actually said Thursday.
Part of the problem is that calendars are ambiguous. If someone says "next Tuesday" on a Monday, do they mean tomorrow or do they mean eight days from now? Both interpretations are correct depending on the region and the person. The agent has to ask. But it cannot ask clumsily, or the caller feels like they are talking to a robot.
Another part is conflict handling. What happens when the only slot the caller wants is already booked? A bad agent says "that is not available, when would you like?" and the caller gives up. A good agent offers two specific alternatives that are close to what the caller wanted. "I have 2 pm the same day, or 10 am the next morning. Which works?" The difference in booking rate between those two approaches is around 20 points in the data we have.
Then there is confirmation. The agent has to repeat the booking back, but it cannot sound like a legal disclaimer. If it says "to confirm, your appointment is Tuesday April 23rd 2026 at 2 pm Mountain Standard Time" you have lost the customer. If it says "great, see you Tuesday at 2," you have a booking and a good feeling.
One more detail almost nobody thinks about. The follow-up. A booking without a confirmation text or email is a booking that has a 30 percent no-show rate. The voice agent has to trigger the confirmation the second the call ends. Not after a batch job runs at midnight. The second. Because if the customer gets in their car 10 minutes later and realises they forgot the time, they need to be able to check their phone and see it.
All of this is plumbing, not AI. But it is the plumbing that decides whether the call turned into money or not.