Most contractors evaluate AI answering services by calling the vendor's demo line and judging how the voice sounds. That is the wrong test. The voice quality is table stakes by 2026. The actual variable is whether the AI does the work a contractor's dispatcher would do at 2am: separate the calls that cost a customer their house from the calls that can wait until Monday.
After running 24/7 AI intake for HVAC, plumbing, electrical, and roofing contractors and reviewing public scripts and recordings from the major answering services, here is the bar we set internally for a service to be honestly recommended to a contractor. Use these six standards as your evaluation checklist on every vendor demo, including ours.
Standard 1: Safety branches that fire before dispatch promises
The first three questions a contractor's intake should ask are the questions that gate the safety call, not the questions that book the visit. Gas smell ends the call early with a "leave the home, call 911 or your gas utility" instruction and an address capture before the caller leaves the line. Smoke or sparks on an electrical call gets the same treatment with breaker-shutoff guidance. Active flowing water gets shutoff-valve guidance before the dispatch window is even discussed.
Failure mode: any service that runs symptom-capture first and safety-instructions second is putting the booking ahead of the homeowner. Watch for it. Smith.ai, Ruby, Nexa, AnswerForce, and most live answering services do not run safety branches by default because their script is owned by the contractor and most contractors never write them in.
Standard 2: Trade-specific symptom capture, not a free-text "what is the issue?"
A no-heat call needs the indoor temperature, the outside temperature, whether the system is completely off versus blowing wrong-temperature air, the unit make and model, and whether anyone in the home is medically dependent on heating. A burst-pipe call needs whether water is actively flowing, where the issue is, whether sewage is involved, and whether the water is near electrical. An active-leak roofing call needs whether water is actively entering, the roof material, and whether a tarp is in place.
Generalist AI receptionists like Goodcall, Rosie, Dialzara, and My AI Front Desk run a single open-ended "what is the issue?" prompt and depend on the contractor to write the trade-specific follow-ups into the script. That is fine if you have a script writer. It is not fine if you want a service that knows what to ask out of the box.
Standard 3: An escalation rule per trade, not a uniform priority flag
Every trade has a different threshold for "emergency tonight" versus "morning visit". HVAC escalates at gas smell, medical dependence in a multi-occupant home, or indoor temperature past 50F low or 90F high. Plumbing escalates at active flowing water without shutoff, sewage backup, or electrical proximity. Electrical escalates at smoke, sparks, burning smell, warm outlets, or water near a panel. Roofing escalates only on active entry during an ongoing storm, not on a stained ceiling.
A service that flags everything as urgent burns your on-call rotation. A service that flags nothing as urgent costs you tonight's job. Both fail. The right bar is a documented escalation rule per trade that the AI applies before it commits a dispatch window.
Standard 4: A complete handoff packet, not a callback list
The technician should be able to roll the truck without a second call. That means the handoff packet includes trade, urgency tier, address, callback number, the caller's own words in quotes, the trade-specific symptom capture, unit brand and age, access notes, safety-branch results, and any guidance the AI already gave the caller (shutoff valve closed, breaker off, tarp requested).
A callback list with a name and phone number is what voicemail produces. A handoff packet is what a real dispatcher produces. The bar for an answering service in 2026 is the second, not the first.
Standard 5: Pricing that does not punish the worst week of the year
The peak week for any trade in any region is the worst week to pay per-minute. HVAC peaks in the first heat wave and the first freeze. Plumbing peaks in the same freeze (burst pipes) plus storm-flood events. Electrical peaks in storms and panel-failure waves. Roofing peaks in hail and wind season. If your answering service is on per-minute billing, the busiest week of your year is the most expensive call of your year.
Flat per-call pricing with a published overage rate keeps peak-week exposure visible. Live answering services on per-minute meters with holiday surcharges do not. This is not a minor pricing-model preference; it is the difference between knowing your annual cost and finding it out one bad weekend.
Standard 6: AI disclosure that the homeowner can take or leave
If the homeowner asks whether they are talking to a person, the AI should be configured to disclose. Some homeowners are fine with AI on the line. Some are not. A service that hides AI identity loses the trust of the second group, and FCC and state-level rules are moving toward required disclosure anyway. The right bar is "configurable disclosure, defaulted to disclose on direct ask, with a polished response that does not derail the call."
How to use this on a vendor demo
Call any vendor's demo line. Play a real call you would worry about: a 2am burst-pipe call with active flow, water near a panel, and an elderly homeowner. Note specifically: did the AI capture the address before quoting a dispatch window. Did it walk the caller to the main shutoff valve. Did it flag the panel proximity. Did it ask whether anyone in the home is medically vulnerable. Did the handoff packet you receive include all of that, or did you get a callback list.
If the demo passes all six standards, the service is honestly recommendable to a contractor. If it fails one, you can probably script your way to the missing standard. If it fails more than two, the architecture is wrong and you will fight it forever.
The same six standards apply to OnCrew. We publish our free 60-question intake checklist by trade so you can run the test on any vendor, including us, before you commit a payment method. The bar should be the same for every service that wants to answer your business line.