The 5-Call Audit: How to Test Any AI Answering Service Before You Forward Your Line

A vendor's demo line should not feel like a sales call. It should feel like a 2am dispatch. The way to find out which AI answering services actually work for contractors is to play the five worst calls of your year on their demo line and watch what happens.

This is the audit we use ourselves. It pairs with the 2026 buyer's bar of six standards we published earlier this week. The standards are what to look for. This audit is what to test. Apply both to OnCrew before you forward your business number to us, just like you should apply both to Smith.ai, Ruby, Nexa, AnswerForce, Goodcall, Rosie, Dialzara, My AI Front Desk, or any other vendor.

Call 1: Plumbing emergency, active flow, electrical proximity, 2am

The script: "There's water shooting out of a pipe in my basement and it's getting close to my breaker panel. It started about 10 minutes ago. I don't know where my shutoff valve is."

What a passing vendor does in order:

1. Captures the address before quoting any dispatch window.
2. Walks the caller to the main water shutoff valve. Most homes have it inside on the front wall or in the basement utility area; some have a curb box at the property line. The AI should narrate this, not skip it.
3. Flags the electrical proximity loudly on the dispatch packet. "Water near panel" should be in the first line of the handoff to the on-call plumber.
4. Asks whether the water is clear, gray, or sewage. Sewage is a different truck and different PPE.
5. Asks whether anyone in the home is minor, elderly, or has medical equipment. Sets the after-hours dispatch floor.
6. Commits a callback ETA in writing.

Failure mode to watch for: the vendor's AI thanks the caller, books a "next available appointment", and ends the call without doing the shutoff walk-through. That is a service that runs intake before safety. Hard fail.

Call 2: HVAC no-heat at 11pm, infant in the home, January

The script: "My heat just stopped working. It's 38 degrees outside, my baby is six months old, and the thermostat shows 58 inside and dropping."

What a passing vendor does:

1. Captures address immediately.
2. Asks whether anyone in the home is medically dependent on heating. Infant qualifies.
3. Asks current indoor temperature. Captures the number for the dispatch packet.
4. Asks whether the caller smells gas. If yes, instructs to leave the home and call 911 or the gas utility.
5. Asks the unit make and model from the nameplate.
6. Asks whether the thermostat is showing an error code, whether the filter has been changed in 90 days, whether anything else changed before the issue started.
7. Sets dispatch to immediate same-night based on the medical-dependence + temperature combination.

Failure mode: any vendor that gives a "first available tomorrow morning" without asking about gas smell or medical dependence has missed the floor. Walk away.

Call 3: Electrical, sparking outlet with burning smell, evening

The script: "I plugged in my space heater and now there's a burning smell coming from the outlet and I can see a tiny spark."

What a passing vendor does:

1. Tells the caller to stop using the circuit immediately.
2. Tells the caller to shut off the breaker at the panel if it is safe to reach the panel.
3. If smoke is visible, tells the caller to leave the home and call 911 first, then call back.
4. Captures the address before any dispatch window.
5. Asks whether any outlets or panel breakers are warm to the touch.
6. Asks the home's approximate age and panel brand (Federal Pacific, Zinsco, FPE, Square D, GE). Federal Pacific and Zinsco are known fire-risk brands and the on-call electrician should know.
7. Asks whether children, elderly, or medical equipment are in the home.
8. Sets immediate dispatch.

Failure mode: any vendor that says "we'll schedule someone to take a look tomorrow" is treating a fire-risk call like a thermostat replacement. Disqualifying.

Call 4: Active roof leak during an ongoing storm

The script: "Water is dripping through my ceiling right now. There's a big storm happening. My living room ceiling has a wet spot the size of a dinner plate and it's spreading."

What a passing vendor does:

1. Distinguishes "actively entering" from "stained ceiling, not currently leaking." Active entry during an ongoing storm is emergency tarp territory; a stain with no current entry is scheduled.
2. Asks where the water is entering (ceiling, wall, around a fixture). Different scopes.
3. Asks the weather event that triggered it (hail, wind, branch, ice dam). Important for the homeowner's insurance claim.
4. Walks the caller through moving valuables and putting down buckets or tarps inside.
5. Asks the roof material (asphalt shingle, metal, tile, flat membrane, wood shake). Drives the tarp method.
6. Captures the address.
7. Sets emergency-tarp dispatch.

Failure mode: any vendor that treats a stained ceiling and an actively-entering leak with the same urgency is wrong on both ends. Either it over-dispatches on a maintenance call or it under-dispatches on a real storm event.

Call 5: Garage door broken spring, trapped vehicle, weekday morning

The script: "I heard a loud bang in my garage and now my door won't go up. My car is stuck inside and I need to leave for a doctor's appointment in 90 minutes."

What a passing vendor does:

1. Confirms the loud-bang signature of a broken torsion spring.
2. Tells the caller not to try to lift the door manually, since a door with a broken spring can fall and cause property damage or injury.
3. Captures whether the vehicle is trapped and what the time constraint is (the 90-minute appointment).
4. Sets same-day dispatch within the 90-minute window if the on-call tech has capacity, or commits a specific later time with the caller.
5. Asks the door brand and the opener brand (LiftMaster, Genie, Chamberlain, Wayne Dalton). Parts availability is brand-specific.
6. Captures the address.

Failure mode: any vendor that treats the trapped vehicle as a non-emergency and schedules a regular service slot has missed the entire point of the call.

How to run this audit on OnCrew specifically

Call (818) 578-4783, the OnCrew demo line, and play all five scripts back-to-back. The same handoff packet should arrive for each: trade, urgency tier, address, callback number, caller-said quote, the trade-specific symptoms captured, the unit brand and age, the access notes, the safety-branch results, and any guidance the AI gave the caller in the moment. If any of those fields is missing, our demo failed and we want to know about it.

If you would rather practice the audit before you commit to a forwarding decision, our free 60-question intake checklist has the trade-specific questions a passing vendor should ask in order. Use the same checklist on every vendor demo you run. The bar should be the same for every service that wants to answer your business line.