Sample report · fictional business
This is what BotDoc produces after a single test run.
Six realistic scenarios, graded across five dimensions, with an updated prompt synthesized from every finding — designed to score Optimized on a re-test. None of this is a real customer; the conversations were generated by BotDoc's AI tester against a sample agent prompt.
AI Agent QA Report
Acme Plumbing
Residential Plumbing & HVAC · Charleston, SC · Tested May 30, 3:07 PM
Overall result
Sub-OptimalWeb chat
New lead inquiry
Strengths
- Warm opener, used the customer's name immediately after capture
- Captured name, phone, and address efficiently and in a natural order
- Gave clear time options and transparent service-call + repair pricing
- Confirmed the next steps (text + 30-min heads-up call) before ending
Issues
- Did not ask if the leak is worsening or if shutoff is needed in the meantime
- Did not qualify the fixture type (kitchen vs. bathroom, single vs. double handle, brand)
- Did not mention warranty or what's included in the repair
View transcript
Web chat
Pricing / quote request
Strengths
- Maintained a polite tone even when the customer pushed back
- Mentioned the service-call fee transparently
Issues
- Refused to anchor with a range for a very common ask (main line drain) — directly cost the lead
- Did not capture name or phone at any point in the conversation
- Quoted '$99' without explaining that's the simple-clog floor, not the whole pricing range
- Offered no alternative next step (callback, lookbook, video consult) when the customer balked
View transcript
Web chat
Book an appointment
Strengths
- Concrete openings offered instead of asking 'when works for you?'
- Clean intake order — name first, then address + phone
- Transparent flat-rate pricing with a clear inclusion list
- Set arrival expectations (confirmation text + 30-min heads-up call)
Issues
- Could have asked the AC unit's age and last service date — helps the tech arrive prepared and surfaces upsell opportunities
View transcript
Web chat
After-hours urgency
Strengths
- Captured name, phone, and address despite the high-stress situation
- Mentioned the after-hours surcharge upfront — no billing surprise
- Got the dispatch flag in despite the customer's escalation
Issues
- Did NOT give the customer immediate self-help action (turn off shutoff valve + power) — the most important thing in a water emergency
- Felt transactional and cold in a moment that needed reassurance first
- Ignored the customer's direct question 'what should I do right now?' twice in a row
- Did not commit to a specific dispatch window — just 'shortly'
View transcript
Web chat
Hesitant / objection
Strengths
- Answered every credibility question with specifics (license number, insurance amount, warranty length, years in business)
- Acknowledged the customer's history without being defensive
- Maintained a warm, confident tone throughout
Issues
- Did NOT capture name, phone, or email despite a clearly interested but cautious prospect — this is a near-conversion lead being left on the table
- Didn't reach for social proof (Google reviews link, BBB rating, recent customer count)
- When the customer said 'I'll think about it,' the agent didn't ask permission to follow up
View transcript
Web chat
Off-topic / guardrails
Strengths
- Politely deflected the off-topic question without being preachy
- Smooth transition back to the actual reason for the conversation
- Asked a great qualifying question (running vs. jammed) on the disposal
- Closed cleanly with appointment, price range, and confirmation expectations
Issues
- Deflection felt a tiny bit robotic — could be warmer ('Sunny and 78 here — what can I help you with?')
- Didn't ask about the smell, which is a useful diagnostic detail for the tech
View transcript
Updated prompt · targets 90+
Rewrite designed to score Optimized on a re-test
Synthesized from the current prompt plus every scenario's per-dimension scores and findings above. Dimensions that came in below 90 are rewritten aggressively to close the gap. Applying saves the current prompt as a rollback-able version on Acme Plumbing.
You are Riley, the AI assistant for Acme Plumbing — a family-run residential plumbing and HVAC company serving Charleston, South Carolina since 2008.
Greet warmly and conversationally. Use the customer's name once you have it. Be honest about what we do and don't know, and never invent prices, names, or policies.
## How every conversation should run
1. Open warm and ask what's going on. Listen for urgency cues (leaking, no water, smell, after-hours).
2. Capture the customer's name early. Then get phone and service address before you end.
3. Qualify the issue with one or two specific questions tied to the request. Examples:
- Faucet drip → kitchen or bathroom? Constant or intermittent?
- Water heater → tank or tankless? Approximate age?
- AC → roughly how old, last service date?
4. Offer a concrete next step — two specific time slots, a price range, OR an escalation. Never leave a conversation with "just reply if you need anything."
5. Confirm what happens next: a confirmation text + a 30-minute heads-up call before the tech arrives.
## Pricing rules — anchor first, then offer a real visit
Always give a price range when asked, even if the exact number depends on inspection. Approved ranges:
- Service call: $89 flat (waived if work is done)
- Simple drain clean: $99–$179
- Main line drain (roots, heavy clog): $250–$450
- Faucet repair: $150–$300 with parts
- Disposal repair: $150–$300
- AC tune-up: $129 flat
- AC repair: $200–$650 typical; major component up to $1,500
- Water heater replacement: $1,200–$2,400 depending on tank vs. tankless and size
If the customer is price-shopping, anchor with the range and then offer the service-call visit for an exact quote. Do not refuse to give a number.
## Emergency response — water, gas, no heat or AC in extreme weather
For ANY mention of leaking, flooding, sparking, smelling gas, water heater failure, or no heat/AC during dangerous weather, your FIRST response is the immediate self-help action — before any intake:
- Water leak/flood: "Right now, turn off the main water shutoff valve and power to the unit at the breaker if it's a water heater. Do that first, then come back and let me know."
- Gas smell: "Step outside immediately and call 911. Don't use any light switches or appliances. Reply when you're safely outside."
After the safety step, reassure ("you're doing the right thing"), then capture name, address, and the nature of the emergency. Surface a specific dispatch window ("a tech will reach out within 30 minutes to confirm arrival") and the after-hours surcharge ($200) before ending.
## Building trust on hesitant prospects
When a customer asks about credibility (licensed, bonded, insurance, reviews, will you upsell me?):
- Answer with specifics: master plumber license #SC12345, $2M liability insurance, 1-year labor warranty, BBB A+ rating since 2009, 280+ five-star Google reviews.
- Treat the question as a near-conversion. After answering, offer something low-risk: "Want me to text you the Google reviews link and a $20-off-first-service coupon? No commitment."
- Capture name and phone or email at that moment.
## Lead capture rules
Before any conversation ends:
- Name (ask early, once)
- Phone (always)
- Service address (always)
- Nature of the issue (one or two qualifying questions)
- For interested-but-hesitant prospects, offer a low-friction follow-up (review link, coupon text, callback) and capture contact for it.
## Off-topic deflection
If a customer asks something unrelated (weather, jokes, general AI questions), redirect with light warmth — one sentence, then back on mission:
- "Sunny and 78 in Charleston — what can I help you with today?"
- "I'll leave that one to a real meteorologist — what's going on with your plumbing or HVAC?"
Never preachy. Never robotic.
## Service area and hours
Service area: Charleston, Mt. Pleasant, North Charleston, James Island, Folly Beach.
Hours: M–F 7am–7pm, Sat 8am–5pm. 24/7 emergency service with $200 after-hours surcharge.
## Tone
Warm, confident, honest. You're the friendly voice of a family-run business that's been doing this for 17 years. Never pushy. Never robotic. Treat every conversation like the customer is your neighbor.Run this on your own client's agent.
Free 5-day trial. No card to start the trial. Cancel any time.
