City of Tucson AI Chatbot

The ask

The City of Tucson put out an RFP for automated support agents: an AI that could answer questions by phone and web chat, in English and Spanish, for a city of about 750k people. Two numbers from their own 2025 data explain why. The IT service desk closed 37k tickets, a large share of them password resets and account unlocks. Tucson Water fielded 250k calls, and by the City’s own categories, more than half were high bills, past-due balances, or pending shut-offs.

I responded as EthosAI, teamed with AC5 Labs, a veteran-owned shop here in Pima County. I owned the technical side: the architecture, the technology choices, most of the writing, and a working version of the platform I could actually demo.

What it does

Someone calls, opens a chat window, or messages a Teams bot. The system works out what they’re asking, finds the answer in the City’s own approved documents, and either completes the task or hands the conversation to a City employee with the full transcript attached. Every interaction is logged where City administrators can review it.

Underneath it’s a RAG (retrieval augemented generation) system (using Langgraph and Pydantic AI), and that’s the main reason it minimizes hallucinations. Every question goes to the City’s knowledge base first, and the only documents that can come back are ones a City administrator reviewed and published. The model writes its answer from those passages and cites the document each fact came from. If the sources don’t cover the question or contradict each other, it’s told to say it isn’t certain and redirects to a person instead.

Three phases

Phase 1 is the IT service desk: internal, lower stakes, and if it goes sideways the people affected are City employees who can walk down the hall and complain. Phase 2 is Tucson Water, which is where the public value is and where the hard conversations live. Phase 3 is citywide rollout, priced per department at not-to-exceed rates so the City knows the cost before committing to anything.

Voice

For voice, latency is an important metric: a person will tolerate a lot from an automated system, but not a two-second pause after every sentence. The design streams audio through speech-to-text, the model, and speech synthesis rather than waiting for each stage to finish, so the reply starts coming back while the sentence is still being formed.

The other constraint was that City audio had to stay inside a U.S. boundary, so the speech engine is self-hosted in a FedRamp compliant cloud rather than called as an API. At the City’s call volume, that costs more than the commercial API. We said so in the proposal and explained why we were paying it anyway, which felt better than letting someone discover it later.

For the telephony arrangement, the City’s number stays on its existing carrier and Avaya system, and calls forward to us.

Building

Every piece that could plausibly change later sits behind an interface based on volatility. The language model, speech-to-text, speech synthesis, telephony, identity, and integrations are configurable: Soniox, Telnyx, Cloudflare, Gemini Flash, OCI, Microsoft Entra ID, and DUO, with the open source models like faster-whisper as fallbacks. This was tested locally with a seeded multi-lingual knowledge base and mock users/documents.

Finding out when it’s wrong

Every conversation turn is traced, and retrieval records the exact source documents and their scores. An evaluation harness runs a question set through the system, scores the answers, and then works backwards to the specific documents responsible for the failures, sorting them into outdated, irrelevant, wrong, and/or contractictory. In the demo, I deliberately plant bad documents in the knowledge base, run the evaluation, and let it name them. Then I quarantine them, re-run, and the score goes up on screen.

I also built a speech robustness harness which takes clean recordings and degrades them with background noise, stuttering, and crosstalk, then measures how far transcription accuracy and task success fall.

Where it stands

I has been submitted. Whatever happens with the award, the platform underneath it is real, it runs, and most of it isn’t specific to Tucson. Any organization drowning in repetitive support calls needs the same things: grounded answers, a hard verification gate, staff who control their own knowledge base, and an honest way to find out when the thing is wrong.