Fraud-Fighting AI Granny
A voice agent designed to consume a scammer’s time without putting a real person—or real personal information—into the conversation.
01 / THE PROBLEM
Make scam attention expensive.
Scam operations scale when human attention is cheap. Every minute a scammer spends with a safe, synthetic counterpart is a minute not spent pressuring a real target.
The product question was not “can an AI make a phone call?” It was whether a bounded voice agent could safely hold the interaction, disclose nothing real, and make the attacker’s economics slightly worse.
02 / THE OUTCOME
A working voice loop, with the code exposed.
A public Node.js prototype that answers a Twilio call, listens, generates an in-character response, synthesizes speech, and keeps the exchange going within a bounded conversation.
Built for fraud teams, trust-and-safety builders, and anyone exploring defensive voice agents.
03 / ARCHITECTURE
Five steps inside one conversational turn.
Twilio forwards an incoming voice call to the Express service.
Twilio speech recognition turns the caller’s response into text.
The language model generates a safe, in-character reply from the running transcript.
Azure Speech renders the response and Twilio plays it back.
Conversation state and a turn cap keep the loop finite.
What the public code does today
- The incoming-call handler greets the caller and opens a speech-gather loop.
- Each transcript is appended to per-caller conversation history before the next model response is generated.
- The system prompt keeps the agent calm, non-disclosing, and focused on consuming the caller’s time.
- Azure Speech writes a generated audio response that Twilio plays into the call.
- The call ends after a bounded exchange, and terminal call states clear the in-memory history.
04 / RECREATE IT
Clone it, supply your own services, inspect the loop.
The repository is a compact Node.js service. You will need a Twilio voice number, OpenAI API access, Azure Speech, Node.js, and a secure public tunnel or deployment for the webhooks.
git clone https://github.com/AjitGaddam/Fraud-fighting-AI-Granny-.git
cd Fraud-fighting-AI-Granny-
npm install
# configure credentials through environment variables
npm startPoint the Twilio voice webhook to /handle_incoming_call. Twilio posts recognized speech to /process_speech; the service returns TwiML that plays the synthesized answer and gathers the next turn.
Read the repository before exposing it publicly. The current code is a prototype and the production controls below are part of the build record—not optional polish.
05 / PRODUCTION NOTES
The honest distance between “works” and “dependable.”
- Validate every Twilio webhook signature before accepting call events.
- Move conversation state from process memory into a durable, expiring store.
- Replace the placeholder session secret with a managed secret and rotate it.
- Add rate limits, abuse controls, redaction, and an explicit audio-retention policy.
- Wire the repository’s scam-pattern detector into the live call path—or remove it until it is a tested control.
- Add operational telemetry, model fallbacks, consent review, and jurisdiction-aware call handling.
06 / LESSONS
What this build makes visible.
- A useful defensive agent needs a narrow job and an explicit stopping condition.
- Voice latency is product behavior: recognition, generation, synthesis, and playback all sit inside one conversational turn.
- A character prompt shapes behavior; deterministic controls still belong outside the prompt.
- Publishing the source makes the gap between an interesting prototype and a dependable service visible—and therefore improvable.
SOURCE / APACHE 2.0