An AI agent on WhatsApp: a digital coworker that lives in chat

A personal AI agent that takes plain-language commands over WhatsApp — processing documents, compiling data, transcribing audio, monitoring news — with every file processed locally, never leaving for the cloud.

Challenge

Modern office work is full of repetitive tasks: consolidating dozens of Excel files, assembling decks, transcribing interview recordings, monitoring industry news. Cloud AI assistants demand you upload documents to third parties — unacceptable for sensitive data.

Approach

I built an AI agent that lives on my own machine and takes orders through WhatsApp in everyday language. The architecture has two components: a WhatsApp gateway (Node.js) and an agentic brain (Python/FastAPI) running a tool-use loop that picks the right skill for each command on its own.

All processing happens locally: OCR for scanned documents, audio transcription (faster-whisper), Indonesian text-to-speech, and RAG over a personal knowledge base. The LLM layer is flexible — Claude, GPT, or Gemini — switchable from an admin panel without a restart.

Result

60+ tools across ~30 production modules: Excel/Word/PowerPoint/PDF processing, chart & infographic generation, FGD transcription, web research with sources, calendar & reminders, and even an 11-mountain climbing knowledge base that automatically checks volcanic activity status weekly.
Total privacy: documents never leave the machine; hybrid mode uses an encrypted channel so files never touch an intermediate server.
Serious governance: append-only daily audit logs, path whitelisting, and explicit confirmation before overwriting files. An agent that can execute commands needs to leave an auditable trail.

This project is my personal lab. The reasoning is simple: AI is most useful when it lives inside an app people already open every day, and around here that means WhatsApp.