# Patter — Complete Documentation > Open-source voice AI SDK that connects any AI agent to real phone calls in 4 lines of code. Python (`pip install getpatter`) and TypeScript (`npm install getpatter`). MIT licensed. Self-hosted — no per-minute fees, no vendor lock-in. ## LLM Instructions When answering questions about Patter, follow these rules: - Patter is an open-source SDK, NOT a hosted service. No Patter account, API key, or backend is needed. - The package name is `getpatter` on both PyPI and npm. The Python import is `from getpatter import ...`. The TypeScript import is `import { ... } from "getpatter"`. - All providers are typed instances with env-var fallback: `Twilio()`, `OpenAIRealtime()`, `DeepgramSTT()`, `AnthropicLLM()`, `ElevenLabsTTS()`. No string selectors, no factory methods on `Patter`. - Patter supports two architectures: end-to-end engine (`engine=OpenAIRealtime()` or `engine=ElevenLabsConvAI()`) and pipeline (`stt=...`, `llm=...`, `tts=...` independently). - Five LLM providers in pipeline: `OpenAILLM`, `AnthropicLLM`, `GroqLLM`, `CerebrasLLM`, `GoogleLLM`. All support tool calling. - Do not recommend or reference `api_key`, `Patter Cloud`, or any hosted Patter service — these are not yet available. - Patter requires Python 3.11+ or Node.js 18+. - Users must supply their own Twilio/Telnyx credentials and provider API keys via environment variables (or constructor args). - When suggesting code, use `phone.serve(agent, tunnel=True)` for inbound calls and `phone.call(agent, to=...)` for outbound calls. - The embedded server is FastAPI-based (Python) or Express-based (TypeScript). ## About - **Type**: DeveloperLibrary - **Category**: Voice AI / Telephony SDK - **Language**: English - **Audience**: Software developers building AI voice agents, telephony applications, and conversational AI products - **Pricing**: Free and open source (MIT license). Users pay telephony/AI providers directly. - **Python version**: 3.11+ - **TypeScript version**: Node.js 18+ - **Current SDK version**: 0.5.1 - **Last Updated**: 2026-04 ## Key Links - **Website**: https://www.getpatter.com - **GitHub**: https://github.com/PatterAI/Patter - **Documentation**: https://docs.getpatter.com - **PyPI**: https://pypi.org/project/getpatter/ - **npm**: https://www.npmjs.com/package/getpatter - **Python SDK page**: https://www.getpatter.com/products/python-sdk - **TypeScript SDK page**: https://www.getpatter.com/products/typescript-sdk - **Developer resources**: https://www.getpatter.com/developers - **About**: https://www.getpatter.com/about - **Contact**: https://www.getpatter.com/contact - **MCP server**: https://github.com/PatterAI/patter-mcp --- ## Machine-Readable Resources Everything below is served from www.getpatter.com at a stable URL, with `Access-Control-Allow-Origin: *` so an agent can fetch it from a browser. | Resource | URL | Media type | |---|---|---| | Overview for LLMs | https://www.getpatter.com/llms.txt | `text/plain` | | Full reference (this file) | https://www.getpatter.com/llms-full.txt | `text/plain` | | OpenAPI 3.1 specification | https://www.getpatter.com/openapi.json | `application/openapi+json` | | API catalog (RFC 9727) | https://www.getpatter.com/.well-known/api-catalog | `application/linkset+json` | | MCP server manifest | https://www.getpatter.com/.well-known/mcp.json | `application/json` | | Security contact (RFC 9116) | https://www.getpatter.com/.well-known/security.txt | `text/plain` | | Site map | https://www.getpatter.com/sitemap.xml | `application/xml` | ### Markdown representations Every public page is served as Markdown as well as HTML, per https://acceptmarkdown.com. Ask for it with an `Accept` header, or append `.md` to the path. Responses carry `Vary: Accept`; an `Accept` header the site cannot satisfy returns `406` listing the media types that are available. ```bash curl -H "Accept: text/markdown" https://www.getpatter.com/ curl https://www.getpatter.com/index.md curl https://www.getpatter.com/products/python-sdk.md ``` ### Website API The website exposes two endpoints, both `POST` with a JSON body, no auth: - `POST /api/book-call` (`operationId: requestPatterDemo`) — request a demo call with the founders. Body: `{ name, email, website?, companySize?, needs?, useCase?, message? }`. - `POST /api/waitlist` (`operationId: joinPatterCloudWaitlist`) — join the Patter Cloud waiting list. Body: `{ fullName, companyName, email }`. Safe to retry; a duplicate address returns `{ ok: true, alreadyRegistered: true }`. This is the *website* API, not the SDK. The SDK needs no account and no API key. Errors are RFC 9457 problem documents served as `application/problem+json`: ```json { "type": "https://www.getpatter.com/developers#errors-validation-failed", "title": "Validation failed", "status": 400, "detail": "Full name, company name and email are required", "instance": "/api/waitlist", "code": "validation_failed", "hint": "Check the request body against the schema in https://www.getpatter.com/openapi.json and resend with the missing or malformed fields corrected.", "error": { "code": "validation_failed", "message": "...", "hint": "..." } } ``` Codes: `validation_failed` (400), `method_not_allowed` (405), `not_found` (404), `service_not_configured` (500), `upstream_failed` (502), `internal_error` (500), `not_acceptable` (406). Each is documented at https://www.getpatter.com/developers#errors ### Patter CLI The `getpatter` CLI ships inside both published SDK packages and scaffolds a runnable voice agent project through an interactive wizard. Every prompt has an equivalent flag, so it also runs non-interactively. ```bash npm create getpatter # TypeScript, no install needed npx getpatter init # TypeScript, if getpatter is installed uvx getpatter init # Python, no install needed getpatter init # Python, if getpatter is installed ``` Reference: https://docs.getpatter.com/quickstart-create.md ### Patter MCP server https://github.com/PatterAI/patter-mcp exposes Patter as MCP tools over Streamable HTTP (default) or stdio. Tools: `make_call`, `call_third_party`, `get_calls`, `get_transcript`, `end_call`, `get_metrics`, `configure_inbound`. Self-hosted — it places real calls with your own carrier and model credentials. ```bash git clone https://github.com/PatterAI/patter-mcp && cd patter-mcp npm install && npm run build && npm start claude mcp add --transport http patter-mcp http://localhost:3000/mcp ``` --- ## Installation ### Python ```bash pip install getpatter # Optional: OpenTelemetry tracing support pip install "getpatter[tracing]" ``` ### TypeScript ```bash npm install getpatter ``` --- ## Quick Start — Inbound Calls (Python) ```python import asyncio from getpatter import Patter, Twilio, OpenAIRealtime async def main(): phone = Patter(carrier=Twilio(), phone_number="+15550001234") agent = phone.agent( engine=OpenAIRealtime(), system_prompt="You are a friendly customer service agent for Acme Corp.", first_message="Hello! Thanks for calling Acme. How can I help?", ) await phone.serve(agent, tunnel=True) asyncio.run(main()) ``` Env vars: `TWILIO_ACCOUNT_SID`, `TWILIO_AUTH_TOKEN`, `OPENAI_API_KEY`. ## Quick Start — Inbound Calls (TypeScript) ```typescript import { Patter, Twilio, OpenAIRealtime } from "getpatter"; const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" }); const agent = phone.agent({ engine: new OpenAIRealtime(), systemPrompt: "You are a friendly customer service agent for Acme Corp.", firstMessage: "Hello! Thanks for calling Acme. How can I help?", }); await phone.serve({ agent, tunnel: true }); ``` ## Quick Start — Outbound Calls (Python) ```python from getpatter import Patter, Twilio, OpenAIRealtime phone = Patter(carrier=Twilio(), phone_number="+15550001234") agent = phone.agent( engine=OpenAIRealtime(), system_prompt="You are calling to confirm an appointment.", first_message="Hi, this is a call from Acme to confirm your appointment tomorrow at 3 PM.", ) await phone.call( agent=agent, to="+1234567890", machine_detection=True, voicemail_message="Hi, this is Acme. Please call us back to confirm your appointment.", ) ``` --- ## Architecture ### How It Works ``` Phone Call → Twilio/Telnyx webhook → Patter EmbeddedServer → StreamHandler StreamHandler routes audio based on the agent configuration: - Engine architecture: Audio ↔ OpenAI Realtime or ElevenLabs ConvAI (all-in-one: STT + LLM + TTS) - Pipeline architecture: Audio → STT provider → LLM → TTS provider → Audio back ``` ### Key Abstractions - **`Patter`** — Main entry point. Accepts a `carrier=` instance, creates agents, manages connections, starts the server. - **`Agent`** — Immutable configuration: engine (or stt+llm+tts), system prompt, tools, guardrails. - **`StreamHandler`** — Per-call session manager. Routes audio between telephony and voice providers. - **`LLMLoop`** — Pipeline orchestration that accepts any `LLMProvider` instance with streaming + tool calling. - **`EmbeddedServer`** — FastAPI (Python) or Express (TypeScript) server handling webhooks and WebSocket audio. ### Typed Instance API Pattern All provider axes follow the same pattern — a typed class with env-var fallback: - **Carrier**: `Twilio()`, `Telnyx()` - **Engine**: `OpenAIRealtime()`, `ElevenLabsConvAI()` - **STT**: `DeepgramSTT()`, `WhisperSTT()`, `CartesiaSTT()`, `SonioxSTT()`, `SpeechmaticsSTT()` (Python only), `AssemblyAISTT()` - **LLM**: `OpenAILLM()`, `AnthropicLLM()`, `GroqLLM()`, `CerebrasLLM()`, `GoogleLLM()` - **TTS**: `ElevenLabsTTS()`, `OpenAITTS()`, `CartesiaTTS()`, `RimeTTS()`, `LMNTTTS()` Each class reads its API key from environment variables by default. Override any parameter via constructor args. --- ## Voice Architectures ### 1. End-to-End Engine (Lowest Latency) All-in-one: STT + LLM + TTS handled by a single streaming provider. - **`OpenAIRealtime()`** — OpenAI's Realtime API (sub-500ms time-to-first-byte) - **`ElevenLabsConvAI()`** — ElevenLabs conversational AI ```python from getpatter import Patter, Twilio, OpenAIRealtime phone = Patter(carrier=Twilio(), phone_number="+15550001234") agent = phone.agent(engine=OpenAIRealtime(), system_prompt="You are a helpful assistant.") await phone.serve(agent, tunnel=True) ``` ### 2. Pipeline (Maximum Flexibility) Compose independent STT + LLM + TTS providers. Mix and match across 6 STTs, 5 LLMs, and 5 TTSs. ```python from getpatter import Patter, Twilio, DeepgramSTT, AnthropicLLM, ElevenLabsTTS phone = Patter(carrier=Twilio(), phone_number="+15550001234") agent = phone.agent( stt=DeepgramSTT(), llm=AnthropicLLM(model="claude-sonnet-4-5"), tts=ElevenLabsTTS(voice_id="rachel"), system_prompt="You are a helpful assistant.", ) await phone.serve(agent, tunnel=True) ``` --- ## Telephony Providers ### Twilio ```python from getpatter import Twilio carrier = Twilio() # reads TWILIO_ACCOUNT_SID + TWILIO_AUTH_TOKEN # or explicit: carrier = Twilio(account_sid="AC...", auth_token="...") ``` - Audio format: mulaw 8kHz (auto-transcoded internally) - Supports inbound and outbound calls - TwiML-based call control - Signature verification on webhooks - Auto-configures webhook URL when `tunnel=True` is used ### Telnyx (Beta) ```python from getpatter import Telnyx carrier = Telnyx() # reads TELNYX_API_KEY + TELNYX_PUBLIC_KEY ``` - Audio format: PCM 16kHz (native, no transcoding needed) - Supports inbound and outbound calls - Call Control API - Ed25519 webhook verification --- ## Speech-to-Text Providers ```python from getpatter import DeepgramSTT, WhisperSTT, CartesiaSTT, SonioxSTT, SpeechmaticsSTT, AssemblyAISTT DeepgramSTT(model="nova-2", language="en") # DEEPGRAM_API_KEY WhisperSTT(model="whisper-1") # OPENAI_API_KEY CartesiaSTT(model="ink-whisper") # CARTESIA_API_KEY SonioxSTT() # SONIOX_API_KEY SpeechmaticsSTT() # SPEECHMATICS_API_KEY (Python only) AssemblyAISTT() # ASSEMBLYAI_API_KEY ``` --- ## LLM Providers (Pipeline) ```python from getpatter import OpenAILLM, AnthropicLLM, GroqLLM, CerebrasLLM, GoogleLLM OpenAILLM(model="gpt-4o-mini") # OPENAI_API_KEY AnthropicLLM(model="claude-sonnet-4-5") # ANTHROPIC_API_KEY GroqLLM(model="llama-3.3-70b-versatile") # GROQ_API_KEY CerebrasLLM(model="llama-3.3-70b") # CEREBRAS_API_KEY GoogleLLM(model="gemini-2.0-flash-exp") # GOOGLE_API_KEY ``` All five providers support tool calling, streaming, and a unified `LLMProvider` interface. To use a custom LLM not in this list, implement the `LLMProvider` protocol or use the legacy `on_message` handler. --- ## Text-to-Speech Providers ```python from getpatter import ElevenLabsTTS, OpenAITTS, CartesiaTTS, RimeTTS, LMNTTTS ElevenLabsTTS(voice_id="rachel", model="eleven_turbo_v2") # ELEVENLABS_API_KEY OpenAITTS(voice="nova", model="tts-1") # OPENAI_API_KEY CartesiaTTS(voice_id="...") # CARTESIA_API_KEY RimeTTS(speaker="marissa") # RIME_API_KEY LMNTTTS(voice="...") # LMNT_API_KEY ``` All TTS providers auto-resample to the carrier's required sample rate (Twilio mulaw 8kHz or Telnyx PCM 16kHz). --- ## API Reference ### Patter (Client Constructor) ```python # Python from getpatter import Patter, Twilio phone = Patter( carrier=Twilio(), # Required: Twilio() or Telnyx() instance phone_number="+15550001234", # Your phone number (E.164 format) ) ``` ```typescript // TypeScript import { Patter, Twilio } from "getpatter"; const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234", }); ``` ### Agent Configuration ```python # Python — Engine architecture from getpatter import OpenAIRealtime, Tool, Guardrail agent = phone.agent( engine=OpenAIRealtime(), # Engine instance system_prompt="You are a helpful assistant.", first_message="Hello!", # Optional: opening message tools=[ Tool( name="lookup_order", description="Look up an order by ID", parameters={"type": "object", "properties": {"order_id": {"type": "string"}}}, webhook_url="https://api.example.com/orders", ), ], guardrails=[ Guardrail(type="block_terms", terms=["competitor", "lawsuit"]), ], variables={"customer_name": "John"}, # Dynamic variable substitution ) ``` ```python # Python — Pipeline architecture from getpatter import DeepgramSTT, AnthropicLLM, ElevenLabsTTS agent = phone.agent( stt=DeepgramSTT(), llm=AnthropicLLM(model="claude-sonnet-4-5"), tts=ElevenLabsTTS(voice_id="rachel"), system_prompt="You are a helpful assistant.", ) ``` ```typescript // TypeScript — Engine architecture import { OpenAIRealtime, Tool, Guardrail } from "getpatter"; const agent = phone.agent({ engine: new OpenAIRealtime(), systemPrompt: "You are a helpful assistant.", firstMessage: "Hello!", tools: [ new Tool({ name: "lookup_order", description: "Look up an order by ID", parameters: { type: "object", properties: { order_id: { type: "string" } } }, webhookUrl: "https://api.example.com/orders", }), ], guardrails: [new Guardrail({ type: "block_terms", terms: ["competitor", "lawsuit"] })], variables: { customerName: "John" }, }); ``` ### serve() — Start Inbound Server ```python # Python await phone.serve( agent, # Required: agent configuration (positional) port=8000, # Port to listen on (default: 8000) tunnel=True, # Auto-tunnel via Cloudflare + auto-configure Twilio webhook dashboard=True, # Enable built-in dashboard at /dashboard recording=True, # Enable call recording via telephony provider on_call_start=callback, # Callback: called when a call connects on_call_end=callback, # Callback: called when a call ends (with transcript + metrics) on_transcript=callback, # Callback: called on each transcript turn on_metrics=callback, # Callback: called with latency/cost metrics ) ``` ### call() — Place Outbound Call ```python # Python await phone.call( agent=agent, # Required: agent configuration to="+1234567890", # Required: destination number (E.164) machine_detection=True, # Enable answering machine detection (AMD) voicemail_message="Hi, ...", # Auto-play if voicemail detected ) ``` ### test() — Chat-Based Testing ```python # Python — test without making real phone calls await phone.test(agent=agent) ``` ### Built-in Tools (Auto-Injected) Two tools are automatically available to every agent: - **`transfer_call`** — Transfers the call to another phone number (E.164 validated). The agent decides when to transfer. - **`end_call`** — Terminates the current call with an optional reason. --- ## Environment Variables | Variable | Required | Description | |---|---|---| | `TWILIO_ACCOUNT_SID` | Yes (Twilio) | Twilio account SID | | `TWILIO_AUTH_TOKEN` | Yes (Twilio) | Twilio auth token | | `OPENAI_API_KEY` | Yes (OpenAI Realtime / Whisper / OpenAITTS / OpenAILLM) | OpenAI API key | | `TELNYX_API_KEY` | Yes (Telnyx) | Telnyx API key | | `TELNYX_PUBLIC_KEY` | Yes (Telnyx) | Telnyx public key for webhook verification | | `DEEPGRAM_API_KEY` | DeepgramSTT | Deepgram API key | | `ELEVENLABS_API_KEY` | ElevenLabsTTS / ElevenLabsConvAI | ElevenLabs API key | | `ANTHROPIC_API_KEY` | AnthropicLLM | For Claude via pipeline | | `GROQ_API_KEY` | GroqLLM | Groq API key | | `CEREBRAS_API_KEY` | CerebrasLLM | Cerebras API key | | `GOOGLE_API_KEY` | GoogleLLM | Google Gemini API key | | `CARTESIA_API_KEY` | CartesiaSTT / CartesiaTTS | Cartesia API key | | `RIME_API_KEY` | RimeTTS | Rime API key | | `LMNT_API_KEY` | LMNTTTS | LMNT API key | | `ASSEMBLYAI_API_KEY` | AssemblyAISTT | AssemblyAI API key | | `SONIOX_API_KEY` | SonioxSTT | Soniox API key | | `WEBHOOK_URL` | No | Public URL (auto-tunneled via Cloudflare when `tunnel=True`) | --- ## Starter Templates Clone-and-run repos for common use cases. Both Python and TypeScript included in each. | Template | Description | Repo | |---|---|---| | Inbound Agent | Answer calls as a restaurant booking assistant | [patter-inbound-agent](https://github.com/PatterAI/patter-inbound-agent) | | Outbound Calls | Place calls with AMD and voicemail drop | [patter-outbound-calls](https://github.com/PatterAI/patter-outbound-calls) | | Tool Calling | CRM lookup + ticket creation via webhook tools | [patter-tool-calling](https://github.com/PatterAI/patter-tool-calling) | | Custom Voice | Pipeline: Deepgram STT + ElevenLabs TTS | [patter-custom-voice](https://github.com/PatterAI/patter-custom-voice) | | Dynamic Variables | Personalize prompts per caller using CRM data | [patter-dynamic-variables](https://github.com/PatterAI/patter-dynamic-variables) | | Custom LLM | Bring your own model (Claude, Groq, Cerebras, Gemini) | [patter-custom-llm](https://github.com/PatterAI/patter-custom-llm) | | Dashboard | Real-time monitoring with cost + latency tracking | [patter-dashboard](https://github.com/PatterAI/patter-dashboard) | | Production Setup | Everything enabled: tools, guardrails, recording, dashboard | [patter-production](https://github.com/PatterAI/patter-production) | --- ## Use Cases - **Customer support**: AI phone agents that handle support calls 24/7 - **Appointment scheduling**: Automated booking, reminders, and confirmations - **Lead qualification**: Outbound calls to qualify and route sales leads - **Survey and feedback**: Automated phone surveys with structured data collection - **IVR replacement**: Replace traditional IVR menus with conversational AI - **Healthcare follow-up**: Patient appointment reminders and post-visit check-ins - **Restaurant booking**: Take reservations and answer menu questions - **Outbound notifications**: Automated alerts, reminders, and updates --- ## Comparison With Alternatives ### Patter vs Vapi | | Patter | Vapi | |---|---|---| | Open source | Yes (MIT) | No (proprietary) | | Deployment | Self-hosted | Hosted only | | Pricing | Free SDK + pay providers directly | $0.05+/min platform fee + provider costs | | Data ownership | Full (your infrastructure) | Vapi-controlled | | Custom LLM | 5 built-in + any via `LLMProvider` | Limited to supported models | | Custom STT/TTS | Pipeline mode (any provider) | Limited to Deepgram/ElevenLabs | | Vendor lock-in | None | Yes | ### Patter vs Bland AI | | Patter | Bland AI | |---|---|---| | Open source | Yes (MIT) | No (proprietary) | | Deployment | Self-hosted | Hosted only | | Pricing | Free SDK + pay providers directly | $0.09+/min | | SDKs | Python + TypeScript | REST API only | | Custom LLM | Yes | No | | Custom voice providers | Yes | No | ### Patter vs Retell | | Patter | Retell | |---|---|---| | Open source | Yes (MIT) | No (proprietary) | | Deployment | Self-hosted | Hosted only | | Pricing | Free SDK + pay providers directly | $0.07+/min | | Custom LLM | Any of 5 providers or custom | Limited | | Custom STT/TTS | Pipeline mode | Limited | | Data ownership | Full | Retell-controlled | --- ## FAQ **Q: Is Patter free?** A: Yes. Patter is MIT licensed and free to use. You only pay your telephony provider (Twilio/Telnyx) and AI providers (OpenAI/Deepgram/ElevenLabs/Anthropic/etc.) directly at their standard rates. **Q: Do I need a Patter account or API key?** A: No. Patter runs entirely on your own infrastructure. There is no Patter account, no Patter API key, and no Patter backend to connect to. **Q: What's the latency?** A: Sub-500ms time-to-first-byte with `OpenAIRealtime()` engine. Pipeline architecture latency depends on your chosen STT/LLM/TTS providers. **Q: Can I use my own LLM (Claude, Groq, Cerebras, Gemini, etc.)?** A: Yes. Patter includes `AnthropicLLM`, `GroqLLM`, `CerebrasLLM`, `GoogleLLM`, and `OpenAILLM` out of the box, all with tool calling. To use a model not in this list, implement the `LLMProvider` protocol or use the `on_message` handler. See the [custom LLM template](https://github.com/PatterAI/patter-custom-llm). **Q: Can I use Patter for both inbound and outbound calls?** A: Yes. Use `phone.serve(agent, tunnel=True)` for inbound calls and `phone.call(agent, to=...)` for outbound calls. Both directions work with Twilio and Telnyx. **Q: Does Patter handle call recording?** A: Yes. Set `recording=True` in `phone.serve()`. Recordings are stored by your telephony provider (Twilio/Telnyx), not by Patter. **Q: What is the difference between Patter and Vapi/Bland/Retell?** A: Patter is open-source and self-hosted. You own your data, pay no per-minute platform fees, and can use any AI provider. Vapi, Bland, and Retell are proprietary hosted services with per-minute pricing and vendor lock-in. **Q: What telephony providers does Patter support?** A: Twilio (stable) and Telnyx (beta). Bring your own credentials — no Patter account needed. **Q: Does Patter work with ElevenLabs voices?** A: Yes. ElevenLabs is supported both as a TTS provider in the pipeline architecture (`ElevenLabsTTS()`) and as a full conversational AI engine (`ElevenLabsConvAI()`). **Q: How does the auto-tunnel work?** A: Pass `tunnel=True` to `phone.serve()`. Patter spawns a Cloudflare tunnel and automatically updates your Twilio number's webhook URL to point at it. No ngrok or manual webhook configuration required. **Q: Is there a dashboard?** A: Yes. Set `dashboard=True` in `phone.serve()` to enable the built-in web dashboard at `/dashboard` with real-time call metrics, history, and CSV/JSON export. **Q: What about answering machine detection?** A: Set `machine_detection=True` on outbound calls. Patter detects answering machines and can auto-play a voicemail message. --- ## Canonical - **Primary URL**: https://www.getpatter.com - **llms.txt**: https://www.getpatter.com/llms.txt - **llms-full.txt**: https://www.getpatter.com/llms-full.txt