Hard-E is a production-grade, multi-tenant conversational AI built for the contracting industry. It connects to a contractor's CRM and operates as a voice-first sales director — executing CRM actions, managing email and SMS, answering product questions from uploaded documentation, and learning from every interaction.
Architecture
Hard-E is built on four layers: the brain (LLM orchestration), the voice (real-time WebRTC pipeline), the memory (persistent learning), and the tenant framework (multi-company isolation).
The Brain
The core intelligence runs on Anthropic Claude Sonnet. When a user speaks or types, Claude classifies intent, selects which CRM tools to call, executes them (in parallel where possible), and formulates a natural-language response. A per-tenant "Soul Document" defines personality, product opinions, and company-specific behavior.
The Voice Pipeline
Real-time, two-way voice conversation is delivered through a full-duplex WebRTC pipeline built on Pipecat, an open-source framework for voice AI agents. The pipeline coordinates four services simultaneously:
- Deepgram — streaming speech-to-text with sub-second transcription
- Claude — intent processing, tool orchestration, response generation
- Cartesia Sonic — text-to-speech with sub-100ms time-to-first-audio
- Silero VAD — voice activity detection enabling natural barge-in (interrupt the AI mid-sentence)
End-to-end voice latency is under 2 seconds. When a voice session ends, the system automatically summarizes the conversation and saves key learnings to persistent memory.
The Memory System
Hard-E uses a three-tier learning architecture that makes it get smarter over time:
- Session Memory — short-term conversational context (24-hour TTL)
- Conversation Learning — automatic extraction of decisions, facts, and action items after sessions go idle
- Knowledge Memory — permanent categorized store: company facts, learned patterns, user preferences, and tool knowledge
Multi-Tenant Framework
Each company operates in complete isolation — encrypted CRM credentials, private knowledge base (S3-backed), separate memory, and customizable personality. A guided onboarding flow configures a new tenant in minutes. Trial accounts are provisioned through invite links with configurable time and message limits.
CRM Integration
Hard-E provides deep, bidirectional integration with Leap CRM through 18 specialized tools:
Custom workflow stages are resolved dynamically — Hard-E fetches and caches each company's stage configuration rather than relying on hardcoded mappings. Compound tools execute multiple API calls in parallel via asyncio.gather, reducing latency by ~50% on multi-step queries.
Direct Tools API
Hard-E exposes a RESTful API that allows external AI agents and automation platforms to execute CRM tools directly — without routing through the LLM layer.
- Discover tools —
GET /api/v1/toolsreturns every available tool with full input schemas - Execute —
POST /api/v1/tools/executefor single tool calls - Batch —
POST /api/v1/tools/batchfor up to 10 tools in one request
API keys are tenant-scoped, revocable, rate-limited, and audit-logged. This API has been validated in production with an external automation agent running 300+ calls per day.
Knowledge Base
Hard-E processes training documents through chunk-based retrieval-augmented generation (RAG). Documents are stored in S3, split into ~1,500-character chunks, scored for keyword relevance against each query, and the top-ranked chunks feed the LLM response. Current product expertise includes:
- James Hardie fiber cement — full Northeast Product Catalog (HardiePlank, HardiePanel, HardieShingle, HardieTrim, Artisan, Architectural Panel)
- CertainTeed — 2026 Siding Collection (vinyl and polymer siding lineup)
- General siding terminology, sales training materials, and competitive positioning
Each tenant can upload their own documents (.txt, .md) to build a private knowledge base specific to their business — pricing sheets, SOPs, product guides, sales scripts.
Security
3-Tier Safety Policy
- Green (auto-execute) — read-only operations: CRM lookups, knowledge queries, memory search
- Yellow (execute + audit) — state-changing but reversible: notes, stage updates, SMS
- Red (require approval) — high-impact or irreversible: customer creation, sending emails, financial operations
Infrastructure
- SSL/TLS on all endpoints with automated certificate renewal
- CORS-restricted origins, per-endpoint rate limiting
- Credential encryption at rest for all tenant API keys
- JWT-based session management with bcrypt password hashing
- Daily automated security monitoring and email reporting
- Fail2Ban intrusion prevention
Performance
| Metric | Performance |
|---|---|
| Voice latency (end-to-end) | Sub-2 seconds |
| TTS time-to-first-audio | ~40–90ms |
| STT transcription | Real-time streaming |
| CRM lookup (parallel compound) | ~3–4 seconds |
| Simple question response | ~1.5–2 seconds |
| Direct Tools API execution | Sub-1 second |
| Knowledge base retrieval | ~2–4 seconds |
Technology Stack
Getting Started
Contractors can try Hard-E through an invite-based trial system. Each trial provides a private tenant with configurable time and message limits. During signup, connecting a Leap CRM API key enables the full 18-tool CRM integration immediately. No Leap-side changes are required — Hard-E operates on Leap's existing published API.
Want to test it? Contact us for a trial link. Plug in your Leap API key during setup and Hard-E connects to your CRM data immediately. Full voice mode, all 18 tools, your own private knowledge base.
Roadmap
- Hover 3D Integration — property modeling and measurement data for automated estimation
- Estimation Engine — combining product knowledge, labor costs, and measurements to generate material and cost estimates
- Multi-CRM Support — extending the plugin architecture beyond Leap
- Agent-to-Agent API — enabling external AI agents to have conversations with Hard-E, not just call tools
- Advanced Analytics — dashboard for conversation metrics, tool usage, and pipeline insights