The Problem: Back-Office Bottlenecks in Austrian Professional Services
A 201-500 employee professional services firm in Austria faces a familiar problem: customer support is a bottleneck. Order and shipment status inquiries arrive via phone, email, and chat, and the back office team spends 3-4 hours daily answering the same questions. The error rate is 8-12%: wrong shipment dates, incorrect order statuses, missed follow-ups. The firm wants round-the-clock response without hiring more staff, but the EU AI Act’s transparency requirements and the need to keep regulated data in-house complicate the solution. Forfis starts with a process audit that maps the top 10 workflows by volume and error cost, then selects order status queries as the pilot: high volume, low complexity, clear success metrics. The 8-week timeline is tight but feasible because the scope is narrow: one workflow, one channel (voice), one integration stack (CRM, ERP, Google Workspace). The audit phase (weeks 1-2) establishes the baseline: 12-minute average cycle time, 8% error rate. The pilot must reduce cycle time to under 2 minutes and error rate to under 1%.
Mechanism: LangGraph Orchestration and the Voice Agent Loop
The voice agent runs on a LangGraph state machine. Each node is a step: ‘transcribe audio’, ‘parse intent’, ‘query CRM’, ‘draft response’, ‘speak response’. Edges are conditional: if the intent is ‘order status’, route to the CRM lookup node; if ‘shipment tracking’, route to the logistics API node; if ‘escalate to human’, route to the operator queue. LangGraph tracks conversation state: which customer is being served, what they’ve already asked, whether the agent has given a response. This is more robust than a simple chain because it handles loops (customer asks a follow-up) and parallel branches (check order AND shipment status). The LLM layer uses OpenAI GPT-4 for intent parsing and response drafting, with a confidence threshold: if the model’s confidence is below 80%, the agent asks a clarifying question or escalates. The speech-to-text layer uses Whisper or a commercial API, targeting under 500ms latency. The text-to-speech engine converts the drafted response to natural speech. The entire loop (transcription, LLM inference, API call, TTS) targets under 3 seconds for a natural conversation feel. The architecture is model-agnostic: if the firm later needs to deploy open-weight models on-premises for regulated data, the LangGraph orchestration layer stays the same; only the LLM node changes.
Trade-offs: Model Choice, Latency, and Human Oversight
The first trade-off is model choice. OpenAI GPT-4 offers the best quality for natural language understanding, but it requires sending data to a third-party API. For a professional services firm handling client data, this may violate internal data governance policies. The alternative is open-weight models (Llama 3, Mistral) on the client’s own hardware, which keeps data in-house but sacrifices some quality. Forfis resolves this by using GPT-4 for the voice agent’s core reasoning (where quality matters most) and open-weight models for data extraction tasks (where speed and privacy matter more). The second trade-off is latency vs. accuracy. A faster model (GPT-3.5) reduces latency but increases error rate. For order status queries, the error cost is low (a wrong shipment date is annoying but not catastrophic), so a faster model is acceptable. For contract or billing queries, the error cost is high, so a slower, more accurate model is required. The third trade-off is automation vs. human oversight. Full automation reduces cycle time but increases risk. The human-in-the-loop model (agent drafts, human approves) adds 30-60 seconds to each interaction but reduces error rate to near zero. For the pilot, Forfis uses full automation for standard queries and human approval for anything touching money or contracts.
Compliance and Recommendation: EU AI Act and the 8-Week Pilot
The EU AI Act’s Article 50 requires transparency for AI systems interacting with humans. The voice agent must clearly state it is an AI, not a human, at the start of the conversation. Forfis builds this into the opening script: ‘You are speaking with our automated assistant. I can help with order status and shipment updates. If you need a human, say so.’ The system logs all interactions, including the AI’s responses and any escalations, for audit purposes. The logs are stored in the firm’s own infrastructure, not a third-party cloud, to comply with data residency requirements. For order status queries, the risk classification is minimal: the agent is not making decisions that affect rights, so it does not trigger the higher-risk obligations under Article 6. However, if the agent is later extended to handle refunds or contract disputes, the risk classification changes, and additional obligations (e.g., human oversight, impact assessment) apply. The recommendation is to build the transparency and logging infrastructure from day one, even if the current use case is low-risk. This avoids a costly re-architecture if the scope expands. The dedicated AI team (technical lead, product designer, 2-3 engineers) works full-time on the pilot for 8 weeks. The cost structure is fixed-scope: the pilot has a defined deliverable (a working voice agent for order status queries, with measured before/after metrics). Rollout and managed operation are separate phases with ongoing costs.
Leave a Reply