Forfis Blog

Tag: Cut First-Response Time

  • Swiss E-commerce Cuts Invoice Cycle Time 92% in a Two-Week ISO 27001-Safe Pilot

    Background: A Swiss Retail Group Under Audit Pressure

    This case study is a composite drawn from patterns observed across multiple engagements. No named customer is represented. The details are plausible and reflect the range of outcomes seen in the field, but they do not describe a single real company.

    The client is a Swiss e-commerce and retail group with roughly 2,400 employees, operating in German, French, and Italian markets. The finance and accounting team handles 18,000 to 22,000 supplier invoices per month across three ERP instances. The stack is a mix of SAP S/4HANA for the core ledger, a legacy document management system for invoice images, and Confluence for internal runbooks and audit documentation. The company holds ISO 27001 certification and is in the middle of a renewal audit. The finance director’s mandate was clear: reduce the average cycle time from invoice receipt to ERP posting without introducing a compliance gap.

    Challenge: 20,000 Invoices a Month and a 90-Day Audit Clock

    The finance team was processing invoices manually: a clerk downloaded the PDF, typed the vendor name, amount, tax code, and cost center into the ERP, and flagged discrepancies for review. The average cycle time was 4 to 6 hours per invoice, with a 3 to 5 percent error rate on a sample of 500 invoices. The error rate was not just a cost issue; it was a compliance issue. ISO 27001 requires documented controls over financial data, and a 4 percent error rate on 20,000 invoices per month meant roughly 800 mis-posted entries that had to be caught in a secondary review. The secondary review was itself a manual process, adding another 2 to 3 hours per flagged invoice. The finance director had a deadline: the ISO 27001 renewal audit was 90 days out, and the auditor had already flagged the manual process as a control weakness.

    Approach: A Two-Week Pilot on the Top Five Vendors

    The engagement started with a three-day process audit. The team mapped the invoice lifecycle from receipt to posting, identified the 12 vendor categories that accounted for 78 percent of volume, and pulled a historical sample of 1,200 invoices for calibration. The pilot scope was fixed: one ERP instance, one vendor category (the top 5 suppliers by volume), and a two-week window. The architecture used the OpenAI API for extraction, with a human-in-the-loop approval queue. The model extracted vendor name, invoice number, amount, tax code, and cost center. A reviewer saw the proposed entry alongside the original PDF and could approve, correct, or reject. The approval log was written to Confluence and to the ERP audit trail. The pipeline connected to the ERP via its REST API and to the document store via SFTP. No new infrastructure was required. The client’s existing IT team handled the API credentials and network access.

    Outcome: 92 Percent Cycle-Time Reduction in 12 Days

    The pilot ran for 12 business days. The model processed 1,840 invoices from the top five vendors. The average cycle time dropped from 4.2 hours to 22 minutes, a 92 percent reduction. The error rate on the pilot sample was 0.8 percent, down from the 3.4 percent baseline. Of the 1,840 invoices, 1,612 were approved with zero edits. The remaining 228 required human correction, mostly on tax codes for cross-border invoices. The approval queue averaged 14 minutes per invoice for the corrected entries. The ISO 27001 audit trail showed 100 percent of inferences logged with timestamp, user ID, and confidence score. The finance director presented the pilot results to the audit committee. The auditor accepted the AI-assisted workflow as a control improvement, conditional on the managed operations SLA being in place before the renewal audit.

    Lessons for Teams Running Similar Pilots

    • The historical sample matters more than the model. The 1,200-invoice calibration sample was the single biggest factor in the 0.8 percent error rate. A team that skips this step and goes live with a generic prompt will see error rates of 8 to 12 percent and lose the human trust needed for the approval workflow.
    • Fix the scope before you start. The two-week window only worked because the pilot was limited to one ERP instance and five vendors. A team that tries to cover all 12 vendor categories in two weeks will spend the time on integration edge cases and miss the baseline measurement.
    • The approval queue is the product, not the model. The model’s extraction quality was good, but the reviewer interface was what made the workflow usable. A team that ships a model without a clean approval UI will see reviewers bypass the system and go back to manual entry.
    • ISO 27001 is a design constraint, not a post-hoc checkbox. The audit trail, the data processing agreement, and the access controls were built into the architecture from day one. Retrofitting them after go-live is 3 to 4 times more expensive and often fails the audit.
    • Managed operations is where the value compounds. The pilot proved the concept. The managed operations SLA, with monthly reports on confidence distribution and error rate, is what keeps the error rate at 0.8 percent instead of drifting to 3 percent as vendor formats change.

  • 8-Week AI Automation Audit: Cutting First-Response Time in a UAE Medtech Firm

    1. Map the ticket flow before touching the model

    The audit phase is where most 11-50 person firms stall. Forfis starts by mapping every ticket that hits the support queue over a 10-day window, tagging each by topic, resolution path, and time-to-first-response. For a UAE medtech company, the data typically shows 60-70% of tickets are “where is the protocol for X” or “what is the warranty window for Y” questions that live in Confluence or Notion but are buried under 200+ pages. The audit output is a ranked list of the top five question categories by volume and time cost, with a measured baseline: average first-response time of 4.2 hours, error rate of 12% on a 200-ticket sample. This baseline is the number the pilot must beat, and it is documented in a one-page report the team signs off on before any code is written.

    2. Build the RAG layer on LangGraph, not a monolith

    The RAG pipeline indexes Confluence and Notion pages into a vector store, chunking at 512 tokens with 64-token overlap. LangGraph orchestrates the retrieval, generation, and scoring nodes. The predictive scoring module evaluates each draft on three axes: retrieval relevance (cosine similarity of the top-3 chunks), answer coherence (a secondary LLM call that checks the draft against the retrieved context), and historical approval rate (a running average from the pilot’s first 50 tickets). Responses scoring below 0.85 route to a human; those above auto-post to the helpdesk. For a 15-person team, this means the AI handles roughly 75% of tickets, and the human agent reviews the remaining 25% in under 5 minutes each. The scoring threshold is tunable in the LangGraph config without redeploying.

    3. Run the pilot with a measured before/after baseline

    The pilot runs for two weeks on a live subset of tickets. The team uses the agent in production, and every interaction is logged: the ticket ID, the retrieved chunks, the draft answer, the predictive score, and whether the human approved, edited, or rejected it. By the end of the soak period, the team has a 200-ticket dataset with before/after metrics. For a UAE medtech firm, the typical result is first-response time dropping from 4.2 hours to 18 minutes, with error rate holding at 11% or below. The 8-week timeline includes a one-week buffer for model tuning if the initial scoring threshold is too aggressive or too conservative. The final deliverable is a one-page baseline report with the numbers, the model used, the cost per 1,000 tokens, and a recommendation on whether to scale to all ticket categories or adjust the scope.

    4. Keep the model layer swappable from day one

    The architecture calls the LLM through an abstraction layer in LangChain, so the model is a config parameter, not a hard dependency. For a UAE healthcare firm with no compliance mandate, starting with OpenAI’s GPT-4o API is the fastest path: no hardware procurement, no MLOps overhead. The audit phase documents the cost per 1,000 tokens (typically $0.03-0.06 for GPT-4o) and the latency (18-25 ms for a 512-token response). If the team later decides to move to an open-weight model like Llama 3.1 70B on their own hardware, the LangGraph nodes do not change. The swap is a one-line config update. This matters for a 15-person team because it removes the risk of being locked into a single vendor’s pricing or API changes mid-engagement.

    5. Plug into the helpdesk, not around it

    The agent does not replace the helpdesk. It plugs into the existing ticketing system via API. When a ticket arrives, the agent retrieves relevant chunks, drafts a response, and posts it as a suggested reply in the ticket. The human agent sees the draft, approves or edits it, and sends it. The agent logs the retrieval context and the predictive score in the ticket metadata, so the team can audit why a particular answer was suggested. For a 15-person team, this means no new UI to learn, no workflow redesign, and no training beyond a 30-minute onboarding session. The agent operates inside the tools the team already uses, which is critical for adoption in a small firm where every hour of context-switching is expensive.

    6. Plan for the knowledge base to change

    The most common failure mode is treating the pilot as a one-time deliverable. For a 15-person UAE medtech firm, the knowledge base changes weekly: new protocols, updated warranty terms, revised SOPs. The RAG pipeline must re-index Confluence and Notion on a schedule (daily or on webhook trigger) to keep the chunks current. The predictive scoring model also drifts: the approval rate that was 75% in week 6 may drop to 60% in week 10 if the team starts asking different questions. The 8-week engagement includes a handover document that specifies the re-indexing cadence, the scoring threshold review schedule (monthly), and the escalation path if error rate exceeds 15% on a rolling 50-ticket window. Without this, the agent degrades silently within 60 days.

    7. Define the success metric before the pilot starts

    The 8-week engagement is not a product launch; it is a measured experiment with a clear success criterion. For a UAE medtech firm, the success criterion is: first-response time under 30 minutes on 80% of tickets, error rate under 12%, and the team reporting that the agent saves at least 3 hours per week per agent. The audit phase sets the baseline, the pilot measures against it, and the final report states whether the criterion was met. If it was, the team decides whether to scale to all ticket categories, add a voice channel, or extend the RAG layer to other internal tools. If it was not, the report identifies which axis failed (retrieval, generation, or scoring) and what the next iteration should target. The engagement ends with a decision, not a demo.

  • 12-Step Checklist: RAG Pilot for Lead Qualification in German Insurance

    Pre-Pilot: Scope and Compliance Setup

    A 2-week fixed-scope pilot in a German insurance firm must produce a working RAG assistant on one workflow, a GDPR-compliant data-flow document, and a measured before/after baseline. The checklist below is operational: each item is a task a team can mark done or not done. It assumes the team uses LangChain and LangGraph, integrates with Slack or Microsoft Teams, and targets lead qualification to cut first-response time. The pilot is not a production deployment; it is a scoped experiment with a clear exit criterion. Work through the items in order. Skipping the audit or the baseline measurement invalidates the pilot’s value as a decision input for rollout.

    Build the RAG Pipeline on LangChain and LangGraph

    The RAG pipeline is the core of the pilot. Build it on LangChain for document chunking, embedding, and vector search, and on LangGraph for the stateful workflow that routes queries, handles multi-turn context, and triggers the human-approval gate. Keep the graph simple: one retrieval node, one generation node, one approval gate. Use a managed vector store in an EU region for the pilot. If the client’s data cannot leave the building, switch to an on-premises vector store and an open-weight model on the client’s GPU hardware. The RAG code is identical; only the embedding and inference endpoints change. Test the pipeline against 20 real lead queries before integrating with Slack or Teams.

    Integrate with Slack or Microsoft Teams

    The pilot must integrate with the channel the team already uses: Slack or Microsoft Teams. Build a bot that receives the lead query, calls the RAG pipeline, and returns the draft qualification score and suggested next step. The bot must include a human-approval gate: if the AI’s confidence drops below a threshold, or if the lead involves health-related data, the bot flags the query for a human agent. Log every human override. The integration must not replace the existing CRM or helpdesk; it plugs into them via their APIs. For a 2-week pilot, use a single OpenAI or Anthropic API endpoint for the LLM layer. Keep the model-agnostic layer thin: a single abstraction over the API call so switching providers later requires only a config change.

    Measure the Before/After Baseline

    Before the pilot starts, measure the baseline: cycle time from lead entry to qualified status, and error rate (misclassified leads) for the 2 weeks prior. Document the sample size, the definition of ‘error,’ and the measurement method. During the pilot, measure the same metrics for the 2 weeks of the pilot. The before/after comparison is the pilot’s primary deliverable. Without it, the client has no objective basis for the rollout decision. The baseline report must include: the number of leads processed, the average cycle time before and after, the error rate before and after, and the number of human overrides. This report is the exit criterion for the pilot.

    Define the Pilot Exit Criterion

    The pilot is a fixed-scope engagement: the vendor delivers a defined set of artifacts within the 2-week deadline. It is not a subscription or managed service. After the pilot, the client decides whether to proceed to rollout. The pilot includes a measured before/after comparison on cycle time and error rate, giving the client objective data to justify or reject the full deployment. The exit criterion is clear: if the pilot reduces cycle time by at least 30% and error rate by at least 20%, the client proceeds to rollout. If not, the pilot ends, and the client retains the baseline report and the RAG pipeline code. The vendor does not retain any client data after the pilot ends.

    Maintain the Checklist Over Time

    The checklist is a living document. After the pilot, review each item: mark what worked, what did not, and what needs adjustment. If the pilot proceeds to rollout, update the checklist to reflect the new scope: additional workflows, multi-language support, production monitoring. If the pilot ends, archive the checklist with the baseline report. Revisit the checklist before any new pilot: the GDPR landscape, the LLM provider landscape, and the integration landscape change. The checklist is not a one-time artifact; it is a tool for continuous improvement in AI-native operations. Keep it in the team’s project management tool, not in a static PDF.