The Problem: Manual Candidate Screening at Scale
A 201-500 person B2B SaaS company in the USA runs candidate screening as a manual, multilingual back-office function: recruiters read resumes, score them against job descriptions, and flag top candidates for interview. The process is slow (median 14 days from application to first review), inconsistent across hiring managers, and non-compliant with GDPR Article 22 if any automated decision triggers rejection without human oversight. The company is at the “Running Isolated Pilots” stage of AI maturity: it has tested a chatbot for customer support but has not yet automated a core HR workflow. The goal is a compliance-safe AI rollout that replaces manual screening with predictive scoring, uses pgvector embeddings for semantic matching, integrates via custom REST API and webhooks into the existing ATS, and supports multilingual applications across 5-10 languages. The delivery model is a dedicated AI team working over 6 months, with human-in-the-loop approval on every screening decision.
Prerequisites Before You Start
Before step 1, confirm the following are in place:
- Access to historical hiring data: at least 12 months of application records, including resume text, job description, hiring outcome (hired/not hired), and 12-month retention status. This is the training set for the predictive scoring model.
- ATS API credentials: your applicant tracking system (Greenhouse, Lever, Workable, or equivalent) must expose a REST API with read/write access to candidate records and job postings. Document the endpoint URLs, authentication method (API key or OAuth 2.0), and rate limits.
- Legal sign-off on GDPR compliance: your DPO or outside counsel must confirm that the screening workflow will include a mandatory human approval gate, that data subjects can request an explanation of the scoring criteria, and that all processing is logged under Article 30.
- A named human reviewer for each role family: the person who will approve or reject AI-scored candidates. This is not optional under GDPR Article 22.
- A Postgres 15+ instance with the pgvector extension installed, or a managed Postgres service (RDS, Cloud SQL, Supabase) that supports pgvector. The embeddings table will live here.
- A dedicated AI team with at least 2 engineers and 1 product lead, engaged for the full 6-month timeline.
Step 1: Audit the Current Screening Workflow
Run a 2-week process audit on your current screening workflow. Map every step from application receipt to first interview scheduling: who touches the resume, how long each step takes, where candidates drop off, and which languages appear in the application pool. Export 200 recent applications across 3 role families (e.g., engineering, sales, customer success) and manually score them using your existing rubric. Record the median cycle time (target baseline: under 14 days), the error rate (how often a manually scored candidate was later found to be a poor fit), and the language distribution. This baseline is your before/after measurement. Without it, you cannot prove the AI outperforms the manual process, and you cannot detect degradation after rollout. The audit also identifies which role families have enough historical data to train a reliable scoring model and which do not.
Step 2: Scope the Pilot on One Role Family
Select one role family for the pilot. The criteria: at least 50 historical hires with 12-month retention data, a clear scoring rubric that hiring managers already use, and a multilingual application volume that justifies the embedding pipeline. For a B2B SaaS company, “Senior Software Engineer” or “Account Executive” are typical first pilots because they have high application volume and well-defined skill requirements. Define the pilot scope in a one-page document: the role family, the ATS endpoints you will use, the scoring criteria (skills match, experience depth, education, semantic similarity to past successful hires), the human reviewer’s name, and the success metrics (target: reduce cycle time from 14 days to under 5 days, reduce error rate by 30%). The pilot ships with a measured before/after baseline on both metrics. Do not expand the scope during the pilot; adding a second role family or a new scoring criterion mid-pilot invalidates the baseline comparison.
Step 3: Build the Document Extraction and pgvector Pipeline
Build the extraction and embedding pipeline. Ingest resumes and job descriptions from the ATS via its REST API. Parse the document text (PDF, DOCX, plain text) using a library like pdfplumber or unstructured to extract structured fields: name, email, skills, work history, education. Store the raw text and extracted fields in Postgres. Embed both the candidate profile and the job description using a multilingual embedding model (e.g., multilingual-e5-large-instruct or BGE-M3) into 1024-dimensional vectors. Store the vectors in a pgvector table: CREATE TABLE candidate_embeddings (id UUID PRIMARY KEY, candidate_id UUID, job_id UUID, embedding vector(1024), created_at TIMESTAMP). Use cosine similarity search to rank candidates: SELECT candidate_id, 1 - (embedding <=> $1) AS similarity FROM candidate_embeddings WHERE job_id = $2 ORDER BY similarity DESC LIMIT 50. This replaces keyword matching with semantic matching, so “managed a $2M budget” matches “financial oversight” without identical terms.
Step 4: Train the Predictive Scoring Model
Train the predictive scoring model on your historical hiring data. The features: skills match score (from the extraction pipeline), experience depth (years in relevant roles), education level, semantic similarity to past successful hires (from the pgvector search), and application completeness. The target variable: 12-month retention (1 if the candidate was still employed after 12 months, 0 otherwise). Use a gradient-boosted classifier (XGBoost or LightGBM) for interpretability; the model outputs a probability score between 0 and 1. Calibrate the score so that the top decile corresponds to candidates with a 70%+ probability of 12-month retention. Document the scoring criteria in a one-page summary that you can share with candidates under GDPR Article 13 (right to information about automated decision-making). The model is retrained quarterly as new hiring data accumulates. Store the model version, training data hash, and feature weights in a metadata table for audit purposes.
Step 5: Integrate via REST API and Webhooks
Build the REST API and webhook integration. Expose three endpoints: POST /api/v1/candidates/screen (accepts candidate ID and job ID, returns score and rationale), GET /api/v1/candidates/{id}/score (retrieves the score and feature breakdown), and POST /api/v1/candidates/{id}/approve (human reviewer approves or rejects, with a comment field). The approval endpoint is the GDPR Article 22 gate: no rejection is sent to the candidate until a human clicks approve. Webhooks push events to your ATS: candidate.scored (when the model outputs a score), candidate.approved (when a human approves), candidate.rejected (when a human rejects). All payloads are logged with timestamps, user IDs, and IP addresses for the Article 30 audit trail. The API is deployed on your existing infrastructure (AWS, GCP, or on-prem) behind your existing authentication layer. Rate limits: 100 requests/minute per API key. Error responses follow RFC 7807 (Problem Details for HTTP APIs).
Leave a Reply