PolicyOps

Tech Stack

The Problem

Insurance fraud costs the industry tens of billions annually, yet conventional detection pipelines rely heavily on rigid rule-based heuristics that organized fraud rings rapidly learn to circumvent. In operational insurance environments, detection models face a severe penalty asymmetry: false negatives result in direct capital leakage from undetected fraudulent payouts, while false positives introduce unjustified claim friction that alienates legitimate policyholders. The core challenge was engineering a reproducible, experiment-tracked machine learning pipeline capable of continuous retraining as adversarial fraud patterns evolve. Crucially, deployment within regulated insurance environments requires transparent, auditable decision boundaries so claims adjusters and compliance officers can substantiate every flagged transaction rather than relying on an opaque black box.

Architecture

PolicyOps implements an end-to-end MLOps pipeline designed to maintain data versioning rigor and explainable decisioning throughout the model lifecycle. The architecture ingests raw claims records managed under DVC version control, passing them into an automated Pandas feature engineering pipeline to generate versioned, reproducible data artifacts. Model exploration was systematically tracked across 40+ MLflow experiment runs comparing baseline classifiers against ensemble methods, culminating in a champion XGBoost model achieving an F1 score of 0.8355 and ROC-AUC of 0.9239 under fraud-weighted recall priorities. A SHAP TreeExplainer layer decomposes every inference into feature contribution scores and waterfall visualizations for regulatory auditors, while a production-grade FastAPI service exposes high-throughput prediction and explanation endpoints consumed by an ink-navy/paper styled React frontend.

Approach

1. 01

   Designed a DVC data pipeline to version raw claims data, feature engineering scripts, and model artifacts — ensuring full reproducibility across training runs.

2. 02

   Ran 40+ MLflow experiments comparing Logistic Regression, Random Forest, and XGBoost on precision-recall tradeoffs relevant to fraud (high recall priority — missing fraud is expensive).

3. 03

   Selected XGBoost as the champion model (F1 = 0.8355, ROC-AUC = 0.9239) and used SHAP TreeExplainer to surface the top 15 driving features for auditor review.

4. 04

   Built a FastAPI serving layer with a /predict endpoint and a /explain endpoint returning SHAP waterfall plots as base64 images.

5. 05

   Designed a React frontend with an ink-navy/paper/brass visual identity featuring a claims submission form, real-time prediction output, and interactive SHAP visualisation.

Outcome

A fully reproducible fraud detection system with documented experiment lineage, production-grade serving, and an explainable output layer that satisfies regulatory review requirements. The SHAP integration reduces the black-box objection common in insurance ML deployments.