The Problem

 

Insurance companies can't investigate every claim, so they need a way to decide which ones deserve a closer look. Only about 6% of claims in this data are fraud, which makes them hard to find. I wanted to see if a model could flag risky claims using only the information already in a claim file.

 

​My Role

 

I did this project on my own for my machine learning class. I chose the topic because of my internship on the Claims Litigation team at USLI, where I saw how much work goes into reviewing claims.

 

1. The Data

 

The dataset has 15,420 real auto claims from a US insurer, 1994 to 1996. I first used a popular Kaggle fraud dataset, but switched after realizing it was likely made up: the columns were too evenly spread, and patterns like "chess players commit fraud" made no sense.

​

Only 6% of claims are fraud. A model that always said "not fraud" would be 94% accurate and completely useless, so I judged my models on how much fraud they caught instead.

 

 

 

 

 

 

 

 

 

 

 

​

2. Cleaning the Data

 

I checked the histograms for outliers and removed one claim with no claim date. I ordinal encoded ranges like vehicle price in their real order, one-hot encoded the rest, and dropped PolicyType because it just combined two other columns. The most interesting part was a hidden missing-values problem:

 

 

​

 

 

 

 

​

3. Looking for Clusters

 

Before using the fraud labels, I ran KMeans to see if claims grouped together on their own. The best split was 2 groups: people with multiple cars or a recent move, and everyone else. That first group had more fraud, but it was still over 90% legit, so clusters alone couldn't find fraud.

 

 

​​

 

 

 

 

 

 

​

4. Training and Testing

 

I split the data 80/20 and trained a logistic regression and a decision tree. Since fraud is so rare, I lowered the cutoff from 50% to 10%, so a claim gets flagged if its fraud chance is above average. I picked 10% before seeing results instead of tuning it on the test data.

 

​​

​

​

​

​

​

​

 

​​

Logistic regression caught 126 of 199 fraud claims. The decision tree caught 92.

​​​

5. What the Tree Learned

​

 

 

 

 

 

 

 

 

 

Coral boxes are groups with over 10% fraud, which the model flags for review.

​

• Liability-only policies were almost never fraud. Liability pays for damage you do to others, not your own car, so there's little to gain.

• The riskiest group was at-fault drivers with All Perils coverage.

• When someone else was at fault, fraud was rare unless the policyholder had moved recently.

​

Result

​

Logistic regression caught 63% of fraud claims, and the decision tree caught 46%. Most flagged claims still weren't fraud, but both models more than doubled the fraud rate compared to picking claims at random. The tree found clear red flags. Liability-only policies were almost never fraud (0.7%). At-fault drivers with All Perils coverage were 15.6% fraud. Not-at-fault claims from people who had recently moved were 29% fraud.

​

What I Learned

​

Accuracy can be really misleading. My first model was 94% accurate but caught only 1 fraud claim. I also learned to check data instead of trusting it: the imputer filled in ages around 30 for drivers who were actually 16 or 17. Getting realistic results on real data taught me more than getting great results on fake data.