Choosing the Right Metrics for iGaming Predictive Models
In the previous article we explained that any machine learning model should beat a simple rule-based baseline before it is deployed. This leads to a practical question.
How do you actually compare them? Which numbers prove that the model is better? The answer is rarely accuracy. In iGaming, a model reporting 99% accuracy can still be useless.
Choosing the right evaluation metric is both a technical and a business decision. Different problems require different ways of measuring success.
Metrics should follow the business objective
Different predictive use cases require different metrics. In fraud detection, recall is critical. Missing an abuser can cause direct financial damage. In automated payouts or bonus eligibility, precision becomes more important. Operators must be confident that bonuses or payouts are not going to players who violated the rules. In marketing or lifecycle segmentation such as early VIP detection or churn prediction, lift and profit curves matter most. The goal is to identify the right group of players while balancing the cost of action such as bonuses against the value preserved or created.
Choosing a metric is not a technical detail. It is a strategic choice that must match the business outcome you care about. This also affects how durable the model will be over time. Earlier we discussed why behavioural patterns outperform rigid rules. Those patterns remain valuable only if the model is optimised for the correct business objective.
Why accuracy can mislead iGaming teams
Many predictive use cases in iGaming involve rare events. Consider a churn model that predicts whether a player will leave within the next seven days. If only 5% of players churn in a typical week, a model that predicts “stay” for everyone will be correct 95% of the time. It will also catch zero churners. Accuracy rewards guessing the majority class. It does not show whether the model identifies the players who matter or whether bonuses are being wasted on the wrong targets.
For this reason, models should be evaluated with metrics that focus on the positive class. Common choices include precision, recall, F1, and lift. These metrics measure how well the model separates signal from noise.
The baseline and the performance bar
Before deploying a model, it should be compared against a simple rule. Start with the strongest single feature available. Sweep through different thresholds and measure how well that rule performs. Choose the threshold that optimises the metric aligned with your business objective.
If balanced performance matters most, optimise for F1.
If sending unnecessary bonuses is expensive, optimise for precision.
If missing VIPs or churners causes the greatest damage, optimise for recall.
This rule becomes your baseline.
The machine learning model must outperform this rule on the same metric and on a held-out dataset. If it does not, the rule should remain in place. If it does, the additional complexity of the model is justified.
Precision, recall and the cost of being wrong
Choosing the right metric ultimately comes down to the cost of mistakes.
A false positive in a churn model may trigger a retention bonus for a player who was never going to leave. The result is wasted incentive spend.
A false negative means failing to identify a player who actually churns. In that case the operator loses the player’s future lifetime value.
Managers need to decide which error has the higher cost. The trade-off between precision and recall reflects that decision. One practical way to evaluate this in iGaming is by adding business costs to the model outputs.
This is where profit curves become useful. They track cumulative profit or cost as you move through the ranked list of players generated by the model. The goal is not to maximise accuracy or even F1. The goal is to find the point where saved lifetime value minus the cost of interventions produces the highest profit.
Different models and thresholds can then be compared on the same curve. The best model is the one that pushes that curve upward.
Lift and the lift curve
Lift measures how strongly the model concentrates the positive class within the highest ranked players. For example, suppose 3% of all players belong to a high-value segment. If the top decile identified by the model contains 15% high-value players, the lift equals 5. This means the model is finding high-value players five times more effectively than random selection. The lift curve shows lift across different percentiles of the scored population. It reveals how well the model segments players. Strong models show high lift near the top of the ranking. If lift stays flat or increases only in the middle of the distribution, the model does not produce useful prioritisation.
Comparing the lift curve of the model to the baseline rule shows whether the model truly adds value.
Translating metrics into business language
Technical metrics alone rarely convince senior management. Measures such as AUC or accuracy summarise statistical performance but say little about financial outcomes. Executives do not need a lecture about ROC curves. They need to understand the revenue impact.
Instead of saying that the model has an AUC of 0.78, the message should focus on outcomes. For example: the model allows the team to retain 20% more churn-risk players while reducing unnecessary bonus spend by 10 percent. Another scenario could show that the same budget can target fewer players while generating more high-value customers.
Metrics that translate directly into costs and revenue help organisations understand why the model matters.
Hands-on: The €15,000 Threshold Decision
Let’s walk through a simple example.
You run a churn prediction model on a segment of 1,000 players. The data science team first shows the result as a standard confusion matrix. Technically everything looks clear. The real question is what those numbers mean for the business.
Step 1: The Technical View
The model identifies 100 players who were actually going to churn. These are True Positives. It also correctly stays silent for 870 players who were going to stay. These are True Negatives. However, the model is not perfect. It misses 20 players who churn anyway. These are False Negatives. It also flags 10 players as risky even though they were never going to leave. These are False Positives.
From a technical perspective, this is simply a classification result. From a business perspective, every one of these categories has a financial consequence.
Step 2: The Economic View
For a manager, these outcomes translate directly into P&L items. Let’s apply simple economics to the model output.
Each saved player (True Positive) generates €200 in LTV, but requires a €50 retention bonus.
Net gain per saved player: €150.
Each missed player (False Negative) represents a €200 loss in future LTV.
Each incorrectly flagged player (False Positive) means a €50 bonus spent unnecessarily.
When you convert the confusion matrix into cash flow, the picture becomes much clearer.
The model saves 100 players, generating €15,000 in net profit.
At the same time, €500 is spent on incorrect bonuses.
The model clearly creates value.
Step 3: The Strategic Choice
Now comes the real decision. Should you lower the model threshold to capture the 20 players who were missed? Those players represent €4,000 in potential future value. At first glance it seems worth trying to capture them. But lowering the threshold has consequences. Suppose catching those additional churners requires sending retention bonuses to 100 more players who were never going to leave. Those bonuses would cost €5,000. In that situation, the attempt to improve recall actually reduces profit by €1,000.
This is exactly why profit curves are useful. They show the point where sending another bonus is still cheaper than the lifetime value you are trying to protect. Once that balance flips, increasing model sensitivity starts destroying value instead of creating it.
Summary
Predictive models in iGaming must ultimately be evaluated through financial impact. Accuracy alone provides little guidance. Metrics such as precision, recall, lift, and profit curves provide a clearer picture of whether a model supports the business. The correct metric depends on the problem. Fraud detection, bonus control, and churn prediction each require a different balance between errors. At the end of the day, management evaluates models through the P&L. The right metrics translate statistical performance into economic outcomes.
When the measurement reflects real business value, the mathematics of machine learning becomes a practical tool for decision making.
