“Can we have only the AI part?”
If you work around data and AI long enough, you start hearing this sentence a lot. It usually comes from a good place. Teams see a demo of an AI agent, like what it does, and want to plug it into their existing stack without “rebuilding everything underneath”.
On the surface it sounds efficient. Why spend months on warehouses, models and semantic layers if an AI layer can just “connect to everything” and give you answers?
The problem is simple and uncomfortable.
AI that is detached from a unified, governed data foundation does not become a shortcut. It becomes a sophisticated way of guessing on top of fragmented information. It can look impressive in a demo. In production it quietly introduces risk into every decision.
Below is a deeper look at why that happens, and what current research actually says about it.
What “AI only” really means in practice
When someone asks for “only the AI part”, they usually imagine something like this:
- an agent that can query all existing systems
- no need to centralise data
- no need to standardise KPIs first
- natural language questions, instant answers
Under the hood, this means the model has to:
- discover relevant sources on the fly
- resolve conflicting values between systems
- infer how to join entities across tables
- guess which version of a metric is the “right one”
All of that without a single place where truth is defined.
Architecturally, you are asking a generative model to solve problems that normally belong to data engineering, governance and BI. It is not designed for that job.
Fragmented data is the first failure point
Most organisations do not have one clean data backbone. They have a zoo of systems:
- CRM and marketing automation
- transactional ledgers and payment providers
- sportsbook or casino platforms
- bonus and promotion engines
- KYC, risk and RG systems
- spreadsheets, CSV exports, ad hoc BI data marts
Each of these uses its own field names, its own timing, and its own assumptions. You often see:
- the same player or transaction represented differently in two systems
- different time zones or aggregation windows
- subtle differences in how amounts, fees and bonuses are booked
- missing or delayed events in one of the feeds
When an AI layer connects to all of this without a warehouse in the middle, it has to unify reality in real time. There is no guarantee it will choose the same path twice.
This is not just anecdotal. The 2025 AI Data Readiness Report found that 69 percent of companies say poor data limits their ability to make informed decisions, and 45 percent report that unstructured, fragmented data is the biggest barrier to AI success.
Other surveys see the same pattern from a different angle. Informatica’s 2024 CDO Insights report highlights data quality as the primary hurdle to generative AI adoption in enterprises. A 2025 analysis of technology leaders’ stats notes that 64 percent of organisations cite data quality as their top data integrity challenge, with poor quality data undermining AI initiatives and analytics accuracy and contributing to an estimated 25 percent revenue loss through inefficiencies and bad decisions.
In short: the majority of organisations openly admit that their data is not ready for serious AI. Fragmentation is not an edge case. It is the norm.
Metrics and definitions are a symptom, not the whole disease
It is tempting to focus the conversation on KPI definitions only.
Anyone who has worked with operators has seen it. Three teams, three versions of NGR. Retention defined one way in CRM, another way in Finance. Margin including or excluding different cost components depending on who built the report.
This matters a lot. If the AI agent is allowed to read dashboards and documentation and “figure out” what NGR is, it might:
- exclude bonus cost because one document does
- include PSP fees because another example does
- treat chargebacks differently depending on which sample it sees
From the outside you still see one label. Under the hood the model is sampling from a family of similar but incompatible formulas.
But definitions are only one layer of the problem. Even a perfectly defined NGR does not help if the underlying deposits, withdrawals, bets and bonuses are inconsistent across sources.
Without unified, reconciled data, metric definitions become a clean interface on top of a messy implementation. The AI can quote the definition correctly and still calculate it against the wrong inputs.
Why language models guess instead of saying “I don’t know”
The second, and often ignored, failure point is in the model itself.
Large language models are not built to say “I do not know”. They are trained to continue text in a way that looks statistically likely. When they face ambiguity or missing information, they do not halt. They hallucinate.
Recent work from OpenAI and academic collaborators spells this out very clearly. In the paper “Why Language Models Hallucinate”, Kalai, Nachum, Vempala and Zhang argue that hallucinations are not quirks of specific architectures, but a direct consequence of the way models are trained and evaluated.
Two points from that work are particularly important for anyone deploying AI on business data:
- Pretrained language models will hallucinate even if training data is perfectly correct. Hallucinations arise as statistical errors in how the model learns to classify statements as likely or unlikely.
- Evaluations and benchmarks often reward answering over abstaining. Models are effectively optimised to be good test takers. Guessing when uncertain improves benchmark scores more than saying “I do not know”.
A large survey called “Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models” reaches similar conclusions at a higher level. It documents how LLMs frequently generate content that diverges from the input, contradicts earlier context, or misaligns with established world knowledge, and classifies hallucinations as a central challenge for real world use.
Popular coverage of this research made it into mainstream science outlets as well. The summary is blunt: today’s systems are structurally inclined to “fake” an answer rather than admit uncertainty, because their training and scoring pipelines treat guessing as success.
So when your AI layer sits on top of conflicting or incomplete data and you ask:
“What was yesterday’s NGR?”
“Which players show early VIP behaviour?”
“Did we see any suspicious patterns in casino yesterday?”
the model is under pressure to produce something plausible, not to perform an audit of whether it has enough reliable information.
When fragmentation meets hallucination
Once you put these two pieces together, the risk becomes obvious.
On the input side:
- data is fragmented, duplicated and partially inconsistent
- systems disagree on exact values and timing
- metrics may not be defined the same way in every place
On the model side:
- it is rewarded for answering, not for staying silent
- it does not know which source is “authoritative”
- it cannot reliably resolve contradictions on its own
The agent will still respond. The chart will still render. The anomaly explanation will still be written in clean English.
Under the surface, several things can go wrong:
- joining the wrong player or ticket across systems
- double counting or missing transactions because of mismatched keys
- using stale data from a slower source instead of fresh data from another
- silently switching definitions between queries depending on context
- inferring trends from partial history because one dataset lags ingestion
In high signal domains, small numeric errors matter. A few percentage points off on churn, NGR or margin may be the difference between “campaign is working” and “campaign is burning money”. A missed pattern in risk metrics may be the difference between catching abuse early and letting it grow.
This is why “AI only” is not neutral. It amplifies whatever is wrong underneath and packages it in a more convincing form.
What the current data and AI surveys are actually telling you
If you look across recent research and industry surveys, most of them are converging on the same message.
- Huble’s 2025 AI Data Readiness Report: 69 percent of companies say poor data limits their ability to make informed decisions and 45 percent cite unstructured, fragmented data as the biggest barrier to AI success. Only 8.6 percent describe themselves as truly AI ready.
- Informatica’s 2024 CDO Insights: data quality is named as the primary barrier to corporate adoption of generative AI, ahead of skills or tooling.
- A 2025 summary of data integrity trends reports that 64 percent of organisations see data quality as their top data challenge and estimates that companies lose around 25 percent of revenue to poor data driven decisions and inefficiencies.
- A 2025 Protiviti study on trustworthy AI finds that among organisations with high AI maturity, 69 percent say they are very confident in their data capabilities, implying that solid data foundations are strongly correlated with AI success.
- Public sector reviews in the UK and elsewhere repeatedly highlight legacy systems and poor quality data as structural obstacles to real AI productivity gains.
This is not a gap that an extra model or a new prompt template will close. It is structural.
How a proper foundation changes the AI behaviour
The good news is that the same research gives a roadmap for what works.
AI becomes significantly more reliable when three things are in place:
A unified data layer
Operational data is collected from all sources, cleaned, deduplicated, reconciled and stored in a warehouse or equivalent architecture. Conflicts are resolved once, not ad hoc on every query.
A semantic layer
Metrics and attributes are defined centrally with clear formulas, filters and ownership. This gives both humans and AI a precise language for questions like “Active Player”, “Churned”, “Net Gaming Revenue” or “High risk”. The AI agent is not allowed to invent formulas. It must call the definitions that exist.
Governance and access control
Datasets, metrics and attributes are versioned, documented and permissions aware. The AI can only query what the user is allowed to see. Changes to definitions are tracked. There is a feedback loop when numbers do not look right.
McKinsey’s recent work on agentic AI makes a similar point from a strategy angle: the most valuable AI agents are custom built around high impact processes and deeply aligned with a company’s specific data flows and business logic, not dropped in as a generic horizontal feature.
In that type of environment, the AI agent still has limitations, but its room for dangerous improvisation is greatly reduced. It is no longer responsible for resolving data conflicts, inventing KPI formulas or reconciling inconsistent definitions. It becomes a consumer of truth, not its creator.
What this means for anyone considering “AI only”
If a vendor offers you an “AI layer” without talking seriously about:
- how your data is ingested and reconciled
- where the single source of truth lives
- how KPIs are defined and governed
- how datasets are documented and versioned
- how access control and lineage are handled
then they are skipping the hardest part of the problem.
The short term result may still look good. Early dashboards and conversations will be built on hand picked, clean subsets. Over time the cracks appear, as harder questions touch the messy parts of the stack.
The long term result is more insidious. People start to trust a system that is structurally encouraged to guess on top of fragmented inputs. The errors are not obvious. They are embedded in day to day decisions.
The conclusion is straightforward.
If you do not define and consolidate your truth, the model will invent one for you. It will do it with confidence. It will sometimes look brilliant. And it will be very hard to spot when it is wrong.
Reliable AI is never just the model or the chat interface.
It is the last layer of a well structured data system.
Everything else is guesswork wrapped in a nice UI.
