Where LLMs actually help in a data platform (and where they don't)

Since building an AI model and running LLM-based analytics at a start-up, I get asked at almost every client whether 'we should be using AI' somewhere in the data platform. Most of the time, the honest answer is no — a well-modelled dimensional layer and a good BI tool solve the actual business question faster and more reliably than a language model would.

Where LLMs genuinely earn their place is unstructured text that would otherwise never make it into the warehouse at all: customer complaints, support tickets, free-text survey responses, contract clauses. Structuring and classifying that kind of data by hand doesn't scale, and traditional NLP techniques need feature engineering that a modern LLM can often shortcut. That's a real, defensible use case — not because it's fashionable, but because there's no other practical way to get that data into a queryable structure.

Where I'm more cautious is using an LLM as a substitute for the modelling work itself — asking a model to 'just answer questions over the warehouse' without a governed semantic layer underneath it. It papers over an unmodelled platform rather than fixing it, and the answers are only as trustworthy as the mess of tables underneath.

The platforms that get the most value from AI are the ones that already have their structured data in good shape, and use the LLM specifically to widen the platform's scope to unstructured sources it couldn't previously reach — not as a shortcut around doing the modelling in the first place.