Semantic Layer for AI: Why Business Context Matters for Enterprise Adoption
Most enterprises now have the AI models. Few have the meaning.
Every executive team weighing an AI rollout has, by now, cleared the technology hurdle. Large language models are capable, cloud providers will happily provision the compute, and there is no shortage of vendors selling copilots for every function. Yet many enterprise AI initiatives still stall before they reach production, and the reason is rarely the model. The problem is that the AI does not know what the business means by “customer,” “active account,” or “quarter.” This is where a semantic layer for AI becomes important.
Without that shared vocabulary, a fluent, confident answer can also be a wrong one.
Why is business meaning essential for reliable enterprise AI?
A language model reasons over words without access to the specific business logic behind them. It has no inherent way to know that “revenue” excludes returns in your general ledger but includes them in your sales dashboard, or that “customer” means a billing account in finance and a household in marketing. Point a capable model at a raw data warehouse and it will still produce an answer, although that answer may be incorrect. The model can also present it with complete confidence. This creates a particular risk for enterprise AI adoption: incorrect answers may go unnoticed and influence decisions across the organization when users trust the output.
What is a semantic layer for AI, and how does it work?
A semantic layer sits between raw, fragmented data and any tool that consumes it, including a dashboard, an analyst, or an AI agent. It translates tables and columns into the business’s own language: approved definitions of key metrics, the relationships between entities like customers, products, and contracts, and the governance rules about who can see what. Instead of guessing which of a dozen tables holds “active users,” an AI agent consults one governed definition and applies it consistently, every time, across every tool that queries it.
Over the past couple of years, the semantic layer has moved from a niche analytics concept to what many enterprise data leaders now consider foundational infrastructure for AI. A semantic layer for AI gives systems access to defined business concepts, allowing them to work with those definitions when querying data.
How do ontologies improve semantic layers and AI reasoning?
A semantic layer solves the vocabulary problem: it gives every metric and entity one governed name and definition. Ontologies go a step further.
A semantic layer defines what “customer” means, while an ontology describes how customers relate to accounts, contracts, products, and regions, including the hierarchy, causality, and business rules connecting those concepts. That structure matters specifically because of how LLMs consume it.
Left alone, a model retrieving context through standard RAG pulls back chunks of text ranked by similarity, with no guarantee the pieces fit together logically. Ontology-grounded retrieval, an approach increasingly referred to as GraphRAG, allows the model to traverse explicit relationships between concepts using connections defined in advance. The ontology disambiguates: it tells the model which of several “policies” or “accounts” a question actually refers to, and which path through the data is the approved one. A 2025 study on ontology-guided knowledge graph construction found that ontology-guided retrieval beat plain vector search on exactly the kind of multi-hop, ambiguous questions that trip up enterprise AI in production (arXiv, 2025). Vendors report the same pattern, with fewer hallucinations among agents that need to reason across many linked tables instead of a single flat file.
In practice, this makes ontologies the reasoning backbone underneath the semantic layer: the semantic layer supplies agreed definitions; the ontology supplies the map of how those definitions connect, so an AI agent can follow a chain of relationships, from a claim to a policy, then to a customer and a region, using those predefined connections.
How are Microsoft and Databricks using semantic layers and ontologies for AI?
The two dominant enterprise data platforms have arrived at the same conclusion from different starting points. Microsoft’s Fabric IQ extends Power BI’s semantic models with ontologies, so AI agents across the Microsoft ecosystem can use a governed business model when querying data from OneLake. Fabric IQ itself went generally available at Microsoft Build 2026, alongside Graph and operations agents, and at FabCon Europe in September 2026 Microsoft made Fabric IQ generally available inside Microsoft 365 Copilot Chat and Cowork and took the Fabric IQ MCP server to general availability. The ontology item, however, is still documented as preview, with general availability promised only in the coming months. An ontology can also be generated directly from an existing Power BI semantic model, allowing teams to reuse business logic already in production when creating an ontology. Since September, Microsoft has also announced the ability to bind existing DAX measures into an ontology (in preview), so established metrics can be reused without redefining them. The positioning is deliberate: semantic models provide the governed business definitions that AI agents query and continue to support dashboard reporting.
Databricks has taken a parallel path through Unity Catalog. Its Metric Views let teams define KPIs like revenue, churn, or margin once, as governed Unity Catalog objects that can then be queried consistently from SQL, BI tools, and AI agents alike. These sit alongside Pages, which define business concepts as an authoritative source, and Domains, which scope that context so each agent only sees the definitions relevant to it. A Glossary of authoritative concepts and taxonomies was announced alongside them, with preview still to follow, and certification flags mark which assets Genie should treat as trusted.
The Genie Ontology, announced at Data + AI Summit in June 2026 and still in public preview, is a self-improving enterprise context layer that automatically derives business context from data assets, pipelines, queries, dashboards, and connected applications.
It captures relationships, business definitions, and usage patterns, drawing on more than 50 connected applications as well as the lakehouse itself. It then ranks them with an algorithm Databricks calls OntoRank, a PageRank-style score of which definition carries the most authority. Databricks argues that this approach can reduce hallucinations and improve answer accuracy. Since late October 2026, the same context has been available to outside agents through the generally available Genie One MCP server. The agent products it feeds, Genie One, Genie Agents, and Genie Code, are already generally available. The underlying ontology remains in public preview.
A similar approach can be found in projects outside the commercial platform roadmaps. OntoBricks, a source-available project in the Databricks Labs GitHub organization, deploys as a Databricks App and turns Unity Catalog tables into a materialized knowledge graph: teams sketch an ontology on a visual canvas or import an industry standard like FIBO, CDISC or HL7 FHIR, map it to their tables, materialize the triples into the Lakehouse, reason over the graph, and expose the result to LLM agents through an auto-generated API and MCP tools. It runs on established standards such as OWL, R2RML, and SPARQL, but executes as Spark SQL, so teams can use those semantics without setting up a separate graph platform or hiring RDF specialists. It is an exploratory Labs project and is not a supported product. Its architecture shows how ontologies can provide an interface between governed enterprise data and the agents querying it.
Microsoft and Databricks both place ontologies within their enterprise AI strategies. Both are betting that enterprises will standardize on platforms that can encode business meaning and provide AI agents with governed access to that structure. For leadership teams, this makes semantics a platform decision that extends beyond the data team’s backlog.
What are the benefits of a semantic layer for AI adoption?
Getting semantics right has three consequences, and each one maps to a real adoption barrier:
- Trust: when every AI answer traces back to one governed definition instead of an ad hoc query, leaders have less reason to second-guess the numbers. This is the single biggest driver of whether people keep using an AI tool after the pilot ends.
- Consistency at scale: without a shared semantic model, every team and every AI agent can develop different definitions of the same metrics. These differences accumulate as more tools and agents are added.
- Fewer hallucinations: grounding a model in structured business definitions and relationships, so it doesn’t have to infer meaning from raw schema, measurably reduces incorrect answers presented with confidence, which can undermine credibility early in a rollout.
A 2026 ROI study conducted by UserEvidence and commissioned by Strategy found that organizations using its universal semantic layer, Mosaic, reported an average net annual impact of $3.4 million and a 551% ROI, with confidence in reported metrics rising from 5 out of 10 to 9 out of 10 (UserEvidence/Strategy, 2026). These data-quality improvements can influence whether leadership decides to expand an AI pilot across the organization or discontinue it after the demonstration.
How can enterprises move from AI pilots to production?
The organizations moving fastest from AI pilots to production aren’t necessarily using the most advanced models. They have invested early in a governed, business-defined semantic layer, and increasingly in an ontology, beneath those models. Platform vendors are already building this conviction into their roadmaps; for most enterprises, the question now is how quickly they can make similar investments in semantics. Before the next AI budget conversation, leadership teams may want to examine whether business data has consistent definitions across the systems using it, alongside their choice of AI model.
Those definitions have a direct impact on whether AI systems can produce reliable results at scale.
How can organizations accelerate semantic layer implementation for AI?
Want to boost your AI solutions, or learn more about expediting semantics inside your organization?
A well-defined semantic layer can help your organization establish consistent business definitions and improve the reliability of AI-driven insights. At Abylon, we can help you explore how to strengthen your existing data foundations and make better use of your AI solutions.
Contact us to discuss your organization’s needs and potential next steps.
Author of the post:
Kristóf Fellegi - Data/Solution Architect at Abylon Consulting. Linkedin Profile
