AI/BI concepts

Databricks AI/BI is a new type of business intelligence product designed to provide a deep understanding of your data's semantics, enabling self-service data analysis for everyone in your organization. AI/BI is built on a compound AI system that draws insights from the full lifecycle of your data across the Azure Databricks platform, including ETL pipelines, lineage, and other queries.

How does AI/BI work?

Compound AI systems combine multiple AI technologies or models to solve complex problems. Instead of relying on a single AI model or algorithm, a compound AI system integrates multiple interacting components to enhance performance and accuracy.

For example, in a system analyzing customer feedback, natural language processing can be used to understand text sentiment, while machine learning models can predict customer behavior based on this sentiment. By leveraging these complementary AI technologies, compound AI systems can deliver more accurate and insightful results than the individual AI models alone.

Traditional BI tools have relied on reports and dashboards, often requiring extensive involvement from data professionals to create new visualizations. While AI assistants have been integrated into BI tools to address this issue, they frequently struggle with real-world data complexities, providing impressive demos but failing in practice.

The semantic model for an organization often relies on the knowledge held by those who work with the data. Databricks AI/BI captures this understanding from interactions across Databricks. It augments the existing context in the Data Intelligence Platform, leveraging this knowledge to provide practical, real-world answers. It powers two complementary product experiences:

• AI/BI dashboards: an AI-powered, low-code dashboarding solution that includes all the conventional BI capabilities for answering a fixed set of business questions.
• Genie spaces: conversational workspaces powered by Genie where users can ask questions about their data in natural language. Genie spaces can answer a much broader set of business questions than dashboards, and improve over time through human feedback and data team configuration.

AI/BI dashboards

Dashboards remain an effective method for operationalizing predefined analytics for regular use. AI/BI dashboards streamline this process with an AI-powered, low-code authoring experience that simplifies data and visualization configuration. These dashboards include standard BI capabilities such as visualizations, cross-filtering, and periodic PDF snapshots via email. They do not require semantic models, data extracts, or additional management services. See Dashboards.

Genie spaces

Genie spaces are conversational workspaces powered by Genie where users can ask questions about their data in natural language. Unlike traditional fixed-chart tools, Genie responds to user queries with adaptable visualizations and suggestions, seeks clarification when needed, and improves over time through human feedback.

Data teams can configure Genie spaces using tools like instructions, trusted assets, confidence voting, and quality monitoring to ensure reliable, well-governed responses. Analysts can integrate trusted logic from sources like Unity Catalog into a Genie space, allowing it to answer questions with validated logic and transparently communicate the trusted status of its answers.

See What is a Genie space to learn more about how Genie spaces work.

Genie in AI/BI

Genie is also integrated directly into AI/BI dashboards. Published dashboards include an Ask Genie button that lets viewers explore data using natural language, without leaving the dashboard. When a user clicks Ask Genie, a companion Genie space opens alongside the dashboard, allowing them to ask follow-up questions or dig deeper into the data beyond what the predefined visualizations show.

When you publish a dashboard, Databricks automatically generates a companion Genie space based on your dashboard's datasets and visualizations. See Genie spaces with dashboards.

Platform integration

Databricks AI/BI is built on top and tightly integrated with the Databricks Data Intelligence Platform. Key features of AI/BI include:

• Unified governance and lineage: AI/BI is seamlessly integrated into Unity Catalog, aligning with its governance framework and adhering to any global policies set by administrators. Through Unity Catalog's lineage visualization, data producers or administrators can track the usage of their data assets within AI/BI. This traceability back to the dataset's ingestion instills confidence in the analysis results.
• Seamless sharing, without new user licenses: AI/BI is built into the Databricks IAM platform, which integrates directly with many popular IDPs so that you can share your analysis with anyone in your organization. Databricks AI/BI does not have seat-based restrictions, so you can add anyone from your organization without the expense of adding new licenses.
• Industry-leading price/performance: AI/BI is tightly integrated with SQL warehouses and the Photon engine, which contain unique optimizations to deliver high-performance interactions. You can expect optimal performance across a wide range of data volumes, from megabytes to petabytes.
• No data extraction required: AI/BI integrates with your existing data ecosystem, eliminating the need to extract datasets to a separate BI engine. This integration improves data freshness and simplifies data governance, leading to a more streamlined data analysis process.