The Future of Microsoft's Data Analytics: Key Takeaways from the Microsoft CAT Team Fabric Presentation

A couple of weeks back, I had the privilege of attending a Microsoft Fabric presentation by the Microsoft CAT Team in Auckland and gained some very insightful knowledge. They covered most of the Fabric features announced at the recent Microsoft Fabric Community Conference 2024 in Las Vegas.

This post is to share some of the useful key features. If you would rather jump to the key takeaway points, scroll down to the bottom of this article.

First, for those who do not know: What is Microsoft Fabric?

Here is the marketing catchphrase: The Unified Data Platform for the Era of AI (Artificial Intelligence).

The official definition: Microsoft Fabric is an all-in-one analytics solution for enterprises that covers everything from data movement to data science, real-time analytics, and business intelligence. It offers a comprehensive suite of services, including data lake, data engineering, and data integration—all in one place. Put simply, Microsoft Fabric aims to be a one-stop shop for data & analytics—from the data ingestion stage, through the data management stages, all the way to the data insight and data presentation stages. It brings together new and existing components such as Azure Data Factory, Data Warehouse, and Power BI into a single integrated SaaS environment.

If you want to know about the basics and components of Fabric (e.g. What is OneLake? What is KQL database?), there is a wealth of information on Microsoft website: https://learn.microsoft.com/en-us/fabric/get-started/ 

For this article, let us jump back to those new features announced.

New or Upcoming Features

Disclaimer: Some of the features I mention here are in Generally Available, Public Preview, or even still under development at the time of writing. Usual caveats apply for non-released features. Refer to these links for further detail:

-           https://learn.microsoft.com/en-nz/fabric/release-plan/overview

-           https://learn.microsoft.com/en-us/fabric/get-started/whats-new

Shortcuts

Shortcuts are objects in Fabric's OneLake that point to other storage locations. These could be other OneLake locations, Azure DataLake Storage (ADLS) Gen2, AWS S3, Google Cloud Storage, or Dataverse. In the future, these could also include on-premises data sources.

This feature is useful for data acquisition into Fabric, feeding data into data pipelines without having to first ingest the data.

Mirroring

Mirroring is a way to replicate your existing data estate directly easily and continuously into Fabric's OneLake. It behaves similarly to the Database Mirroring feature in SQL Server; under the hood it uses database replication technology that relies on the source database’s Change Data Capture (CDC) technology.

Initial support is for mirroring from Azure Cosmos DB, Azure SQL, and Snowflake.  

This is one of the ways to do data ingestion into Fabric.

COPY INTO

The COPY INTO feature has been available for a while now in Fabric, but it is worth noting that it behaves very similarly to one in other cloud data platforms. It is used for loading data into tables from external file storage using a T-SQL statement.

Only Parquet and CSV file formats are supported, and only ADLS Gen 2 storage account type is supported; for all other storage accounts, you will have to use Shortcuts.

Zero-copy Clone and Time Travel

Zero-copy Clone and Time Travel have also been available for a while now in Fabric, and for those familiar with other cloud data platforms, these features have the same names and behaviours. A zero-copy clone is a replica of a table by copying the metadata, while still referencing the same data files in OneLake. Time Travel, meanwhile, is a way of going back to a past state of a table at a certain point in time (up to 7 days currently).

External Data Sharing

External Data Sharing is a feature for making data in Fabric available to another Fabric tenant. In-place (not copy) and read-only data that can be consumed by any OneLake compatible Fabric workload in the external tenant. The sharing creates a OneLake shortcut in the external tenant that points back to the original data in the sharer's tenant.

AI-Powered Productivity features include:

-           Copilot for Data Warehouse and Power BI: explain or comment SQL or DAX queries, write SQL or DAX queries based on instructions or comments.

-           Copilot for PowerBI Desktop: create reports and diagrams. generate a summary for the report page that you just created

-           Coming up: Custom Generative AI chat to ask questions and get answers based on your data in Fabric.

Direct Lake in Power BI

Direct Lake is a fast-path to load Parque-formatted datafiles from the lake straight into the Power BI engine, ready for analysis.

There is a lot of excitement for, and potential misperception of, this feature. To be clear, Direct Lake is NOT a replacement for Import mode nor for DirectQuery mode. Rather than reiterating the detail, I will just direct you to this awesome blog post by the famous Marco Russo: https://www.sqlbi.com/blog/marco/2024/04/06/direct-lake-vs-import-mode-in-power-bi/

Other Features

There are also more features, such as:

-           Online Spark debugging within Fabric

-           AI based auto optimisation of Spark jobs

-           Rich Power BI modelling on the web (without using Power BI Desktop)

-           Create semantic models on any table types in Fabric, and run SQL queries over the models

-           GitHub integration (in addition to Azure DevOps integration)

-           Network Security features, Private Link

-           Governance and Access Control

-           Object-level Lineage – I’d like to see column/field-level lineage as well in the future.

Key Takeaway Points

Fabric as a Modern Data Platform option

It is great to see Microsoft investing and innovating at a fast pace on the Fabric platform and that is a remarkable thing for us technical users.

Some of the features added are overdue, but others are innovative. Overall, they all make Microsoft Fabric a highly viable modern data platform option especially for customers who are fully invested in the Microsoft and Azure ecosystem and have preference to keep all their data and their workloads within the same platform.

Synapse Analytics

This is a big question in a lot of people’s minds. Microsoft has made it clear that there is no plan currently to deprecate the Synapse Analytics platform, but it is also clear that “the future is Fabric.”, Microsoft’s development and innovation is focussed on Fabric.

So, if you currently use Synapse Analytics but you are starting a new project on the Microsoft platform, I highly recommend for you to seriously consider Fabric.

Microsoft is also looking into migration processes or tools to help customers migrate existing Synapse workloads to Fabric.

----

If you would like to know more about Fabric, consider it as an option for your modern data platform, or would like to consider your migration options, please reach out to us and we will be happy to help you.

For more info on Microsoft Fabric Community Conference 2025: https://fabricconf.com/

‍

Written by Ferry Nugraha, Data Domain's Head of Technology & Capability