Dataflow Gen2 (Microsoft Fabric)

What is Dataflow Gen2?

Dataflow Gen2 is the self-service data-preparation item in Microsoft Fabric. It lets you connect to data, clean it, combine it, and write the result to a chosen destination without writing code.

You work in Power Query in the browser: the same family of editor people know from Power BI and Excel. You connect to a source, build transformations step by step, and choose where the output should land. After that, the dataflow repeats those steps on refresh, manually or on a schedule.

Microsoft positions Dataflow Gen2 inside the Fabric Data Factory workload. It is the low-code side of Fabric data engineering: analysts can shape tables with clicks, while engineers may use pipelines, notebooks, SQL, or Spark for heavier work.

What can it write to?

The big difference from older Power BI dataflows is that Dataflow Gen2 is not trapped inside Power BI's internal dataflow storage. It can write output to several destinations, including Fabric lakehouses, Fabric warehouses, KQL databases, Fabric SQL databases, Azure SQL, Azure Data Explorer, Azure Data Lake Storage Gen2, SharePoint, and Snowflake, depending on configuration and support.

That matters because the dataflow can become part of a broader Fabric architecture. A team can use a dataflow to clean source data and store the result as tables that Power BI, notebooks, SQL endpoints, or other workloads can reuse.

Dataflow Gen2 versus Dataflow Gen1

Dataflow Gen1 is the original Power BI dataflow experience. It is useful for reusable Power Query logic and shared preparation for Power BI reports, but the output mostly stays inside the Power BI dataflow world.

Dataflow Gen2 opens that model up. The main practical differences are:

• Destinations. Gen2 can write to Fabric and external destinations instead of only being consumed through the Power BI dataflow connector.

• Fabric integration. A Gen2 dataflow can be used as an activity in a Fabric pipeline, so it can run as one step in a larger orchestrated chain.

• Monitoring. Refresh history and run details appear in Fabric monitoring experiences.

• Draft and publish flow. Gen2 saves work as you build and publishes validated changes.

• Lifecycle support. Dataflow Gen2 supports modern Fabric lifecycle features such as deployment pipelines and Git integration where available.

One trade-off: DirectQuery against a Power BI dataflow is a Gen1-style pattern. With Gen2, the usual design is to write to a lakehouse or warehouse and build the semantic model on that destination.

Dataflow, pipeline, or notebook?

Fabric has overlapping tools, and the right one depends on where the complexity sits.

Dataflow Gen2
Choose a dataflow when the work is mainly data preparation and the team knows Power Query: filtering, merging, splitting columns, deduplicating, type conversion, lookup logic, and business categories.

Data pipeline
Choose a pipeline when orchestration or large-scale movement is the main job: copy data, call other Fabric items, run a notebook, execute a stored procedure, branch on conditions, loop over files, and monitor the chain.

Notebook
Choose a notebook when the transformation needs code, Spark, Python libraries, complex logic, experimentation, or reusable functions that are awkward in clicks.

In practice, teams combine them. A pipeline copies raw files into a lakehouse, triggers a Dataflow Gen2 to clean and shape the data, then refreshes a semantic model or starts a notebook for the next step.

When does Dataflow Gen2 fit an SME?

For a smaller business without a full data engineering team, Dataflow Gen2 can be the shortest route from messy source data to usable tables.

• Accounting data for reporting. Load revenue and cost data, map account numbers to readable categories, and write the result to a lakehouse table.

• CRM cleanup. Standardise company names, split address fields, remove duplicates, and prepare clean customer tables for reporting.

• Excel consolidation. Combine monthly files from SharePoint into one repeatable table instead of copy-pasting every month.

• Medallion layers. Use dataflows to move raw data into cleaner silver tables when the logic fits Power Query.

The best use case is repeatable transformation work that a business analyst understands and that does not need heavy custom code.

What to watch out for

Capacity use
Dataflows consume Fabric capacity while they refresh. Watch runtime, staging behaviour, and concurrency before putting many heavy dataflows on a small capacity.

Query folding
Power Query is fastest when steps can be pushed back to the source. A step that breaks folding can pull too much data into the engine and make refreshes slow.

Large copy jobs
If the task is mostly moving huge volumes, a pipeline copy activity may be a better fit than a dataflow.

Generated destination tables
Be careful when a dataflow controls the destination table. If the table is recreated, downstream relationships, measures, or manual changes may disappear. Use a stable destination design when others depend on the output.

Know when to move to code
A dataflow with dozens of complex joins and custom functions can become hard to debug. When the logic becomes software, treat it like software: version it, test it, and consider a notebook, SQL model, or pipeline pattern.