Integrating Data with Azure Data Factory


Azure Data Factory (ADF) is a data integration service that enables customers to move, process and store large quantities of data with ease.

This demo will show how to setup a complete ADF pipeline. In this pipeline, we will:

  1. Load data from a SQL Server table into the raw zone of a data lake
  2. Conform the data in the data lake using an Azure Data Lake Analytics job
  3. Load the conformed data into a Azure SQL Data Warehouse for use with BI tools
Along the way, we'll touch on subjects such as writing U-SQL queries, automating pipelines with triggers, and some of the pros and cons of serverless technologies.

Azure Data Factory is a great solution when: 

  • I need to continuously load data from on-premise into the cloud, or from one place in the cloud to another
  • I need the scalability and flexibility to accommodate any amount of data stored in any number of storage technologies
  • I want to minimize the amount of custom code I develop so I can focus on the business logic