Azure Data Engineer Associate (DP 203)
COURSE OVERVIEW
Duration: 5 Days
The DP-203T00: Data engineering on Microsoft Azure course is designed to impart the knowledge and skills necessary to design and implement Data engineering solutions on Azure. It covers a comprehensive range of Azure services including Azure Data Lake Storage Gen2, Azure Synapse Analytics, Azure Stream Analytics, and Azure Databricks among others.
Learners will gain practical experience through labs that reinforce the lessons, such as querying data with Serverless SQL pools, performing Data engineering with Azure Synapse Apache Spark Pools, and implementing Real-time analytics with Azure Stream Analytics.
The course also dives into security and compliance with modules on End-to-end security, and it explores Integration with Power BI for reporting and Azure Synapse Analytics for machine learning processes. By the end of this course, participants will have a strong foundation in Data engineering practices, enabling them to build scalable and secure data solutions in the cloud. This course is beneficial for professionals looking to leverage Azure for data processing, analytics, and to gain insights that can drive business value.
It is highly recommended that candidates pursuing this course have a fundamental understanding of data or have done the DP-900 course and AZ-900.
The primary audience for this course is data professionals, data architects, and business intelligence professionals who want to learn about data engineering and building analytical solutions using data platform technologies that exist on Microsoft Azure. The secondary audience for this course includes data analysts and data scientists who work with analytical solutions built on Microsoft Azure.
Job role: Data Engineer Preparation for exam: DP-203
DP-203 - Azure Data Engineering Course Objectives
- Explore compute and storage options for data engineering workloads in Azure
- Run interactive queries using serverless SQL pools
- Perform data Exploration and Transformation in Azure Databricks
- Explore, transform, and load data into the Data Warehouse using Apache Spark Ingest and load Data into the Data Warehouse
- Transform Data with Azure Data Factory or Azure Synapse Pipelines
- Integrate Data from Notebooks with Azure Data Factory or Azure Synapse Pipelines
- Support Hybrid Transactional Analytical Processing (HTAP) with Azure Synapse Link
- Perform end-to-end security with Azure Synapse Analytics
- Perform real-time Stream Processing with Stream Analytics
- Create a Stream Processing Solution with Event Hubs and Azure Databricks
Introduction to data engineering on Azure
- Identify common data engineering tasks
- Describe common data engineering concepts
- Identify Azure services for data engineering
Introduction to Azure Data Lake Storage Gen2
- Describe the key features and benefits of Azure Data Lake Storage Gen2
- Enable Azure Data Lake Storage Gen2 in an Azure Storage account
- Compare Azure Data Lake Storage Gen2 and Azure Blob storage
- Describe where Azure Data Lake Storage Gen2 fits in the stages of analytical processing
- Describe how Azure data Lake Storage Gen2 is used in common analytical workloads
Introduction to Azure Synapse Analytics
- Identify the business problems that Azure Synapse Analytics addresses
- Describe core capabilities of Azure Synapse Analytics
- Determine when to use Azure Synapse Analytics
Build data analytics solutions using Azure Synapse serverless SQL pools
1. Use Azure Synapse serverless SQL pool to query files in a data lake
- Identify capabilities and use cases for serverless SQL pools in Azure Synapse Analytics
- Query CSV, JSON, and Parquet files using a serverless SQL pool
- Create external database objects in a serverless SQL pool
2. Use Azure Synapse serverless SQL pools to transform data in a data lake
- Use a CREATE EXTERNAL TABLE AS SELECT (CETAS) statement to transform data Encapsulate a CETAS statement in a stored procedure
- Include a data transformation stored procedure in a pipeline
3. Create a lake database in Azure Synapse Analytics
- Understand lake database concepts and components
- Describe database templates in Azure Synapse Analytics
- Create a lake database
4. Secure data and manage users in Azure Synapse serverless SQL pools
- Choose an authentication method in Azure Synapse serverless SQL pools Manage users in Azure Synapse serverless SQL pools
- Manage user permissions in Azure Synapse serverless SQL pools
Perform data engineering with Azure Synapse Apache Spark Pools
1. Analyze data with Apache Spark in Azure Synapse Analytics
- Identify core features and capabilities of Apache Spark
- Configure a Spark pool in Azure Synapse Analytics
- Run code to load, analyze, and visualize data in a Spark notebook
2. Transform data with Spark in Azure Synapse Analytics
- Use Apache Spark to modify and save dataframes
- Partition data files for improved performance and scalability
- Transform data with SQL
3. Use Delta Lake in Azure Synapse Analytics
- Describe core features and capabilities of Delta Lake
- Create and use Delta Lake tables in a Synapse Analytics Spark pool
- Create Spark catalog tables for Delta Lake data
- Use Delta Lake tables for streaming data
- Query Delta Lake tables from a Synapse Analytics SQL pool
Transfer and transform data with Azure Synapse Analytics pipelines
1. Build a data pipeline in Azure Synapse Analytics
- Describe core concepts for Azure Synapse Analytics pipelines
- Create a pipeline in Azure Synapse Studio
- Implement a data flow activity in a pipeline
- Initiate and monitor pipeline runs
2. Use Spark Notebooks in an Azure Synapse Pipeline
- Describe notebook and pipeline integration
- Use a Synapse notebook activity in a pipeline
- Use parameters with a notebook activity
Implement a Data Analytics Solution with Azure Synapse Analytics
1. Introduction to Azure Synapse Analytics
- Identify the business problems that Azure Synapse Analytics addresses
- Describe core capabilities of Azure Synapse Analytics
- Determine when to use Azure Synapse Analytics
2. Use Azure Synapse serverless SQL pool to query files in a data lake
- Identify capabilities and use cases for serverless SQL pools in Azure Synapse Analytics
- Query CSV, JSON, and Parquet files using a serverless SQL pool
- Create external database objects in a serverless SQL pool
3. Analyze data with Apache Spark in Azure Synapse Analytics
- Identify core features and capabilities of Apache Spark
- Configure a Spark pool in Azure Synapse Analytics
- Run code to load, analyze, and visualize data in a Spark notebook
4. Use Delta Lake in Azure Synapse Analytics
- Describe core features and capabilities of Delta Lake
- Create and use Delta Lake tables in a Synapse Analytics Spark pool
- Create Spark catalog tables for Delta Lake data
- Use Delta Lake tables for streaming data
- Query Delta Lake tables from a Synapse Analytics SQL pool
5. Analyze data in a relational data warehouse
- Design a schema for a relational data warehouse
- Create fact, dimension, and staging tables
- Use SQL to load data into data warehouse tables
- Use SQL to query relational data warehouse tables
6. Build a data pipeline in Azure Synapse Analytics
- Describe core concepts for Azure Synapse Analytics pipelines
- Create a pipeline in Azure Synapse Studio
- Implement a data flow activity in a pipeline
Initiate and monitor pipeline runs
Work with Data Warehouses using Azure Synapse Analytics
1. Analyze data in a relational data warehouse
- Design a schema for a relational data warehouse
- Create fact, dimension, and staging tables
- Use SQL to load data into data warehouse tables
- Use SQL to query relational data warehouse tables
2. Load data into a relational data warehouse
- Load staging tables in a data warehouse
- Load dimension tables in a data warehouse
- Load time dimensions in a data warehouse
- Load slowly changing dimensions in a data warehouse
- Load fact tables in a data warehouse
- Perform post-load optimizations in a data warehouse
3. Manage and monitor data warehouse activities in Azure Synapse Analytics Scale compute resources in Azure Synapse Analytics
- Pause compute in Azure Synapse Analytics
- Manage workloads in Azure Synapse Analytics
- Use Azure Advisor to review recommendations
- Use Dynamic Management Views to identify and troubleshoot query performance
4. Secure a data warehouse in Azure Synapse Analytics
- Understand network security options for Azure Synapse Analytics
- Configure Conditional Access
- Configure Authentication
- Manage authorization through column and row level security
- Manage sensitive data with Dynamic Data masking
- Implement encryption in Azure Synapse Analytics
Work with Hybrid Transactional and Analytical Processing Solutions using Azure Synapse Analytics
1. Plan hybrid transactional and analytical processing using Azure Synapse Analytics
- Describe Hybrid Transactional / Analytical Processing patterns
- Identify Azure Synapse Link services for HTAP
2. Implement Azure Synapse Link with Azure Cosmos DB
- Configure an Azure Cosmos DB Account to use Azure Synapse Link
- Create an analytical store enabled container
- Create a linked service for Azure Cosmos DB
- Analyze linked data using Spark
- Analyze linked data using Synapse SQL
3. Implement Azure Synapse Link for SQL
- Understand key concepts and capabilities of Azure Synapse Link for SQL Configure Azure Synapse Link for Azure SQL Database
- Configure Azure Synapse Link for Microsoft SQL Server
Implement a Data Streaming Solution with Azure Stream Analytics
1. Get started with Azure Stream Analytics
- Understand data streams
- Understand event processing
- Understand window functions
- Get started with Azure Stream Analytics
2. Ingest streaming data using Azure Stream Analytics and Azure Synapse Analytics
- Describe common stream ingestion scenarios for Azure Synapse Analytics Configure inputs and outputs for an Azure Stream Analytics job
- Define a query to ingest real-time data into Azure Synapse Analytics
- Run a job to ingest real-time data, and consume that data in Azure Synapse Analytics
3. Visualize real-time data with Azure Stream Analytics and Power BI
- Configure a Stream Analytics output for Power BI
- Use a Stream Analytics query to write data to Power BI
- Create a real-time data visualization in Power BI
Govern data across an enterprise
1. Introduction to Microsoft Purview
- Evaluate whether Microsoft Purview is appropriate for your data discovery and governance needs
- Describe how the features of Microsoft Purview work to provide data discovery and governance
2. Discover trusted data using Microsoft Purview
- Browse, search, and manage data catalog assets
- Use data catalog assets with Power BI
- Use Microsoft Purview in Azure Synapse Studio
3. Catalog data artifacts by using Microsoft Purview
- Describe asset classification in Microsoft Purview
4. Manage Power BI assets by using Microsoft Purview
- Register and scan a Power BI tenant
- Use the search and browse functions to find data assets
- Describe the schema details and data lineage tracing of Power BI data assets
5. Integrate Microsoft Purview and Azure Synapse Analytics
- Catalog Azure Synapse Analytics database assets in Microsoft Purview
- Configure Microsoft Purview integration in Azure Synapse Analytics
- Search the Microsoft Purview catalog from Synapse Studio
- Track data lineage in Azure Synapse Analytics pipelines activities
Request A Call Back Now!
Just fill in the form and we will get back to you soonest!