I will develop etl pipeline using azure data factory and databricks

NOTE: Please reach out to me before placing any order using the CONTACT ME button on the upper right corner to discuss the requirements. The price mentioned on the gig is an hourly rate.

Professional Azure Data Engineer with 3+ years of experience and hands-on developing highly complex ETL pipelines.

• Experienced in implementing scalable end-to-end optimized workflows on Azure Data Factory / Databricks to extract, process, and store data from diverse sources, which resulted in an average 40% reduction in data processing time for my clients.
• Experienced in migration of data from legacy systems (On-premise) and APIs to Azure Data Lake, performing complex transformations on data using DataBricks (Python and PySpark), moving the processed data to SQL Server / PostgreSQL / Data or Delta Lake, and writing SQL Stored Procedures / Functions that can transform the data and perform some calculations on it based on some business logic.
• I am also skilled in developing and maintaining DeltaLake architectures using Databricks and leveraging DataLakes to store the underlying data and enforce security standards.
• Scheduling workloads using Azure Data Factory triggers or Databricks workflows.