Data Engineering
Learn to design and build scalable data pipelines and infrastructure.
About This Course
Data engineers build the systems that move data from where it is created to where it can be used, and keep those systems reliable as volumes grow. This data engineering course covers pipeline design, ETL, and cloud data platforms at an advanced level, taught by an instructor who has built these systems in production.
What the course covers
- Pipeline design: batch and streaming patterns, choosing between ETL and ELT, and designing pipelines that can be rerun safely when something fails.
- Data modeling: structuring data for analytics in a warehouse or lakehouse so queries stay fast and results stay trustworthy.
- Processing at scale: distributed processing with Apache Spark, and working with data in Databricks.
- Streaming: moving events through Apache Kafka and handling data as it arrives.
- Transformation: building tested, version-controlled transformations with dbt.
- Cloud data platforms: designing on AWS and Microsoft Azure, and understanding the trade-offs between services.
- Reliability: data quality checks, monitoring, and alerting, so problems are caught before a dashboard shows the wrong number.
By the end you will be able to
- Design a pipeline for a given source, volume, and freshness requirement
- Choose between batch and streaming approaches and justify the choice
- Build transformations that are tested and repeatable
- Put monitoring in place so failures surface early
Who it's for
- Software engineers and developers moving into data engineering
- Data analysts and BI developers who want to own the pipelines behind their reports
- Database and ETL developers bringing their skills to cloud platforms
This is an advanced course, so you should already be comfortable with SQL and at least one programming language. Python is the usual choice.
How it's taught
On demand, so you can work through it at your own pace. Your instructor is Masood, a senior data engineer with 20 years of experience in financial services, telecommunications, e-commerce, and healthcare. His work spans data pipeline design, ETL architecture, and cloud data platforms on AWS and Azure, using tools such as Spark, Kafka, dbt, and Databricks, and he is fluent in Python, Java, C#, and C++.