Skip to content

Commit

Permalink
add ep180 (#416)
Browse files Browse the repository at this point in the history
  • Loading branch information
iMeriem authored May 22, 2024
1 parent f19986b commit 4458207
Showing 1 changed file with 73 additions and 0 deletions.
73 changes: 73 additions & 0 deletions blablas/ep180/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
date: 2024-05-12
time: 20h:00min
duration: "1:45:17"
title: "Data engineer 101"
tags: ["dev"]
category: "dev"
isNext: false
youtube: https://www.youtube.com/live/mxV9Bx1ZsZg?si=5QnDE6RCcNOBuW1T
published: true
featured: false
---

Data engineering is a critical field in data science that involves preparing the "big data" infrastructure to be analyzed by data scientists. In this episode we are discussing the differences and how important each is with our guests.

## Guests

- [Mahmoud Fettal](https://twitter.com/mahmoudfettal)

- [Salim Jannah](https://www.linkedin.com/in/salim-janah)

- [Omaima Khalil](https://twitter.com/BadQuinn3)


## Notes

0:00:00 - Introduction and welcoming

0:02:50 - What is data engineering?

0:08: 43 - What are the key skills required for a data engineer?

0:16:40 - How does data engineering differ from data science?

0:20:00 - Data analyst vs data engineer vs data scientist

0:22:41 - What are the common tools used in data engineering?

0:28:57 - What are data pipelines?

0:34:54 - What challenges do data engineers face?

0:42:12 - Q&A

0:53:42 - How important is real -time data processing in data engineering?

1:02:35 - What is a data lake, and how does it differ from a data warehouse?

1:12:52 - How do data engineers use machine learning?

1:18:01 - Types of projects really involved with Data engineering

1:32:17 - What future trends should data engineers be aware of?

1:41:00 - Geeksblabla Picks

2:18:30 - Conclusion and Goodbye


## Links

- [Apache Airflow vs Mage.ai](https://www.cidrdb.org/cidr2021/papers/cidr2021_paper17.pdf)

- [Lakehouse paper](https://medium.com/odicis-data-engineering/apache-airflow-vs-mage-ai-in-data-engineering-745c040a05e8)

- [Open Source Agent for Data Analysis](https://pandas-ai.com/)

- [Simplifying Data Engineering and Analytics with Delta](https://www.packtpub.com/product/simplifying-data-engineering-and-analytics-with-delta/9781801814867)


## Prepared and Presented by

- [Meriem Zaid](https://twitter.com/_iMeriem)

0 comments on commit 4458207

Please sign in to comment.