Data Engineering

This section contains the Data Engineering stream, an eleven-notebook sequence on everything that happens to data before it becomes a CSV file you load in COMET or prAxIs. The stream runs in four arcs: Notebooks 1 to 4 cover querying data (why databases exist, SQL from a first SELECT through window functions, and cleaning a messy real-world file), Notebooks 5 to 7 cover designing and storing data (schemas and constraints, transactions and indexes, and the split between transactional and analytical databases), Notebooks 8 and 9 cover keeping data flowing (pipelines as DAGs, data quality, and lineage), and Notebooks 10 and 11 cover data for AI (embeddings, vector search, and a capstone that builds a full retrieval-augmented generation system on real consumer complaints). The notebooks are meant to be read in order.

Modules