Big Data Analytics with Python¶
A Masters course at the African Institute for Mathematical Sciences (AIMS) Rwanda. We start from the question "why does my laptop crash on this file?" and finish with distributed machine learning, streaming pipelines and satellite analytics on real African data.
How the course is built¶
Every chapter follows the same rhythm, so you always know where you are:
- Concepts — the theory and the mathematics, explained with pictures and small examples.
- Code — runnable Python you can paste into Jupyter or Google Colab.
- Lab — a guided exercise on a real dataset (30–90 minutes).
- Check yourself — review questions, some with hidden answers.
The learning path¶
Foundations Chapters 1–3: what makes data "big", making Python fast, and storing data well
Scaling out Chapters 4–7: MapReduce, Polars, DuckDB, Dask and Apache Spark
Analytics at scale Chapters 8–11: machine learning, streaming, geospatial, text and graphs
Start here¶
- Read the syllabus for the schedule, prerequisites and assessment.
- Follow the environment setup before the first lab — it takes about 30 minutes.
- Browse the catalogue of big data sources, with special attention to African and Rwandan datasets you can use for assignments and the capstone.
No powerful laptop? No problem.
Every lab can run on Google Colab (free) or the Databricks Free Edition. Chapters note where a dataset should be subsampled for a modest machine.
Prerequisites in one sentence¶
You are comfortable with Python, pandas and basic statistics or machine learning. Everything else — distributed systems, Spark, columnar storage, streaming — is taught from first principles.