Skip to content

Further reading

Books

  • Kleppmann, M. Designing Data-Intensive Applications. O'Reilly. — The best conceptual foundation for distributed data systems.
  • Chambers, B. & Zaharia, M. Spark: The Definitive Guide. O'Reilly.
  • Damji, J. et al. Learning Spark (2nd ed.). O'Reilly. — Free PDF available from Databricks.
  • Leskovec, J., Rajaraman, A. & Ullman, J. Mining of Massive Datasets. Cambridge University Press. Free at mmds.org, with lecture slides and videos.
  • Reis, J. & Housley, M. Fundamentals of Data Engineering. O'Reilly.
  • Akidau, T., Chernyak, S. & Lax, R. Streaming Systems. O'Reilly.
  • Rey, S., Arribas-Bel, D. & Wolf, L. Geographic Data Science with Python. Free online.
  • Dwork, C. & Roth, A. The Algorithmic Foundations of Differential Privacy. Free online.
  • Barocas, S., Hardt, M. & Narayanan, A. Fairness and Machine Learning. Free at fairmlbook.org.

Foundational papers

  • Ghemawat, Gobioff & Leung (2003). The Google File System. SOSP.
  • Dean & Ghemawat (2004). MapReduce: Simplified Data Processing on Large Clusters. OSDI.
  • Zaharia et al. (2012). Resilient Distributed Datasets. NSDI.
  • Armbrust et al. (2015). Spark SQL: Relational Data Processing in Spark. SIGMOD.
  • Zaharia et al. (2016). Apache Spark: A Unified Engine for Big Data Processing. Communications of the ACM.
  • Akidau et al. (2015). The Dataflow Model. VLDB.
  • McSherry, Isard & Murray (2015). Scalability! But at what COST? HotOS.
  • Armbrust et al. (2021). Lakehouse. CIDR.
  • Raasveldt & Mühleisen (2019). DuckDB: an Embeddable Analytical Database. SIGMOD.
  • Flajolet et al. (2007). HyperLogLog. AofA.

Big data for development and Africa

  • Blumenstock, Cadamuro & On (2015). Predicting poverty and wealth from mobile phone metadata. Science.
  • Jean et al. (2016). Combining satellite imagery and machine learning to predict poverty. Science.
  • Yeh et al. (2020). Using publicly available satellite imagery and deep learning to understand economic well-being in Africa. Nature Communications.
  • Nekoto et al. (2020). Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages. Findings of EMNLP (Masakhane).
  • Taylor, L. (2017). What is data justice? The case for connecting digital rights and freedoms globally. Big Data & Society.

Online documentation and courses