Description
Welcome to the
Building Big Data Pipelines with PySpark & MongoDB & Bokeh
course. In
this course we will be building an intelligent data pipeline using big data technologies like
Apache Spark
and
MongoDB
.
We will be building an
ETLP
pipeline,
ETLP
stands for
Extract Transform Load
and
Predict
.
These are the different stages of the data pipeline that our data has to go through in order for it
to become useful at the end. Once the data has gone through this pipeline we will be able to
use it for building reports and dashboards for data analysis.
The data pipeline that we will build will comprise of data processing using
PySpark
, Predictive
modelling using Spark’s
MLlib
machine learning library, and data analysis using
MongoDB
and
Bokeh
.
You will learn how to create data processing pipelines using PySpark
You will learn machine learning with geospatial data using the Spark MLlib library
You will learn data analysis using PySpark, MongoDB and Bokeh, inside of jupyter notebook
You will learn how to manipulate, clean and transform data using PySpark dataframes
You will learn basic Geo mapping
You will learn how to create dashboards
You will also learn how to create a lightweight server to serve Bokeh dashboards
Requrirements
Requirements
Basic Understanding of Python
Little or no understanding of GIS
Basic understanding of Programming concepts
Basic understanding of Data
Basic understanding of what Machine Learning is