Participants will have the opportunity to put this knowledge into practice through hands-on exercises. Group interaction and instructor feedback make up an important component of the class.
The course starts with an introduction to elemental concepts of Big Data, then progresses into the programming languages and methodologies used to perform Data Analysis. Finally, we discuss the tools and infrastructure that enable Big Data storage, Distributed Processing, and Scalability.
- What Makes Big Data "Big"?
- Velocity, Volume, Variety, Veracity (VVVV)
- Limits to Traditional Data Processing
- Distributed Processing
- Statistical Analysis
- Types of Machine Learning Analysis
- Data Visualization
- Administrators
- Developers
- Data Analysts
- R Language
- Why R for Data Analysis?
- Data manipulation, calculation and graphical display
- Python
- Why Python for Data Analysis?
- Manipulating, processing, cleaning, and crunching data
- Statistical Analysis
- Time Series analysis
- Forecasting with Correlation and Regression models
- Inferential Statistics (estimating)
- Descriptive Statistics in Big Data sets (e.g. calculating mean)
- Machine Learning
- Supervised vs unsupervised learning
- Classification and clustering
- Estimating cost of specific methods
- Filtering
- Natural Language Processing
- Processing text
- Understaing meaning of the text
- Automatic text generation
- Sentiment analysis / topic analysis
- Computer Vision
- Acquiring, processing, analyzing, and understanding images
- Reconstructing, interpreting and understanding 3D scenes
- Using image data to make decisions
- Data Storage
- Relational databases (SQL)
- MySQL
- Postgres
- Oracle
- Non-relational databases (NoSQL)
- Cassandra
- MongoDB
- Neo4js
- Understanding the nuances
- Hierarchical databases
- Object-oriented databases
- Document-oriented databases
- Graph-oriented databases
- Other
- Relational databases (SQL)
- Distributed Processing
- Hadoop
- HDFS as a distributed filesystem
- MapReduce for distributed processing
- Spark
- All-in-one in-memory cluster computing framework for large-scale data processing
- Structured streaming
- Spark SQL
- Machine Learning libraries: MLlib
- Graph processing with GraphX
- Hadoop
- Scalability
- Public cloud
- AWS, Google, Aliyun, etc.
- Private cloud
- OpenStack, Cloud Foundry, etc.
- Auto-scalability
- Public cloud
- A general understanding of math.
- A general understanding of programming.
- A general understanding of databases.
35 hours (usually 5 days including breaks)
