Introduction to Python and PySpark Development

Expand all

Course Description

This Introduction to Python and PySpark Development training course is designed to provide Developers and/or Data Analysts a gentle immersive hands-on introduction to the Python programming language and Apache PySpark.

Course Outline

Chapter 1. Introduction to Python

•     What is Python
•     Installing Python (minimal with mention of Anaconda)
•     Using the Python shell
•     IPython
•     Jupyter notebooks
•     Getting help
•     Lab – learning the environment

Chapter 2. Writing scripts

•     What are scripts
•     Writing scripts
•     Running scripts

Chapter 3. Data types and variables

•     Understanding basic Python types
•     Variables
•     String literals
•     Numbers

Chapter 4. Control statement

•     If statement
•     Handling exception with try

Chapter 5. Looping

•     While loop
•     For-in loop
•     Range function
•     Break and continue (optional)

Chapter 6. Functions

•     What are functions
•     Creating a function
•     Arguments and return values
•     Variable scope

Chapter 7. List and Tuples

•     What are lists
•     Creating list
•     Working with lists
•     Sorting lists
•     Introduction to tuples

Chapter 8. Sequences

•     What are sequences
•     Lists, sets and strings
•     Slicing
•     List comprehension

Chapter 9. Sets and Dictionary

•     What are sets
•     Working with sets
•     Sets vs lists
•     Introduction to dictionaries
•     Using dictionaries

Chapter 10. Data structures

•     Comparison of list, tuples, sets and dictionaries
•     Combining different data structures together
•     Converting from one to another
•     Common use cases

Chapter 11. Reading and writing text files (optional)

•     Opening and closing files
•     Reading from a file
•     Writing to a file
•     Using the with statement (optional)

Chapter 12. Functional Programming Primer

•     What is functional programming
•     Functions as data
•     Functions as arguments
•     Lambda expressions
•     Using map, filter and other functions
•     Difference between simple loops and map/filter/etc

Chapter 13. Hadoop/MapReduce Introduction

Chapter 14. Introduction to PySpark

•     What is Apache Spark
•     Spark use cases
•     Architectural overview

Chapter 15. PySpark Shell

•     What is the PySpark Shell
•     Starting and using the shell
•     Spark context
•     PySpark Shell vs Spark Shell

Chapter 16. Resilient Distributed Dataset

•     What are Resilient Distributed Dataset (RDD)
•     Creating RDDs
•     Transformations and operations

Chapter 17. Parallel Processing

•     Spark cluster
•     Data partitioning
•     Applications, jobs and tasks

Chapter 18. Shared Variables

•     What are shared variables
•     Broadcast variables
•     Accumulators

Chapter 19. Spark SQL

•     What is Spark SQL
•     Uniform data
•     Hive
•     SQL Context object

Chapter 20. Putting it all together

•     Case study
•     Final project

Optional Topics:
Chapter 21. Advanced Functions

•     Default arguments
•     Arbitrary arguments
•     Keyword arguments
•     What are modules
•     Importing modules
•     Creating simple modules

Chapter 22. Modules

Learning Path

• Please contact us for more suggestions.

• Advanced Python 3
• Applied Data Science with Python
• Big Data Analysis with Python

Ways to Attend

• Attend a public course, if there is one available. Please check our schedule, or register your interest in joining a course in your area.
• Private onsite Team training also available, please contact us to discuss. We can customise this course to suit your business requirements.