Deep Learning in Python

This is the repository for D-Lab’s six-hour Introduction to Deep Learning in Python workshop. View the associated slides here.

Objectives

Convey the basics of deep learning in Python using keras on image datasets. Students are empowered with a general grasp of deep learning, example code that they can modify, a working computational environment, and resources for further study.

Content outline

• Installation
  • Jupyter Notbook
  • Keras and Tensorflow
  • Helper packages
• What is “deep” learning?
• Understanding the dataset
• Dataset splitting: training, test, cross-validation
  • Defining moving parts of a deep learning model
  • Understanding a loss function, activation function, and metrics
  • Performance evaluation
• Part 1
  • MNIST 0-9 hand-written digit example
  • Feed Forward (Vanilla) Neural Networks
• Part 2
  • CIFAR10 - 10 image type classification
  • Convolutional Neural Networks

Prerequisites

This is an advanced level workshop. Participants should be intermediate Python users and have had some prior exposure to machine learning.

We assume the following background:

• D-Lab's Python Machine Learning Fundamentals (6 hours)
• Or, comparable experience/training, assuming familiarity with:
  • Basic Python syntax
  • Train/validation/test splitting
  • Dataset cleaning
  • Overfitting / underfitting / generalization
  • Hyperparameter customization
  • Basic linear algebra (vector, matrix, etc.)
  • Basic statistics (linear regression)

If you are not comfortable installing packages, writing your own Python code, and using Jupyter Notebooks, this will not be a good workshop for you.

Getting Started

1. Datahub - If you are a UC Berkeley student, use D-Lab's Datahub (highly recommended), click this link:

2. Google Colab - If you are not a UC Berkeley student, use the following links to open up each Jupyter Notebook in a Google Colab session:

   • 01-Vanilla-Neural-Networks.ipynb
   • 02-Vanilla-Convolutional-Neural-Network-Comparison.ipynb
   • 03-Convolutional-Neural-Networks.ipynb

3. If you would like to install packages locally on your computer, see the installation instructions.

   • This process can take about 30-60 minutes, so be sure to try and do this before class!

Resources

• Massive open online courses

  • fast.ai - Practical Deep Learning for Coders
  • Kaggle Deep Learning
  • Google Machine Learning Crash Course
  • See this sweet interactive learning rate tool
  • Google seedbank examples
  • DeepLearning.ai

• Workshops

  • Nvidia's Modeling Time Series Data with Recurrent Neural Networks in Keras

• Stanford

  • CS 20 - Tensorflow for Deep Learning Research
  • CS 230 - Deep Learning
  • CS 231n - Neural Networks for Visual Recognition
  • CS 224n - Natural Language Processing with Deep Learning

• Berkeley

  • Machine Learning at Berkeley ML@B
  • CS 189/289A

• UToronto CSC 321 - Intro to Deep Learning

• Books

  • F. Chollet and J.J. Allaire - Deep Learning with Python
  • Charniak E - Introduction to Deep Learning
  • I. Goodfellow, Y. Bengio, A. Courville - www.deeplearningbook.org
  • Zhang et al. - Dive into Deep Learning

Contributors

• Sean Perez
• Pratik Sachdeva