Deep Learning

Course Leader: Dr Lela Mirtskhulava

Home Institution: Iv. Javakhishvili Tbilisi State University, Georgia

Course pre-requisites: Data Mining or Machine Learning.

Course Overview
Deep neural networks and their applications to various problems, e.g., speech recognition, image segmentation, and natural language processing. Will cover underlying theory, the range of applications to which it has been applied, and learning from very large data sets.

Learning Outcomes

  1. Identify the proper techniques and algorithms needed to prepare data for deep learning.
  2. Understand advantages and disadvantages of dimensionality reduction as a component or deep learning.
  3. Identify the correct class of methods that can be used to efficiently build a shallow and deep neural network.
  4. Discuss and apply fundamental optimization concepts, such as regularization and early convergence, needed to train a neural network model.
  5. Quickly get accustomed to any deep learning software library and be able to use it for in-depth analysis.
  6. Gain hands-on experience by performing an extensive analysis using deep learning techniques, in individual and group projects.
  7. Effectively present and communicate the knowledge they have acquired in the course.

Course Content

Week

Date

Topics, Readings, Assignments, Deadlines

 

1

 

7/20

Introduction to Deep Learning

Linear regression

Cost function, Gradient Decent,

Learning rate, Normal Equation

1

 

7/21

 

Overfitting, Underfitting,

Training/Validating/Testing

Regularization,

Bias-variance Trade-off

1

 

7/22

 

Logistic Regression,

Binary classification

Softmax,

Multiclass classification

1

 

7/23

 

Neural Networks , Hidden layers

Shallow Neural Networks

Backpropagation

Initialization: Xavier and He

1

 

7/24

 

Batch Normalization

Optimizers

Regularization

Convolutional Neural Networks

2

 

7/27

 

Midterm Exam

Midterm Exam

Convolutional Layer, Pooling Layer

CNN architectures

2

 

7/28

 

Recurrent Neural Networks

Memory cells, Input and Output Sequences

Group Project proposals

Group Project proposals

2

 

7/29                

 

Training RNNs

LSTM, GRU

Autoencoders

Reinforcement Learning, Learning to Optimize Rewards, Policy Search

 

2

 

 

7/30

 

Introduction to OpenAI Gym

Neural Network Policies, Evaluating Actions

Group Project Presentations

Group Project Presentations

Group Project Presentations

 

2

 

 

7/31

 

Learning to Play Ms. Pac-Man Using Deep Q-Learning

Policy Gradients

Markov Decision Processes

Final Exam

Final Exam

Instructional Method
This course requires the student to have a personal computer that is installed with a modern operating system. The lectures will be delivered in the classroom, however the students might be asked to use their laptops or smart devices during the class, or offline in order to participate in the class assignments.

Required Course Materials
This class uses ‘Hands-On Machine Learning with Scikit-Learn & TensorFlow’ (by Aurelien Geron) as a textbook. It is not required to purchase the textbook, but some contents of lecture slides and assignments would be used from the textbook.

Other technology requirements / equipment / material
Programming languages, platforms, as well as software applications and tools, such as Python, Jupyter Notebook, TensorFlow, etc. that will be required for this class are free to download. Students will be informed in class and via Google Classroom ahead of time in order to install all required software.

Assessment

These classes are designed such that in order to be successful, it is expected that students will spend a minimum of forty-two hours for each unit of credit (normally three hours per unit per week), including preparing for class, participating in course activities, completing assignments, and so on.

In-class & discussion forum participation: Students will be evaluated based on their participation in in-class and discussion forum.

Individual written/programming assignments: Students will be provided with handouts describing the assignments and how they will be graded. These assignments will be in-class or take-home written assignments, programming assignments.

Exams: Two Exams will be provided, which can be a combination of multiple choice, short or long answer questions, and programming.   The exams will be based on the homework assignments and course material covered in class.

Determination of Grades

Quiz                                                          10%

Homework assignments                       20%

Class Activity                                           10%

Midterm exam I                                      20%

Final exam                                               40%