README.md

Deep Learning Specialization Coursera [UPDATED Version 2021]

This repo contains all of the solved assignments of Coursera's most famous Deep Learning Specialization of 5 courses offered by deeplearning.ai

Instructor: Prof. Andrew Ng

What's New

This Specialization was updated in April 2021 to include developments in deep learning and programming frameworks. One of the most major changes was shifting from Tensorflow 1 to Tensorflow 2. Also, new materials were added. However, Most of the old online repositories still don't have old codes. This repo contains updated versions of the assignments. Happy Learning :)

Programming Assignments

• Course 1: Neural Networks and Deep Learning

  • W2A1 - Logistic Regression with a Neural Network mindset
  • W2A2 - Python Basics with Numpy
  • W3A1 - Planar data classification with one hidden layer
  • W4A1 - Building your Deep Neural Network: Step by Step¶
  • W4A2 - Deep Neural Network for Image Classification: Application

• Course 2: Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization

  • W1A1 - Initialization
  • W1A2 - Regularization
  • W1A3 - Gradient Checking
  • W2A1 - Optimization Methods
  • W3A1 - Introduction to TensorFlow

• Course 3: Structuring Machine Learning Projects

  • There were no programming assignments in this course. It was completely thoeretical.
  • Here is a link to the course

• Course 4: Convolutional Neural Networks

  • W1A1 - Convolutional Model: step by step
  • W1A2 - Convolutional Model: application
  • W2A1 - Residual Networks
  • W2A2 - Transfer Learning with MobileNet
  • W3A1 - Autonomous Driving - Car Detection
  • W3A2 - Image Segmentation - U-net
  • W4A1 - Face Recognition
  • W4A2 - Neural Style transfer

• Course 5: Sequence Models

  • W1A1 - Building a Recurrent Neural Network - Step by Step
  • W1A2 - Character level language model - Dinosaurus land
  • W1A3 - Improvise A Jazz Solo with an LSTM Network
  • W2A1 - Operations on word vectors
  • W2A2 - Emojify
  • W3A1 - Neural Machine Translation With Attention
  • W3A2 - Trigger Word Detection
  • W4A1 - Transformer Network
  • W4A2 - Named Entity Recognition - Transformer Application
  • W4A3 - Extractive Question Answering - Transformer Application
  • W4A4 - Embedding Plus Positioning Encoding - Transformer Application

DeepLearning.ai - Deep Learning Specialization

This is a list of research papers referenced by the Deep Learning Specialization course.

Convolutional Neural Networks

Week 2

Classic Networks

• Lacuna et al, 1998 - Gradient-based learning applied to document recognition
• Krizhevsky et al 2012 - ImageNet classification with deep convolutional neural networks
• Simony & Zisserman 2015 - Very deep convolutional networks for large scale image recognition - He et al, 2015

Resnets

• Lin et al, 2013 - Deep residual networks for image recognition

Networks in Networks and 1x1 Convolutions

• Szegedy et al, 2014 - Network in network - Going deeper with convolutions

Networks in Networks and 1x1 Convolutions

Min Lin, Qiang Chen, Shuicheng Yan - "Network In Network"

Inception Networks

• Christian Szegedy, and lots of others - "Going Deeper with Convolutions":

Week 3

Convolutional Implementation of Sliding Windows

• Servant et al, 2014 - OverFeat: Integrated recognition, localization and detection using convolutional networks

Bounding Box Predictions

• Redmon et al, 2014 - You Only Look Once: Unified real-time object detection
• Redmon et al, 2016 - YOLO9000: Better, Faster, Stronger

Region Proposals

• Girishik et al, 2013 - Rich feature hierarchies for accurate object detection and semantic segmentation
• Girshik, 2015 - Fast R-CNN
• Ren et al, 2016 - Faster R-CNN: Toward real-time object detection with region proposal networks

Week 4

Siamese Network

• Taigman et al, 2014 - DeepFace: Closing the Gap to Human-Level Performance in Face Verification

Triplet Loss

• Schroff et al. - FaceNet: A Unified Embedding for Face Recognition and Clustering

What are deep ConvNets learning?

• Zeiler and Fergus, 2013 - Visualizing and understanding convolutional networks

Neural Style

• Gates et al, 2015 - A neural algorithm of artistic style
• Harish Narayanan - Convolutional neural networks for artistic style transfer
• Log0, TensorFlow Implementation of "A Neural Algorithm of Artistic Style"

Image Recognition

• Karen Simonyan and Andrew Zisserman (2015). Very deep convolutional networks for large-scale image recognition

NLP Sequence Models

Week 1

GRU

• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches - Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, Yoshua Bengio
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling - Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, Yoshua Bengio

LSTM

• LSTM - Schmidhuber and Hochreiter

Week 2

Skip-Grams, Hierarchical Softmax

• Efficient Estimation of Word Representations in Vector Space

Word Embeddings

• Linguistic Regularities in Continuous Space Word Representations - Tomas Mikolov, Wen-tau Yih, Geoffrey Zwei
• A Neural Probabilistic Language Model - Bengio et al

Negative Sampling

• Distributed Representations of Words and Phrases and their Compositionality Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, Jeffrey Dean

Glove

• GloVe: Global Vectors for Word Representation Jeffrey Pennington, Richard Socher, Christopher D. Manning

Debaising Word Embeddings

• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings

Week 3

Sequence to Sequence Model

• Sequence to Sequence Learning with Neural Networks - Ilya Sutskever, Oriol Vinyals, Quoc V. Le

• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation - Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio

Image Captioning

• Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN) - Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, Alan Yuille

• Show and Tell: A Neural Image Caption Generator - Oriol Vinyals, Alexander Toshev, Samy Bengio, Dumitru Erhan

• Deep Visual-Semantic Alignments for Generating Image Descriptions - Andrej Karpathy, Li Fei-Fei

Bleu Score

• BLEU: a Method for Automatic Evaluation of Machine Translation Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu

Attention based intuation

• Neural Machine Translation by Jointly Learning to Align and Translate - Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio

• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention - Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, Yoshua Bengio

Speech Recognition

• Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks

Further Reading List

• http://deeplearning.net/reading-list/