DeepLearning Note

SoA

2017.06.12. Attention Is All You Need: https://arxiv.org/abs/1706.03762
2019.02.14. GPT-2, Better Language Models and Their Implications: https://openai.com/blog/better-language-models
2020.06.11. GPT-3, OpenAI API: https://openai.com/blog/openai-api
2021.01.05. DALL·E: Creating Images from Text: https://openai.com/blog/dall-e
2021.06.29. Introducing GitHub Copilot: your AI pair programmer: link
2021.08.10. OpenAI Codex: https://openai.com/blog/openai-codex
2021.10.29. Solving Math Word Problems: https://openai.com/blog/grade-school-math
2021.12.31. A Neural Network Solves and Generates Mathematics Problems by Program Synthesis: https://arxiv.org/abs/2112.15594
2022.02.02. Solving (Some) Formal Math Olympiad Problems: https://openai.com/blog/formal-math
2022.02.02. Competitive programming with AlphaCode:https://deepmind.com/blog/article/Competitive-programming-with-AlphaCode?fbclid=IwAR1E9sSTt3XiEXn70d2YO8bx8ruBNKArzLhJ8WuZwpclXCK58qihhzgjeVE

mook

Stanford CS229: Machine Learning
Applied Machine Learning
Practical Deep Learning for Coders (2020)
Machine Learning with Graphs (Stanford)
Probabilistic Machine Learning
Introduction to Deep Learning (MIT)
Deep Learning: CS 182
Deep Unsupervised Learning
NYU Deep Learning SP21
CS224N: Natural Language Processing with Deep Learning
CMU Neural Networks for NLP
CS224U: Natural Language Understanding
CMU Advanced NLP
Multilingual NLP
Advanced NLP
Deep Learning for Computer Vision
Deep Reinforcement Learning
Full Stack Deep Learning
AMMI Geometric Deep Learning Course (2021)

Pytorch

Christian Perone - 발표자료: https://speakerdeck.com/perone/pytorch-under-the-hood

블로그: http://blog.christianperone.com/ Build Deep Learning Projects (Complete Video Series for FREE ) ==============================================

Making your RL Projects in 20 Minutes : https://www.edyoda.com/course/1421
Style Transfer, Face Generation using GANs in 20 minutes : https://www.edyoda.com/course/1418
Language and Machine Learning in 20 minutes : https://www.edyoda.com/course/1419
AI Project - Web application for Object Identification : https://www.edyoda.com/course/1185
Dog Breed Prediction : https://www.edyoda.com/course/1336

DL papers

ast-SCNN: Fast Semantic Segmentation Network.
- 123 fps on 2048x1024 images (2x faster than current state-of-the-art).
- https://arxiv.org/abs/1902.04502
Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs (https://arxiv.org/pdf/1606.00915.pdf) Source code: https://github.com/vietnguyen91/Deeplab-pytorch
Correlational Neural Network. CV, TL, RPL.
Reasoning With Neural Tensor Networks for Knowledge Base Completion. NLP, ML. Blog-post
Learning to Rank Short Text Pairs with Convolutional Deep Neural Networks. NLP, DL, CQA. Code
Common Representation Learning Using Step-based Correlation Multi-Modal CNN. CV, TL, RPL. Code
ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs. NLP, AT, DL, STS. Code
Combining Neural, Statistical and External Features for Fake News Stance Identification. NLP, IR, DL. Code
WIKIQA: A Challenge Dataset for Open-Domain Question Answering. NLP, DL, CQA. Code
Siamese Recurrent Architectures for Learning Sentence Similarity. NLP, STS, DL. Code
Teaching Machines to Read and Comprehend. NLP, AT, DL. Code
Improved Representation Learning for Question Answer Matching. NLP, AT, DL, CQA. Code
Map-Reduce for Machine Learning on Multicore]. map-reduce, hadoop, ML.. MR, ML. Code
Convolutional Neural Tensor Network Architecture for Community Question Answering. NLP, DL, CQA. Code
External features for community question answering. NLP, DL, CQA. Code
Language Identification and Disambiguation in Indian Mixed-Script. NLP, IR, ML. Blog-post
Construction of a Semi-Automated model for FAQ Retrieval via Short Message Service. NLP, IR, ML. Code

data science tutorial

Tracking Bird Migration Using Python 3 Source Code & Tutorial: https://goo.gl/BS4rQc

Data Science Tutorial Read Here: https://goo.gl/ZPyZBX

Annotation detect

Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series
- paper

Tensorflow

The TenSorFlow is an Open Soruce Software Library for Machine Intellience. This repository are many jupyter note-pad like TesroFlow turorials, step books, and others.

DeepMind's WaveNet

Code: source

DeepVoice

Deep Voice: Real-Time Neural Text-to-Speech for Production

Sercan O. Arik, Mike Chrzanowski, Adam Coates, Gregory Diamos, Andrew Gibiansky, Yongguo Kang, Xian Li, John Miller, Jonathan Raiman, Shubho Sengupta, Mohammad Shoeybi
[paper]
[Ref.code

Very simple TensorFlow examples

Code: https://github.com/nlintz/TensorFlow-Tutorials

GAN

Style-based GAN

Document

Pytorch a2c

Advantage Actor Critic (A2C), a synchronous deterministic version of A3C
- Volodymyr Mnih1
- Adria Puigdomenech Badia1
- Mehdi Mirza1,2
- Alex Graves1
- Tim Harley1
- Timothy P. Lillicrap1
- David Silver1
- Koray Kavukcuoglu1
code

Sequence to Sequence -- Video to Text

Subhashini Venugopalan, Marcus Rohrbach, Jeff Donahue, Raymond Mooney, Trevor Darrell, Kate Saenko, arxiv, 2015
[code]
[paper]

Sequence to Sequence -- chatbot

Oriol Vinyals, Quoc V. Le, arxiv, 2015
[code]
[paper]

Show and Tell: A Neural Image Caption Generator

Oriol Vinyals, Alexander Toshev, Samy Bengio, Dumitru Erhan, arxiv, 2015
[code]
[paper]

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, Yoshua Bengio, ICLR, 2014
[code]
[paper]

Learning Deep Features for Discriminative Localization

Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, Antonio Torralba, CVPR, 2016
[code]
[paper]

Deep Visual Analogy-Making

Scott Reed, Yi Zhang, Yuting Zhang, Honglak Lee, NIPS, 2015
[code]
[paper]

Deep Convolutional Generative Adversarial Networks

Alec Radford, Luke Metz, Soumith Chintala, arxiv, 2015
[code]
[paper]

End-To-End Memory Networks

Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, Rob Fergus, NIPS, 2015
[code]
[paper]

Character-Aware Neural Language Models

Yoon Kim, Yacine Jernite, David Sontag, Alexander M. Rush, AAAI, 2016
[code]
[paper]

Deep Reinforcement Learning

Human-level control through deep reinforcement learning

Volodymyr Mnih, et al, 2014
[code1], not trained on atari
[code2]
[paper]

Deep Reinforcement Learning with Double Q-learning

Hado van Hasselt, Arthur Guez, David Silver, 2015
[code]
[paper]

Using Deep Q-Network to Learn How To Play Flappy Bird

Kevin Chen, Deep Reinforcement Learning for Flappy Bird, Report from http://cs229.stanford.edu/ 2015 project
[code]
[report]

Semi-Supervised Learning with Ladder Network

A Rasmus, H Valpola, M Honkala, M Berglund, and T Raiko, NIPS, 2015
[code]
[paper]
[Additional Material]

Convolutional Neural Networks for Sentence Classification

Yoon Kim, EMNLP, 2014
[code]
[paper]
[Additional Material]

Black-Box Adversarial Perturbations

Implementation of Simple Black-Box Adversarial Perturbations for Deep Networks in Keras fork from link

python cifar100.py to train a basic CNN for cifar100 and save that file.
python find_better.py <model> to go through cifar100 test dataset and find a good image (as defined in the paper).
python per.py <KERAS_MODEL> <IMAGE_in_NUMPY> : currently works for cifar images only.

Deep Residual Learning for Image Recognition

K He, X Zhang, S Ren, J Sun
[code] for cifar10
[paper1], [paper2]

colornet - Neural Network to colorize grayscale images

DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients

Shuchang Zhou, Zekun Ni, Xinyu Zhou, He Wen, Yuxin Wu, Yuheng Zou, 2016
[code]
[paper]

A Neural Algorithm of Artistic Style

Leon A. Gatys, Alexander S. Ecker, Matthias Bethge
[code - Neural style in TensorFlow!]
[blog - http://www.anishathalye.com/2015/12/19/an-ai-that-can-mimic-any-artist/]
[A Neural Algorithm of Artistic Style]

콘크리트 구조물 균열 탐지 및 분석

https://github.com/Garamda/SPARK

csv ttols

https://github.com/eBay/tsv-utils.git

Sequence Generative Adversarial Networks

Lantao Yu, Weinan Zhang, Jun Wang, Yong Yu
[code]
[paper]
[paper]

https://github.com/shawLyu/HR-Depth

Reference

CV | Computer Vision
TL | Transfer Learning
RPL | Representation Learning
CQA | Community Question Answering
STS | Sentence Text Similarity
IR | Information Retrieval
AT | Attention
MR | Map Reduce
ASR | Acoustic Scene Recognition
DL | Deep Learning
NLP | Natural Language Processing
ML | Machine Learning.

Name		Name	Last commit message	Last commit date
Latest commit History 686 Commits
0_datasets		0_datasets
1_ducumentation		1_ducumentation
2_tutorials		2_tutorials
3_ETRI-lecture		3_ETRI-lecture
4_wavenet		4_wavenet
5_campsule_net		5_campsule_net
6_tensorflow_10weeks		6_tensorflow_10weeks
08_dimensionality_reduction.ipynb		08_dimensionality_reduction.ipynb
BERT.ipynb		BERT.ipynb
BERT.md		BERT.md
CIFAR-10.py		CIFAR-10.py
CNN.md		CNN.md
Computational_Imageing.md		Computational_Imageing.md
Datasets.md		Datasets.md
Encoder.md		Encoder.md
GAN.md		GAN.md
Image Data Augmentation with Keras.ipynb		Image Data Augmentation with Keras.ipynb
Image Data Augmentation with Keras.py		Image Data Augmentation with Keras.py
Intel_VNNI.md		Intel_VNNI.md
LLM.md		LLM.md
MLOps.md		MLOps.md
ModelTransfer.md		ModelTransfer.md
README.md		README.md
RL.md		RL.md
Sentiment_Analysis.md		Sentiment_Analysis.md
TPUs.md		TPUs.md
VideoSR.md		VideoSR.md
ai_game.md		ai_game.md
bc.md		bc.md
biology.md		biology.md
blackbox.md		blackbox.md
cifar100.py		cifar100.py
code.md		code.md
cv-sat.md		cv-sat.md
deep-geometry.md		deep-geometry.md
deeplearning_open-book.md		deeplearning_open-book.md
diffusion.md		diffusion.md
download_glue.py		download_glue.py
edge.md		edge.md
espnet.md		espnet.md
facetensor.py		facetensor.py
find_better.py		find_better.py
gpu.md		gpu.md
horovod.md		horovod.md
install_cudnn.md		install_cudnn.md
install_tensorflow.md		install_tensorflow.md
jetson-nano.md		jetson-nano.md
jetson.md		jetson.md
keras-bert-toxic-model-bert-fine-tuning-with-keras.ipynb		keras-bert-toxic-model-bert-fine-tuning-with-keras.ipynb
knlp.md		knlp.md
korean-한국어.md		korean-한국어.md
label.md		label.md
lstm.ipynb		lstm.ipynb
medicine.md		medicine.md
modelConvert.md		modelConvert.md
mudi.md		mudi.md
nlp.md		nlp.md
paper.md		paper.md
per.py		per.py
pipelind.md		pipelind.md
process.md		process.md
quantum.md		quantum.md
scrape-line-stamp.py		scrape-line-stamp.py
service.md		service.md
sin.py		sin.py
sound.md		sound.md
stck.ipynb		stck.ipynb
step.md		step.md
superresolution.md		superresolution.md
text_classification.py		text_classification.py
tf.distribute.Strategy.py		tf.distribute.Strategy.py
transformer.md		transformer.md
tts.md		tts.md
txt2imgd.py		txt2imgd.py

kaggledevs/deeplearning-note

Folders and files

Latest commit

History

Repository files navigation

DeepLearning Note

SoA

mook

Pytorch

DL papers

data science tutorial

Deep Learning (CS 1470)

Deep Learning Book

Dive into Deep Learning

Neural Network Design

Neural Networks and Deep Learning

Theories of Deep Learning (STATS 385)

Theoretical Principles for Deep Learning (IFT 6085)

A collection of links of videos(youtube) by course

A collection of tutorial Jupyter notebooks

the matrix calculus

etc