State-of-the-art result for all Machine Learning Problems

LAST UPDATE: 17th November 2017

NEWS: I am looking for a Collaborator esp who does research in NLP, Computer Vision and Reinforcement learning. If you are not a researcher, but you are willing, contact me. Email me: [email protected]

This repository provides state-of-the-art (SoTA) results for all machine learning problems. We do our best to keep this repository up to date. If you do find a problem's SoTA result is out of date or missing, please raise this as an issue (with this information: research paper name, dataset, metric, source code and year). We will fix it immediately.

You can also submit this Google Form if you are new to Github.

This is an attempt to make one stop for all types of machine learning problems state of the art result. I can not do this alone. I need help from everyone. Please submit the Google form/raise an issue if you find SOTA result for a dataset. Please share this on Twitter, Facebook, and other social media.

This summary is categorized into:

• Supervised Learning
  • Speech
  • Computer Vision
  • NLP
• Semi-supervised Learning
  • Computer Vision
• Unsupervised Learning
  • Speech
  • Computer Vision
  • NLP
• Transfer Learning
• Reinforcement Learning

Supervised Learning

NLP

1. Language Modelling

Research Paper	Datasets	Metric	Source Code	Year
BREAKING THE SOFTMAX BOTTLENECK: A HIGH-RANK RNN LANGUAGE MODEL	PTB WikiText-2	Perplexity: 47.69 Perplexity: 40.68	Pytorch	2017
DYNAMIC EVALUATION OF NEURAL SEQUENCE MODELS	PTB WikiText-2	Perplexity: 51.1 Perplexity: 44.3	Pytorch	2017
Averaged Stochastic Gradient Descent with Weight Dropped LSTM or QRNN	PTB WikiText-2	Perplexity: 52.8 Perplexity: 52.0	Pytorch	2017
FRATERNAL DROPOUT	PTB WikiText-2	Perplexity: 56.8 Perplexity: 64.1	Pytorch	2017
Factorization tricks for LSTM networks	One Billion Word Benchmark	Perplexity: 23.36	Tensorflow	2017

2. Machine Translation

Research Paper	Datasets	Metric	Source Code	Year
WEIGHTED TRANSFORMER NETWORK FOR MACHINE TRANSLATION	WMT 2014 English-to-French WMT 2014 English-to-German	BLEU: 41.4 BLEU: 28.9	NOT FOUND	2017
Attention Is All You Need	WMT 2014 English-to-French WMT 2014 English-to-German	BLEU: 41.0 BLEU: 28.4	PyTorch Tensorflow	2017
NON-AUTOREGRESSIVE NEURAL MACHINE TRANSLATION	WMT16 Ro→En	BLEU: 31.44	NOT FOUND	2017
Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets	NIST02 NIST03 NIST04 NIST05	38.74 36.01 37.54 33.76	NMTPY	2017

3. Text Classification

Research Paper	Datasets	Metric	Source Code	Year
Learning Structured Text Representations	Yelp	Accuracy: 68.6	NOT FOUND	2017
Attentive Convolution	Yelp	Accuracy: 67.36	NOT FOUND	2017

4. Natural Language Inference

Leader board:

Stanford Natural Language Inference (SNLI)

MultiNLI

Research Paper	Datasets	Metric	Source Code	Year
NATURAL LANGUAGE INFERENCE OVER INTERACTION SPACE	Stanford Natural Language Inference (SNLI)	Accuracy: 88.9	Tensorflow	2017

5. Question Answering

Leader Board

SQuAD

Research Paper	Datasets	Metric	Source Code	Year
Interactive AoA Reader+ (ensemble)	The Stanford Question Answering Dataset	Exact Match: 79.083 F1: 86.450	NOT FOUND	2017

6. Named entity recognition

Research Paper	Datasets	Metric	Source Code	Year
Named Entity Recognition in Twitter using Images and Text	Ritter	F-measure: 0.59	NOT FOUND	2017

7. Abstractive Summarization

Research Paper	Datasets	Metric	Source Code	Year
Cutting-off redundant repeating generations for neural abstractive summarization	DUC-2004 Gigaword	DUC-2004 ROUGE-1: 32.28 ROUGE-2: 10.54 ROUGE-L: 27.80 Gigaword ROUGE-1: 36.30 ROUGE-2: 17.31 ROUGE-L: 33.88	NOT YET AVAILABLE	2017
Convolutional Sequence to Sequence	DUC-2004 Gigaword	DUC-2004 ROUGE-1: 33.44 ROUGE-2: 10.84 ROUGE-L: 26.90 Gigaword ROUGE-1: 35.88 ROUGE-2: 27.48 ROUGE-L: 33.29	PyTorch	2017

8. Dependency Parsing

Research Paper	Datasets	Metric	Source Code	Year
Globally Normalized Transition-Based Neural Networks	Final CoNLL ’09 dependency parsing	94.08% UAS accurancy 92.15% LAS accurancy	SyntaxNet	2017

Computer Vision

1. Classification

     

Research Paper	Datasets	Metric	Source Code	Year
Dynamic Routing Between Capsules	MNIST	Test Error: 0.25±0.005	PyTorch Tensorflow Keras Chainer List of all implementations	2017
High-Performance Neural Networks for Visual Object Classification	NORB	Test Error: 2.53 ± 0.40	NOT FOUND	2011
ShakeDrop regularization	CIFAR-10 CIFAR-100	Test Error: 2.31% Test Error: 12.19%	NOT FOUND	2017
Aggregated Residual Transformations for Deep Neural Networks	CIFAR-10	Test Error: 3.58%	PyTorch	2017
Random Erasing Data Augmentation	CIFAR-10 CIFAR-100 Fashion-MNIST	Test Error: 3.08% Test Error: 17.73% Test Error: 3.65%	Pytorch	2017
Dynamic Routing Between Capsules	MultiMNIST	Test Error: 5%	PyTorch Tensorflow Keras Chainer List of all implementations	2017
Learning Transferable Architectures for Scalable Image Recognition	ImageNet-1k	Top-1 Error:17.3	Tensorflow	2017
Squeeze-and-Excitation Networks	ImageNet-1k	Top-1 Error: 18.68	CAFFE	2017
Aggregated Residual Transformations for Deep Neural Networks	ImageNet-1k	Top-1 Error: 20.4%	PyTorch	2016

2. Instance Segmentation

Research Paper	Datasets	Metric	Source Code	Year
Mask R-CNN	COCO	Average Precision: 37.1%	TensorFlow (This is not official. Performance is worse than the reported AP)	2017

3. Visual Question Answering

Research Paper	Datasets	Metric	Source Code	Year
Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challenge	VQA	Overall score: 69	NOT FOUND	2017

4. Person Re-identification

     

Research Paper	Datasets	Metric	Source Code	Year
Random Erasing Data Augmentation	Market-1501 CUHK03-new-protocol DukeMTMC-reID	Rank-1: 89.13 mAP: 83.93 Rank-1: 84.02 mAP: 78.28 labeled (Rank-1: 63.93 mAP: 65.05) detected (Rank-1: 64.43 mAP: 64.75)	Pytorch	2017

Speech

Speech SOTA

1. ASR

Research Paper	Datasets	Metric	Source Code	Year
The Microsoft 2017 Conversational Speech Recognition System	Switchboard Hub5'00	WER: 5.1	NOT FOUND	2017

Semi-supervised Learning

Computer Vision

     

Research Paper	Datasets	Metric	Source Code	Year
DISTRIBUTIONAL SMOOTHINGWITH VIRTUAL ADVERSARIAL TRAINING	SVHN NORB	Test error: 24.63 Test error: 9.88	Theano	2016
Virtual Adversarial Training: a Regularization Method for Supervised and Semi-supervised Learning	MNIST	Test error: 1.27	NOT FOUND	2017
Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro	Market-1501 CUHK-03 DukeMTMC-reID CUB-200-2011	Rank-1: 83.97 mAP: 66.07 Rank-1: 84.6 mAP: 87.4 Rank-1: 67.68 mAP: 47.13           Test Accuracy: 84.4	Matconvnet	2017

Unsupervised Learning

Computer Vision

1. Generative Model

Research Paper	Datasets	Metric	Source Code	Year
PROGRESSIVE GROWING OF GANS FOR IMPROVED QUALITY, STABILITY, AND VARIATION	Unsupervised CIFAR 10	Inception score: 8.80	Theano	2017

NLP

Machine Translation

Research Paper	Datasets	Metric	Source Code	Year
UNSUPERVISED MACHINE TRANSLATION USING MONOLINGUAL CORPORA ONLY	WMT16 (en-fr fr-en de-en en-de) Multi30k-Task1(en-fr fr-en de-en en-de)	BLEU:(32.76 32.07 26.26 22.74) BLEU:(15.05 14.31 13.33 9.64)	NOT FOUND	2017

Transfer Learning

Research Paper	Datasets	Metric	Source Code	Year
One Model To Learn Them All	WMT EN → DE WMT EN → FR (BLEU) ImageNet (top-5 accuracy)	BLEU: 21.2 BLEU:30.5 86%	Tensorflow	2017

Reinforcement Learning

Research Paper	Datasets	Metric	Source Code	Year
Mastering the game of Go without human knowledge	the game of Go	ElO Rating: 5185	C++	2017

Email: [email protected]