Transformers

[kitagrawal]

Maybe create a separate section on Transformers (in your to-do list). Recently, they have been getting a lot of attention.

[alvations]

@ankit--agrawal "attention", nice pun =)

[kitagrawal]

:-P
There are some good blog posts on the topic but I will be particularly interested to know some good resources (lecture series) on this topic.

[kmario23]

Hey @ankit--agrawal ,
thanks for your suggestion! Since this is a specialized topic, let's maintain it in this thread, at least for now.

Here is a preliminary list of lectures:

• Transformers(Self-Attention)-Deep-Learning-NLP

Please feel free to suggest if I've overlooked any worthwhile lectures!!

[kitagrawal]

Thank you so much for these. I will update the thread if I find something worthwhile. :)

[georgezoto]

Could you please make the Transformer list of lecture available on the main page?

Also how about adding content from the original attention papers:

1. Neural Machine Translation by Jointly Learning to Align and Translate
   https://arxiv.org/pdf/1409.0473.pdf

2. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
   https://arxiv.org/pdf/1502.03044.pdf

Thanks in advance and let us know how we can lep further,
George

[kmario23]

Could you please make the Transformer list of lecture available on the main page?

This is a nice suggestion! I've been thinking of a neat way to add it on the main page.

Also how about adding content from the original attention papers:

1. Neural Machine Translation by Jointly Learning to Align and Translate
   https://arxiv.org/pdf/1409.0473.pdf
2. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
   https://arxiv.org/pdf/1502.03044.pdf

I'm unsure about this since adding papers is not the goal of this repo!

Thanks in advance and let us know how we can lep further,
Contributions & suggestions are always welcome :)
George

[georgezoto]

I understand, there are plenty of lecture series on Transformers and blog posts that break down the original paper. I respect the framework you have chosen and I look forward your neat idea.