Advanced-Deep-Learning-Techniques-CNNs-and-Character-Level-Translation

Authors: Afonso Alemão, Rui Daniel

Overview

This project dives deep into Convolutional Neural Networks (CNNs), attention mechanisms, and character-level machine translation, giving students a comprehensive understanding of these advanced deep-learning techniques.

Contents

1. Convolutional Neural Network Analysis

• Analyze a simple CNN designed for 3-class image classification.
• Understand dimensions and calculations associated with convolutional layers, activation functions, and max-pooling.
• Compare parameters in different network architectures.
• Delve into the theory behind attention probabilities and outputs in flattened sequences.

2. CNN Image Classification

• Use CNNs for image classification with the Kuzushiji-MNIST dataset.
• Discuss the merits of CNNs over fully-connected networks in terms of parameters and generalization.
• Evaluate CNN performance with non-image data.
• Implement a specific CNN architecture, train, and assess its performance.
• Interpret CNN activation maps.

3. Character-level Machine Translation

• Address the challenge of machine translation at the character level.
• Contrast LSTM-based and masked self-attention-based models.
• Implement and evaluate a character-level machine translation system using transformers.

Datasets and Tools

• Kuzushiji-MNIST: Handwritten Japanese characters dataset.
• PyTorch: Recommended framework for deep learning.

Key Concepts

• Convolutional Neural Networks
• Max Pooling
• Activation Maps
• Attention Mechanism
• LSTM Networks
• Transformers and Self-Attention