AI Caption Generator

This is a web app that allows you to upload an image file, generate a caption for it using deep learning, then automatically post the image an caption to our Twitter bot @Bot2021U.

• Flask backend
• Keras for deep learning, starter code from this tutorial
• current model based on Flickr8K dataset
• tested another model based on InstaCities1M dataset

Developed for University of Iowa Hackathon 2021 by Raymond Yang, Mitchell Hermon, Ethan D'Allesandro, and Riley Bridges. Devpost link here.

Host the App

1. Make a secrets.py file to store your Twitter API keys:

API_KEY = ''
API_SECRET_KEY = ''
BEARER_TOKEN = ''
ACCESS_TOKEN = ''
ACCESS_TOKEN_SECRET = ''

2. Set Flask app to app.py:

   • export FLASK_APP=app.py on linux

3. flask run

Train the Model

1. Change image_prep.py, text_prep.py, progressive_loading.py, evaluate.py, create_tokenizer.py to work with the desired dataset
2. bash train_ai.sh