Psych.ai

Contributers : Sree Harsha Nelaturu, Aparna Krishnakumar, Anith Patel, Maneesha Dendukuri.

Making use of Deep Learning in Image Processing specifically Neural Style Transfer in order to create a tool which can be used to act as an aid for people with hearing disabilities and serve as a bridge for people of different cultures.

Getting Started

Clone the Repository, following which please clone the following repository:

Fast Style Transfer - For fast style transfer which is applied to frames. Create an Azure account and acquire a key for the Microsoft Cognitive Emotion API which needs to be used in the EmotionAPIIntegration.py

Prerequisites

1. Tensorflow
2. Numpy
3. OpenCV
4. CUDA

File Requirements and Description:

1. videofolder.py - creates a folder containig frames of the image.
2. create_array.py - creates numpy array of images
3. EmotionAPIIntegration.py - Uses Microsoft Cognitive Emotion API in order to analyze sentiments of the frames in image folder.
4. imageworkfinal.py - Main script to be utilized for the completely automated process of selection and input of video, and subsequent generation of video.

Working of the Model :

The input video is taken from the side of the user, he selects an enhancement he requires, which consists of "Scary" and "Comic" in the current iteration of the software, following this, the video is converted into frames and stores them in it a folder, following this it runs the emotion API on the frames, and the frames which contain the emotions related to enhancement sought are copied to a new folder, following which style transfer is applied, and subsequently they are replaced in the original folder and a video is compiled. Fully automated, works with Python 3.5.2 and runs on Tensorflow-GPU.

Testing the Model:

In order to use the system please clone the repository and that of fast style transfer, replaced directory links to the folders Create two folders, "styleimage" and "styletransferimages" and place the checkpoint file for style in the checkpoints files. Checkpoints for "Scary" and "Comic" have been provided. Following this run imageworkfinal.py, ensure CUDA is installed for speedy computatition.