B2Detect

Data Science

Object Detection on steaming images using Tensorflow and Apache NiFi

Introduction - B2 Stealth Bomber Social Media Photo Detector

The goal of this project is to build an end-to-end project to detect images that contain B2 Stealth Bomber from popular social media sites.

Language: Python

Requirements:

• Python 2.7
• Tensorflow 1.13

Author: Ian R Brooks

Follow: LinkedIn - Ian Brooks PhD

Table of Contents

1. Introduction

2. Links

   1. Apache NiFi
   2. Tensorflow
   3. Additional

3. Setup Environment

   1. Download Python Libraries
   2. Download NiFi Processor
   3. Download Project
   4. Upload NiFi Templates

4. NiFi Configuration

5. Tensorflow Serving

6. Results - Images Posted To Slack

Links

NiFi Links:

• Posting Images with Apache NiFi 1.7 and a Custom Processor
• Post Images To Slack Custom Processor
• Great Read On Posting Images to Slack from Apache NiFi Using Custom Processor
• Uploading NiFi Template

TensorFlow Links:

• Tensorflow Serving
• Tensorflow Serving Using Docker

Additional Links:

• Ian's Tensorflow Object Detection Tutorial
• Social Searcher
• Slack

Setup Environment - Download Everything!

Download Python Libraries

Run at terminal prompt

pip install requests
pip install pillow
pip install numpy
pip install image
pip install Pillow-PIL

Download NiFi Processor

Run at terminal prompt

#Download Post Image Processor nar  - Thank You Tim Spann! 
wget https://github.com/tspannhw/nifi-postimage-processor/releases/download/1.0/nifi-postimage-nar-1.0.nar \
-O /usr/hdf/current/nifi/lib/nifi-postimage-nar-1.0.nar

Download Project

Download the project using the git url for here.

Upload NiFi Template

NiFi flow template is called B2DetectFlow_BaseTemplate.xml

Once the template has been loaded, you should see the following NiFi flow

NiFi Configuration

Set Social Seacher API Token in InvokeHTTP processor

Set Social Searcher token value

http://api.social-searcher.com/v2/search?q=B2+Stealth+Bomber&type=photo&key=<YOUR TOKEN VALUE HERE>

SSL Context Configuration in InvokeHTTP processor

Post Image Processor Configuration

Update Slack API Token in URL value:

https://slack.com/api/files.upload?token= <YOUR KEY HERE> &channels=b2detect&filename=${absolute.path}${filename}&files:write:user&pretty=1

Put Slack Processor Configuration

Update Webhook URL value

Tensorflow Serving Using Docker

Run at terminal prompt. Note the path need to point to the location of saved_model directory in this github repo.

docker pull tensorflow/serving

#Adding the Version number on model target path is VERY important! 
docker run -p 8900:8500 -p 8501:8501  --mount type=bind,source=/saved_model,target=/models/saved_model/1 \
-e MODEL_NAME=saved_model -t tensorflow/serving &

Configure Execute Stream Command Processor

1. Download (or copy) callTFModel.py python script to the path set in the Exectute Stream Command processor, which is used to call the Tensorflow model.

2. In callTFModel.py, set the URL of the Tensorflow Serving Docker container

import PIL.Image
from PIL import ImageDraw
import numpy
import requests
import time
import json
import sys

imagePath = str(sys.argv[1])
threshold=0.95 
timeTheashold = 2.5

image = PIL.Image.open(imagePath)  
image_np = numpy.array(image)
draw = ImageDraw.Draw(image)

payload = {"instances": [image_np.tolist()]}
start = time.time()
res = requests.post("<URL OF DOCKER CONTAINER>:8501/v1/models/saved_model:predict", json=payload)
processTime = time.time()-start

jsonStr= json.dumps(res.json())
jsonDict = json.loads(jsonStr)

predScore = jsonDict['predictions'][0]['detection_scores'][0]

if((predScore >= threshold) and (timeTheashold >= processTime)):
	response = {"response":"B2Found","confidence": predScore , "duration": processTime }

else:
	response = {"response":"B2NotFound","confidence":predScore,"duration":processTime}

jsonresponse = json.dumps(response)
print(jsonresponse)

Results - Images Are Posted To Slack Channel