Thready – Threat Detection Using Drones

This was our submission to IBM Drone Hackathon held on 30-31 January during ηvision2016 at IIT Hyderabad. You can access the project folder on my Github account here.

Prologue

We received a mail regarding the Drone Ideathon (Phase-I) in the last week of December, 2015. Students had to submit ideas as to what would they use a drone for, which would then be reviewed by IBM delegates.

Around 20 ideas were submitted out of which 5 were selected for the Phase-II (hackathon). These were the teams which IBM would be actually providing drones to work on. Ours was one of them. We had around 2 weeks to prepare a working demonstration and test it.

IBM provided us with one DJI Phantom 3 and one Parrot AR 2.0 drone. We were in a team of 2 consisting me and my friend Prathmesh.

The Project

Our idea was to develop an IoT device for in-house detection of threat using drones named ‘Thready‘. For our prototype, we chose to focus on the concentration of poisonous gases in the atmosphere. It was somewhere inspired by the recent Pathankot attack incident where a soldier died while he was inspecting the dead terrorists and a triggered-bomb exploded.

Hardware – 

Firstly, we needed stack where we can connect our sensors to wifi. The first plausible option that comes is to use Raspberry Pi. But the sensors being used were analog in nature. Raspberry Pi is quite complicated to work around with common sensors.

Another option was to use ESP2866 wifi module for Arduino but one would rather build a time machine that to figure out how to work with this (sarcasm intended!). Given the strict timeline of the project besides hectic academic schedule, we certainly had to look for alternatives.

Particle Photon 

‘Particle Photon‘ came as a savior for us. Its a recently launched Arduino-based micro-controller with inbuilt wifi module. All you’ve to do is register your device on https://build.particle.io with provided credentials.

There you can start a project and compile your codes in their inbuilt editor. Clicking on the settings tab, you can know your ‘device id’ and ‘access token’ which would be required in the later stage.

It also has a dedicated android app for you to connect your particle device with the wifi router. You can find it on Play Store here. Once connected, particle will start emitting its own signal which you can connect to and kickstart your IoT project.

Circuit –

Our circuit was quite simple. We were using MQ-2 MQ2 Gas Sensor Module for smoke detection. It can measure the concentrations of upto 8 distinct gases given you can distinguish or better use different unit for each gas you want to measure.

This would have certainly complicated the circuit. We decide to go with measuring ‘carbon dioxide‘ measurement for this was one gas we could easily test out for. Just blow your breath out and there the concentration shoots up FTW!

Since we were developing an in-house application, we thought of incorporating obstacle avoidance technique but that meant delving into the whole complex drone-working mechanism which obviously we didn’t have the time for. Nonetheless, I had procured an HC-SR04 Ultrasonic Distance Measuring Transducer.

It didn’t take me much time to figure out working with it. Just make few connections, write a simple Arduino code and there you have any obstacle distance displayed on the serial monitor. Knowing something doesn’t go wasted and I used this sensor in my ‘Smart Dustbin’ project for Inter-IIT Tech Meet. You can find the sample code here.

	#define trigPin 13
	#define echoPin 12
	#define led 11
	#define led2 10
	#define threshold_distance 10
	void setup() {
	Serial.begin (9600);
	pinMode(trigPin, OUTPUT);
	pinMode(echoPin, INPUT);
	pinMode(led, OUTPUT);
	pinMode(led2, OUTPUT);
	}
	void loop() {
	long duration, distance;
	digitalWrite(trigPin, LOW);
	delayMicroseconds(2);
	digitalWrite(trigPin, HIGH);
	// delayMicroseconds(1000);
	delayMicroseconds(10);
	digitalWrite(trigPin, LOW);
	duration = pulseIn(echoPin, HIGH);
	distance = (duration/2) / 29.1;
	if (distance < 4) { // This is where the LED On/Off happens
	digitalWrite(led,HIGH); // When the Red condition is met, the Green LED should turn off
	digitalWrite(led2,LOW);
	}
	else {
	digitalWrite(led,LOW);
	digitalWrite(led2,HIGH);
	}
	if (distance >= 200 || distance <= 0){
	Serial.println("Out of range");
	}
	else {
	Serial.print(distance);
	Serial.println(" cm");
	}
	if(distance < threshold_distance){
	print();
	}
	delay(500);
	}
	void print(){
	Serial.print("Dustbin Full");
	Serial.print("\n");
	}

view raw

distance.ino

hosted with ❤ by GitHub

Interface 

Normal users don’t understand the code and stuff. You have to provide them with simple interface that shows how plausible the danger is.

Once we got the sensor data successfully displayed on the serial monitor, our next task was to transmit it over wifi to be displayed on the web & Android app we made.

Back-end

Particle photon has amazing support for code deployments and querying. Obviously we had to read its documentation to know all of its feature since we were using it for the first time.

Initially we wanted to use something like Flaskr – A SQLite based blogging application based on Flask (A python micro-framework for web) for our web app. Its because we wanted to store our data over time as well. Flask is quite easy to use and you may have your web app running in no time. But beware if you’re new to this. Some part of the flask server setup may bug you to hell even if you think you’re following the documentation properly. You can read my experience here.

After lots of head-scratching moments, we chucked the idea of using flask considering the time it would have taken to code the database support and get it working properly. Reading through the ‘particle photon’ documentation, we came across Particle Cloud API – https://api.particle.io/v1/devices/230024001447343339383037/events/?access_token=77394ac5db47xxxxxxxxxxxxxxx6c74f5c1d15c7 (its ours, you can’t see the access token :P)

Though normally one would have gone with API testing alternatives like hurl.it or requestbin but Particle Cloud API was perfect for us. Learning about the data publishing using spark photon was a challenge in itself. You can read snippets of my on-the-go learning experience here.

After lots of juggling through forums and discussion, we found the trick that did the work – spark.publish() function.

	#include "application.h"
	unsigned int nextTime = 0; // Next time to contact the server
	const int gasOutput = A0;
	// int DangerLight = D7;
	float MidConcThreshold = 650;
	// float HighConcThreshold =1000;
	float CurrentValue=0;
	int i=0;
	void setup() {
	Serial.begin(9600);
	// pinMode(DangerLight, OUTPUT);
	}
	void loop() {
	CurrentValue = analogRead(gasOutput);
	Serial.println("Keep Calm. no danger !");
	Spark.publish("Gas_Concentration",String(CurrentValue));
	delay(1000);
	}

view raw

detection.ino

hosted with ❤ by GitHub

Front-end

This was a cake-walk for me once we figured out the back-end. I have been into web development for around two years now. Being a design enthusiast, I knew how I want my interface to be. But wait there is a catch!

The chart. Yes this stupid chart gave me a hard time. By now the gas concentration was successfully being displayed on the interface but I wanted a real-time dynamic chart rendering to let the user easily visualize the data. Once the value exceeds the threshold (which may differ place to place which you could measure by flying the drone for sometime and noting down the average of the data – we were able to successfully answer it when asked by the evaluation team), the data and chart changes color, preferable the line & text turns red.

First I needed to decide what kind of chart would be the perfect fit. Initially I though pie-chart would be the best with its segment expanding and contracting with the values. But there was another learning chapter waiting for us to read.

I had thought using Google charts would do the work easily but that was my misconception. I couldn’t find any effective method to display a dynamic chart in the entire documentation. Even if I did somewhere hidden in the distant corners of the forums, it didn’t work out and I was stuck. Jumping between different types of charts and experimenting the code everytime only wasted our precious time. But it was a lesson worth to take.

Then I moved on to the alternatives like plot.ly and HighCharts. Since the time was less, I thought of sticking with the one that looked more promising – it was HighCharts for me.

I wrote the JavaScript code for querying the Cloud API URL mentioned earlier and get the data in JSON format. Parsing the JSON was easy as I had done it numerous times earlier. Further I wrote the code for chart rendering and fed the data to it in terms of variable I had stored the concentration in. Also I switched to more interactive ‘line chart‘ which was being updated with the upcoming data over time-axis.

When you’ve done everything right, you expect the system to work but when it doesn’t, you feel miserable. I was facing the similar situation when I was expecting the real time chart to be displayed successfully on the webpage but everytime the variable changed, the line disappeared and only the timeline seemed to be moving.

I was unable to figure out where the problem was even after lots of debugging and adding breakpoints. In the scarcity of time, I tweaked with the code numerous times even doing the changes where it was least expected, yet no result.

Finally I decided to round off the concentration value using Math.round() method, just to see since nothing was working out. I re-ran the web app and voila! There it was – my beautiful dynamic chart being plotted in real time.

var x = (new Date()).getTime(), // current time
y = Math.round(parseInt(conc_value)/10);

That was probably the most relieving ‘localhost:3000‘ I visited in recent days. I still haven’t figure out as to why only rounded value worked out but I was finally ready for my confident presentation with a working prototype besides. Finally it was ready for actual testing on drone.

Mounting

We tested our project on DJI Phantom 3 drone. The organizers have made a packaging box where our equipment could be kept and then it was firmly mounted on the drone. I quickly setup the connections, dumped the code and ran the web app.

As the drone flied, we were able to see the concentration continuously changing over time. Since we were flying outdoor where it was a bit dusty, I could see the large variation in the data but monitoring the gas-prone zones in the campus was sometimes I was least concerned about, at that time.

Future Work

• Make it more portable. Getting rid of the wired connection and making a compact circuit on a PCB.

• As pointed earlier, we wanted to incorporated obstacle avoidance feature. This is really a big area where the researchers are still working on. DJI Phantom 4 was recently launched with obstacle avoiding object tracking functionality.

• The database for storing the fluctuation over a large time frame to monitor the environment of a location. This would be probably the first and the easiest task to accomplish.

Learning

This project was a huge learning experience for me. Given the compact timeline, figuring out things (when you’re working with them for the first time) becomes a bigger challenge. There were times when we felt helpless given no one was there to guide us. The relief that I got after successful demonstration in indescribable but it has a long way to go.

When you build something end-to-end, only then you know what it takes to get the entire system working.