EatTrack

Logo: Tracking Location by Friedrich Santana from the Noun Project
Hello world
stats website
ask quantity
guess from image

Inspiration

During a conversation with a friend, we realized how hard was to keep track of your diet. Right now, the vast majority of apps to track what do you eat require a lot of effort from the user: manually input product name, expect the app to know what is it, then try to estimate how much did you take from that. It can be really difficult to keep it, requires a really strong will.

Following that problem, I decided to try to build a bot that minimizes the number of interactions required to get a result.

What it does

EatTracker is an almost public messenger bot that recognizes tracks the calories you take. It does it by scanning food from the pictures you send him and approximate quantity you send him. EatTracker gives you an immediate approximate number of calories. In addition, it also offers nice dashboard to check your eating history, together with a (for the moment) small amount of statistics.

How I built it

The whole backend runs on Django. For the image recognition process, I used Clarifai Food pre-trained neural network, through its REST API. To extract average nutrition values, I used Wolfram Alpha results API. In order to build the bot, I set up a Messenger bot plugged to the backend previously mentioned.

Challenges I ran into

Probably the most challenging part was to use the predicted data and improve it with user input, minimizing the user efforts. Also, my initial idea was to build it on my own partial-trained convolutional neural network. However, due to my laptop having low resources (4Gb of RAM) and taking forever to install tensor_flow source library, I decided to avoid doing it on my own, as I wasn't certain that it would be ready on time. In addition, my teammates, who knew more than me regarding neural networks, disappeared, which made me realize I had to reduce the scope of my project.

Accomplishments that I'm proud of

Some of my favorite accomplishment is probably the "interface"/"interaction". I really think that its fast process helps the user in a great way.

What I learned

I've never programmed a bot before, especially a Messenger bot. I was really looking forward to do it and I'm glad I did it. Also, even though I couldn't apply it to this project, I learned the basics of a neural network. Specially I learned a bit more about convolutional networks and how to do knowledge transfer process retraining the last layer for a different feature recognition (my plan was to do this with the 101_Food public dataset and Google's pre-trained Inception v3 network).

What's next for EatTrack

Train the neural network using user's live feedback to improve the classification
Use a self-hosted neural network
Add statistics to user dashboard
Use user data to provide feedback on diet and improve the user diet
Predict future diet distortions in user diet
Add gamification to improve user engagement
Add product classification (vegetable, meat, fruit...) in order to study user's habits
Create models of eating habits for users (hourly...)
MAKE THE BOT PUBLIC ONCE FACEBOOK ACCEPTS ME

I guess there would be even more ways to continue with EatTrack, but that's all I got in my head right now.

Logo: Tracking Location by Friedrich Santana from the Noun Project

Also: bought eattrack.tech but it's not a good idea to change domain of a bot at this hours :D