I realized that the database I found for food nutrition is not enough for the calorie estimation so I looked for more resources. I started learning SQL.

Finished propsal first draft. Explored softwares for image processing.

I discussed my new idea with Charlie and Xunfei. I searched for more papers about 3D modeling and volume estimation but could not find a lot. I will be creating an Andriod application, so I looked into Andriod camera API and found that I can specify the distance between the food and phone camera until it satisfies the requirement. I plan to include face recognition as authentication for privacy purposes and found a GitHub repo for it that I can use. I also found a paper that is more closely related than what I have found so far.

After coming back from CMU workshop for CS researches, I have decided to modify my idea a bit to integrate more CV into the project. From recipe recommendation and calorie estimation, I have decided to focus only on calorie estimation. There are many calorie estimation software that requires users to have a reference object when taking a picture of food. As much as this method has brought food calorie estimation to a new level of accuracy, it is inconvenient for users as they need to have the reference object with them at all times.

In my project, I aim to solve this problem as well as to bring the accuracy of calorie estimation to another level. Users will scan the reference object the first time they set up the application. The scanned object will be saved in the database as a 3D object with its area and volume. Next time the user scans the food, the object will appear next to food. These two will be compared and extract the volume of food from it. From volume, the calorie of food will be estimated.

I read and did more research on different algorithms of recipe recommendation. I removed some papers from my box that turned out to be not quite related and added some more papers. I also wrote the final version of my literature review.

I have decided on the project I will be working on as my senior project. I have talked to Charlie about it, discuss my ideas regarding this project. He will be my advisor for the project. I have found 10 more papers and a couple of technologies I might be using. I have also found the datasets of food and recipes I will be using for my project.

My final idea is nutrition management and recipe recommendation system. Users will be able to scan the ingredients they have using the app and the app will recommend recipes using the user input they have put before such as any allergies, or food they don’t want to or cannot consume. The next step of my project will be the calorie estimation of food the user will consume. For this part, I plan to use a texture mapping and scanning for the optimum estimation of calories, and ingredients. For the privacy issues, I plan to have users scan their face on the first use of the app and have an API that will determine whether the current user is the user of this account. I am still thinking about possible ways to detect liquid ingredients and seasonings of the food.

I have talked to Xunfei about my 3 ideas and decided to discard one of my 3 ideas because of overhead issues. I wrote 2 annotated bibliographies of my 2 ideas with 3 papers each. I have also done more research on my 2 ideas.

I wrote 3 annotated bibliographies, one for each idea, and each of the annotated bibliographies is composed of 2 papers I have found for my ideas. I went to San Diego to attend Tapia conference of diversity in computing.

Talked to Charlie about my ideas and got feedback from him. Found 5 papers for each idea.

1. My first idea is to create an application that scans the picture of foods and let users know what ingredients are in the dish. I am still deciding whether I want my app to be used as a diet and nutrition guide or aid for visually impaired people. This application will be available in multiple languages (at least 3). Xunfei had provided me more questions to explore as feedback. I will integrate computer vision and natural language processing in this project.
2. My second idea is a program that detects prank calls made to 911 or other emergency centers. I will focus on details of the caller’s speech such as the urgency, intonations, breathing, etc. as well as background noises like whether the background is too quiet or too loud or is there any footsteps, etc.
3. My third idea is to generate speech from a user’s hand gestures in several languages. I plan to piggyback an already existing and working program that translates hand gestures to speech. My main focus will be on improving that program and working to the accuracy of language translations.