Uploaded papers I have finalized for the proposal. I will meet Xunfei to confirm the papers and talk more about the first draft. I have made an appointment with the writing center for the first draft. I spoke to the library desk and Jose regarding the project and got suitable feedback.
I have narrowed my project to studying Facebook Reactions and how reactions may differ based on the gender of the post creator. I have also found papers that focus on facebook reactions. Because Facebook Reactions were released as a feature in 2016, the papers on the subject are limited, and I haven’t found any relating to gender. However, some papers I found analyze facebook reactions in a way that would be interesting to compare between the gender of the post creator. For example, one paper uses the reactions to measure the controversy of a post, so I could measure if posts by women are more controversial in general than that of men. I also found tools from some of these papers that I could use in my project.
After meeting with Xunfei, I decided to modify my diagram a bit so I redesigned it from my practice proposal. I also collected more papers that could be used in my proposal and read more articles and research papers. I also found an online tutorial of a project that is closely related to mine, so I enrolled in the course for free and downloaded the jupyter file to play with it on my own. I think doing this now will help me figure out some possible options and directions for next semester. I also made a timeline of my work for this semester as well as next semester. I asked Xunfei some remaining questions about the proposal and my project in general to clarify my thoughts. I also checked out the rubric for project proposal and brainstormed ideas for my first draft of proposal.
In order to get more familiar with neural networks I decided to use a program that lets you create neural networks. In order to do this I started reading about tensorflow and tensorflow graphs and their inner workings like variables, constants and operations. I read some tutorials on tensorflow and also studied about the Keras model subclassing API which is one of the building blocks of tensor flow to start building a simple neural network. I also read I also searched for more papers that are similar to my research and read Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification, Semantic Clustering and Convolutional Neural Network for Short Text Categorization in order to familiarize myself more with neural networks that are used for text classification.
I have been looking more into the image processing part. I have created my first draft of the code to alter the colors of an image. I have also looked into the rotating of the food in the image. This seems not doable (in the way and scope I wanted to), so I changed my framework to take a video as an input, instead of an image. The video can then be split into images, and the images from the better angels will be picked. I have also written to the Gourmet Food Dataset researchers to ask for their dataset, but have not received a reply yet. I have been looking at the yelp dataset. I have found an online project that assumed all images taken with DSLR cameras were good, and the rest wasn’t. This seems to have worked pretty well for the classifying. I will look into that.
I discussed my new idea with Charlie and Xunfei. I searched for more papers about 3D modeling and volume estimation but could not find a lot. I will be creating an Andriod application, so I looked into Andriod camera API and found that I can specify the distance between the food and phone camera until it satisfies the requirement. I plan to include face recognition as authentication for privacy purposes and found a GitHub repo for it that I can use. I also found a paper that is more closely related than what I have found so far.
After coming back from CMU workshop for CS researches, I have decided to modify my idea a bit to integrate more CV into the project. From recipe recommendation and calorie estimation, I have decided to focus only on calorie estimation. There are many calorie estimation software that requires users to have a reference object when taking a picture of food. As much as this method has brought food calorie estimation to a new level of accuracy, it is inconvenient for users as they need to have the reference object with them at all times.
In my project, I aim to solve this problem as well as to bring the accuracy of calorie estimation to another level. Users will scan the reference object the first time they set up the application. The scanned object will be saved in the database as a 3D object with its area and volume. Next time the user scans the food, the object will appear next to food. These two will be compared and extract the volume of food from it. From volume, the calorie of food will be estimated.
I read and did more research on different algorithms of recipe recommendation. I removed some papers from my box that turned out to be not quite related and added some more papers. I also wrote the final version of my literature review.
During this past week, I have revised how I want to implement my social engineering attack. I want to use what is called a USB rubber ducky where you insert a MicroSD card into the USB. This card has payloads on it which you insert into the victims computer and then the payload is executed. Many different types of payloads can be written. These scripts are written in a language called duck script.
Charlie and I also discussed how to better implement my physical attack. This includes using a wireless adapter as well as ethernet cords to jack into ports around campus and see how easily I can get in.
In the past week, I have:
For this week, I started working on NetLogo, the software that I plan to use to create the simulation model for my project. I looked into the tutorials and the sample models library. NetLogo has its own programming language and development environment, so I spent quite some time to study its ecosystem. I also created a simple simulation model that read a file containing elevation information, display the elevation in different shades of green, built some fire sources and let them spread to places where elevation was smaller than 500. All of my notes for NetLogo can be found in Box.
Charlie and I also discussed my design for the project. For now I will focus on four types of input: Wind, Elevation, Temperature, and Humidity. First I will explore them individually to see how each affects my model. Then I will combine them, two at a time, to explore their combined effect on the model.
I found a MFCC library in GitHub and explored it a little bit. It directly takes a wav file as input and returns one N*1 array (a sequence of acoustic vector). I recorded my voice and converted to a wav file. I briefly tested the code. It took my wav file and return an array containing a sequence of vectors. I will use this library in my project. But there are many related factors that i need to study. I also wrote the timeline for the rest of this semester and next semester. My next step is to keep working on this MFCC library and explore the Dynamic Time Warping library in GitHub.
I finished a first pass of all but one of the papers in my reading list, and also read some of the papers that are highly relevant to my project, which I had already read for a first pass, for a second or third pass depending on how relevant the content seemed. I have also outlined the introduction and motivation for my project, and am working through the related works section. Apart from the project proposal, I have also spent some time trying to find some ‘gadgets’ that will assist with the proof for a simpler variant of parks puzzle, which is an effort that has not yet borne much fruit.
This week I have:
Next week:
This week, I worked more on my proposal outline. Tuesday morning, I met with somebody from EPIC to go over my grant application to go to GDC, at which I may try getting some playtesting data from my project from professionals. This morning, I met with Xunfei to look at my proposal outline before revising and finalizing it.
During this week, I continued work on my literature review after meeting with Xunfei. After finishing the review, I also started work on my proposal outline, and continued looking for more resources to use in my project. I specifically need to find more procedural generation source code for game stages, I’m fairly happy with my two music generation methods.
I am working on finding more papers that study gender bias in social media posts and narrowing my idea further. One challenge is finding a feasible way to collect data from a site (since APIs have limits), or finding an existing data set or web scraper that fits my needs. I am also looking for authors that have published their code for their work and/or who have described their methods in detail.
This past week I worked on revising my literature review as well as writing my proposal outline that will serve as a starting point for my first proposal draft. I met with my advisor who helped me to come up with a good starting dataset for my initial neural network. I will continue read about neural networks and maybe try to implement a simple one in the upcoming weeks.
I went over the broad categories that need to be addressed by the project proposal, and created a proposal outline for Assignment 7. I also worked with my advisor, Igor, to find a good candidate for the reduction adn start working on the proof. We found that there was a natural way of reducing a subset of 3SAT, 2SAT with distinct variables was, to an instance of Parks Puzzle. For next week I hope to generalize the technique to a larger subset of 3SAT.
This week, I changed my method from hybrid to content-based filtering because there isn’t much research done in the hybrid method. So I chose to improve the content-based filtering instead. I also wrote my proposal outline and revised my diagram with the help of Xunfei. I might explore more ways to improve the existing method and see if there is anything else I can add.
For this week, I wrote my proposal outline. During the next two weeks I will use this to construct the 1st draft of my proposal. I also spoke with Charlie about taking a different approach to the social engineering aspect of my project. Most of these I have found YouTube videos to demonstrate and describe the process but I have yet to find any hard research.
I also constructed more details for implementing the technical part of my project. This will also be discussed with Craig.
I finished my proposal outline. The next step is to write my proposal draft. I also discussed with Xunfei and she help me drew a better flowchart. I gained a clearer understanding about the flow of my project. I downloaded a SDK of the iFlytek company’s voiceprint recognizer product for reference.
I met with Charlie for the weekly meeting. We discussed different designs for my simulation model. I will first create some input data, which includes creating dummy values for the base map instead of using a real map. The main program will contain simple transition functions. This is to make sure that I can produce a simple version of a simulation model. I will also have to look more into NetLogo, especially into its fire model libraries.
During my weakly meeting with Igor, he brought to my attention a better way to increment the ranking algorithm. In the first round, a certain number of image processing techniques will be applied to the original image and the top 10 or so images will be passed on to the next round. For each round after, permutations of the image processing techniques will be applied to the images, and the next 10 winners will be promoted to the next round. This way, we can keep applying several techniques on the images, and find the best combination. The process would go on until either the machine has a confidence interval beyond a certain threshold, or a certain number of rounds have passed. The latter is important so the machine does not keep going for ever (or for too long) if the image is simply to bad be made decent. This brings me to the question of what to do if no food is found in the image. Should it return an error, or maybe apply the process to the image and see how it turns out? It is possible that the user submitted an image that contains an unusually morphed food, which the AI might not recognize as food, but still be able to make look good.
I also have heard about genetic algorithms, and will look into those as a safety net/supplement.
I met with Xunfei to improve my design outline for the proposal outline. I completed the outline on the box site. I am trying to find more ways to improve the existing system and see if there is anything else I can add. I found additional papers on QR code security, so I am learning how to avoid thefts with QR codes.
In the past few weeks I have settled on my topic being exploring gender bias on a website using a combination of computational linguistics and quantitative analysis. After writing my literature review on work that explored a variety of sites, I decided this week to focus on a social media site for my project. My next step is to explore more papers focused on analyzing social media and the APIs available for different sites in order to choose which site I want to focus on.
I read over 20 papers in the last two weeks to work on my literature review. I met with my advisor and talked to him about my proposal and what could be improved in my literature review. I have found some projects/papers that touch upon what my project aims to be. This will help me find and establish a starting point once I start working on my project. One of the challenges that I am currently facing is finding a dataset. I have come across a few datasets that I can use from kaagle.com.
This is what my initial model looks like (This does not delve deeper into how the neural networks are configured.)
While preparing my diagram for the quiz, I got a much better conceptual understanding of what I want my project to look like. I have also found nice papers this week. I started thinking of a few different image processing techniques that might help with making the image, and picked a computer vision algorithm for my AI (AlexNet) . I also decided on an image ranking algorithm to decide which image to return, a binary comparison. I feel significantly more comfortable about my project now that I have a more concrete idea for my software architecture, even though I am still fuzzy on the implementation details/
This week, I mainly worked on reviewing my papers carefully and summarizing them for the literature review. I also met and talked with Xunfei about my proposal idea and came up with a preliminary idea. I will have to work more to develop it.
This week, I wrapped up my first draft of the literature review. I’ll be meeting with the writing center as I embark on my final draft. I’ve continued working on nailing down my exact idea that I’ll be proposing, as well as looking into the available resources found throughout the papers I’ve read, from algorithms to source codes. I’ve done some basic work on a prototype game, but have been too busy to make much progress yet. I think a good portion of my project may include comparisons between different methods and combinations of methods between the PCG-G (different algorithms, mostly) and music generation (mostly grammar-based versus machine learning).
This week I worked on improving my understanding of the Parks Puzzle and exploring possible proof techniques to show that it is NP complete from two directions. I continued working on the Time Complexity chapters of “Introduction to The Theory of Computation” by Michael Sipser to round out my theoretical understanding, while also solving many instances of the puzzle using an app on my phone. I came onto one general idea for the proof involving only ‘AND’ and ‘OR’ gadgets that I discussed with my advisor, who made some suggestions involving an ‘IFF’ gadget, which I am going to continue working on. I also received feedback on my literature review, which showed some significant problems that I corrected according to the grading rubric.
I’ve finished writing the Literature Review for my idea “Fire Spread Simulation Using Cellular Automata.” After reading the papers for my research, I found a recent paper on this topic which used Machine Learning to solve the drawbacks of previous research. However, I could only find one paper using this technique so I will have to dig deeper to find more related materials. Charlie has suggested that I should categorize the papers based on the input data (terrain, weather condition, etc).
I’ve chosen “Fire Simulation Using Cellular Automata” as my final idea. I have also met with Charlie and decided to meet every week on Monday. I will also meet with Xunfei regularly for my research. Xunfei has suggested that I should look into ArcGIS for the simulation part of the research and also suggested me to talk to Jose as ArcGIS would require funding.
I finalized my proposal to “Applying Voiceprint Recognition Technology to Identity Verification”. The keywords are voice recognition, voiceprint, feature extraction, voice detection, voice verification. The difficulty I might encounter is that there may be background noise in the voice input. If the noise is loud, it may affect the feature extraction and voice recognition. I probably need to explore methods for removing noise.
Received feedback for LR. I understood the mistakes I made and working on the revised LR. I also found additional papers that I will use. Met with Xunfei to discuss proposal outline and LR.
I did not add an update to week 6 due to the long weekend but had been working on my literature review, which I finished today. It was very useful to read and re-read certain articles and realise some are useful and some are not. I now have further inspiration with where I can take my idea and am happy with its process. I will be looking in the next week or two to start looking into potential technologies to use for my project, which currently seems to be leaning on public Python libraries.
I have decided on the project I will be working on as my senior project. I have talked to Charlie about it, discuss my ideas regarding this project. He will be my advisor for the project. I have found 10 more papers and a couple of technologies I might be using. I have also found the datasets of food and recipes I will be using for my project.
My final idea is nutrition management and recipe recommendation system. Users will be able to scan the ingredients they have using the app and the app will recommend recipes using the user input they have put before such as any allergies, or food they don’t want to or cannot consume. The next step of my project will be the calorie estimation of food the user will consume. For this part, I plan to use a texture mapping and scanning for the optimum estimation of calories, and ingredients. For the privacy issues, I plan to have users scan their face on the first use of the app and have an API that will determine whether the current user is the user of this account. I am still thinking about possible ways to detect liquid ingredients and seasonings of the food.
I now know what idea I am going to go with, it’s a new idea and is not related to any of my old ideas. My new idea is about using neural networks and natural language processing to predict a better way to write emails or other forms of text in order to better engage the reader. This will be focused on business emails and other forms of business-related texts. I have read a lot of papers on neural networks in the past week and have spent most of my time writing my literature review on it.
I have talked to Xunfei about my 3 ideas and decided to discard one of my 3 ideas because of overhead issues. I wrote 2 annotated bibliographies of my 2 ideas with 3 papers each. I have also done more research on my 2 ideas.
I wrote 3 annotated bibliographies, one for each idea, and each of the annotated bibliographies is composed of 2 papers I have found for my ideas. I went to San Diego to attend Tapia conference of diversity in computing.
I’ve finalized the base project idea – I’m going with the one involving music generating AI. Additionally, I’ve officially gotten a proposal adviser, Xunfei, who I’ll be meeting with every Wednesday morning. I’m having some issues in limiting the scope and application of my project, which I’ll be focusing on while I finish up my literature review. In terms of the review, I’ve read 9 of the 10 papers, so I only need to read the last one and put the notes I have into the literature review format.
My project is using Voice Print Recognition technology to check if the voiceprint of the input match the corresponding one in the database. This technology can be used in many identity verification scenes like customer services for bank, door lock, business transaction. The main steps approximately will be: take voice input -> (remove noise -> ) extract voice features -> building models with selected algorithms -> compare voice features -> check if voiceprint match. The possible algorithms might be: VQ, MFCC, DTW.
This week I mainly finished my LR and spoke in detail regarding the project idea with Xunfei. It seems I like I know which technology to progress with and eliminate the two I was debating before. I also worked on my basic design outline.
I read more papers on my ideas, met with my advisor to schedule a weekly meeting and discussed a new idea proposed by the company I interned with this summer.
The new idea is a Neural Network driven A.I. that can learn and predict better ways to write emails, marketing campaigns and other forms of communication with the customer.
This week, I finalized who my advisor will be (Charlie). I also decided that I will be working on my security testing idea as my main project. To start this, I spoke with Brendan Post (IT) to discuss my ideas. He was happy to help me and I will be in further discussion with him as I move forward with my proposal.
I also found a couple other papers related to my idea. There was one that had much more lower level detail and actuially described the implementation of their testing. The researchers used Kali Linux to hack into a router through different ways such as SSH, Telnet, and SNMP. There were images that showed the commands they used. It was the first article I found to have a lot of low-level detail.
This week I spent some time finalizing my proposal idea. I discussed the Parks Puzzle with Igor and come to the conclusion that I should work on proving its NP-Completeness for my final project. I had to discuss this idea with my advisor Igor, as well as Charlie and Xunfei before I could finalize this plan. Once I had this confirmed by Xunfei, I put aside my work on the other ideas and started to solely focus on NP-Completeness. My first task is to go through the relevant chapters of “Introduction to The Theory of Computation” by Michael Sipser nad working through problems to clear up my understanding of the problem, which I have started to work on.
In the past week, I searched and skimmed lots of papers to use for my own research. I also talked with Dave and Xunfei to refine my ideas. I also found a data scraping software to extract product information from Sephora.
I am going with the auto-image processing for food. I have been looking more into the literature, and there is limited literature that deals with making food look better, and they are all relatively recent publications. However, there is literature about how to train machines to assess how aesthetically pleasing images of food are, and also on how to make regular images more aesthetically pleasing.
I will start experimenting with different machine learning algorithms, and also image processing techniques.