This week I worked on the final proposal. Feedback from the second draft indicated that there were some grammar and structure errors. I added the testing and abstract sections. I hope to finish the proposal by the end of this week.
- I focused this week on fixing my first proposal.
- I re-did all of my diagrams so that they would use the proper shapes
- I re-wrote my design section
- I add more to my introduction to better explain the importance and the gaps
- I elaborated about the timeline and gave a high level overview by month
- I also did research into the postgres database using SQL because that seems like the best tool for my project.
- Next week over the break, I hope to go more in depth into my readings and start to finalize the tools I want to use
I read some new papers and research about different modeling algorithms and started to worry about the accuracy on my system. The accuracy is not only rely on the modeling but also based on the dataset for training and the quality of acoustic input (the speaking environment). But selecting a suitable modeling algorithm is important. Now the popular models are: HMM, VQ, DTW, GMM, UBM, i-Vector. I temporarily chose hybrid GMM-UBM. I might change in the future or mix other modeling to enhance the accuracy. My goal is to reach an accuracy at least 90%.
This week, I continued working on my project proposal, submitting my second draft after some much-needed updates. I still need to work further on the Related Works section. I additionally continued working on early implementation of the project. Lastly, I prepared a first draft of my presentation slides.
I have mostly worked on the second draft of my proposal in the past week. I simplified and restructured my framework model. I also looked for more use cases of my project and decided what frameworks to use while implementing it.
This week I have not been able to do much progress. I have decided to scrap the idea to use Machine Learning in the module for altering images, due to difficulty in implementation. Besides that, I have worked on the second draft of my proposal.
This week, I worked on the similar project posted online. While working on it, I found some challenges in modifying the content-based data set to fit the collaborative-filtering method. I might end up modifying my project from a hybrid recommender to a content-based recommender. But I will keep looking for alternatives to make it possible.
During this week I met with Charlie to review feedback for the first draft of the proposal. I reconstructed this version for the 2nd draft. I also read some papers for the next pass. These were more directly related to the software I am going to use.
I found an API for food calories. I worked on proposal second draft. I explored an image editing software I am thinking to use.
Still working on the second draft. Rebuilding the diagram of SLMS. Also ready with the draft for the presentation.
I realized that the database I found for food nutrition is not enough for the calorie estimation so I looked for more resources. I started learning SQL.
I discussed my proposal draft with my advisor. I got her feedback and suggestion, and knew how to revise and improve my proposal. In the past week, I read more papers about the GMM-UBM modeling method that I plan to use for my project. I understood the specific procedure now but it is still hard to fully understand this principle… Now my another problem is to find a suitable dataset and decide if my system is text-dependent. There are three primary ways for speaker verification now: text-dependent, mixed, text-independent. The text-independent way is very difficult and complicated to do because user can say anything to pass the verification. But text-dependent way is restricted and not safe for spoofing attacks. For example, people can replay pre-recorded voice to pass the verification. Therefore, the mixed way is better. It restricts the text in a way but safe for spoofing attacks. For example, they user can only speak numbers one – ten, but every time the text is random. But it is hard to find a dataset of all audio file in numbers in English. Now I need to decide which text way my system will use.
This previous week, the work I’ve done has been two-pronged, as has become the norm and will continue to be for the rest of this semester. First, I continued work on the basic implementation of the game. I currently have the control module working, as well as a looping stage that I created in order to test the controls. On the proposal side, I’ve been making edits based on the in-class peer review that we did, as well as working more recently based on the feedback given by Xunfei. I also met with Xunfei to go over her feedback of my first draft, and updated her on my progress.
This week, I investigated the technologies being used in my found papers more closely to find which technologies would be more feasible for my project. For data collection, I have found that the facebook-sdk python library (https://pypi.org/project/facebook-sdk/) used by Pool and Nissim is the best option to connect to the Facebook Graph API, since it looks well documented and has all the options I might need. I also decided to use the Facebook Pages of politicians as my dataset. I reread As the Tweet, So the Reply?: Gender Bias in Digital Communication with Politicians by Mertens et al. to see if their methods could be adapted to my project. I will need to look at their references for methods in more detail to see if I can feasibly apply them to my project.
This past week, I worked on mainly reading my new papers. I did a third pass reading on all my old papers and did at least second pass reading on the new ones. I tried finishing more than half of the existing project on python notebook and played with the data set. I now have a better sense of how to start my project next semester. I also met Xunfei and updated my progress to her. As soon as the feedback for proposal draft 1 comes out, I will be revising my writing and finishing the existing project I have been working on. I also plan to test the existing project on collaborative filtering to make sure it works with a different data set.
In the past week, I have spent most of my time working on the first draft of the proposal. I decided to research and include a new category of papers in my proposal that I had not spent a lot of time before on. The new category that I included was “Sentiment Analysis.” While working on the proposal and refining the design of my framework, I realized that sentiment analysis, something that has been thoroughly covered by researchers of neural networks is very close to my research since I also need to know the sentiment behind the email/piece of text that is to be improved.
This week I have
- Continue with the Operating Systems with the focus on File Systems through studying and reimplementing the Unix Version 6.
- Continue with the Networking parts.
This week I have:
- Studied the general knowledge of blockchain
- Read about File System in an Operating System
- Read about the basics of Computer Networks
In the past week, I have used my peer review from Jordan, as well as my own proof-reading of a physical copy of my draft to fix a lot of errors. I wrote my draft in a bit of a rush, and as a result, there were a lot of formatting errors, most of which I have now fixed. I have also updated some of my diagrams in accordance with feedback I have received and expanded some content in my draft that needed to be clarified.
In addition to working on my draft, I have been working on my project itself (preliminary work can be found in the git repo). I created a mockup GUI to give me some ideas about how I want to design the actual version next semester, as well as testing some implementations of different filters, operators, and edge detectors. Some of these results will hopefully be represented in the next version of my draft.
During the past week, I finished the first draft of my proposal and started to make those changes for the second draft. I have also continued reading some papers for their next pass. I continued to watch videos and read content related to the USB Rubber Ducky. I have started to put together some scripts that I would like to use for the attack. I also spoke with Charlie to refine my methods for the physical attacks I am going to implement. I now have a better/ more related CS implementation for this attack than what I previously had. During this next week, I am going to be working more with Metasploit on Kali Linux.
For this week I looked into many different datasets, including gis fire map data https://frap.fire.ca.gov/mapping/gis-data/ and Kaggle dataset https://www.kaggle.com/elikplim/forest-fires-data-set but couldn’t find what I was looking for.
I picked The Ranch Fire in California but couldn’t find good datasets for it. I was trying to find I’m trying to find elevation, wind direction, humidity, and vegetation.
All of them have to contain coordinates so I can layer them together. Also, I need each set at different time stamps for the simulation. I will discuss this during the next weekly meeting.
For this week I have done the following tasks:
Learned to view shapefile contents with Netlogo and Python library pyshp. A shapefile is an Esri vector data storage format for storing the location, shape, and attributes of geographic features. This type of file is quite complicated so it took me sometimes to understand the format and its contents.
I had trouble finding the right dataset for my project. Charlie suggested that I look into https://www.frames.gov/afsc/partners/fmac/guides-products. I downloaded the data for Alaska but it does not have the contents that I was looking for. Finding the right data is currently a big challenge.
I am also getting more familiar with Netlogo. Using Netlogo, I could view the content of the data for Alaska and also extract the metadata using the command line provided with NetLogo.
I have been continuing learning machine learning with Python, specifically PyTorch. I started with PyTorch because it has a less steep learning curve compared to Tensorflow (the alternative). However, there are more tutorials for Tensorflow and I might pivot next semester as the image processing gets more complicated and I need more resources in incorporating image processing into the machine learning. I think I will be able to build both a ResNet and AlexNet algorithm and compare them to decide which one to use. I have also written the code for video editing in Python to convert the input video into frames. For this task I am using OpenCV. It is straightforward to do this. I have not yet decided how many frames I will take in the first round.
- I spent a lot of time this week trying to closely read the texts I’ve found already to try and find any mention of the data set they are using. This was very difficult because the research articles usually don’t name what their data set was called or don’t explain where to find the data set they were using. There is not a lot of details in these papers about the researchers’ process and methodology in a way that would allow me to replicate their results. This made finding fake news data sets extremely difficult. However, through the close reading and intense web searches, I have found 21 fake news related data sets.
- I also spent a lot of time researching what would perhaps be the best machine learning tool to use for my project. I’ve narrowed it down to these possibilities: Oryx 2, Tensorflow, Azure ML Studio, Weka, Shogun, AWS CLI, TensorBoard, Kerras, Caffe2. I think that I might be able to use more than one for my project to get the best results but more research still needs to be done about which tool is better for the type of data set I have (which is not a timeseries data set).
Discussed the feedback on the first draft with Xunfei. Got valuable feedback and planning to implement them in the second draft. I am searching for the algorithms to be used in the SLMS and working on the second draft.
Finished propsal first draft. Explored softwares for image processing.
- Started my work in reviewing new found research that has more relevant research about a fake news detector application
- Finished my first draft for my project proposal
- I will need to update my related works section because of the new research I’ve found
- After the peer review session, I will need to go back and redesign my figures so that they are easier to read
- Started finding datasets which is very difficult because the research papers never tell you where to find the data set they use and often they never name the dataset either.
- However, I was able to find some github repositories with datasets and some websites of the authors in the research papers that actually linked to the dataset of their works
Finished my first draft of proposal. I read some blogs about speaker verification tech and found out that I was wrong on some aspects (actually I was confused). Those blogs help me understand more and deeper about speaker verification. So I revised my framework and flowcharts: take voice input -> feature extraction -> modeling -> database. The modeling part is the most difficult part in speaker verification. The most popular models are: Hidden Markov Model, Gaussian Mixture Model, Vector Quantization, etc. I am not sure which one I will use for sure. It all depends on my dataset and customer need. I need to experiment several models to know which one I want the best. But I chose GMM temporarily on my proposal.
I only worked on finishing the first draft of the proposal this week.
This past week, I’ve finalized the basic design for the game I will be implementing. It will be a horizontal auto-runner, where the player ducks/jumps to avoid obstacles to the beat of the music in order to keep playing. I continued familiarizing myself with Unity2D, and plan on starting work on the game this upcoming week. Additionally, I wrote up the first draft of my project proposal.
This past week, my work has been split in two directions: First, I’ve been refamiliarizing myself with Unity, by means of going through my Game Design second project. Further than that, I’ve been familiarizing myself with Unity2D for the first time, which I plan on using for the senior project due to the simplicity as compared to Unity3D. Besides getting used to the main software engine I will be using, I also continued reflection on my proposal outline; I’ve been looking more into different PCG-G algorithms and have decided on using the chunk paradigm as my second stage generation algorithm. Its stages won’t be as directly aligned to the music, but it should improve efficiency.
I extensively worked on the proposal last week, reading more papers and writing out what I plan to do helped me figure out the scope of the proposed project. I also experimented a bit more with tensorflow. I made some changes to my initial framework design to now include a frontend and backend for the end-user to interact with.
This week, I revised my diagram again and selected papers to use for my proposal. I read through all of them carefully and wrote an outline for my proposal. I discussed the details of my ideas with Xunfei, modified some, and finished writing the proposal.
Much of my work this week has consisted on working through the first draft of the proposal as well as reading some papers for the next pass. I also found a couple other sources to use. These new sources were not research papers but rather articles related to my project.
I finally received the Gourmet Dataset. In fact, I received a devised version that has twice as many images as the original one. I also have the Yelp dataset, although that dataset has not be curated by humans, I am hoping to use it for training my algorithm in addition/instead of ImageNet or AVA.
Since I already have gotten access to the datasets, I have been reading about ResNet/AlexNet implementations, which was my goal for next week.
This week I met with Charlie to discuss my project design. We also talked about GIS extension, which is a library to handle GIS data for NetLogo. Charlie talked about how to layer different types of data on a base map. The most important tasks for the upcoming weeks are to figure out how to find different types of data for a fire location and how to process the data.
I worked extensively on finishing the first draft. I rewrote the design and related works section.
Uploaded papers I have finalized for the proposal. I will meet Xunfei to confirm the papers and talk more about the first draft. I have made an appointment with the writing center for the first draft. I spoke to the library desk and Jose regarding the project and got suitable feedback.
I have narrowed my project to studying Facebook Reactions and how reactions may differ based on the gender of the post creator. I have also found papers that focus on facebook reactions. Because Facebook Reactions were released as a feature in 2016, the papers on the subject are limited, and I haven’t found any relating to gender. However, some papers I found analyze facebook reactions in a way that would be interesting to compare between the gender of the post creator. For example, one paper uses the reactions to measure the controversy of a post, so I could measure if posts by women are more controversial in general than that of men. I also found tools from some of these papers that I could use in my project.
After meeting with Xunfei, I decided to modify my diagram a bit so I redesigned it from my practice proposal. I also collected more papers that could be used in my proposal and read more articles and research papers. I also found an online tutorial of a project that is closely related to mine, so I enrolled in the course for free and downloaded the jupyter file to play with it on my own. I think doing this now will help me figure out some possible options and directions for next semester. I also made a timeline of my work for this semester as well as next semester. I asked Xunfei some remaining questions about the proposal and my project in general to clarify my thoughts. I also checked out the rubric for project proposal and brainstormed ideas for my first draft of proposal.
In order to get more familiar with neural networks I decided to use a program that lets you create neural networks. In order to do this I started reading about tensorflow and tensorflow graphs and their inner workings like variables, constants and operations. I read some tutorials on tensorflow and also studied about the Keras model subclassing API which is one of the building blocks of tensor flow to start building a simple neural network. I also read I also searched for more papers that are similar to my research and read Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification, Semantic Clustering and Convolutional Neural Network for Short Text Categorization in order to familiarize myself more with neural networks that are used for text classification.
- This past week I have been trying to find more research papers that discuss how to create a prototype fake news detector instead of papers that just talk about how to discern fake news from media.
- The hunt for those kinds of papers is more difficult than the one for methods of fake news detection which tells me that my work in creating a user application for fake news detection is very much needed.
- It also seems like a lot of these papers are geared specifically towards twitter and so hopefully my research can fill a gap.
- I have also been trying to figure out a good timeline for myself both for this semester and the next. I do believe this project is feasible if I can just find some data sets in a timely manner.
I have been looking more into the image processing part. I have created my first draft of the code to alter the colors of an image. I have also looked into the rotating of the food in the image. This seems not doable (in the way and scope I wanted to), so I changed my framework to take a video as an input, instead of an image. The video can then be split into images, and the images from the better angels will be picked. I have also written to the Gourmet Food Dataset researchers to ask for their dataset, but have not received a reply yet. I have been looking at the yelp dataset. I have found an online project that assumed all images taken with DSLR cameras were good, and the rest wasn’t. This seems to have worked pretty well for the classifying. I will look into that.
I discussed my new idea with Charlie and Xunfei. I searched for more papers about 3D modeling and volume estimation but could not find a lot. I will be creating an Andriod application, so I looked into Andriod camera API and found that I can specify the distance between the food and phone camera until it satisfies the requirement. I plan to include face recognition as authentication for privacy purposes and found a GitHub repo for it that I can use. I also found a paper that is more closely related than what I have found so far.
After coming back from CMU workshop for CS researches, I have decided to modify my idea a bit to integrate more CV into the project. From recipe recommendation and calorie estimation, I have decided to focus only on calorie estimation. There are many calorie estimation software that requires users to have a reference object when taking a picture of food. As much as this method has brought food calorie estimation to a new level of accuracy, it is inconvenient for users as they need to have the reference object with them at all times.
In my project, I aim to solve this problem as well as to bring the accuracy of calorie estimation to another level. Users will scan the reference object the first time they set up the application. The scanned object will be saved in the database as a 3D object with its area and volume. Next time the user scans the food, the object will appear next to food. These two will be compared and extract the volume of food from it. From volume, the calorie of food will be estimated.
I read and did more research on different algorithms of recipe recommendation. I removed some papers from my box that turned out to be not quite related and added some more papers. I also wrote the final version of my literature review.
During this past week, I have revised how I want to implement my social engineering attack. I want to use what is called a USB rubber ducky where you insert a MicroSD card into the USB. This card has payloads on it which you insert into the victims computer and then the payload is executed. Many different types of payloads can be written. These scripts are written in a language called duck script.
Charlie and I also discussed how to better implement my physical attack. This includes using a wireless adapter as well as ethernet cords to jack into ports around campus and see how easily I can get in.
In the past week, I have:
- Finished a simple Hangman Game in Elixir, I coded it up during the weekend. It can be found here: https://github.com/hungphi98/Hangman_Elixir. I learned a lot of things while making this tiny game: decoupling design; BEAM processes and concurrency; distributed client-side; and a couple of Elixir native concepts: Supervisor, Agent, and Application. I think these lessons will be extremely valuable to me once I started implementing my idea.
- I started picking up three books and reading them at the same time, two books by Tanenbaum: Distributed System and Computer Network, another one is Handbook of Peer to peer networking. I find it much more useful to learn from reading books other than papers. They gave me a foundation of knowledge, and I don’t feel starved like I do when reading the papers. Some of the stuff I have picked up so far:
- Scaling techniques
- Basics of network security
- Requirements and implications of a P2P network
- I started seriously looking at the implementation of Blockchain. I have learned their basic protocol and had a rough idea of how they work. I have coded a basic blockchain using Flask but too shy to upload to GitHub since the code is extremely messy – I might clean it – probably not.
For this week, I started working on NetLogo, the software that I plan to use to create the simulation model for my project. I looked into the tutorials and the sample models library. NetLogo has its own programming language and development environment, so I spent quite some time to study its ecosystem. I also created a simple simulation model that read a file containing elevation information, display the elevation in different shades of green, built some fire sources and let them spread to places where elevation was smaller than 500. All of my notes for NetLogo can be found in Box.
Charlie and I also discussed my design for the project. For now I will focus on four types of input: Wind, Elevation, Temperature, and Humidity. First I will explore them individually to see how each affects my model. Then I will combine them, two at a time, to explore their combined effect on the model.
I found a MFCC library in GitHub and explored it a little bit. It directly takes a wav file as input and returns one N*1 array (a sequence of acoustic vector). I recorded my voice and converted to a wav file. I briefly tested the code. It took my wav file and return an array containing a sequence of vectors. I will use this library in my project. But there are many related factors that i need to study. I also wrote the timeline for the rest of this semester and next semester. My next step is to keep working on this MFCC library and explore the Dynamic Time Warping library in GitHub.
I finished a first pass of all but one of the papers in my reading list, and also read some of the papers that are highly relevant to my project, which I had already read for a first pass, for a second or third pass depending on how relevant the content seemed. I have also outlined the introduction and motivation for my project, and am working through the related works section. Apart from the project proposal, I have also spent some time trying to find some ‘gadgets’ that will assist with the proof for a simpler variant of parks puzzle, which is an effort that has not yet borne much fruit.
This week I have:
- Learned about the P2P and different Protocols in the implementation
- Learned about the history of P2P, and different architectures in implementing a fully functional P2P File Sharing Application. It is quite colorful actually.
- Made a rough draft of Outline Proposal
- Learned about design patterns in Elixir. I am still very very new to the language. The backend architecture of Elixir is actually a lot different from all the web servers backend I have seen.
- The code is really really tight – there is should be no place for redundancy, ever. A good Elixir code should be decoupled as much as possible for scaling later.
- Perfect for real-time processing, and concurrency handling. – An absolutely perfect choice for this project. Fun fact, Elixir is built on top of Erlang VM. And Erlang is what helped WhatsApp become the WhatsApp we know today – at times WhatsApp had 1 million new users every day – and the scaling power of Erlang enabled instantaneous communication, with little to no failure of nodes. AT&T also used Erlang as its backbone for telecommunication – in Erlang, your application still runs while being updated! (That’s why you can make phone calls why AT&T updates their software in the background – mindblown!)
- Supervision tree which automatically respawns failing nodes/processes –> guarantee availability.
- Running on Erlang VM means the code in Elixir is compiled into Erlang bytecode, which runs on BEAM VM. Erlang processes are implemented entirely by the Erlang VM and have no connection to either OS processes or OS threads. So even if you are running an Erlang system of over ten million processes it is still only one OS process and one thread per core and completely isolated to your actual OS. Amazing!
- There are more wonderful things about Elixir but I guess I stop ranting here.
- I am trying to scale my current implementation of Baby Distributed Hash Table (still very early stage and primitive) to more nodes but there are some unexpected bugs that I have to study further. The Distributed Hash Table will become a crucial part of the query later in my application, along with the Merkle Tree.
- I am studying the architecture of Napster – the killer P2P application that appears in the early 2000s that paved the way to P2P. However, I kinda want to improve/ or rather try with a slightly modified architecture – to negate the delegator/gateway node to make my system completely decentralized to absolutely destroy the single point of failure. I am not sure if this is possible. I might not have time to actually implement it in the near future but might be good to keep in mind.
- Continue with studying Elixir. The more I learn about it, the more I fall in love with this language. So elegant yet doing so much.
- Continue to do more research on applications that have P2P architectures. I already saw some “grid architecture”, which is slightly different from P2P but have not taken a deep dive in it yet. So perhaps one of the things to look at. I have also looked at some apps other than Napster, and I wonder why most of them were implemented for Windows.
- Contemplate over the current architecture that I have in mind and its purpose. Most of the P2P apps have faced a challenge in the legal issues regarding copyright. I guess I have to repurpose my app such that no such thing would happen if I decide to do hardcore and actually deploy it into use.
This week, I worked more on my proposal outline. Tuesday morning, I met with somebody from EPIC to go over my grant application to go to GDC, at which I may try getting some playtesting data from my project from professionals. This morning, I met with Xunfei to look at my proposal outline before revising and finalizing it.
During this week, I continued work on my literature review after meeting with Xunfei. After finishing the review, I also started work on my proposal outline, and continued looking for more resources to use in my project. I specifically need to find more procedural generation source code for game stages, I’m fairly happy with my two music generation methods.
- I’ve been working on the Lit Review and Proposal Outline this week and I finally finished the Lit Review fully.
- My Lit Review included sections on:
- Data sets: what kind of data has been tested and what do they extract from the data to use as a way of identifying misinformation
- Identification/Classification Methods: what approaches did people take to test the data and have it respond with whether or not it was fake news
- Prototype Design Consideration: some papers outlined what a good prototype detector should have or be used for and that is something I want to deliver on so it was important I note what I found
- The Proposal was a bit challenging
- While drawing designs, I realized that there are a lot of parts to my idea (not that it’s unfeasible though) so I’ll have to sit down and not only figure out a good overall framework but good designs for all the smaller parts
- I also struggled with the methodology, budget, and timeline section because my framework is very much in flux since I actually need to figure out what works before doing the meat of my project.
I am working on finding more papers that study gender bias in social media posts and narrowing my idea further. One challenge is finding a feasible way to collect data from a site (since APIs have limits), or finding an existing data set or web scraper that fits my needs. I am also looking for authors that have published their code for their work and/or who have described their methods in detail.
This past week I worked on revising my literature review as well as writing my proposal outline that will serve as a starting point for my first proposal draft. I met with my advisor who helped me to come up with a good starting dataset for my initial neural network. I will continue read about neural networks and maybe try to implement a simple one in the upcoming weeks.
I went over the broad categories that need to be addressed by the project proposal, and created a proposal outline for Assignment 7. I also worked with my advisor, Igor, to find a good candidate for the reduction adn start working on the proof. We found that there was a natural way of reducing a subset of 3SAT, 2SAT with distinct variables was, to an instance of Parks Puzzle. For next week I hope to generalize the technique to a larger subset of 3SAT.
This week, I changed my method from hybrid to content-based filtering because there isn’t much research done in the hybrid method. So I chose to improve the content-based filtering instead. I also wrote my proposal outline and revised my diagram with the help of Xunfei. I might explore more ways to improve the existing method and see if there is anything else I can add.
For this week, I wrote my proposal outline. During the next two weeks I will use this to construct the 1st draft of my proposal. I also spoke with Charlie about taking a different approach to the social engineering aspect of my project. Most of these I have found YouTube videos to demonstrate and describe the process but I have yet to find any hard research.
I also constructed more details for implementing the technical part of my project. This will also be discussed with Craig.
I finished my proposal outline. The next step is to write my proposal draft. I also discussed with Xunfei and she help me drew a better flowchart. I gained a clearer understanding about the flow of my project. I downloaded a SDK of the iFlytek company’s voiceprint recognizer product for reference.
I met with Charlie for the weekly meeting. We discussed different designs for my simulation model. I will first create some input data, which includes creating dummy values for the base map instead of using a real map. The main program will contain simple transition functions. This is to make sure that I can produce a simple version of a simulation model. I will also have to look more into NetLogo, especially into its fire model libraries.
During my weakly meeting with Igor, he brought to my attention a better way to increment the ranking algorithm. In the first round, a certain number of image processing techniques will be applied to the original image and the top 10 or so images will be passed on to the next round. For each round after, permutations of the image processing techniques will be applied to the images, and the next 10 winners will be promoted to the next round. This way, we can keep applying several techniques on the images, and find the best combination. The process would go on until either the machine has a confidence interval beyond a certain threshold, or a certain number of rounds have passed. The latter is important so the machine does not keep going for ever (or for too long) if the image is simply to bad be made decent. This brings me to the question of what to do if no food is found in the image. Should it return an error, or maybe apply the process to the image and see how it turns out? It is possible that the user submitted an image that contains an unusually morphed food, which the AI might not recognize as food, but still be able to make look good.
I also have heard about genetic algorithms, and will look into those as a safety net/supplement.
I met with Xunfei to improve my design outline for the proposal outline. I completed the outline on the box site. I am trying to find more ways to improve the existing system and see if there is anything else I can add. I found additional papers on QR code security, so I am learning how to avoid thefts with QR codes.
- I wrote my project proposal this week and I already had most of the info ready.
- The sections that I didn’t feel really prepared for were the design and what software/hardware do you need sections.
- I didn’t have a good idea of what should go into my design and what counts as a component
- I’m also not sure what kind of software/hardware I need because I’m not completely sure what my own unique approach will be so I don’t know which software/hardware is the best for my approach yet
- After reading articles for my literature review, I see that there are a lot of different ways to identify and classify media with misinformation.
- I will need to do a bit of work to be able to combine all the methods in a way that it will be able to identify and classify media of all different topics and types
- I’ve also split up my project into smaller more manageable goals to accomplish
- First Level Basic Goals:
- Find a large enough dataset that is properly vetted as credible and one that is properly vetted as not credible
- I want this dataset to be on a variety of topics
- Find an algorithm/classifier that is accurate 80%-100% of the time on the dataset with a variety of topics
- Find the key features that are the most reliable for classifying
- Find a large enough dataset that is properly vetted as credible and one that is properly vetted as not credible
- Second Level Goals:
- Expand the dataset to include pictures where the text was extracted from it
- Re-test the algorithm/classifier to make sure a drop in accuracy hasn’t occurred
- Re-test that the key features for articles can apply to the text within a picture
- Third Level Goals:
- Expand the dataset to include videos where they are transcribed as accurately as possible.
- Re-test the algorithm/classifier to make sure a drop in accuracy hasn’t occurred
- Re-test that the key features for articles can apply to the transcriptions
- Fourth Level Goals:
- Create an app/website where you can upload a piece of media and the app will use the algorithm/classifier and tell you if it is credible or not
- Fifth Level Goal:
- The app/website will keep a record of things that have been deemed credible or not
- Create a browser extension that will take the media from the current tab and check if it is credible
- Sixth Level Goal:
- The app/website and browser extension will scan and search for certain keywords set by the user and check new content that’s been uploaded
- First Level Basic Goals:
- I’ve chosen Charlie as my adviser for my project
- The idea that I picked for my Capstone is my Fake News Detection Idea
- The basic idea is that I would create a website/application and a website extension that takes mediums as input and will tell the user if it is factual or not
In the past few weeks I have settled on my topic being exploring gender bias on a website using a combination of computational linguistics and quantitative analysis. After writing my literature review on work that explored a variety of sites, I decided this week to focus on a social media site for my project. My next step is to explore more papers focused on analyzing social media and the APIs available for different sites in order to choose which site I want to focus on.
I read over 20 papers in the last two weeks to work on my literature review. I met with my advisor and talked to him about my proposal and what could be improved in my literature review. I have found some projects/papers that touch upon what my project aims to be. This will help me find and establish a starting point once I start working on my project. One of the challenges that I am currently facing is finding a dataset. I have come across a few datasets that I can use from kaagle.com.
This is what my initial model looks like (This does not delve deeper into how the neural networks are configured.)
While preparing my diagram for the quiz, I got a much better conceptual understanding of what I want my project to look like. I have also found nice papers this week. I started thinking of a few different image processing techniques that might help with making the image, and picked a computer vision algorithm for my AI (AlexNet) . I also decided on an image ranking algorithm to decide which image to return, a binary comparison. I feel significantly more comfortable about my project now that I have a more concrete idea for my software architecture, even though I am still fuzzy on the implementation details/
This week, I mainly worked on reviewing my papers carefully and summarizing them for the literature review. I also met and talked with Xunfei about my proposal idea and came up with a preliminary idea. I will have to work more to develop it.
This week, I wrapped up my first draft of the literature review. I’ll be meeting with the writing center as I embark on my final draft. I’ve continued working on nailing down my exact idea that I’ll be proposing, as well as looking into the available resources found throughout the papers I’ve read, from algorithms to source codes. I’ve done some basic work on a prototype game, but have been too busy to make much progress yet. I think a good portion of my project may include comparisons between different methods and combinations of methods between the PCG-G (different algorithms, mostly) and music generation (mostly grammar-based versus machine learning).
This week I worked on improving my understanding of the Parks Puzzle and exploring possible proof techniques to show that it is NP complete from two directions. I continued working on the Time Complexity chapters of “Introduction to The Theory of Computation” by Michael Sipser to round out my theoretical understanding, while also solving many instances of the puzzle using an app on my phone. I came onto one general idea for the proof involving only ‘AND’ and ‘OR’ gadgets that I discussed with my advisor, who made some suggestions involving an ‘IFF’ gadget, which I am going to continue working on. I also received feedback on my literature review, which showed some significant problems that I corrected according to the grading rubric.
I’ve finished writing the Literature Review for my idea “Fire Spread Simulation Using Cellular Automata.” After reading the papers for my research, I found a recent paper on this topic which used Machine Learning to solve the drawbacks of previous research. However, I could only find one paper using this technique so I will have to dig deeper to find more related materials. Charlie has suggested that I should categorize the papers based on the input data (terrain, weather condition, etc).
I’ve chosen “Fire Simulation Using Cellular Automata” as my final idea. I have also met with Charlie and decided to meet every week on Monday. I will also meet with Xunfei regularly for my research. Xunfei has suggested that I should look into ArcGIS for the simulation part of the research and also suggested me to talk to Jose as ArcGIS would require funding.
I finalized my proposal to “Applying Voiceprint Recognition Technology to Identity Verification”. The keywords are voice recognition, voiceprint, feature extraction, voice detection, voice verification. The difficulty I might encounter is that there may be background noise in the voice input. If the noise is loud, it may affect the feature extraction and voice recognition. I probably need to explore methods for removing noise.
Received feedback for LR. I understood the mistakes I made and working on the revised LR. I also found additional papers that I will use. Met with Xunfei to discuss proposal outline and LR.
I did not add an update to week 6 due to the long weekend but had been working on my literature review, which I finished today. It was very useful to read and re-read certain articles and realise some are useful and some are not. I now have further inspiration with where I can take my idea and am happy with its process. I will be looking in the next week or two to start looking into potential technologies to use for my project, which currently seems to be leaning on public Python libraries.
I have decided on the project I will be working on as my senior project. I have talked to Charlie about it, discuss my ideas regarding this project. He will be my advisor for the project. I have found 10 more papers and a couple of technologies I might be using. I have also found the datasets of food and recipes I will be using for my project.
My final idea is nutrition management and recipe recommendation system. Users will be able to scan the ingredients they have using the app and the app will recommend recipes using the user input they have put before such as any allergies, or food they don’t want to or cannot consume. The next step of my project will be the calorie estimation of food the user will consume. For this part, I plan to use a texture mapping and scanning for the optimum estimation of calories, and ingredients. For the privacy issues, I plan to have users scan their face on the first use of the app and have an API that will determine whether the current user is the user of this account. I am still thinking about possible ways to detect liquid ingredients and seasonings of the food.
I now know what idea I am going to go with, it’s a new idea and is not related to any of my old ideas. My new idea is about using neural networks and natural language processing to predict a better way to write emails or other forms of text in order to better engage the reader. This will be focused on business emails and other forms of business-related texts. I have read a lot of papers on neural networks in the past week and have spent most of my time writing my literature review on it.
I have talked to Xunfei about my 3 ideas and decided to discard one of my 3 ideas because of overhead issues. I wrote 2 annotated bibliographies of my 2 ideas with 3 papers each. I have also done more research on my 2 ideas.
I wrote 3 annotated bibliographies, one for each idea, and each of the annotated bibliographies is composed of 2 papers I have found for my ideas. I went to San Diego to attend Tapia conference of diversity in computing.
I’ve finalized the base project idea – I’m going with the one involving music generating AI. Additionally, I’ve officially gotten a proposal adviser, Xunfei, who I’ll be meeting with every Wednesday morning. I’m having some issues in limiting the scope and application of my project, which I’ll be focusing on while I finish up my literature review. In terms of the review, I’ve read 9 of the 10 papers, so I only need to read the last one and put the notes I have into the literature review format.
- The First Responders Tech idea is not panning out in terms of finding any relevant research articles so that may be way to difficult for me to achieve in this time frame
- The Secure Paperless Voting Machine is working out in terms of finding research articles but upon reading a couple of the them, I’ve realized how complex the issues are. Both the hardware itself and the software need to be more secure. This might be too big of a project.
- There are a lot of articles and info about my Fake News Detector idea and my Ancient Sites in VR idea so that bodes really well for the feasibility of those two ideas as my official capstone idea
Updates in Ideas
- My first “Fake News Detector” idea has remained mostly the same
- I still have not come up with a more doable idea for the secure paperless voting machines because there’s so much oversight with voting regulations and laws and because there’s an added obstacle of different voting systems/mechanisms.
- My “911 Tech” has hit a wall in terms of find research articles on it
Updates in Process
- I talked to Charlie about my three ideas and he suggested a change from 911 specific tech to First Responders/Disaster Relief tech because they actually use open source technology and it might be more feasible to create something and find research on it.
- I also created a fourth idea in case the First Responders idea doesn’t pan out. My idea was creating very realistic recreations of ancient archaeological sites that you can interact with in VR.
My project is using Voice Print Recognition technology to check if the voiceprint of the input match the corresponding one in the database. This technology can be used in many identity verification scenes like customer services for bank, door lock, business transaction. The main steps approximately will be: take voice input -> (remove noise -> ) extract voice features -> building models with selected algorithms -> compare voice features -> check if voiceprint match. The possible algorithms might be: VQ, MFCC, DTW.
This week I mainly finished my LR and spoke in detail regarding the project idea with Xunfei. It seems I like I know which technology to progress with and eliminate the two I was debating before. I also worked on my basic design outline.
I read more papers on my ideas, met with my advisor to schedule a weekly meeting and discussed a new idea proposed by the company I interned with this summer.
The new idea is a Neural Network driven A.I. that can learn and predict better ways to write emails, marketing campaigns and other forms of communication with the customer.
This week, I finalized who my advisor will be (Charlie). I also decided that I will be working on my security testing idea as my main project. To start this, I spoke with Brendan Post (IT) to discuss my ideas. He was happy to help me and I will be in further discussion with him as I move forward with my proposal.
I also found a couple other papers related to my idea. There was one that had much more lower level detail and actuially described the implementation of their testing. The researchers used Kali Linux to hack into a router through different ways such as SSH, Telnet, and SNMP. There were images that showed the commands they used. It was the first article I found to have a lot of low-level detail.
This week I spent some time finalizing my proposal idea. I discussed the Parks Puzzle with Igor and come to the conclusion that I should work on proving its NP-Completeness for my final project. I had to discuss this idea with my advisor Igor, as well as Charlie and Xunfei before I could finalize this plan. Once I had this confirmed by Xunfei, I put aside my work on the other ideas and started to solely focus on NP-Completeness. My first task is to go through the relevant chapters of “Introduction to The Theory of Computation” by Michael Sipser nad working through problems to clear up my understanding of the problem, which I have started to work on.
In the past week, I searched and skimmed lots of papers to use for my own research. I also talked with Dave and Xunfei to refine my ideas. I also found a data scraping software to extract product information from Sephora.
I am going with the auto-image processing for food. I have been looking more into the literature, and there is limited literature that deals with making food look better, and they are all relatively recent publications. However, there is literature about how to train machines to assess how aesthetically pleasing images of food are, and also on how to make regular images more aesthetically pleasing.
I will start experimenting with different machine learning algorithms, and also image processing techniques.
In the past week, I have found papers for the literature review, started working on the paper. I have to talk with my advisor regarding the papers. I also found that the QR code would be a viable option rather than a barcode. QR code can store data horizontally and vertically whereas only vertically in the barcode.
Most of the articles I have read this week have been quite informative and take the time to explain even the most basic things. However, this has not been true for all the articles that I have read. Some of these articles assume that the reader is already familiar with the terminologies and technologies used in their research or study. Although this helps the reader dig deeper into the topic and come out with a better understanding of the study or research, it also means that doing a 1 or 2 par reading often does not suffice in these cases. Along with reading research papers this week, I also worked on some technical diagrams for my projects this week.
In the past week, I explored different ideas, talked to the faculty, and found more papers to read. I found lots of interesting paper related to my personalized skin care product recommendation idea. I found out that using content-based filtering to recommend products might be helpful, and I could modify the algorithm to have better performance.
Some papers I found useful include:
 Recommender System By Grasping Individual Preference and Influence from other users
 Recommendations System for Purchase of Cosmetics Using Content- Based Filtering
 Item Clustering as An Input for Skin Care Product Recommended System using Content Based Filtering
These papers gave me an idea for what algorithm to use for the recommendation system and how to modify it according to my need. Some of the research was done using five skin types, but I’m thinking to increase it to 16 or more.
I am pretty certain at this point that I want to do something food and image processing related. This evolved out of my first idea (MyOrder), but is not quite the same.
I want to do some sort of auto image processing to make food look better, and provide an interface/API to use the service. I have found a very large dataset that could help with that, albeit unlabeled. There also is research done on labeling food images as good-loking/not-good-lookingm, which I could use to label the food.
I have two other ideas I will consider (as long as I am still allowed to consider). The first is an app that can tell the ingredients of food and/or some kind of shazaam for food that tells what dish a certain dish is. The other is an app that counts calories by looking at pictures of the food you eat.
I haven’t decide my topic yet, but I was reading papers related to my three ideas to gain a deeper understanding on these ideas.
My first idea is an AI tech for voice print recognition. It can be used for avoiding voice spoofing attacks on business, banks (like mobile phone customer service), etc. The main steps for voice recognition is: take vocal input -> identity-feature analysis -> deviating feature selection -> deviating feature comparison -> distance to reference pattern estimate -> check if voice match. The main algorithms for feature extractions are: GMM, JFA, GMM-SVM, etc. On the paper “Vulnerability of speaker verification systems against voice conversion spoofing attacks: The case of telephone speech”, the authors experimented several algorithms and concluded that although JFA has a high inaccuracy but the converted samples with JFA sounds very mechanical so human can easily distinguish. The authors of paper “Voice command recognition system based on MFCC and VQ algorithms” discuss and examine two significant modules: MFCC and DTW. Their results were good. So I will consider use these two modules.
For my second idea which is creating an AI tool for safety driving, the key tech is 3D dynamic facial recognition. I learned that the most Facial recognition tech can be decided into two main parts: facial detection and facial recognition. I can use open sources like opencv and dlib to do facial detection. There are 3 factors i need to care about: detection rate, misdetection rate, false alarm rate. The authors of the paper “BP4D-Spontaneous: A high-resolution spontaneous 3D dynamic facial expression database” reported a newly developed spontaneous 3D dynamic facial expression database in their paper. I am not sure if I can or should use their new database. Although the paper “Real time facial expression recognition in video using support vector machines” primary discuss detecting emotion from facial expression, it provides some facial recognition tech info for me.
My third idea is creating a smart tool to grade algebra on handwritten homework. The APP takes a photo of the handwritten homework and using OCR tech to extract the texts and grade them. The main tech is just OCR. Although the two papers I read both talk about their own APP and OCR system, I can refer some technologies they used, like matrix matching, fuzzy logic for facial extraction.
This week I researched deeper into my three ideas, and now have developed preferences and better understandings of them. Specifically, I believe the natural language processing idea where I will create summaries of subjects in PDF’s seems to not only be the most interesting to me, but the most achievable and most understandable of the ideas. It requires a lot more CS than the other two, which I like. The others, especially the math proof assistant idea, seem like they will contain much more design choices and therefore will distract from the CS side of it. I did make more progress though on finding approaches to the math notes to LaTeX idea, as there is a decent amount of research in symbol recognition, however handwritten was harder to find as opposed to digitally drawn with electric pens. The proof assistant idea is the hardest to research but I am making slow steady progress in this field.
In the past two weeks, I’ve read through a total of 12 sources, four for each idea, and created annotated bibliographies based on them. While some of these papers are more useful than others, each has been helpful in one way or another – and, indeed, I’ve been able to find more papers to look into further as I modify my ideas based off of both peer feedback and my own research. Some papers have given me actual algorithms to either implement or look at as a building block for my own- others have shown me, for example, how much one of my ideas (copyright detection) is already extensively researched, and pointed me towards new directions in which I could modify the project to make it something original.
I spoke with Charlie this past week and we both decided that my first idea is my best one. So, I decided this was the one I was going to pursue my penetration testing idea and found 6 more articles for this.
A couple of the articles I found were about professors having/teaching a first-time hacking course. It was really cool to see the different designs of the classes. There was a general theme of keeping a subset of computers in a controlled environment and then allowing the students to work from there. One article included a complete description of their syllabus and the assignments of the course. This was helpful as it gave me an idea as to what software was being used in these courses.