I have started working on passing images to the ranking algorithm.
I also have found some online food-photography courses I want to look at. Learning that will be helpful in knowing how to improve my images.
I have started working on passing images to the ranking algorithm.
I also have found some online food-photography courses I want to look at. Learning that will be helpful in knowing how to improve my images.
This past month i have been mostly working with getting everything for my project working and fighting some major issues. The first issue that has been almost solved is that the NorthStar uses DisplayPort out while my computer only has an HDMI port and a MiniDisplayPort in. Turns out HDMI outputs are not compatible with DisplayPort and so the adapter i got to do that does not appear to work. I am investigating getting the proper port by the end of this current week.
The other issue i had to fight was the fact that for a week and a half, i did not have my primary computer since it was broken and needed to be repaired. I had a much less good backup computer that i used to test the hypothesis above, so i was not entirely useless during that time.
This coming week and weekend, i am hoping to have a fully functioning environment running and have the ability to display tracked hands in the headset, depending on the availability of an adapter.
Last weekend, I spent time with a small group of friends filling out a spreadsheet of information for 2020 Senate candidates. So far, 154/348 filed candidates have been added to the sheet. During that time, we discovered that a few candidates operate their campaign on a public Facebook profile instead of a Facebook page. In talking with Charlie, he guessed that the process in the API to collect profile data shouldn’t be too different from page data. Therefore, I am planning to collect this data as well, while noting the names with profiles in case their results are drastically different from overall results. Next, I plan to develop the scripts to start collecting and analyzing small amounts of data, planning to scale and automize them later.
This week was a big planning week for me. I spent a lot of time writing down notes and ideas, as well as researching the details of what I need for my project. I also spent some time gathering resources for my project in the form of data from Iceland. A combination of 2018 and 2019 data will provide me a much-needed training/testing case.
I have progressed in my implementation, further streamlining the process of creating various edge detections of original images. This week I added the Prewitt edge detection algorithm and improved my Caney edge implementation to have a tight, wide, and auto mode.
I have also been researching technologies for image recognition via machine learning with multiple channels. This is the idea that a single “object” in the AI can have multiple images associated with it, and it is necessary for my project.
This week I was able to create a saved checkpoint of my learning model for semantic relation extraction. This hopefully means I won’t need to train it further and can now focus on feeding it my data, which now needs to be pre-processed before being fed into the model. A basic GUI window was also up and running this week with PyQt5 which was great to see! I will be writing more code in the coming weeks now so I need to ensure that my project files are organized.
This week, I tried to implement some models and was hoping to get it on our Layout server with GPUs. However, the system admins were still working on that and I could not ssh to the server. Therefore, I created a google cloud free trial account and started writing and testing my model on their server.
Since my project involves a significant part that’s marketing, I was advised by my instructor to talk to Seth and other professors about how I should approach a dataset. After talking to them, I have decided that a good approach would be creating a dataset using the readability formulas. First I will calculate the average readability and then filter the dataset using that average readability. A marketing dataset has been extremely hard to find, but asking around has led me to the Groupon API – it lets me get 100 deals per second which will help me easily scrape millions of deals in a few days. I plan to run a script in the background that does it. Since last week, I have also successfully implemented word2vec using Genism – a python library.
In the past week, I worked on calculating the cosine similarity between the ingredient composition of an inputted item and that of the rest of the items in the data. I am struggling to decide on which formula to use for this, since the related project used the equation different from the “typical” formula used to compute cosine similarity. I will need to look into this more next week.
A Real Time Fall Detection System to Assist the Elderly Using Deep Neural Networks
The elderly have a high chance of falling and get injured or faint. This might put them to danger if they are alone. One way that can help the elder people is having a system that can monitor their actions, detect the falling action and other behaviors after falling down, classify the levels of severity and send an alert to their emergency contacts or the emergency room if the level is serious.
Deep learning, pattern recognition, image processing
Python, PyTorch (or Keras)
I might also need a CCTV camera if I decide to build the actual device.
First I will apply some image processing techniques to enhance the images and videos quality. If the dataset is small, I will use of image data augmentation techniques to produce more data. Then train the model that detect the person falling in the photo frame using deep neural networks, then use the people falling photos and videos to train a model that classify the level of severity. When the index of severity passes a threshold, send out the alert.
There are several projects that work on the similar problem. Most of them work on detecting the falling action only. In this project, I hope to build a system that is more detail and can decide whether it is an emergency case.
I might not be able to find a big enough dataset to train the model.
I worked with possible ways of proving that non-contiguous Parks is NP-Complete, and found one good avenue for exploration. Over the week I produced a general technique to convert any instance of 3-SAT to an instance of the non-contiguous Parks Puzzle, thus proving that it is NP-Complete, our first major result. I am working to modify the proof, or try similar techniques for the contiguous case this week.
Automating laptop checkouts from CST front desk using image recognition
Although we need a human to address the needs of guests in the welcome desk of CST, it would be ideal for the worker and students if we can automate the process of MACs’ checkout. Humans are prone to error and we do not want any student worker to be liable of errors that could cost them thousands of dollars. So this project would allow a machine to handle the checkout using a camera to identify the laptop and the student wishing to check out the laptop and remove the process from the desk worker completely.
Image recognition, machine learning models,
This would need a good quality camera, python, and some database management software
Have a camera stationed above the cst desk. Also I think it would be beneficial to change the barcodes in the laptops to bigger QR codes for easy recognition and better visuals for the camera. Use various machine learning models to train the software to recognize students and identify unique laptops. This product should also send out emails regarding reservation details to the students like the current system does.
This is different implementation from the current process we have in that we are removing human responsibility from this procedure. This will hopefully reduce human error in the process and decrease financial liability to student worker and the institution. It is also scalable to other use cases (like Runyan desk for example) to increase automation and improve efiiciency.
The problem I anticipate is making sure the model I have does not mis-identify students checking out the laptop or mistaking someone walking by the cst desk as someone checking out a product. Lighting might also be some issue as the desk is besides huge windows and so lighting is very different in night vs day, or summer vs winter. Another issue to consider is the camera quality (need to get good camera under reasonable budget)
Ans: Using trained neural nets to be able to tell when a statement/sentence is sarcasm
Ans: NLTK and
Ans: Botmock is the only software that will be needed for this project
Ans: I plan on making this an extension of Botmock
Ans: With my project, I am using the same method of using CNN model hierarchy when it comes sentiment analysis to learn the context and space in which the sentence exists
Ans: Every sarcasm exist in a defined space one difficulty of this project is trying to build a barrier for that space. Another problem would be getting access to Botmock’s API to make this application compatible.
This past week I have really dug into my physical testing. Using Kali Linux and a wireless adapter (supports monitor mode), I was able to use commands to see which networks were available and from there, I could see all of the clients connected to each network. However, I only could see the BSSID (MAC Address) of each device, nothing more. I then went in to WireShark which showed me a little more data. I could potentially see what type of device it was. However, all data was encrypted in ECSecure. Trying to break the encryption was hard as we have hundred of users with different passwords. It’s not just a single password for the ECSecure network (that would be too easy to break). I plan to continue this testing and see what else I can find through ECOpen.
I have also started to set-up my Social Engineering experiment that way everything is ready when the start date arrives.
I worked on the login page and setup and almost done with the forget password setup. I have to decide on the database for the login, whether it will be single data base or sql for the whole application. I plan on working on the User interface features in the coming week. This involves setting up a db to store books, setting storage attributes etc. This is the main portion of the project and shall take the most time.
I have spent some time thinking about how to split up the timeline into more detail. I have met with Charlie, and decided that the program should take a bulk of images as an input rather than a video. The next step is to learn more on the photography aspect of things.
This week I have looked at some papers of most recent models for classifying images to build for my dataset. I encountered some challenges while reading those papers since there were terms that were hard to understand. Next week, I will continue to work on the image dataset and model.
Visual representation of nation’s development level
The goal of this project is to use the various world bank data that is available to evaluate different development metrics for each nation. Then I want to use visualization tools to effectively communicate to the interested audience. The visuals will change as the indicators for the countries change so the website would be a ‘live image’
Api, data visualization tools like Tableau or python, statistical tools to calculate the indicators and compare between nations.
Python, SQL, json. Maybe some database management system to store the data. Tableau for visuals.
I want to pull the data from various data sources like the world bank website using api and load it into some sort of database. Using this data, I want to use some tools to calculate and compare the indicators of development for various countries. The output from these calculations would be then visualize in a website live and these visuals would change based on any changes noticed in the world bank dataset.
I want to create a live version of this problem. I found a few websites that visualize these metrics or tabulate them, but it is hard to interpret for people who are not very informed about the topics involved. I want to make my website very intuitive so people with different experience levels can look and interpret the data intuitively.
The problem I anticipate is figuring out how to have the database where I store my data update in a lively manner so that any changes in the data bank is represented instantly in the website without any intervention required. I will have to learn various methods that are hopefully available readily that can make this possible for me
Due to a lack of available usable datasets, after talking to my advisor and instructor I decided to modify my project to focus on readability and sentiment instead. I researched papers on readability and sentiment this last week and have starting writing code using python(Keras). My next week’s goals are to have some working code for a trained network that produces more readable code. I still need to look a bit more into what constitutes as readable when it comes to marketing material.
Driver Drowsiness Detection Using Deep Neural Networks
Driving while feeling sleepy or tired is one of the main causes of traffic accidents. One solution for this might be having a device in the car that monitor drivers’ behaviors and facial expressions and ring the alarm if the drivers tend to fall asleep.
Dataset of facial expressions (images and videos)
Python, PyTorch (or Keras)
Build a pipeline that first apply some image processing techniques to improve the quality of the images, then train a model (using neural networks) to detect and locate face position in the images, and the last step is to build a model (also using deep neural networks) to classify the behaviors and facial expressions.
Most relevant projects track the drivers’ eyes to see if they close their eyes. I am considering checking eyes movements and also other behaviors such as yawning or nodding off in order to improve the classification performance.
There might not be a big dataset for me to use.
This past week, I have started phase 1 of my project, testing the physical security of the network. Along with starting this phase, I started to write the Google survey that will be used w/ the social engineering experiment. I also ordered the hardware needed for the social engineering test. I have not encountered any obstacles. This next will I will continue to use WireShark to test the physical network using both a wireless and ethernet adapter.
In the past week, I have spent most of my capstone time organizing my project and testing some options for the machine learning component. I have been working with fast.ai and ImageAI python packages, trying to set up some groundwork for when I have data ready.
I have also organized all the algorithms that I want to try, at least until after I can compare some results (after I see the results, I may opt to implement more)
My hope for the next week is to make progress on acquiring training data with drones, or at least narrow down where I might want to survey.
I forked MLMAN, a PyTorch model that achieved the second-highest accuracy of validation on the FewRel dataset for semantic relation extraction. Running locally with a useful amount of iterations, it took to long to train, so I will be training the module on hopper and saving the model there to fetch for local use. With this saved model, I hope to start pre-processing and feeding sentences into it for validation.
Over this week I finished up an non-contiguous IFF and OR gadgets, however I came to the conclusion, after meeting with Igor, that there does not seem to be a way to effectively put together these two gadgets. However, we also concluded that in most cases, it is not a particularly difficult challenge to find a gadget for contiguous parks, if one already knows the equivalent gadget for contiguous parks. Since I have reached a dead end, over the next week I am going to try out one promising new direction, and hopefully by close to proving the result for non-contiguous parks within the next two weeks.
My project is to collect and study the Facebook Reactions and comments on posts by U.S. politicians to see if bias exists based on the gender of the politician. I have decided with Charlie’s advice to focus my project on the 2020 Senate races. The 2020 Presidential election doesn’t have enough candidates to be a good sample size. The 2020 House races would likely have a wide variety of candidate strategies based on the district, many districts with no competition, and less voters per race. By contrast, the Senate races have enough candidates to be a good sample size, while also having more voters per race, meaning there should be more Facebook Pages with enough user activity to be used in my dataset.
This week I found sources for the Senate races, created a spreadsheet for candidates, and decided on which relevant columns should be in the spreadsheet. I am filling out the sheet first for races where the filing deadline has passed for the primary first. Next, I plan to learn how to access the Facebook API using the Facebook SDK Python library, and to collect sample data for candidates I have already added to the spreadsheet.
I decided to change my modeling method to neural networks. I have read a paper called Text-Independent Speaker Verification Using 3D Convolutional Neural Networks and checked their resources on GitHub. I tried to run their demo but required packages couldn’t be installed on my laptop. i probably need to request a place to run on CS/Cluster from the SysAdmins. I also found other similar resources on GitHub. My next step is to run them with testing files. I also had the first weekly meeting with my advisor Xunfei to discuss timeline and future plans.
I made a visualization (plot) displaying ingredient composition similarity between different products and skin types. I attached two drop-down options for users to select from product categories and skin types. I also attached labels to the graph so that it displays the product’s name, brand, price, and rank.
Working on the login and sign up system. Reading existing papers that talk about such system. Sign up will be via Zimbra only since using Facebook and other applications could lead to fraud accounts. The home page would be ready by next week.
Detect and Translate Chinese text in images
Lately many translator applications have introduced the new feature that can scan a document or take an image with texts to detect and translate the texts into another language.
Many of these applications perform well with very neat and clear handwriting or high quality images but not quite well with cursive handwriting or low quality images. My research goal is to improve the detection performance in these cases.
Chinese – English Dictionary API
Python, PyTorch, matlab
Build a pipeline that first enhance the quality of the image data using image processing techniques, then feeds data to a deep neural network model (maybe CNN) to detect the Chinese characters and connect to a dictionary API to translate the text into English.
The current applications do not perform very well on low quality images, so my goal is to find solutions to this limit of the translation apps.
I did some experiments and found that big apps like Google Translate still had trouble detecting the not-very-neat handwriting. Therefore it could be very challenging to achieve my research goal.
I go through the project again because it has been a while since I had CS 388 last Spring. I downloaded the data set and started doing some data manipulation and preprocessing. I will start looking at the models for image data set next week.
It was the first week, and we had out presentations. I also found an advisor. Everything is swell.
For the past week, I went back to my materials in CS388 and re-read my proposal along with the research papers in the proposals. In the following week, I need to obtain the dataset and learn (at least partially), the tools/ML models I will need for the project.
This week I worked on setting up Keras and completed a course on deep learning using Keras (Learn Keras: Build 4 Deep Learning Applications). As I prepped for implementing the project, one of the significant challenges I have encountered is finding an appropriate dataset to train my neural network. Since my project aims to make a business’s marketing material more engaging, an appropriate dataset with labeled data to set up a clear definition of what counts as engaging and what counts as non-engaging is necessary. After some research and talking to my advisor and the instructor, one of the parameters that I am now looking for while searching for datasets is data that might be labeled based on reading level/hard to read/easy to read. The main goal for next week as I move forward with my project is to have a concrete dataset that I can train my neural network with.
In the past week, I loaded the data, extracted ingredients from products, and made a document-term matrix containing product names and ingredient composition. I plan to visualize ingredient similarity between products this week. I haven’t faced many obstacles yet, but I want to finish things earlier than planned to allow some time for future obstacles.
This week I created the presentation for Wednesday, which helped to make clear to me my new current goal after work done over break. I have found some new datasets and repositories for models online, which I will be presenting to my advisor to figure out which best suits my project. I have also tried to better breakdown my timeline following the selection of a module for the following month, and have personal project goals. I researched some libraries for GUI implementations, currently leaning towards Electron (Java) or PyQt5 (Python).
This was the first week so I worked on getting back up to speed with the research, and on creating presentations for the first class.
This week was mainly for refreshing myself on the details of my project. I finalized Charlie to be my advisor for 488 and set up a weekly meeting time with him. I also completed the 3 slide powerpoint in preparation for the presentation in the joint class of 388/488. I adjusted my timeline and plan to start the first phase of my project on Monday. I did not have any obstacles this week. Within this next week I plan to start the physical testing phase of my project.
This week has been mostly organizational for me. I found some more resources on Github that I want to try and make use of, and I worked on my design plan for implementation. I talked with Igor about technologies I can use, and what I might need to use them effectively.
The main obstacle right now is the amount of structure that my project requires, which is why I am taking my time to create a solid plan for how things will connect to one another.
Next week, as my design becomes concrete, I will start coding different segments of my project, using some of the preliminary work I have done as a guide.
First of all, I decided my advisor to be Xunfei who was my advisor as well last semester. We decided our weekly meeting time. I have read some new papers and decided to change my modeling method from GMM-UBM to Neural Networks, and combine with i-vectors or x-vectors. I have found related code sources about Deep Neural Networks/Convolutional Neural Networks for speaker verification on GitHub. GMM-UBM is one of the most classical and dominant methods for speaker verification, but its accuracy decreases as the amount of users increases. Nowadays, there are new methods performs better than it, like Deep Neural Networks/Convolutional Neural Networks. This change on my project might be more challenging because I am using a new method which probably has fewer recourses. But I really want to make the accuracy for speaker verification higher than 90%.
I am getting familiar with Android studio. As per my timeline, the first step in the application is to implement the login system. Aim is to decide by end of this week whether to use Firebase and SQL or only SQL. I have to speak to Charlie regarding this. I revised my project through the first presentation, submitted the advisor form. Next week, work on the application should begin!
I just worked on my final paper this week. I met with Xunfei to ask questions about it.
I have been working on my final proposal this week. I will post it after the deadline for the assignment.
I finished my final proposal and I’m rechecking everything for submission this week.
In the past week, I have been working mostly on my presentation and
my proposal. My proposal is close to a finished state, but I am still
working on collecting preliminary results. I have also been trying to
create new figures (images and charts) which are easier to read on
printed copies of my proposal.
For the implementation itself, I am still working on the things I outlined in the first section of my project timeline (setting up the pipeline of the project without adding all the features at each stage), to try and get a minimum version working. I think that this will take a couple more weeks, but I am hopeful that it will lead to me having some buffer time next semester during my implementation of the project.
I finished my presentation. My next step is to add abstract and more introduction to my proposal paper, and finish the final version of it. I have done more research in the past week and planed to change my modeling method from GMM-UBM to Convolutional Neural Network or Deep Neural Network. GMM-UBM is very classical but also “old-fashioned”. CNN and DNN are newer and better. GMM-UBM’s performance lowers as the amount of speakers increases. But I do not have enough time to change method for this semester. I will do more research during winter break and probably change next semester.
I made final edits to my presentation and finished reading 3rd passes for all papers I have found. I have also revised my design by adding some more details to it. I have found a book about OpenCV projects so I have started implementing an application for image recognition. I am still working on my final draft proposal. More research is done on Android camera API, to see what I can use and what I cannot for my application. I plan to implement small chunks of my senior project during winter break, so I am looking for online resources to walk me through the process.
I have researched and read a few more papers in the last week. I have expanded upon my analyze -> split -> replace modules with actual implementation details using an encoder-decoder model to swap less engaging text with more engaging text. In order to do this, the text needs to be vectorized and then trained. I have also found a module that can help me achieve that. I have also extensively worked on my proposal presentation. I also met with my advisor and went over the presentation and was advised to explain the slides in a way that a person with no understanding of neural networks can understand what is being communicated.
In the past week, I browsed to see if I could find a better data set for my project. I wanted to find a data set with users’ purchase history as well as the product information, but I could not find the appropriate data set to apply hybrid filtering. So I ended up modifying it to content-based recommender. I thought of more details to add to the project and discussed with Xunfei about ways to expand it. I ended up revising my diagram and some parts of the design. I also worked on creating slides for the presentation.
This past week, I spent time working on the presentation. I also met with Charlie to discuss the presentation slides. Charlie told me to replace a table in the Motivation section with graphs to show how wildfires have increased overtime. He also told me to cut down some texts in the Related Work sections. For the Proposed solution, he told me to redesign my graph. I also had to add more details in the Timeline and Budget sections. I also worked on the final paper during the break by adding the new requested sections.
This week I have been working on my presentation and have refined my proposal a bit. I will keep working on my presentation until the upcoming Wednesday as I wait for feedback on my second draft of my proposal
This week I have worked on the presentation. That’s it.
This week I worked on the final proposal. Feedback from the second draft indicated that there were some grammar and structure errors. I added the testing and abstract sections. I hope to finish the proposal by the end of this week.
I read some new papers and research about different modeling algorithms and started to worry about the accuracy on my system. The accuracy is not only rely on the modeling but also based on the dataset for training and the quality of acoustic input (the speaking environment). But selecting a suitable modeling algorithm is important. Now the popular models are: HMM, VQ, DTW, GMM, UBM, i-Vector. I temporarily chose hybrid GMM-UBM. I might change in the future or mix other modeling to enhance the accuracy. My goal is to reach an accuracy at least 90%.
This week, I continued working on my project proposal, submitting my second draft after some much-needed updates. I still need to work further on the Related Works section. I additionally continued working on early implementation of the project. Lastly, I prepared a first draft of my presentation slides.
I have mostly worked on the second draft of my proposal in the past week. I simplified and restructured my framework model. I also looked for more use cases of my project and decided what frameworks to use while implementing it.
This week I have not been able to do much progress. I have decided to scrap the idea to use Machine Learning in the module for altering images, due to difficulty in implementation. Besides that, I have worked on the second draft of my proposal.
This week, I worked on the similar project posted online. While working on it, I found some challenges in modifying the content-based data set to fit the collaborative-filtering method. I might end up modifying my project from a hybrid recommender to a content-based recommender. But I will keep looking for alternatives to make it possible.
During this week I met with Charlie to review feedback for the first draft of the proposal. I reconstructed this version for the 2nd draft. I also read some papers for the next pass. These were more directly related to the software I am going to use.
I found an API for food calories. I worked on proposal second draft. I explored an image editing software I am thinking to use.
Still working on the second draft. Rebuilding the diagram of SLMS. Also ready with the draft for the presentation.
I realized that the database I found for food nutrition is not enough for the calorie estimation so I looked for more resources. I started learning SQL.
I discussed my proposal draft with my advisor. I got her feedback and suggestion, and knew how to revise and improve my proposal. In the past week, I read more papers about the GMM-UBM modeling method that I plan to use for my project. I understood the specific procedure now but it is still hard to fully understand this principle… Now my another problem is to find a suitable dataset and decide if my system is text-dependent. There are three primary ways for speaker verification now: text-dependent, mixed, text-independent. The text-independent way is very difficult and complicated to do because user can say anything to pass the verification. But text-dependent way is restricted and not safe for spoofing attacks. For example, people can replay pre-recorded voice to pass the verification. Therefore, the mixed way is better. It restricts the text in a way but safe for spoofing attacks. For example, they user can only speak numbers one – ten, but every time the text is random. But it is hard to find a dataset of all audio file in numbers in English. Now I need to decide which text way my system will use.
This previous week, the work I’ve done has been two-pronged, as has become the norm and will continue to be for the rest of this semester. First, I continued work on the basic implementation of the game. I currently have the control module working, as well as a looping stage that I created in order to test the controls. On the proposal side, I’ve been making edits based on the in-class peer review that we did, as well as working more recently based on the feedback given by Xunfei. I also met with Xunfei to go over her feedback of my first draft, and updated her on my progress.
This week, I investigated the technologies being used in my found papers more closely to find which technologies would be more feasible for my project. For data collection, I have found that the facebook-sdk python library (https://pypi.org/project/facebook-sdk/) used by Pool and Nissim is the best option to connect to the Facebook Graph API, since it looks well documented and has all the options I might need. I also decided to use the Facebook Pages of politicians as my dataset. I reread As the Tweet, So the Reply?: Gender Bias in Digital Communication with Politicians by Mertens et al. to see if their methods could be adapted to my project. I will need to look at their references for methods in more detail to see if I can feasibly apply them to my project.
This past week, I worked on mainly reading my new papers. I did a third pass reading on all my old papers and did at least second pass reading on the new ones. I tried finishing more than half of the existing project on python notebook and played with the data set. I now have a better sense of how to start my project next semester. I also met Xunfei and updated my progress to her. As soon as the feedback for proposal draft 1 comes out, I will be revising my writing and finishing the existing project I have been working on. I also plan to test the existing project on collaborative filtering to make sure it works with a different data set.
In the past week, I have spent most of my time working on the first draft of the proposal. I decided to research and include a new category of papers in my proposal that I had not spent a lot of time before on. The new category that I included was “Sentiment Analysis.” While working on the proposal and refining the design of my framework, I realized that sentiment analysis, something that has been thoroughly covered by researchers of neural networks is very close to my research since I also need to know the sentiment behind the email/piece of text that is to be improved.
This week I have
This week I have:
In the past week, I have used my peer review from Jordan, as well as my own proof-reading of a physical copy of my draft to fix a lot of errors. I wrote my draft in a bit of a rush, and as a result, there were a lot of formatting errors, most of which I have now fixed. I have also updated some of my diagrams in accordance with feedback I have received and expanded some content in my draft that needed to be clarified.
In addition to working on my draft, I have been working on my project itself (preliminary work can be found in the git repo). I created a mockup GUI to give me some ideas about how I want to design the actual version next semester, as well as testing some implementations of different filters, operators, and edge detectors. Some of these results will hopefully be represented in the next version of my draft.
During the past week, I finished the first draft of my proposal and started to make those changes for the second draft. I have also continued reading some papers for their next pass. I continued to watch videos and read content related to the USB Rubber Ducky. I have started to put together some scripts that I would like to use for the attack. I also spoke with Charlie to refine my methods for the physical attacks I am going to implement. I now have a better/ more related CS implementation for this attack than what I previously had. During this next week, I am going to be working more with Metasploit on Kali Linux.
For this week I looked into many different datasets, including gis fire map data https://frap.fire.ca.gov/mapping/gis-data/ and Kaggle dataset https://www.kaggle.com/elikplim/forest-fires-data-set but couldn’t find what I was looking for.
I picked The Ranch Fire in California but couldn’t find good datasets for it. I was trying to find I’m trying to find elevation, wind direction, humidity, and vegetation.
All of them have to contain coordinates so I can layer them together. Also, I need each set at different time stamps for the simulation. I will discuss this during the next weekly meeting.
For this week I have done the following tasks:
Learned to view shapefile contents with Netlogo and Python library pyshp. A shapefile is an Esri vector data storage format for storing the location, shape, and attributes of geographic features. This type of file is quite complicated so it took me sometimes to understand the format and its contents.
I had trouble finding the right dataset for my project. Charlie suggested that I look into https://www.frames.gov/afsc/partners/fmac/guides-products. I downloaded the data for Alaska but it does not have the contents that I was looking for. Finding the right data is currently a big challenge.
I am also getting more familiar with Netlogo. Using Netlogo, I could view the content of the data for Alaska and also extract the metadata using the command line provided with NetLogo.
I have been continuing learning machine learning with Python, specifically PyTorch. I started with PyTorch because it has a less steep learning curve compared to Tensorflow (the alternative). However, there are more tutorials for Tensorflow and I might pivot next semester as the image processing gets more complicated and I need more resources in incorporating image processing into the machine learning. I think I will be able to build both a ResNet and AlexNet algorithm and compare them to decide which one to use. I have also written the code for video editing in Python to convert the input video into frames. For this task I am using OpenCV. It is straightforward to do this. I have not yet decided how many frames I will take in the first round.
Discussed the feedback on the first draft with Xunfei. Got valuable feedback and planning to implement them in the second draft. I am searching for the algorithms to be used in the SLMS and working on the second draft.
Finished propsal first draft. Explored softwares for image processing.
Finished my first draft of proposal. I read some blogs about speaker verification tech and found out that I was wrong on some aspects (actually I was confused). Those blogs help me understand more and deeper about speaker verification. So I revised my framework and flowcharts: take voice input -> feature extraction -> modeling -> database. The modeling part is the most difficult part in speaker verification. The most popular models are: Hidden Markov Model, Gaussian Mixture Model, Vector Quantization, etc. I am not sure which one I will use for sure. It all depends on my dataset and customer need. I need to experiment several models to know which one I want the best. But I chose GMM temporarily on my proposal.
I only worked on finishing the first draft of the proposal this week.
This past week, I’ve finalized the basic design for the game I will be implementing. It will be a horizontal auto-runner, where the player ducks/jumps to avoid obstacles to the beat of the music in order to keep playing. I continued familiarizing myself with Unity2D, and plan on starting work on the game this upcoming week. Additionally, I wrote up the first draft of my project proposal.
This past week, my work has been split in two directions: First, I’ve been refamiliarizing myself with Unity, by means of going through my Game Design second project. Further than that, I’ve been familiarizing myself with Unity2D for the first time, which I plan on using for the senior project due to the simplicity as compared to Unity3D. Besides getting used to the main software engine I will be using, I also continued reflection on my proposal outline; I’ve been looking more into different PCG-G algorithms and have decided on using the chunk paradigm as my second stage generation algorithm. Its stages won’t be as directly aligned to the music, but it should improve efficiency.
I extensively worked on the proposal last week, reading more papers and writing out what I plan to do helped me figure out the scope of the proposed project. I also experimented a bit more with tensorflow. I made some changes to my initial framework design to now include a frontend and backend for the end-user to interact with.
This week, I revised my diagram again and selected papers to use for my proposal. I read through all of them carefully and wrote an outline for my proposal. I discussed the details of my ideas with Xunfei, modified some, and finished writing the proposal.
Much of my work this week has consisted on working through the first draft of the proposal as well as reading some papers for the next pass. I also found a couple other sources to use. These new sources were not research papers but rather articles related to my project.
I finally received the Gourmet Dataset. In fact, I received a devised version that has twice as many images as the original one. I also have the Yelp dataset, although that dataset has not be curated by humans, I am hoping to use it for training my algorithm in addition/instead of ImageNet or AVA.
Since I already have gotten access to the datasets, I have been reading about ResNet/AlexNet implementations, which was my goal for next week.
This week I met with Charlie to discuss my project design. We also talked about GIS extension, which is a library to handle GIS data for NetLogo. Charlie talked about how to layer different types of data on a base map. The most important tasks for the upcoming weeks are to figure out how to find different types of data for a fire location and how to process the data.
I worked extensively on finishing the first draft. I rewrote the design and related works section.
Uploaded papers I have finalized for the proposal. I will meet Xunfei to confirm the papers and talk more about the first draft. I have made an appointment with the writing center for the first draft. I spoke to the library desk and Jose regarding the project and got suitable feedback.
I have narrowed my project to studying Facebook Reactions and how reactions may differ based on the gender of the post creator. I have also found papers that focus on facebook reactions. Because Facebook Reactions were released as a feature in 2016, the papers on the subject are limited, and I haven’t found any relating to gender. However, some papers I found analyze facebook reactions in a way that would be interesting to compare between the gender of the post creator. For example, one paper uses the reactions to measure the controversy of a post, so I could measure if posts by women are more controversial in general than that of men. I also found tools from some of these papers that I could use in my project.
After meeting with Xunfei, I decided to modify my diagram a bit so I redesigned it from my practice proposal. I also collected more papers that could be used in my proposal and read more articles and research papers. I also found an online tutorial of a project that is closely related to mine, so I enrolled in the course for free and downloaded the jupyter file to play with it on my own. I think doing this now will help me figure out some possible options and directions for next semester. I also made a timeline of my work for this semester as well as next semester. I asked Xunfei some remaining questions about the proposal and my project in general to clarify my thoughts. I also checked out the rubric for project proposal and brainstormed ideas for my first draft of proposal.
In order to get more familiar with neural networks I decided to use a program that lets you create neural networks. In order to do this I started reading about tensorflow and tensorflow graphs and their inner workings like variables, constants and operations. I read some tutorials on tensorflow and also studied about the Keras model subclassing API which is one of the building blocks of tensor flow to start building a simple neural network. I also read I also searched for more papers that are similar to my research and read Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification, Semantic Clustering and Convolutional Neural Network for Short Text Categorization in order to familiarize myself more with neural networks that are used for text classification.
I have been looking more into the image processing part. I have created my first draft of the code to alter the colors of an image. I have also looked into the rotating of the food in the image. This seems not doable (in the way and scope I wanted to), so I changed my framework to take a video as an input, instead of an image. The video can then be split into images, and the images from the better angels will be picked. I have also written to the Gourmet Food Dataset researchers to ask for their dataset, but have not received a reply yet. I have been looking at the yelp dataset. I have found an online project that assumed all images taken with DSLR cameras were good, and the rest wasn’t. This seems to have worked pretty well for the classifying. I will look into that.