CS488 Update
by on with No Comments

In the past week, I worked on modifying my diagram and finished generating results. I also tried to draw meaningful insights from the validation results. After chatting with Xunfei, I finalized my poster and thought about elements to add to my paper.

CS488 – Week 15
by on with No Comments

This week I prepared my poster for the final submission, while editing also reworking my paper for the final submission next week. Last week I received feedback regarding user interaction and did research this week into what is the best way to build a GUI for my software.

CS 488 – Week 13
by on with No Comments

I have been working on wrapping up my project and worked on finishing up the final paper and the poster. The obstacle remains the same where the model gives out very erratic outputs for sentences that are not in its vocabulary but works well with sentences that it has seen before. I will keep continuing to keep working on this and try to figure out a way to make it work, but this goal might be outside the scope of the project given the current workload and time. 

CS488 – Week 14 – Update
by on with No Comments

During the past week, I have submitted the second version of my paper. After submission, I have continued working on the final parts of the paper. These parts include finishing the social engineering results and making the recommended changes to certain images to enhance my paper. With the given feedback, I have also started making changes to my senior poster which is due Sunday. 

CS488 – Week 14
by on with No Comments

This week I submitted my second draft of the paper, which required lots of results production as well as time to write. I graphed prediction trends between different materials of foreign data with my model which also was demonstrating how my model was performing. I received great feedback from my advisor regarding the poster and paper and will look to improve these both this week for final submission.

CS 488 – Week 12
by on with No Comments

I worked on finishing the frontend and wrapping up my project and figuring out what metrics to use measure and compare my models with. I also worked on the poster for my project. This upcoming week I will work on wrapping up my final paper and add the description of my second model to the final paper. 

CS488 – Week 13
by on with No Comments

This week we had the first draft of the poster due, which meant producing and visualizing a lot of results from my project. From this motivation, I compared my predictions across very different data (news articles, fictional novels, ect) and also was able to produce a convolution matrix that showed just how accurate my model was. This coming week I want to transfer these results into the next paper draft and continue with the user flow of the software.

CS 488 Update
by on with No Comments

I’ve mostly been working on making the poster and validating results. I found out that my original plan for validation would not work as nicely, so I will discuss with Xunfei to figure out what I can do. My diagram also needs to be tweaked.

CS488 – Update
by on with No Comments

I couldn’t do much due to comps, but collected all information from participants to conduct the validation process. I also ran their results on my software and recorded the results. This week, I will work on extracting relevant reviews from Sephora to evaluate the efficiency of my method.

CS488 – Week 12
by on with No Comments

This week I further improved the pre-processing of sentences so that they are cleaner and easier to read on output. I then downloaded some previous year project posters to help with designing my own and have already completed half of it. It showed me that now I need to work further on results to present, on the accuracy of my model outside of its dataset.

CS 488 – Week 11
by on with No Comments

I have finished the front end of the project, and am trying to wrap it up. One obstacle that I am facing is that my project proposed to have human testing, which will not be possible due to the current situation. I will be working on the poster in the next few days.

CS 488 – Update – April 6th
by on with No Comments

This past week I mainly worked on the first draft of my poster. It was much easier to complete since I have the majority of my project finished. In the coming week, I plan to continue my testing with my virtual servers, Kali and Metasploitable. Luckily, I have not encountered any obstacles when trying to use these two for testing. I also plan to continue work on the second draft of my paper. 

CS 488 Update
by on with No Comments

In the past week, I mainly worked on implementing a simple interface for my program. I decided to take text inputs for skin type and beauty effects and use a button that returns the recommended products when clicked. To test my program, I collected more input data from participants. I will be using them in the upcoming week to validate my methods.

CS488 – Week 11
by on with No Comments

During the past couple of weeks, I made some good progress on my project. I now have a functioning driver file that the user will run to train or validate a model, or to predict from their input data. This has allowed me to tie up different parts of my software into one functioning project. I also have predictions working (big step) and am currently working on the best way for the user to view their results.

CS 488 – Week 9
by on with No Comments

This past week I worked on my other model that uses both engaging/readable and non-engaging/readable advertisements, I found that this model does not perform well because of the lack of parallel data and sentence structures between the two different sentences. For the next week, I will work on refining my initial model as well as writing the frontend for the project. 

CS488 – Week 8 – Updates
by on with No Comments

The feature extraction module is finished (ready to use) now, but I am still stuck on the modeling module… The model I am using is called VGGVOX which is available on Keras. I am stuck on input pre-processing. The bug is on a function on BatchNormalization(). This function normalizes the activations of the previous layer at each batch. But the issue is not on this function, instead, it is on some deep layers of tensorflow innate functions… which i cannot modify. I am kind of lost which step is wrong exactly.

Project Description: Gender Bias Detection Using Facebook Reactions
by on with No Comments

Gender bias on Facebook might be measured by analyzing the difference in reactions on posts by women or men. My project is studying bias on Facebook pages of United States politicians using Facebook Reactions and post comments. Specifically, I am focusing on politicians running for US Senate in 2020. Data is being collected from Facebook pages of the politicians using a crawler and will be into a database. 

The data will be analyzed by performing sentiment analysis on the comments and using an entropy function on the reactions for each post. The comment analysis is both focused on whether a comment contains more negative or positive words, and if it contains more personal or professional related words. My hypothesis is that female politicians may have comments directed at them that are both more negative, and more focused on personal issues. I am using an entropy function on the reactions to each post to measure how divided the reactions are. Related work used an entropy function on reactions to measure the controversy of a post. My hypothesis is that, in general, posts by female politicians will be more controversial than posts by male politicians.

CS 488 – Succinct description
by on with No Comments

My project aims to develop models where we can predict the risk of having cancer based on both numerical data and image data. After training all the models, they will be analyzed to see which has the best accuracy and possible ideas to improve the accuracy. After that we can decide the best model to use if we want to predict the risk of having cancer.

CS 488 – Update – Week 8
by on with No Comments

Since I uploaded the architecture design last week, this will I will go back to posting the normal updates here – I have been slowly working on my second model that I will compare my initial mode to. I have not faced any obstacles yet except the learning curve that comes with learning Keras, but since Keras is well documented it does not take much time for me to figure out something that I am stuck in. In the upcoming week, I will keep working on the second model and plan to have it finished by the end of spring break.

CS488 – Elevator Pitch
by on with No Comments

Information informs our entire lives. Information shapes public opinion which shapes things like public policy, elections, the health and safety of the public, and more. No one is above the harm that can come from misinformation, which is why we need to fight against its spread.

Fake News as an area of research is relatively new and so some of the aspects are not very well researched, and new aspects to research pop up. Some existing problems in this research are that all of the solutions to these aspects are made in isolation, therefore no one solution can be used to find all instances of fake news, and that most solutions do not have an accessible, comprehensive user platform to disseminate their solution to the people.

This solution that I will provide will be a functional model of a user platform that demonstrates how an engaging and accessible one-stop-shop for fake news detection can work. It allows the user to interact in many different ways that require different levels of effort and is able to scale to include many different automatic detection methods.

CS 488 – Week 8 Update
by on with No Comments

In the past week, I worked to finish implementing non content-based filtering which recommends products based on the user’s skin type and desired beauty effects. I was able to apply the concept of TF-IDF to judge which ingredients are heavily related to each beauty effect. Now that all my methods are working, I will implement widgets to the python notebook to create some sort of interface so that I don’t need to change the input each time. I will also start revising the paper and validating my method.

CS488 – Elevator Pitch
by on with No Comments

My project is about extracting features from images. Using low-cost collection techniques such as satellite imagery or drone surveys, a database of positive and negative cases can be created. Additional information will be extrapolated from each image in the database using a combination of modern algorithms and combined back into a single imager as different colored layers of a JPEG image. These processed images, the goal of which is to provide as much information as is possible, are used to train a machine learning model. Hypothetically, the additional information provided by the edge detection algorithms will enhance the accuracy and reliability of the machine learning model, reducing the need for expensive surveying equipment.

CS488 – Week 8
by on with No Comments

This week I focused more on refining my idea and how it would flow for a user, which then helped me to create a flow diagram for this week. During this process, I realized some flows in my code were inefficient, so I changed the flow of information through certain functions to match up with my flow diagram.

I created a validation function to test a loaded model and also an argument parser to make it easier to pass values for different and important variables into the code.

CS488 – Week 7 – Updates
by on with No Comments

I developed the feature extraction module for my project and it is working. It now converts a voice input file (.wav) to a sequence of acoustic feature vectors. I tested with my own voice. The two files of my voice recording produce two very different sequences of vectors. But I think we cannot tell my looking at these numbers. They are just a list of numbers of the .wav file. I am still having bugs on my modeling module. I followed Charlie’s suggestion to learn TensorFlow from the basic. I build and trained a model with TF’s dataset and it worked. But this is just a basic try. I will keep looking at it.

CS488 – Week 6 – Updates
by on with No Comments

Last week CS was down so I couldn’t post my week6 updates. I finally finished the environment setting for my modeling module code. I am using a model called VGGVox Models which are created by the same authors of the dataset I am using. I almost gave up this resource because it is written in Matlab which I have never used before. But then I found a python resource guiding me how to import this model. However, I am still having bugs running this model. It says the true_fn and false_fn have different data types. I tracked the error and found that the error is in TF innate files which i cannot modify. But I don’t know which step that I pass data incorrectly.

Elevator Pitch
by on with No Comments

My senior project is to develop a technology that provides higher performance and security for target applications. It is called unikernel which is an optimized library operating system. Unikernel consists of the minimum set of components that a target application requires from a complete operating system. Unikernel is light weight and has higher isolation than containers. It will be the trend of running environment for applications in many fields such as cloud computing and high performance computing in the near future.

CS488 – Week 7 Updates
by on with No Comments

In the past week, I worked on creating a survey to take inputs for content-based filtering, modified the skin type test questions, and obtained some responses. I also worked on implementing non-content-based filtering using TF-IDF which I am struggling with. I will be meeting with Xunfei on Thursday and try to finish this part as soon as possible.

CS 488 – Elevator Pitch
by on with No Comments

My project aims to create a skincare product recommender system based on the user’s skin type and ingredient composition of a product. The main component of the project is content-based filtering and the secondary component is non content-based filtering. For content-based filtering, a user provides his or her skin type and selects a skincare product from sephora.com. The system then identifies the chemical components of products and uses cosine similarity to recommend products that have similar ingredient compositions. 5 recommendations for each product category are then made and returned to the user. Non content-based filtering allows users not to input the product if they lack knowledge or have not found a product they like. A user provides his or her skin type and desired beauty effect to obtain top 5 product recommendations across all 6 categories.

CS488 – Week 7
by on with No Comments

This week I began creating a model using the Keras Python library. I have been training it on the SemVal Task 8 2010 dataset, with accuracies of around 90% during training and 5 epochs and 60-65% validation accuracy. I was successfully able to save and reload the model.

I will be working on increasing the accuracy of this model in the coming week before applying it outside of its dataset.

CS488 – Elevator Pitch
by on with No Comments

My project aims to see how applicable semantic relation extraction models are outside of their dataset. Semantic relations are how we draw knowledge and facts from a text and no text is the same and when we research we usually look for these relationships regarding certain subjects in the text important to us. I want to see if a normal user can use state-of-the-art semantic models outside of their dataset to decrease the time needed to find specific knowledge about any entity in an unstructured text.

CS 488 – Week 7 – Elevator Pitch
by on with No Comments

My project aims to develop a reproducible penetration test that can help secure a large network. Tests will come from three different avenues- physical and technical testing, as well as social engineering. The results from these tests will be put together in a final report and given to the appropriate people who can make appropriate changes as needed.

CS 488 – Week 6
by on with No Comments

— Elevator Pitch — 

My project aims to use a sequence to sequence encoder-decoder model to make text-based advertisements more engaging and readable. This will help businesses get an edge over their competitors by attracting new customers as well as retaining their existing customers by making sure that their advertisements are readable and engaging to their target audience. This will be done through the analysis of pre-existing advertisements which will then be used to train the model on how to restructure sentences to make them more readable and engaging. 

CS 488 – Update – Week 6
by on with No Comments
  • This week I focused on getting part of the PolicialNews Data set from Castello et al. to work with Weka to be able to see if I can recreate the results used by their classification methods
    • Downloaded a tool to combine excel files into one sheet without data loss, manually added headers and an extra column denoting which was fake and which was real
    • But Weka still won’t load the data so that I can test it
  • Next week I will focus on making smaller versions of the data set to see what features are the issue for Weka and testing features individually; I will also look into Keras as a machine learning tool and see what kind of testing can be done

CS488 – Week 6
by on with No Comments

This week I was trying to run two different TensorFlow models with checkpoints, however, I could get the checkpoints to work which is key to my project. After a discussion with Dave, I have decided to implement a simple model myself using the Keras library since it is more abstracted and well documented, so it shouldn’t take too much time. I will be aiming for a minimum of 50% accuracy with my model and the SemVal-2010 Task 8 Dataset, which I think is the best dataset choice for this task.

Following this implementation, I want then start testing my model outside its dataset.

CS 488 – Week 6
by on with No Comments

This week I started working on writing the first draft of the paper. The models seemed to run probably when I tried to test with the small data set. However the accuracy was not what I wanted since there were not enough data. I will tried to implement these models on an online cloud computing system while waiting for Layout to be available.

CS 488 – Week 5 – Updates
by on with No Comments

I wrote the outline of the senior project paper. It is similar to my proposal but i changed my modeling method. i need to rewrite the modeling from GMM-UBM to Convolutional Neural Networks. But I am still having bugs on the CNN resource I gained from GitHub. I am pretty sure I will use MFCC as feature extraction. But if i cannot get proper resource of CNN I probably need to change to DNN or others. But I will try my best cuz I want to use CNN.

CS488 – week 5 – Update
by on with No Comments

In the last week, I have spent more time learning about using fastai with a convolutional neural network, specifically the resnet34 and resnet50 models, which I think will be ideal for my purposes. I have also been working with Jordan to get the modules for this set up on Lovelace.

I have also been scavenger hunting for more data. I have data from Iceland, and a few confirmed spots on campus which I can use for both positive and negative training cases, but more data is better for this kind of AI. My search has lead me to reach out to Tom Hamm, Greg Vaughn, The Earlham Library, and the Geology department.

Over the next week, I will continue learning about fastai and start implementing my model to be trained over the data that I have already.

CS 488 – Week 5
by on with No Comments

This week, I focus on the writing parts including the outline for the capstone paper. I also start implementing some neural network models for the image data set such as MobileNet and EfficientNet. I will try to test the model using a small sample training data set while waiting for Layout to be available so I can train the whole large data set. 

CS 488 – Week 5
by on with No Comments

I finished writing my initial model and have trained it using a small sample dataset. The accuracy of my initial model is quite bad. I am currently researching how I can modify and improve the accuracy of my model as well as downloading a larger dataset to increase the accuracy. I also worked on the outline of the first draft of the final paper. Currently, the obstacle is the lack of accuracy of my model. This next week I will work on finishing up the first draft of the final paper as well as increasing the accuracy of my model.

CS 488 – Updates – Week 5
by on with No Comments
  • I found a data set that would be the easiest to recreate results with
    • I just need to merge the data set of credible news and the data set of non-credible news with an added column denoting whether it was real or fake to be able to test
  • However, I ran into a major hiccup because Weka crashed and I can no longer open it.
    • I am documenting the errors and trying to reinstall it and fix but because for some reason I can’ t delete some of the old folders and so I still have not gotten Weka to start back up again
    • My hypothesis is that one of the packages is failing the whole opening process because I don’t have R on this machine
    • I am also afraid of force deleting the folder because I have no idea how it will affect Weka or my computer
  • I did successfully complete my outline but I do not have enough information to fill out the Results section and anything regarding my exact methodology.

CS 488 – Week 5 – Update
by on with No Comments

This week I spoke with Charlie and gained clarity into how Earlham handles guest connections through ECOpen and from there does not allow access to the network within Earlham but simply gives access to the outside internet. I also started my Social Engineering test on Thursday. I ran into some speed bumps with that and am currently resolving. For this next week, I plan to talk to Aaron in ITS about NMAP and its use to possibly check for any ports that are left open and could be vulnerable. I plan to meet with him sometime early next week.

CS488 – Week 5
by on with No Comments

This week I first worked on creating an outline for my final paper, which was useful as it sharpened my current understanding of my project and where it is headed. I also was working with a new model and was able to successfully train it, save checkpoints and load them. I also created basic pre-processing functions for my data to match the format of input sets.

Loading of weights did seem to not work with this model. When I reached a checkpoint with 80%+ accuracy and saved the weights, I followed up with loading the weights and feeding in test data from the dataset, but accuracy dropped to 5%. This was extremely confusing and is my priority to understand this week otherwise I will have to find another model.

CS 488 – Week 4 – Updates
by on with No Comments

TensorFlow is working fine on Lovelace now. But I just found that the demo uses TensorFlow 1 while the latest version installed on Lovelace is TF2………. The demo has a lot of code. I am not sure if i should work on this one and update all codes to TF 2, or just find another resource…… I talked to Xunfei, she told me to try other resource briefly. Because update that demo is not a small work.

CS488 – Week 3 – Update
by on with No Comments

I am still having issue on running the demo code from GitHub. I requested installation of TensorFlow in python3 on lovelace but it seems there’s still error. It is probably the issue of environment setting. I will communicate with the admins.

TensorFlow is working fine on Lovelace now. But I just found that the demo uses TensorFlow 1 while the latest version installed on Lovelace is TF2………. The demo has a lot of code. I am not sure if i should work on this one and update all codes to TF 2, or just find another resource…… I talked to Xunfei, she told me to try other resource briefly. Because update that demo is not a small work.

CS488 – Week 4
by on with No Comments

This week I did a lot of research and work on the more anthropological side of my project. I emailed Tom Hamm and Greg Vaughn and got some great information about where I could find the foundations of old buildings around campus that I could use for my project. This information will hopefully be detailed enough for me to create some labeled training images.

I also spent some time this week learning fast.ai, which I have settled on for now as the best option for identifying images. The library is extensively documented, and extremely robust. As soon as Layout or a similar machine is back up, I will be able to start testing code, but for now, learning the library is just as important.

CS 488 – Week 4
by on with No Comments

Last week I worked on collecting and preprocessing the data using Groupon API. I also started learning about and implementing my autoencoder model. So far the obstacle has been the learning curve but I have been extensively reading about neural networks and Keras and should be able to continue working on the project without any hiccups. Next week I plan to start my first draft of the paper as well as have a somewhat working version of the autoencoder model.

CS 488 – Week 4
by on with No Comments

This week, I started the data preprocess for my image data. The steps include, resizing, cropping, normalizing and lastly change to tensor value so that it can be fit in a neural network. For the numerical data set, I started looking into different algorithms which are not as computationally expensive as neural network such as k-nearest neighbor, support vector machine. In that way, I can test it when Layout server is still not available.

CS 488 – Updates – Week 4
by on with No Comments
  • I have spent this week analyzing the data sets that I have to see if there are any outside things that I need for these data sets to be able to be tested using Weka.
  • I have found that some required me to have my app registered with Facebook Developers and Disqus and some were not actually in proper .csv format and so Weka (the tool that I am using to test classification methods) could not read it.
    • This meant that I have a lot smaller pool of articles that I am able to replicate.
    • I have found 27 different data sets but I haven’t read all the papers those data sets are used in and some of the papers that mention the data sets are just explaining how they created the data sets and not how to use them in this context.
  • Because of all of these little setbacks, I am working on just finding smaller sample data to test Weka with, so that I can make sure Weka is working and I am focusing on recreating the results from Castello et al.’s work for the moment.
  • Castello et al.’s data format is different than what I have used for Weka before and I have to do some more digging to see if I need to combine the fake news data set with the credible news data set for each year first before sending it through Weka, or if I can just open both within Weka and tell it how to find what it needs.

CS 488 – Week 4
by on with No Comments

In the past week, I worked on generating five recommendations from each of the the six product categories. I still have a confusion about the cosine similarity formula so I’m planning to meet with other faculties in the following week while keep working on the next task. Other than that, there wasn’t any obstacle and I just need to make the function return the results in a nice and clean way.

CS488 – Week 4
by on with No Comments

This week I made efforts to get predictions from my model that was trained last week. However, after some hours spent understanding the code, I realized that this model is not for practical use but rather theoretical predictions, as each query set requires a supporting set.

Following this setback, I have now found some models to train from a smaller dataset in comparison to FewRel. I believe these models are able to be used practically on random query sets. With the smaller training time required for them, I should be able to verify which is best for my project this week.

CS 488 – Week 4 – Update
by on with No Comments

This week, I continued my testing for the physical aspect of my project. During this testing, I tried to focus on ECOpen since it says there is no encryption associated with the network. Come to find out, there is still an authentication process that one must go through when trying to connect to ECOpen. So when I ran a packet sniffer on a device that was on an ECOpen channel, I could not see any data. (This is a good thing and was noted). I also finished preparing my social engineering test which will begin tomorrow, February 12th. This next week will consist of my social engineering test, processing results from physical test, and working on my paper.  

CS 488 – Week 3 – Updates
by on with No Comments

Last weekend, I spent time with a small group of friends filling out a spreadsheet of information for 2020 Senate candidates. So far, 154/348 filed candidates have been added to the sheet. During that time, we discovered that a few candidates operate their campaign on a public Facebook profile instead of a Facebook page. In talking with Charlie, he guessed that the process in the API to collect profile data shouldn’t be too different from page data. Therefore, I am planning to collect this data as well, while noting the names with profiles in case their results are drastically different from overall results. Next, I plan to develop the scripts to start collecting and analyzing small amounts of data, planning to scale and automize them later.

cs488 – Week 3
by on with No Comments

This week was a big planning week for me. I spent a lot of time writing down notes and ideas, as well as researching the details of what I need for my project. I also spent some time gathering resources for my project in the form of data from Iceland. A combination of 2018 and 2019 data will provide me a much-needed training/testing case.

I have progressed in my implementation, further streamlining the process of creating various edge detections of original images. This week I added the Prewitt edge detection algorithm and improved my Caney edge implementation to have a tight, wide, and auto mode.

I have also been researching technologies for image recognition via machine learning with multiple channels. This is the idea that a single “object” in the AI can have multiple images associated with it, and it is necessary for my project.

CS488 – Week 3
by on with No Comments

This week I was able to create a saved checkpoint of my learning model for semantic relation extraction. This hopefully means I won’t need to train it further and can now focus on feeding it my data, which now needs to be pre-processed before being fed into the model. A basic GUI window was also up and running this week with PyQt5 which was great to see! I will be writing more code in the coming weeks now so I need to ensure that my project files are organized.

CS 488 – Week 3
by on with No Comments

This week, I tried to implement some models and was hoping to get it on our Layout server with GPUs. However, the system admins were still working on that and I could not ssh to the server. Therefore, I created a google cloud free trial account and started writing and testing my model on their server. 

CS 488 – Week 3
by on with No Comments

Since my project involves a significant part that’s marketing, I was advised by my instructor to talk to Seth and other professors about how I should approach a dataset. After talking to them, I have decided that a good approach would be creating a dataset using the readability formulas. First I will calculate the average readability and then filter the dataset using that average readability. A marketing dataset has been extremely hard to find, but asking around has led me to the Groupon API – it lets me get 100 deals per second which will help me easily scrape millions of deals in a few days. I plan to run a script in the background that does it. Since last week, I have also successfully implemented word2vec using Genism – a python library. 

CS 488 – Week 3 – Update
by on with No Comments

In the past week, I worked on calculating the cosine similarity between the ingredient composition of an inputted item and that of the rest of the items in the data. I am struggling to decide on which formula to use for this, since the related project used the equation different from the “typical” formula used to compute cosine similarity. I will need to look into this more next week. 

1 2 3