CS388 – Week 4 – Update

with No Comments

I am reading papers for my first idea, which is “Detect and Translate Chinese text in images”. One research that I read was about improving the performance of the Optical Character Recognition for Chinese books that are in precarious conditions. Instead of trying to enhance the image quality, their research applies N-gram, long short-term memory, and backward and forward N-gram statistics text model to develop a more accurate OCR model.

CS388 – Week 3 – Third Idea

with No Comments
  • Name of Your Project

A Real Time Fall Detection System to Assist the Elderly Using Deep Neural Networks

  • What research topic/question your project is going to address?

The elderly have a high chance of falling and get injured or faint. This might put them to danger if they are alone. One way that can help the elder people is having a system that can monitor their actions, detect the falling action and other behaviors after falling down, classify the levels of severity and send an alert to their emergency contacts or the emergency room if the level is serious.

  • What technology will be used in your project?

Deep learning, pattern recognition, image processing

  • What software and hardware will be needed for your project?

Python, PyTorch (or Keras)

I might also need a CCTV camera if I decide to build the actual device.

  • How are you planning to implement?

First I will apply some image processing techniques to enhance the images and videos quality. If the dataset is small, I will use of image data augmentation techniques to produce more data. Then train the model that detect the person falling in the photo frame using deep neural networks, then use the people falling photos and videos to train a model that classify the level of severity. When the index of severity passes a threshold, send out the alert.

  • How is your project different from others? What’s new in your project?

There are several projects that work on the similar problem. Most of them work on detecting the falling action only. In this project, I hope to build a system that is more detail and can decide whether it is an emergency case.

  • What’s the difficulties of your project? What problems you might encounter during your project?

I might not be able to find a big enough dataset to train the model.

Idea 3

with No Comments
  1. Name of Project

Automating laptop checkouts from CST front desk using image recognition

  • What research topic/question your project is going to address?

Although we need a human to address the needs of guests in the welcome desk of CST, it would be ideal for the worker and students if we can automate the process of MACs’ checkout. Humans are prone to error and we do not want any student worker to be liable of errors that could cost them thousands of dollars. So this project would allow a machine to handle the checkout using a camera to identify the laptop and the student wishing to check out the laptop and remove the process from the desk worker completely.

  • What technology will be used in your project?

Image recognition, machine learning models,

  • What software and hardware will be needed for your project?

This would need a good quality camera, python, and some database management software

  • How are you planning to implement?

Have a camera stationed above the cst desk. Also I think it would be beneficial to change the barcodes in the laptops to bigger QR codes for easy recognition and better visuals for the camera. Use various machine learning models to train the software to recognize students and identify unique laptops. This product should also send out emails regarding reservation details to the students like the current system does.

  • How is your project different from others? What’s new in your project?

This is different implementation from the current process we have in that we are removing human responsibility from this procedure. This will hopefully reduce human error in the process and decrease financial liability to student worker and the institution. It is also scalable to other use cases (like Runyan desk for example) to increase automation and improve efiiciency.

  • What’s the difficulties of your project? What problems you might encounter during your project?

The problem I anticipate is making sure the model I have does not mis-identify students checking out the laptop or mistaking someone walking by the cst desk as someone checking out a product. Lighting might also be some issue as the desk is besides huge windows and so lighting is very different in night vs day, or summer vs winter. Another issue to consider is the camera quality (need to get good camera under reasonable budget)

Idea Number 3

with No Comments
  1. Name of Your Project

Ans: SARS

  • What research topic/question your project is going to address?

Ans: Using trained neural nets to be able to tell when a statement/sentence is sarcasm 

  • What technology will be used in your project?

Ans: NLTK and 

  • What software and hardware will be needed for your project?

Ans: Botmock is the only software that will be needed for this project

  • How are you planning to implement?

Ans: I plan on making this an extension of Botmock

  • How is your project different from others? What’s new in your project? 

Ans: With my project, I am using the same method of using CNN model hierarchy when it comes sentiment analysis to learn the context and space in which the sentence exists

  • What’s the difficulties of your project? What problems you might encounter during your project?

Ans: Every sarcasm exist in a defined space one difficulty of this project is trying to build a barrier for that space. Another problem would be getting access to Botmock’s API to make this application compatible.

CS 388 Idea 2

with No Comments
  1. Name of Project

Visual representation of nation’s development level

  • What research topic/question your project is going to address?

The goal of this project is to use the various world bank data that is available to evaluate different development metrics for each nation. Then I want to use visualization tools to effectively communicate to the interested audience. The visuals will change as the indicators for the countries change so the website would be a ‘live image’

  • What technology will be used in your project?

Api, data visualization tools like Tableau or python, statistical tools to calculate the indicators and compare between nations.

  • What software and hardware will be needed for your project?

Python, SQL, json. Maybe some database management system to store the data. Tableau for visuals.

  • How are you planning to implement?

I want to pull the data from various data sources like the world bank website using api and load it into some sort of database. Using this data, I want to use some tools to calculate and compare the indicators of development for various countries. The output from these calculations would be then visualize in a website live and these visuals would change based on any changes noticed in the world bank dataset.

  • How is your project different from others? What’s new in your project?

I want to create a live version of this problem. I found a few websites that visualize these metrics or tabulate them, but it is hard to interpret for people who are not very informed about the topics involved. I want to make my website very intuitive so people with different experience levels can look and interpret the data intuitively.

  • What’s the difficulties of your project? What problems you might encounter during your project?

The problem I anticipate is figuring out how to have the database where I store my data update in a lively manner so that any changes in the data bank is represented instantly in the website without any intervention required. I will have to learn various methods that are hopefully available readily that can make this possible for me

CS388 – Week 2 – Second Idea

with No Comments
  1. Name of Your Project

Driver Drowsiness Detection Using Deep Neural Networks

  • What research topic/question your project is going to address?

Driving while feeling sleepy or tired is one of the main causes of traffic accidents. One solution for this might be having a device in the car that monitor drivers’ behaviors and facial expressions and ring the alarm if the drivers tend to fall asleep.

  • What technology will be used in your project?

Dataset of facial expressions (images and videos)

  • What software and hardware will be needed for your project?

Python, PyTorch (or Keras)

  • How are you planning to implement?

Build a pipeline that first apply some image processing techniques to improve the quality of the images, then train a model (using neural networks) to detect and locate face position in the images, and the last step is to build a model (also using deep neural networks) to classify the behaviors and facial expressions.

  • How is your project different from others? What’s new in your project?

Most relevant projects track the drivers’ eyes to see if they close their eyes. I am considering checking eyes movements and also other behaviors such as yawning or nodding off in order to improve the classification performance.

  • What’s the difficulties of your project? What problems you might encounter during your project?

There might not be a big dataset for me to use.

CS388 – Week 1 – First Idea

with No Comments
  1. Name of Your Project

Detect and Translate Chinese text in images

  • What research topic/question your project is going to address?

Lately many translator applications have introduced the new feature that can scan a document or take an image with texts to detect and translate the texts into another language.

Many of these applications perform well with very neat and clear handwriting or high quality images but not quite well with cursive handwriting or low quality images. My research goal is to improve the detection performance in these cases.

  • What technology will be used in your project?

Chinese – English Dictionary API

  • What software and hardware will be needed for your project?

Python, PyTorch, matlab

  • How are you planning to implement?

Build a pipeline that first enhance the quality of the image data using image processing techniques, then feeds data to a deep neural network model (maybe CNN) to detect the Chinese characters and connect to a dictionary API to translate the text into English.

  • How is your project different from others? What’s new in your project?

The current applications do not perform very well on low quality images, so my goal is to find solutions to this limit of the translation apps.

  • What’s the difficulties of your project? What problems you might encounter during your project?

I did some experiments and found that big apps like Google Translate still had trouble detecting the not-very-neat handwriting. Therefore it could be very challenging to achieve my research goal.

CS 388 – Week 14 – Updates

with No Comments

This past week, I spent time working on the presentation. I also met with Charlie to discuss the presentation slides. Charlie told me to replace a table in the Motivation section with graphs to show how wildfires have increased overtime. He also told me to cut down some texts in the Related Work sections. For the Proposed solution, he told me to redesign my graph. I also had to add more details in the Timeline and Budget sections. I also worked on the final paper during the break by adding the new requested sections. 

CS 388 – Week 11 – Updates

with No Comments

For this week I looked into many different datasets, including gis fire map data https://frap.fire.ca.gov/mapping/gis-data/ and Kaggle dataset https://www.kaggle.com/elikplim/forest-fires-data-set but couldn’t find what I was looking for.

I picked The Ranch Fire in California but couldn’t find good datasets for it. I was trying to find I’m trying to find elevation, wind direction, humidity, and vegetation. 
All of them have to contain coordinates so I can layer them together. Also, I need each set at different time stamps for the simulation. I will discuss this during the next weekly meeting.

CS 388 – Week 12 – Updates

with No Comments

For this week I have done the following tasks:

Learned to view shapefile contents with Netlogo and Python library pyshp. A shapefile is an Esri vector data storage format for storing the location, shape, and attributes of geographic features. This type of file is quite complicated so it took me sometimes to understand the format and its contents.

I had trouble finding the right dataset for my project. Charlie suggested that I look into https://www.frames.gov/afsc/partners/fmac/guides-products. I downloaded the data for Alaska but it does not have the contents that I was looking for. Finding the right data is currently a big challenge.

I am also getting more familiar with Netlogo. Using Netlogo, I could view the content of the data for Alaska and also extract the metadata using the command line provided with NetLogo.

CS388 – Week 10 – Update

with No Comments

This week I met with Charlie to discuss my project design. We also talked about GIS extension, which is a library to handle GIS data for NetLogo. Charlie talked about how to layer different types of data on a base map. The most important tasks for the upcoming weeks are to figure out how to find different types of data for a fire location and how to process the data.

CS 388 – Week 9 Updates

with No Comments

I discussed my new idea with Charlie and Xunfei. I searched for more papers about 3D modeling and volume estimation but could not find a lot. I will be creating an Andriod application, so I looked into Andriod camera API and found that I can specify the distance between the food and phone camera until it satisfies the requirement. I plan to include face recognition as authentication for privacy purposes and found a GitHub repo for it that I can use. I also found a paper that is more closely related than what I have found so far.

CS 388 – Week 8 Updates

with No Comments

After coming back from CMU workshop for CS researches, I have decided to modify my idea a bit to integrate more CV into the project. From recipe recommendation and calorie estimation, I have decided to focus only on calorie estimation. There are many calorie estimation software that requires users to have a reference object when taking a picture of food. As much as this method has brought food calorie estimation to a new level of accuracy, it is inconvenient for users as they need to have the reference object with them at all times.

In my project, I aim to solve this problem as well as to bring the accuracy of calorie estimation to another level. Users will scan the reference object the first time they set up the application. The scanned object will be saved in the database as a 3D object with its area and volume. Next time the user scans the food, the object will appear next to food. These two will be compared and extract the volume of food from it. From volume, the calorie of food will be estimated.

CS388 – Week 9 – Update

with No Comments

For this week, I started working on NetLogo, the software that I plan to use to create the simulation model for my project. I looked into the tutorials and the sample models library. NetLogo has its own programming language and development environment, so I spent quite some time to study its ecosystem. I also created a simple simulation model that read a file containing elevation information, display the elevation in different shades of green, built some fire sources and let them spread to places where elevation was smaller than 500. All of my notes for NetLogo can be found in Box.

Charlie and I also discussed my design for the project. For now I will focus on four types of input: Wind, Elevation, Temperature, and Humidity. First I will explore them individually to see how each affects my model. Then I will combine them, two at a time, to explore their combined effect on the model.

CS388 – Week 8 – Update

with No Comments

I met with Charlie for the weekly meeting. We discussed different designs for my simulation model. I will first create some input data, which includes creating dummy values for the base map instead of using a real map. The main program will contain simple transition functions. This is to make sure that I can produce a simple version of a simulation model. I will also have to look more into NetLogo, especially into its fire model libraries.

CS388 – Week 7 – Update

with No Comments

I’ve finished writing the Literature Review for my idea “Fire Spread Simulation Using Cellular Automata.” After reading the papers for my research, I found a recent paper on this topic which used Machine Learning to solve the drawbacks of previous research. However, I could only find one paper using this technique so I will have to dig deeper to find more related materials. Charlie has suggested that I should categorize the papers based on the input data (terrain, weather condition, etc).

CS388 – Week 6 – Update

with No Comments

I’ve chosen “Fire Simulation Using Cellular Automata” as my final idea. I have also met with Charlie and decided to meet every week on Monday. I will also meet with Xunfei regularly for my research. Xunfei has suggested that I should look into ArcGIS for the simulation part of the research and also suggested me to talk to Jose as ArcGIS would require funding.

CS 388 – Week 6 Updates

with No Comments

I have decided on the project I will be working on as my senior project. I have talked to Charlie about it, discuss my ideas regarding this project. He will be my advisor for the project. I have found 10 more papers and a couple of technologies I might be using. I have also found the datasets of food and recipes I will be using for my project.

My final idea is nutrition management and recipe recommendation system. Users will be able to scan the ingredients they have using the app and the app will recommend recipes using the user input they have put before such as any allergies, or food they don’t want to or cannot consume. The next step of my project will be the calorie estimation of food the user will consume. For this part, I plan to use a texture mapping and scanning for the optimum estimation of calories, and ingredients. For the privacy issues, I plan to have users scan their face on the first use of the app and have an API that will determine whether the current user is the user of this account. I am still thinking about possible ways to detect liquid ingredients and seasonings of the food.

CS388-Week5-Update

with No Comments

Here are my comments for the papers I read this week.

[1] Forest fires spread modeling using cellular automata approach.

They described a method using cellular automata to simulate how fire spread over an area of island Brac in Croatia. The paper had a great overview of Classification of forest fire models, explanation of cellular automata, well-known Neighborhood Templates, and how Landscapes can be represented as cellular automata. They mentioned that only vegetation characteristics and wind conditions were taken into account as input parameters. I might include more input data if I use this model.

[2] Computer vision system for fire detection and report using UAVs Special Issue for Submission.

The main concerns of the paper was how to detect fire using computer vision techniques as well as hardware systems. The paper serves as an explanation to their system rather than how their system is compared to other fire detection models. I might use this pa- per for my research if I want to establish a communication system later on.

[3] Using cellular automata to simulate wildfire propagation and to assist in fire management.

Unlike the cellular automaton mentioned in the other two papers, this one did not take into consideration the state of stress of vegetation and the meteorological condition. If it possible, I would like to develop a system that can output different simulations based on different cellular automata models based the ones in this paper and in the other two mentioned above.

[4]  An FPGA processor for modeling wildfire spreading.

The model was designed to not require too much computational resources and computational power so that it could describe fire behavior in real time. I might use this model if I want to design my simulation model in real time.

[5]  A Cellular Automata model for fire spreading prediction.

The result was a model of cells that evolve with given transition rules. This model forms the basic foundation my research. I can implement a similar model with these transition rules.

[6] Forest fire spread simulating model using cellular automaton with extreme learning machine Extracting Traffic Events and Human Mobility Patterns in Geosocial Media Data for Assessing Real-time Road Traffic View project Understanding human activity pattern.

They mentioned that the accuracy of this model was between 58.45 and 82.08%. I do not think a simulation accuracy of 58.45% is a reliable. This research also used cellular automaton to pre- dict fire propagation, which is similar to the paper ”Forest fire spread simulation algorithm based on cellular automata.”

CS 388 – Week 4 – Update

with No Comments

Below are the papers I read for each idea this week. I also read more papers for my third idea, “Fire Spread Simulation” and found the cellular automata algorithm mentioned in one of the papers quite interesting. The basic idea behind the model is to break down the images of the forest fire into different cells and apply the same algorithms onto each cell, similar to divide and conquer. The authors have also developed an improved algorithm based on this model to not only predict how fire spreads but also trace the source of fire.

Improve Fire Identification Mapping and Monitoring Algorithm (FIMMA) 
[1]  Z. Li, W. Jiang, F. Wang, Q. Meng, X. Zheng, and B. Liu, “GIS based dynamic modeling of fire spread with heterogeneous cellular automation model and standardized emergency management protocol,” Proceedings of the 3rd ACM SIGSPATIAL International Workshop on the Use of GIS in Emergency Management, EM-GIS 2017, 2017.
[2]  X. Xie, J. Wang, H. Qin, and X. Cheng, “The simulation and research of fire spread situation based on osg,” ACM International Conference Proceeding Series, pp. 156–159, 2019.

Fire Detection Using A Combination of Different Image Analysis Techniques

[1]  L. Giglio, W. Schroeder, and C. O. Justice, “The collection 6 MODIS active fire detection algorithm and fire products,” Remote Sensing of Environment, vol. 178, pp. 31–41, jun 2016.
[2]  Y. Long and X. Hu, “Spatial partition-based particle filtering for data assimilation inwildfire spread simulation,” ACM Transactions on Spatial Algorithms and Systems, vol. 3, no. 2, aug 2017.

Fire Spread Simulation 
[1]  X. Rui, S. Hui, X. Yu, B. Wu, and G. Zhang, “Forest fire spread simulation algorithm based on cellular automata,” Natural Hazards, vol. 91, no. 1, pp. 309–319, 2018. 
[2]  S. K. Singh and S. Kanga, “Forest Fire Simulation Modeling using Remote Sensing & GIS,” International Journal of Advanced Research in Computer Science, vol. 8, no. 5. [Online]. Available: https://www.researchgate.net/publication/325848449 

CS388- Week3- Update

with No Comments

For this week, I read the papers on Moodle, met with Charlie to discuss my ideas, and read papers related to them. Below are some updates for each topic.

Idea 1 Title: Improve Fire Identification Mapping and Monitoring Algorithm (FIMMA)

Description: Enhance FIMMA to reduce the number of false-positive results and apply the algorithm to detect fire in urban areas.

Update: This algorithm has been developed for the past 20 years by NASA. It has gone through many modifications and enhancements, and is currently producing quite accurate results. The main reason why it cannot detect fire instantly is because of the time the satellites take to circle the Earth.

Idea 2 Title: Fire Detection Using A Combination of Different Image Analysis Techniques.

Description: The research aims to detect fire by analyzing different attributes (heat, temperature of the surrounding area, temperature and color of the smoke) at different ranges of the electromagnetic spectrum (UV, visible, and infrared).

Update: There have been numerous unmanned vehicles being developed to take images of a fire scene at different electromagnetic spectrum. A drawback of detecting fire using image analysis might potentially lie with the hardware rather than the analysis techniques. For example, the resolution of the images and the hardware of the camera are usually affected when the vehicles try to take photos too close to the fire source.

Idea 3 Title: Real-time Fire Tracking System.

Description: The research aims to create a system that can provide current data for active fire as well as calculate the direction it is moving. The system can combine the data from Google Earth and NASA’s Fire Map to provide the current condition of the fire. It will also need real-time data of a given location, particularly weather (wind, humidity) and local condition (population, buildings) in order to make predictions. 

Update: Since last week, I have been focusing on this idea more than the other two. What I am planning to do is to build a system that can produce a simulation of how an active fire spreads in a specific fire situation, particularly in forest fire. Many factors will have to be considered to build this simulation, especially data about the surrounding environment, weather, and terrain, etc. I plan to use archived data so that I can compare my predictive result to the actual fire spread direction.

CS 388 – Week 2 (3 Ideas)

with No Comments
  1. My first idea is to create an application that scans the picture of foods and let users know what ingredients are in the dish. I am still deciding whether I want my app to be used as a diet and nutrition guide or aid for visually impaired people. This application will be available in multiple languages (at least 3). Xunfei had provided me more questions to explore as feedback. I will integrate computer vision and natural language processing in this project.
  2. My second idea is a program that detects prank calls made to 911 or other emergency centers. I will focus on details of the caller’s speech such as the urgency, intonations, breathing, etc. as well as background noises like whether the background is too quiet or too loud or is there any footsteps, etc.
  3. My third idea is to generate speech from a user’s hand gestures in several languages. I plan to piggyback an already existing and working program that translates hand gestures to speech. My main focus will be on improving that program and working to the accuracy of language translations.

CS388 – Week2 – Three Ideas

with No Comments

1 ) Name of Your Project: Improve Fire Identification Mapping and Monitoring Algorithm (FIMMA).

Enhance FIMMA to reduce the number of false-positive results and apply the algorithm to detect fire in urban areas.

2 ) Name of Your Project: Fire Detection Using A Combination of Different Image Analysis Techniques.

The traditional methods to detect forest fires such as using mechanical devices or humans are not effective on a global scale. With the advancement of technology, detecting forest fire using image analysis has proven to be promising due to its low cost and effectiveness on a global scale. However, these analyses generally focus on only one technique, either analyzing the images in one range of the electromagnetic spectrum or study the heat signature of the fire. The research aims to detect fire by analyzing different attributes (heat, temperature of the surrounding area, temperature and color of the smoke) at different ranges of the electromagnetic spectrum (UV, visible, and infrared).

3) Name of Your Project: Real-time Fire Tracking System.

A wildfire usually spreads rapidly within hours from the start, which means responding quickly to the fire can lead to fewer damages and casualties. The research aims to create a system that can provide current data for active fire as well as calculate the direction it is moving. The system can combine the data from Google Earth and NASA’s Fire Map to provide the current condition of the fire. It will also need real-time data of a given location, particularly weather (wind, humidity) and local condition (population, buildings) in order to make predictions.

CS388 – Week 1 – First Idea

with No Comments
  • Name of My Project

Managing Fire From Space

Improve Fire Identification Mapping and Monitoring Algorithm (FIMMA).

  • What research topic/question is my project going to address?

Space technology has been used to improve life on Earth through many applications. One of them is to use Earth observation satellites to detect forest fires and alert local authorities in real time.

NASA has been developing FIMMA algorithm that analyzes data taken from Earth observation satellites to detect possible forest fires.

The FIMMA algorithm has several limitations, which often leads to false-positive results. The research aims to address and find possible solutions to the current limitations and produce a better algorithm.

  • What technology will be used in your project?

Fire Information for Resource Management System (FIRMS).

  • What software and hardware will be needed for your project?

Data manipulation and graphing tools.

  • How are you planning to implement?

Compare the current implementation of FIMMA algorithm with other fire detection algorithms and fire products.

  • How is your project different from others? What’s new in your project? 

The algorithm is currently only accurate over forested regions. The algorithm may miss real fires over urban areas, as well as many agricultural burns. A focus of the research is to improve the fire detection accuracy of these regions.

  • What’s the difficulties of your project? What problems you might encounter during your project?

The research requires a deep understanding of the algorithm, geology, satellite technology, and data science.

It can be challenging to obtain the most updated progress of the algorithm.