This week, I started working on my poster, and revised my paper based on the feedback I received from Dave and Ajit
- The spring break week dealt with mapping MIDI control into the acoustics of the three-dimensional space, which I have successfully done.
- I have also successfully added an FIR based buffer into the program as well to track analytics. You use buffir~ object when you need a finite impulse response (FIR) filter that convolves an input signal with samples from an input buffer. With that, you can look at an FIR filter response graph in MaxMSP which calculates the filter response of the FIR in the main sound patch for a given number of samples (coefficients)
Created and ran experiment. Noted the results down.
Finished the first draft of the Capstone paper. Currently revising and polishing it.
This week I have been:
– Continuing experimenting with different hyperparameters of my models.
– Writing analysis based on the results.
– Continuing to work on the paper.
This week I turned in a new draft of my paper and fixed a few bugs. I am now ready to collect most of the data I’ll need.
For the past week I:
- Completed implementing trading simulation
- Worked on the initial draft
- Started working on SVR
This week I will do:
- Will keep working on SVR
- Work on the results part of my paper
This week I tested out the reader with student interface, and checked if self-check in and check-out worked properly. I also met with Craig and discussed plan to migrate the application to the server and perhaps have it ready for the EPIC expo.
This week I have:
– Reran the tests of my model and continued analyzing the results.
– Worked on my paper.
This week I spent most of my time working on the paper, and finished the implementation and design sections. I also started working on the administrator interface and got a good portion of it done.
For the past week, I worked on implementing the trading simulation. It has been taking more than what I thought but I plan to complete it by the end of this week.
I was not able to work as much as I hoped due to the other commitments I had before the spring break. But as I plan to stay on campus over the break, I will try to complete the implementation part as much as possible and will complete the initial draft due during the break.
This week I have dealt with working on improving my final paper and working on the MIDI mapping and audio of my project inside MaxMSP. I primarily worked on getting the audio format correctly to be able to loop and connect each file with the object associated in the three-dimensional space within MaxMSP. I secondly worked on the formatting of my paper; it needed to flow from topic to topic more also recognizing why it was relevant to the project as a whole.
This week I have been working on my paper both revising the submitted partial draft and adding the portions that were not done yet.
I have also done some work on my project itself. I have been doing a lot of refactoring. I made it so all the variations I am testing can be turned on and off via toggle buttons and all things that need to be initialized to run the patch are accessible in the main patch window instead of having to open the subpatches. I also revised portions of my project that had large repeated sections of code.
This week I have been:
– Continue with the implementation of my design.
– Set up a hyperparameter space to analyze how they affect the model.
– Test the model using different parameter combinations from that set.
– Continue with my paper.
This week I
-implemented feedback from the first draft
-started writing the rest of the paper
-did some minor debugging and commenting
Built a functioning, testable (not yet accurate) Neural Network that takes in my input of heuristic data and output a direction.
Since the puzzle states were strings, I wanted the output to be the value added to the index of the “_” to move to a new position but negative values cannot be used as targets or labels in neural network so instead I coded them using values 0-3 to represent each move.
I created a function that takes in the output of the neural network and converts it to a move for the puzzle.
Also know which specific layers I need to experiment with make my neural network be accurate.
- Found some more research papers to read which were related to the topic that I chose.
- Worked on the literature review.
In past week, I have spent time doing research to find interesting papers that I think will be related to my ideas. For each of my ideas there are at least five papers to take a look on at thee moment. However, for the last idea about a mobile application for time scheduling, I could only find some topics about effective time scheduling and mostly are just about general mobile application development. I have also read some paper for the first pass in order to briefly know what the authors were proposing in their papers and how well are those paper fit to my topics. I am also looking for available data sets for my first two ideas.
I have continued working on my annotated bibliography, but I have not finished it yet. I am finding it hard to focus on and be motivated for a project that I can not physically work on. I am still missing a couple sources for the annotated bibliography and I have been a bit lazy about looking for them. I think I will start by looking at other papers that I have found citation’s.
This week mainly dealt with the writing of my final paper. My paper was mainly a mostly complete draft that worked for my initial goal and vision of SoundStroll 2.0. However, in its finality, it will be called SonoSpatial Walk and and it will be its completely own project. The added changes allow for generation of objects through MIDI, and hopefully other things will be allowed with the source code that was completed changed by myself as well. Hopefully, with the object generation, I will be allowed to work on the sound properties that play and loop through triggers. Also with the sound properties, I’m hoping I can resynthesize sound in real time using Fourier Transformations in order to complete change the sound as well.
Over this week I worked on finishing a first draft of my paper. Also I worked on debugging code and trying to install some packages.
This week I:
-worked on my paper draft
-worked on fixing bugs in my software
For this week, I have been:
– Finish writing the first draft of my paper.
– Continue reading ways to determine implicit ratings from the data.
– Test and analyze the initial results of the recommender system.
Read the CS papers help documents to understand how to read and understand the long papers.
Finding difficulties in searching for papers directly related to my ideas. Read a couple of papers according to the instructions which helped understand the material.
Met with Dave and Ajit to discuss the 3 ideas. The first 2 ideas were supported and I received some extra information about some features I can add to the project.
ACM membership is set- up and I am trying to refine the 3 ideas by looking up CS papers and discussing with fellow peers.
For the past week, I primarily worked on finishing up the partial draft of my paper.
I reread some of the relevant papers for my research and modified the related work section so that the connection between my research and each cited work is more clear.
Also in my topic proposal, I had a huge list of related works. However, in my partial draft for the capstone paper, I only cited the papers which are most important for my research and modified them to fit into the new format.
This week I finished implementing a rough version of the student user interface. I spent a considerable time discussing the logic behind to student check out and check and what measures were necessarily to put in. I received feedback from Ajit on the design, and modified my approach based on that.
This week I have been:
- Working on the first draft of my paper.
- Reading a research paper that use the same dataset as my project to derive a formula to get the implicit rating from the song play count.
- Writing an initial version of the collaborative filtering algorithm, will try to test it next week.
I have been a little bit behind on synthesizing my articles into the annotated bibliography, I mostly have just been trying to collect more and more articles. The gesture control Idea and the educational app idea seem a little difficult to complete in a semester of work so I think I will probably continue forward with the PCG for my video game, It is something I am definitely passionate about, I cant say the say for the others.
-I created a large variation to my algorithm that spacializes the visualization based on the ratios just intonation ratio instead of being directly correlated to the frequencies. This involved implenting a new module that calculates the just intonation ratio and scales the sine wave visuals to that.
-I created tables to use in my paper draft and started turning my outline into my draft.
This week, while I was not able to work on the coding aspect to my project, I did receive a synthesizer from Professor Forrest Tobey, and have been working extensively in MaxMSP to get it to react to my program.
Also, I have been working on my paper to get it ready for the 02/25/19 deadline.
I’m still trying to get the data over. I’ve been working on it for like 2 to 5 hours everyday but errors keep arriving. I’ve also researched about algorithms and have a better idea of my project scope.
I tested that the last of my dependencies is on the cluster and functional. Started implementing the second major piece of my approach, SURF. Taking a little longer than expected but not a major delay.
Changed a piece of my pipeline and created some issues. Working on fixing those, and hopefully improving the overall functionality in the process.
For the past week I:
- Limited the training data to consider only events after the crises in 2008
- Tried to predict the value for after 1 to 20 days and examined how accuracies changed
- Started working on trading simulation (Bollinger band)
This week I will:
- Complete the partial draft, which is due Feb 25th
- Complete implementing the trading simulation
Managed to have Keras, an open source network library, installed in Jupyter.
Currently focusing on building a sample neural network, adjusting the data into a format that can be used by the network, iterating on the architecture required for the Neural Network and writing a draft of the first few sections of the Capstone paper.
This week I discussed the structure of my paper with Ajit, and received feedback on how to explain the design and implementation sections. I also started working on implementing the student user interface.
For this week, I have been looking for research papers on the three topics that I have chose, specifically for the Text Categorization and the Air Quality Monitoring system. Two things I found out:
– Air Quality Monitoring system:There have been many research on a low-price air quality system, some of which use the same technology as I do. This gives me two advantages: First is that I can reference and study those paper for my research. Secondly, I can compare the results I made on my model with that of other researchers so that my model monitoring system would be the same as the others (which would indicate that my model is working correctly).
– Text Categorization: It has been a research and application topic for over a decade. Many researchers have made considerable amount of progress on this topic, which provides me with more insight into the way to approach, which model should I use, etc… Furthermore, I have not found any documents about the application of Text Categorization on Social Media. Therefore my application can be a newly added topic. One thing that I need to worry about is how I am able to get the post and analyze them. Also, I need a pre-classified dataset to train my model in, which I haven’t found. Other than that, I think this can be a great topic to look into.
This week, I was able to realize partial spatialzation in MaxMSP using the new tool that I had found previously, HOA Library. Right now I am trying to get my object driver to work to be able to create multiple figures. Also, Forrest wanted me to map the knobs of a MIDI keyboard to Max and the patch.
I have started collecting papers on my three ideas.
There are many different sources that have used gesture controlled navigation in the past so I think it will be hard to differentiate from them, but most of them seem to have to use another device. There are a couple that are just controlled by a camera, this would be a neat way to solve this problem, and wouldn’t require people to buy anything.
For my second Idea of PCG for my video game, I have gathered sources where people use pcg to remake games or even make new ones. None of them seem super similar to what I would like to implement. I have also found a few good overviews and lit reviews of the subject. These will come in handy.
The papers I found for my last idea, of an educational app for ultimate frisbee, mostly just include the theorys and methods for educational applications. these include apps such as duo lingo and other language learning sources. I figure a lot of this could be mapped over to any educational application. It seems like a lot of the problems in this field deal with retention.
I have been trying to get the sensor to give me significant results and have had a lot of trouble with that. I mostly did research and experimentation to try and get it to work.
This week I finished my implementation of the SIFT algorithm, starting to work on implementing SURF next.
Also looking ways to organize images to improve accuracy. Currently planning to get keypoints, then organize photos based on similar keypoints rather than locality.
This week I did the following tasks:
-Met with Dave about the structure of my outline and my experimental results section
-Met with Forrest to get feedback on my work.
-Began to work on a new variation of my algorithm based on Forrest’s feedback
-Read chapter 5 in writing for computer science
For the past week I:
– Implemented MLP ‘Regressor’ instead of classifier. The accuracy increased significantly.
– Wrote functions to generate financial indicators from close prices
– Implemented lag
This week I will do:
– Limit the training data to consider only events after the crises in 2008.
– prediction after 5 days is with accuracy +-2.5%, I will predict 10 days and see how accurate the predictions are.
– selection approach for simulator. When simulator is going to decide to sell or buy?
– explanation / justification of your design choices for simulator.
These are the things I did for this week:
– Read Chapter 5 and did the quiz.
– Reworked the representation of the 2D matrix since the original method uses too much memory and not very practical.
– Started implementing the collaborative filtering algorithm for music recommendation.
I have talked with Ajit about all three ideas and received some feedbacks and suggestion. Also, I have started finding research papers that are related to my ideas to read for next week.
This week I finalized the schema for the database with Ajit, and familiarized myself with the PostgreSQL commands after receiving the log in information from Craig. I faced an unexpected challenge with the ordering of the RFID device from Ebay. Instead I researched for two days and found a few other cost friendly options in the US, and have proposed to the department to purchase one of them.
I am planning to finish all the software end of the project by the time I get my hands on the device!
I decided to pivot on my ideas a little bit because 2 of them were already solved and I didn’t feel that I had anything to add to the field. I have found papers and starting to read them on my new ideas, ideas listed below.
Idea 1 : Gesture Controlled Mouse and Keyboard
Description: The Idea is to fully replace the keyboard and mouse of a computer by using a 2D camera to track hand motions. I really like the idea of incorporating the feature of swipe to text as seen on some mobile phones, this could possibly increase typing speed.
Idea 2 : Procedural Level generation for 2D platform game
Description: Last semester I built a Game in Unity with 2 friends, we have continued to work on our game with the intention that one day we sell it. A large part of this process will be the generation of new levels. One idea is to develop an algorithm that procedurally generates the levels. While procedurally generated games are fairly common, procedurally generated platformers are far less common. There are a few challenges involved that I think would make this interesting.
Idea 3: Educational app for learning the rules of Ultimate Frisbee
Description: My other new idea, is to create an app that is maybe similar to other educational apps such as Duo-lingo, but with the intention of teaching the rules of ultimate frisbee. Since it is Self-refereed sport it is much more important that the players know the rules themselves. However these rules in the rule book are very wordy and hard to follow sometimes, this would provide an easily accessible way to learn the rules without having to sit down and read a book.
This week I made a first draft of an outline for the paper. Also worked on implementing the first of three algorithms I plan to use in my research, Scale Invariant Feature Transformation. While not completely finished I have most of the framework complete, and am just about where on schedule.
This week offered me a chance to get into detail with spatialization libraries:
- Jamoma – C++ and Max/MSP general purpose audio and spatialization toolset with standard interfaces, requires custom installation depending on the version of Max though. This library is needed to use most of Tom’s Max patches.
- NASA Slab – (Older) open source project for testing spatial auditory displays, requires registration to NASA open source archive.
- CICM Higher order ambisonics library – SuperCollider(under development), CSound, Pd, Faust, oFx, Max/MSP and C++ archive of useful ambisonics stuff, renders down to binaural well but computationally quite intensive. This library is required to use most of Tom’s Max patches.
- Sound Field Synthesis Toolbox for MATLAB – Python version exists as well btw. Sound Field Synthesis Toolbox (SFS) for Matlab/Octave gives you the possibility to play around with sound field synthesis methods like Wave Field Synthesis (WFS), or near-field compensated Higher Order Ambisonics (NFC-HOA). There are functions to simulate monochromatic sound fields for different secondary source (loudspeaker) setups, time snapshots of full band impulses emitted by the secondary source distributions, or even generate Binaural Room Scanning (BRS) stimuli sets to simulate WFS with the SoundScape Renderer (SSR).
- MIAP – Max/MSP objects, not spatial audio per se but pretty cool. More spatial parameter space exploration, though the binaural example is in the pack.
- Octogris – OSX DAW 8 channel spatialization plugin
- Spatium – Plugin (AU), Max/MSP, Standalones (OSX). Modular open source software tools for sound spatialization: renderers, interfaces, plugins, objects. Got some nice processing based physic interactions for spatial control.
- ambiX – Ambisonics spatialization Plugin compatible with Reaper, Ardour, MaxMSP, Bidule or as standalone applications with Jack.
- HOA – Higher Order Ambisonics (HOA) resources for Pure Data and Max from Paris Nord university.
- ATK – Ambisonic Toolkit for Reaper and SuperCollider.
- Sonic Architecture – resources for ambisonics in Csound and the blue environment from Jan Jacob Hofmann.
- Iannix- a graphical open source sequencer for digital art. It requires sound making software or hardware connected to the sequencer. The sequencer sends instructions (e.g.OSC) and allows to create 2D and 3D scores programming the behavior of sliders and triggers.
- Zirkonium – tool from zkm to spatialize music.
- NPM – web audio classes for Ambisonic processing FOA and HOA.
- [omnitone] (https://github.com/GoogleChrome/omnitone) – spatial audio on the web – by Google.
Adapted from: https://github.com/darkjazz/qm-spatial-audio/wiki/Open-source-free-spatialisation-tools-around-the-web
What I decided to go for was was HOA Library, because HoaLibrary is a collection of C++ and FAUST classes and objects for Max, PureData and VST destined to high order ambisonics sound reproduction. It won “Le Prix du Jeune chercheur,” awarded by the AFIM in 2013. This library is free, open-source and made available by CICM, the research center of music and computer science of the Paris 8 University. WIth using that, I know that I can make a lot of edits to it, and many people have used it in a concert or installation setting. Now that it’s decided, I can work towards the connection between MIDI and the spatialization of objects.
Here are my ideas for topics! Another post with some cost analyses is coming soon!
Topic Name: Real-time management using Augmented Reality
Topic Description: Examine the applications of Head-Mounted augmented reality displays such as HoloLens or Project Northstar in real-time management scenarios like Theatre Stage Management or NASA rocket launch management and implement a basic proof of concept software to eventually be used in the Theatre department as part of my Theatre Capstone.
Topic Name: Using real-time spacial mapping to improve calling for stage managers during performances
Topic Description: Examine the feasibility of using technologies such as Kinect 2.0 in the area of theatre to allow stage managers to keep better track of the positions etc of their actors, allowing them to make more accurate cue calls when their vision of the actors might be obscured, and implement a proof-of-concept application for use in my Theatre Capstone.
Topic Name: Using micro controllers to facilitate cross-device communication and control
Topic Description: Use Arduinos or Raspberry Pis to allow two or more very different devices to be controlled by another device. For example, allow Qlab on a computer and cues on a light board to be controlled from a single application running on a different computer. If feasible, make a proof-of-concept for use in my Theatre Capstone.
For the past week I:
- Completed the outline
- Worked on neural network implementation
- Thought about trading strategy
This week I will do:
- Complete on MLP implementation
- Start working on the paper (background, motivation)
This week I accomplished several things:
-Reconfigured the way my patch combines matrices to avoid issues I was having w crossfade
-implenented horizontal movement after note press
-Organized parts of my project into sub-patches and cleaned up some stuff
-Implemented envelopes connected to sound and video out
-Created an outline for my paper
This week I haven’t been able to get as much work done as I planned on. These are things that I did:
– Reread the proposal to answer some questions in the outline.
– Finish writing the outline.
– Make some minor changes to the program to reformat the data as suggested by Xunfei.
– Tried to run the program with the large dataset but wasn’t successful.
This week I talked to Dave about the delay on my database. I am waiting to hear back from Charlie andor Craig so I can start working on building the database and connecting the dots. Until then I have read up on some more algorithms, specifically Reddits and ELO’s. I am also writing up my outline for CS488 and cleaning up some paragraphs from last years paper.
Built a prototype testing agent for the Capstone that should in theory take in a file of problem states and go through them with each Heuristic(Neural Network not yet ready) then output the solution size, number of nodes visited and number of nodes that are waiting for a visit. Haven’t gotten to properly test this one so will need to make time for that.
Tried writing a high level outline of my Capstone.
Further deliberated on my 3 ideas, I really enjoy the idea about replacing the keyboard and mouse set up with a gesture recognition system from a 2D camera, most gesture recog systems require another device that one must buy, but cameras are already built into most things. My other favorite is the personal budgeting app, I am interested in this one because it applies to my life, It would be software that I would use to help me save money.
I have changed two of my ideas upon re-evaluating the achievability of the project given my skill and the time limit I have
This week, I finally got my own laptop so I redownloaded all the necessary software and libraries for my gesture recognition system. Using the OpenCV library in Python, I was finally able to use the camera to detect the hand by separating the background with the foreground via thresholding. Then, I used findContours to identify the hand within a region of interest (roi). Finally, I made a copy of the the frame containing the roi and displayed the video in binary black and white. My code can now detect the hand with a static background.
This week I met with Ajit and discussed some of the necessary features for the user interface that the administrator will be using. This involved seeing a list of recently checked out items, adding new objects, and adding new users. I also met with Craig and discussed the back-end work. We decided to use Django and PostgreSQL.
In terms of implementation, I have created 4 different heuristics functions for my Capstone that take in a state and output a value based on how good the state is. with 4 being inadmissible. Technically 5 as 2 heuristics are additions to a single heuristic.
Modified my admissible heuristic function so it outputs a move (left=-1,right=+1,up=-4,down=+4) based on what move a search algorithm would have performed if it is was taking an action in that state.
Created my training agent function that outputs a file contains vectors of these heuristic outputs. One per state
Did some research into activation functions and Neural Network types to figure out what initial design I should go with for my Network.
This week I worked on developing a script that takes a directory and determines if the contents are all images of the same type, and if they aren’t converts them to jpgs. I also meet with my adviser and starting working on an outline that will be due next week.
This week dealt with the focus of the FFT and other conversion formulas. In recognizing that the Fast Fourier Transform is an algorithm that computes the discrete Fourier transform (DFT) of a sequence, or its inverse (IDFT), you understand how that mainly relates to Fourier analysis and converting a signal from its original domain (often time and/or space) to a representation in the frequency domain and vice versa. With my current project, I am trying to understand how this relates between the signal processing and MIDI, and how that will allow me to transfer the signals into the shapes that I will be making in Jitter and MaxMSP. In understanding the multiple ways in which the transforms work I can use the signals that I get by decomposing a sequence of values into components of different frequencies, which will then generate multiple sounds, and that hopefully can be somewhat composed into a song or piece that is made through the patch. Currently SoundStroll 2.0, but it might end up being a whole different concept and Max patch when it’s done. In understanding what changes are being made, the new product could be thought of as a completely different three-dimensional audiovisual spatialization software. My main work has been with working to create a different audiovisual projection whilst working with MIDI and other signals to get different spatialization in MaxMSP, this was concurrently added to my paper, and it was also worked on in MaxMSP and Jitter.
Here is what I worked on for my project this week:
-Worked with poly tool to process two simultaneous notes.
-Got it working but temporarily disabled the circle of fifths map table-Now midi messages are sent to two paths for separate notes.
-Using xfade to combine matrices is not good, colors are still evenly distributed throughout the whole matrix instead of following freq mapping.
-Patch is getting crowded need to figure out in and out tool to split up and organize project.
-Talked with Xunfei about getting Max installed on some Earlham laptops
For this week, I have been mainly working on:
– Make a few changes on the coding environment for my project from last week for better efficiency.
– Follow Xunfei’s advice to consider some other music datasets. I eventually decided to use the Echo Nest Taste Profile Subset. This dataset is larger and more widely used than the LFM-1b dataset which I had been using.
– Design an efficient method to convert the original data into matrix form in order to use collaborative filtering.
For the past week I did:
- Completed data collection from Alpha Vantage API
- Started working on neural network implementation
- Met with my advisor and went over my main argument for my project
This week I will do:
- Keep working/complete on neural network implementation
- Complete the outline
Idea 1: Optical Character Recognition using Machine Learning
Optical character recognition is not a completely new field of research. However, with the advance of today’s technology, we can apply machine learning to optical character recognition and therefore can recognize not only the printed characters but the more difficult handwritten characters or characters from different type of documents such as certificate or receipt. With the help of OCR, a lot of human work can be reduce. I would like to research about that specific use of machine learning in OCR to handle those harder cases.
Idea 2: Disease prediction
People have always want to know their health status therefore they can prepare and take a better care of their body. It is not difficult to know a person’s current health status since he/she can just go to hospital to have a general medical check. However, people also want to know what type of disease they may have the future, this is when disease prediction using machine learning will be useful. I would like to research about this topic but the first things I found to be difficult will be how to get the appropriate the training data set.
Idea 3: Time management application
Since students sometimes need help with planning the classes and studying for exams, I want to create an application that at first can suggest the best time for students to go class or what time to study. After that, each term, the student can input how well did he/she perform on those classes/exams and from that in the next term/semester the application may suggest a more specific timetable for that specific students to have a better performance.
Topic Name: Earlham Tennis App
Developing an app to use it as a way of improvement for the tennis players. Entering data during matches to get score stats, errors and serve percentages. All this data will be stored in a database and could be accessed at any time.
Topic Name: Box Office App
Again, develop a mobile application to manage the Earlham Box Office. Users would be able to look at the event schedule, purchase tickets. To make it efficient, the student data will have to be obtained in order to find from the directory.
Topic Name: Sensor
If possible, make a sensor that would be attached to the tennis racquet determining how much strength is used and how many balls were hit in a span of time. This would be collected and according to the body analytics of the player, it can be determined how to improve the performance. The stats can then be seen on a web application.
Researched how image processing works (tutorials, examples using python and open-source software OpenCV). ~20 minutes on 1/19/19, ~30 minutes on 1/21/19
Made sure all necessary software was downloaded to my computer. ~30 minutes on 1/17/19
Adjusted timeline. ~15 minutes on 1/23/19
Found an advisor, and set up a meeting time to plan my project. Tested that the libraries I plan on using are available and functional on the cluster. Finally reviewed my project to make review the goals I’ve set myself are achievable and reasonable.
This is my first post for CS488.
- Met with Ajit to plan the next few weeks, and decided to start the data collection this week.
- Edited the timeline a little bit to include implementation of the project.
- I’ve shared my box folder with Dave and Charlie and will soon be contacting Andy Moore for access to volumetric data collected by the Geology students across campus.
This first week, I tried to focus
my project into something tangible. Initially, I was not sure about how
I wanted to go about editing and making SoundStroll 2.0, and I had
talks with advisors who told me that I tried to achieve too much in a
short period and that I also needed to find a focus. Interestingly
enough, I think that working with and creating a bandpass filter, in
addition to other improvements, can produce different effects in
SoundStroll 2.0, and I’m curious to see how that might change
SoundStroll 2.0, regarding effects and three-dimensional objects that
are spatialized with the signals.
In understanding what needs to be done for the project, I still have many main steps:
1. Completely rehaul and add a new more advanced spatialization toolset/visualizer to work with the updated software.
2. Add vocal recognition through a speech processing patch that’s made by myself to the updated Max project and get them to work together.
3. Add new additions that track the Fourier Transforms for analysis that allow the original and added vocal recognition through a speech processing patch.
4. Make sure that vocal, MIDI, and OpenGL components are working so that objects are spatialized through vocal component or the MIDI interface.
5. Potentially add virtual reality/VR component to the project to let you traverse the world.
In finality, I want SoundStroll 2.0 to be a logical successor to its former self. SoundStroll 2.0, or the new edition that I plan to create will use Fourier Transforms for vocal recognition through a speech processing application, which can also be analyzed for more profound findings audially or mathematically in relation to sound; a different, more advanced spatialization toolset to create a scene that you can traverse; and a more connected environment to create different ways to traverse with Max for Live, Ableton Live, and Reason, which will allow new sounds and objects to be triggered and spatialized through SoundStroll 2.0.
This week I looked over the specific dataset that I want to use for my project. I reached out to sources to obtain that dataset. Similarly, I read various researches and, checked existing experiments and projects to learn implementation of Convolutional Neural Network (CNN) in python using Tensorflow and Keras. Read documentations and went over tutorials to learn Tensorflow and OpenCv. Also, I setup Tensorflow in my local machine to work with OpenCv library.
Just sharing a helpful tutorial link below to understand about OpenCV
My first idea is for a application that allows the computer to be navigated with gesture control, initial thought is to use the camera that is on almost every laptop to map the mouse pointer between say the thumb and the forefinger, and when the thumb and forefinger touch emulate a the click of the mouse. Further interface could also be implemented such as a virtual keyboard or talk to text features, basically attempting to replace a mouse and keyboard, further research needed.
My second idea is either a stand alone software or a Photoshop add on for real time pixel art animation editors. Given a sequence of images with a specified distance apart, color pallet and speed at which to move through the images, one could make a change and the animation would update real time, also allowing the change of color pallets.
My third idea is a personal budget planning and expense tracking app. I person can track what they buy by inputting the cost of an item and categorize that item falls into (possibly further subcategories for more in depth statistics) ie $16.69 on groceries on 1/21/19, $32.55 on cloths on 1/22/19 etc. One can input there salary and how much they want to not spend and the app could keep track and suggest a budget for you, give statistics about your spending patterns etc.
The tangible work I have done this week on my project is finding the data sets and started setting up a work flow to automate the process. I created the connection between SQL and my simulation so that the simulation takes 2 arguments, a name of the database to store the timestamp data as well as the result data and the file path of a csv containing data. This sets up a database for the data with three tables, two of which contain results and one with the actual timestamp data. I have also started to take notes as I go so that I can refer to them when I start to write my paper.
I have gotten off to a relatively smooth start on my project. Xunfei is going to be my adviser and I have scheduled a meeting with her for tomorrow morning. I have prepared to present on my topic in class tomorrow. The majority of my time this week I spent starting my project implementation. I am still learning how to use Max/Jitter effectively but here are some of the things I have done:
-Map color values to MIDI notes.
-Map a saw waveform function over the x axis with scale relative to note frecuency
-Make the saw function scroll horizontally at a rate relative to frequency
-Crossfade the color and waveform matrices
With all of this together, my projects output visual output represents a single note at a time. Obviously it is a pretty rough version of what I want the eventually single note visualization to look like but the foundation of representing tone through color and pitch through wave frequency is functional and has no noticeable latency between note press and display.
This week I met with Ajit for an hour. We went over timeline and design of my project. I also met with Craig and ordered the RFID reader and tags after approving them.
This week I reviewed my project proposal and started setting up the coding environment. I also asked Xunfei to be my advisor for the project. During our meeting we discussed the project’s timeline for the first month and I made some changes in my timeline according to Xunfei’s suggestions. I’m also preparing for the project presentation in the next class.
First post of Capstone progress. First thing I did was review my Capstone proposal paper to re familiarize myself. Immediately I begun work on the work I outlined for Week 1 in my timeline.
I built a generator for random initial board states for an 11 sliding tile-puzzle. I chose to have each board state be a solved state that was then subjected to a random number of moves. This is ensure that each state is solvable without having to check first. Another choice I made was to have each state be represented as a string with each tile being represented by a hexadecimal number and position represented by its index in string.The blank space is represented by “_”. This was done because a number of states will be generated during the AI’s decision making process so the states should be as small and compact as possible.
I also started to look into the tools for creating the neural network. So far I am looking at Tensorflow, Keras in combination with Python.
For the past week I did:
- Obtained data from Alpha Vantage API
- Updated the project timeline for the semester.
- Prepared a topic presentation
- Met with my advisor and went over my updated project timeline.
This week I will do:
- Read, experiment, and implement neural network
This week I haven’t been able to do a lot since I have a lot of exams in other classes as well. But I have been working mainly on reviewing everything I have accomplished in this class and drafting a quick notes for what I should prepare for in CS488.
This week I have been reading the feedback of the second draft of my proposal. I’m working on revising the paper to complete the final version for the paper.
I also talked with Dave about the paper and adjust the timeline according to his suggestions.
Finished working on the final paper.
This week, I attempted to collect preliminary data, which made me realize that I would need an air compressor in order to obtain meaningful data. I added this to my proposal and made some modifications to improve it overall.
This week after receiving feedback from Xunfei on the second draft of my paper, I updated it with respect to her comments. I also followed up with Craig; he is going to purchase the equipment early next semester.
Got feedback for my second draft of my proposal. Ready to make these final changes and submit on time! Started installing the dependencies so that I can get familiar with the tools I’ll be using over the break. Not planning to commit too much effort into this in the midst of finals, but will redouble my efforts after the tests are over.
This week I received feedback on the second draft of my proposal from Xunfei. I spent time updating the proposal with respect to her feedback. I also met with Craig and decided on a set of RFID reader and tags. Craig is going to purchase them before the end of the semester.
For this week, I have been discussing with Dave about details of my proposal paper, especially what I need to add/edit for the second draft and the final draft. I have also been reading some new research papers that have implementations of collaborative filtering that I can learn from.
This and the previous week, I spend my time trying to beat my way through a variety of problems related to one specific module of my project’s code. After talking over it with my adviser I have shifted both that piece of code and what should be taking my priority. The original model of my project was that, after the blockchain authenticated a ‘website’ so that it could access a user’s ‘cookie’ file, the functions would be passed over to the user from the website’s server to the user’s. My adviser suggested I simply change this so that the user has both the file and the functions, and that the website simply sends a signal if the blockchain transaction was approved. The actual process of the functions on the cookie file are not as important as the blockchain authentications, which is the center of the project.
The refocusing suggestion was also on everything related to my experiment, such as the details of my experimentation, and the discussion of further research. Leave it so that I have as much done as possible, with that window left for my experimental results.
This week, I was focused on making the poster and including the final results of my research.
During the past 2 weeks I have:
– Been able to program the communication between the Chrome extension and the classifier, with some modification in the project design. Instead of using Native Messaging, I set up a Flask server and make the two programs communicate through the HTTP POST method.
– Worked on the front end of the Chrome extension which displays the percentage breakdown of political ideologies in a news article.
– Worked on my poster and submitted it to my advisor for feedback.
In the coming week, I plan to work on the final draft of my paper given the feedback I received from the 2nd draft. I will also finalize the project as well as the poster
I completed the 2nd draft of my topic proposal.
In particular, I added and fixed the followings:
- A table that summarises what evaluation methods are used in previous work
- Added reference links to all the tables and figures
- Fixed all the comments I received on my first draft
This week I haven’t been able to work much because of some traveling. However, I was able to think more about my design decision including the RFID reader that I will buying. I have selected a device, and discussed with Xunfei. I am planning to meet with Craig and possibly purchase the equipment before the end of the semester.
This week I received feedback from Xunfei on my initial draft of proposal. I spent the last few days thinking more about design and trying to answer some questions that Xunfei raised. I tried referring back to my sources and it has been very helpful.
This week I haven’t been able to do much due to some travelling. However, I still managed to design some methods to improve the running time of the collaborative filtering algorithms. I also worked on editing the second draft of my proposal and have a lot of progress in that.