Further deliberated on my 3 ideas, I really enjoy the idea about replacing the keyboard and mouse set up with a gesture recognition system from a 2D camera, most gesture recog systems require another device that one must buy, but cameras are already built into most things. My other favorite is the personal budgeting app, I am interested in this one because it applies to my life, It would be software that I would use to help me save money.

In terms of implementation, I have created 4 different heuristics functions for my Capstone that take in a state and output a value based on how good the state is. with 4 being inadmissible. Technically 5 as 2 heuristics are additions to a single heuristic.

Modified my admissible heuristic function so it outputs a move (left=-1,right=+1,up=-4,down=+4) based on what move a search algorithm would have performed if it is was taking an action in that state.

Created my training agent function that outputs a file contains vectors of these heuristic outputs. One per state

Did some research into activation functions and Neural Network types to figure out what initial design I should go with for my Network.

This week I worked on developing a script that takes a directory and determines if the contents are all images of the same type, and if they aren’t converts them to jpgs. I also meet with my adviser and starting working on an outline that will be due next week.

This week dealt with the focus of the FFT and other conversion formulas. In recognizing that the Fast Fourier Transform is an algorithm that computes the discrete Fourier transform (DFT) of a sequence, or its inverse (IDFT), you understand how that mainly relates to Fourier analysis and converting a signal from its original domain (often time and/or space) to a representation in the frequency domain and vice versa. With my current project, I am trying to understand how this relates between the signal processing and MIDI, and how that will allow me to transfer the signals into the shapes that I will be making in Jitter and MaxMSP. In understanding the multiple ways in which the transforms work I can use the signals that I get by decomposing a sequence of values into components of different frequencies, which will then generate multiple sounds, and that hopefully can be somewhat composed into a song or piece that is made through the patch. Currently SoundStroll 2.0, but it might end up being a whole different concept and Max patch when it’s done. In understanding what changes are being made, the new product could be thought of as a completely different three-dimensional audiovisual spatialization software.  My main work has been with working to create a different audiovisual projection whilst working with MIDI and other signals to get different spatialization in MaxMSP, this was concurrently added to my paper, and it was also worked on in MaxMSP and Jitter.

Found an advisor, and set up a meeting time to plan my project. Tested that the libraries I plan on using are available and functional on the cluster. Finally reviewed my project to make review the goals I’ve set myself are achievable and reasonable.

This first week, I tried to focus my project into something tangible. Initially, I was not sure about how I wanted to go about editing and making SoundStroll 2.0, and I had talks with advisors who told me that I tried to achieve too much in a short period and that I also needed to find a focus. Interestingly enough, I think that working with and creating a bandpass filter, in addition to other improvements, can produce different effects in SoundStroll 2.0, and I’m curious to see how that might change SoundStroll 2.0, regarding effects and three-dimensional objects that are spatialized with the signals.

In understanding what needs to be done for the project, I still have many main steps:

1. Completely rehaul and add a new more advanced spatialization toolset/visualizer to work with the updated software.

2. Add vocal recognition through a speech processing patch that’s made by myself to the updated Max project and get them to work together.

3. Add new additions that track the Fourier Transforms for analysis that allow the original and added vocal recognition through a speech processing patch.

4. Make sure that vocal, MIDI, and OpenGL components are working so that objects are spatialized through vocal component or the MIDI interface.

5. Potentially add virtual reality/VR component to the project to let you traverse the world.

In finality, I want SoundStroll 2.0 to be a logical successor to its former self. SoundStroll 2.0, or the new edition that I plan to create will use Fourier Transforms for vocal recognition through a speech processing application, which can also be analyzed for more profound findings audially or mathematically in relation to sound; a different, more advanced spatialization toolset to create a scene that you can traverse; and a more connected environment to create different ways to traverse with Max for Live, Ableton Live, and Reason, which will allow new sounds and objects to be triggered and spatialized through SoundStroll 2.0.

This week I looked over the specific dataset that I want to use for my project. I reached out to sources to obtain that dataset. Similarly, I read various researches and, checked existing experiments and projects to learn implementation of Convolutional Neural Network (CNN) in python using Tensorflow and Keras. Read documentations and went over tutorials to learn Tensorflow and OpenCv. Also, I setup Tensorflow in my local machine to work with OpenCv library.

Just sharing a helpful tutorial link below to understand about OpenCV

Link: https://www.lynda.com/Python-tutorials/OpenCV-Python-Developers/601786-2.html

The tangible work I have done this week on my project is finding the data sets and started setting up a work flow to automate the process. I created the connection between SQL and my simulation so that the simulation takes 2 arguments, a name of the database to store the timestamp data as well as the result data and the file path of a csv containing data. This sets up a database for the data with three tables, two of which contain results and one with the actual timestamp data. I have also started to take notes as I go so that I can refer to them when I start to write my paper.

First post of Capstone progress. First thing I did was review my Capstone proposal paper to re familiarize myself. Immediately I begun work on the work I outlined for Week 1 in my timeline.

I built a generator for random initial board states for an 11 sliding tile-puzzle. I chose to have each board state be a solved state that was then subjected to a random number of moves. This is ensure that each state is solvable without having to check first. Another choice I made was to have each state be represented as a string with each tile being represented by a hexadecimal number and position represented by its index in string.The blank space is represented by “_”. This was done because a number of states will be generated during the AI’s decision making process so the states should be as small and compact as possible.

I also started to look into the tools for creating the neural network. So far I am looking at Tensorflow, Keras in combination with Python.