Music Analyzer for Visualization Systems

• music is analyzed with a neural network which allows for the recognition of much more complex sounds than your average "beat detector"
• the output of the network can be used to trigger multiple "effectors" -- discrete, or variable
• training process is manual
  1. listen to a song and pick out a reoccurring sound which you want the system to recognize
  2. create a new untrained detector (either a discrete "hit" detector, or a variable "strength" detector)
  3. listen to the song and set the desired output of the detector, so that it corresponds to what you hear in the music
     • for a "strength" detector, you'd have a long line that spans the entire song which you'd adjust with control points to set the intensity. discrete detectors would have a series of "hit points" that could could add, move, or delete.
  4. after you've finished creating this "training data", the neural network will analyze the music and train itself against it
  5. the user can then verify if the network was trained properly:
     • the recognizer can be tested with a library of example visualizers to see how it'll look when it's running
     • the user can spit out "sound patterns" from the network and play them back to see what they sound like (a sound pattern is the most common generalization of the sounds they have trained into the network)
     • the user can run the detector on the training song, or a test song, and see what the hitpoints/waveform looks like
  6. the user will be able to adjust and retrain the network for invalid or weak matches
     • dynamic retraining? (as you adjust the slider, the network relearns)
• once they have trained a network to recognize their sound, they can use it in their favorite mp3 player, send it to other users, etc.