Visualizing Notes & Timing

A post I wrote in September about Calculating Beats, generated quite a few questions from readers. The general consensus was that people wanted to not only program sequences of notes in a musical fashion, but also how to visualize and analyze the playback of the programmed sound. While similar, the two tasks are handled separately.

To program audio you generate blocks of sound data before any audible sound occurs. Over time the data is read and, amazingly, turned into sound from your speakers. Visualizing that data in real time happens as the music is playing. If we want to display information about the currently playing audio we need to look at the position of the sound as it plays back and do some calculations to determine where in “music time” the audio is.

In AS3 a Sound object will dispatch a SampleDataEvent.SAMPLE_DATA event that indicates when to generate new sound data. Likewise, another event can be handled for visualizing this data in real time. To do so, all we need to do is set up a handler for Event.ENTER_FRAME. Within the handler we can look to the position of the audio playback, which in AS3 is the SoundChannel.position property. The position property is the elapsed time in milliseconds. With this value we can calculate some useful numbers. Below are a few examples:

If we know the bpm of the audio we can calculate the length, in samples, of each quarter note:

quarterNoteLength = SAMPLE_RATE * 60 / _tempo;

To find out how many samples have elapsed, where position is the elapsed milliseconds of the sound:

samplesElapsed = ( position / 1000 ) * SAMPLE_RATE;

Using the values above we can determine the current quarter note:

currentQuarterNote = samplesElapsed / quarterNoteLength;

To find the current measure we can just divide the current note by however many notes are in a measure:

currentMeasure = quarterNoteLength * 4;

The example below demonstrates how musical time can be visualized in a grid. The top row represents 1/8 notes, the middle represents 1/4 notes and the bottom row represents measures. Press play and move the slider to adjust the tempo. No actual audio is produced.

Demo Source Code Here

One of the questions I received asked about using an audio loop for the sound source. One of the obstacles in working with audio and the Flash Player is that the player only natively recognizes mp3 files. Mp3 files are inherently difficult to loop due to meta data at the beginning of the file. This is a widely known and discussed issue. There are workarounds. If you find a solution for loading PCM data such as a .wav file that is precise in its duration, you could easily use many of the calculations above and in the linked source code to determine the current notes.

If you are dealing with the Flash and mp3s there is another solution. You can load the entire loop and cache its data. Using some of the techniques in this post and other posts, you can use data from the cached source and essentially splice out the the parts you want ( minus the meta data ) and fill an audio buffer with it.

As I write this, I am beginning to see the makings of a new article. I think I’ll stop here. Hopefully this information is helpful to the readers that had questions. Don’t forget to check out the source code, as I think it will definitely shed some light on these topics.