Sync test screenshot
It’s not that easy to synchronize programmed animation and dynamic sound in Flash. To have something happen on screen at exactly the same time a sound starts to play.
I thought about it a bit and came up with two different scenarios where audio and visuals need to be synchronized:
- The first is where a sound plays and something visual on screen happens to accompany that audio event.
- The second is the other way around. Something happens on screen that should be accompanied by a sound.
In the first case the animation is the initiator, in the second case the sound. For the second case I included a Flash example here with the source code available.
Yes I know, it’s the hype of 2009 and we’re in 2010 now, but I haven’t seen anything yet where augmented reality is combined with sound. If you do know examples of augmented reality and audio, please leave a comment. I’d love to see what other people do with it.
This is a small experiment I started half a year ago but never finished. I looked it up again last week because I needed a Flash AR example for a project at work. Here I improved it a bit to make it more presentable.
MP3 wave display is orange.
Wave Display 2 is an improved version of the last one (in the previous post). A few things had to be rewritten in the code for waveform rendering, and there were some extra features I wanted to include to make the application a bit more interesting to use.
The new features in this version:
- Audio playback. You can now listen to the loaded MP3 file. Click ‘play mp3’ or press the SPACE bar to listen.
- Draggable start and end locators to select only part of the sound. Drag the ‘S’ and ‘E’ locators to set the start and end point.
- Reverse playback. Click the ‘reverse’ button to toggle reverse playback.
- Adjustable playback speed. Drag the ‘transpose’ slider to change the playback speed. The speed is adjusted so that the pitch changes from one octave down to one octave up in semitone steps.
- Drag to zoom. Press and drag the mouse on the display area for finer control: Drag vertical to zoom and horizontal to move the waveform. I always really liked this behaviour in Steinberg WaveLab where I first saw it, and where I now copied it from.
The waveform display
2010! A new year, a new decennium even! This year I want to be more consistent in publishing articles on this weblog. That’s my new year’s resolution.
The first one of the new year is more of a utility than a complete application. It’s a wave display; It loads an MP3 file and displays the sound as a graph of the waveform. It makes no sound. A simple waveform display like this might be a useful component in a larger application where audio needs to be edited. Here it’s on its own because I was figuring out how to best program such a component. So this application is more experimental than useful.
The past weeks I’ve been learning PureMVC. It had been on my mind for some time, and now that a project (a pianoroll type MIDI editor) became too complicated and the lack of structure took the fun out of programming I figured it was time for a nice new clean framework.
Part of the sequencer
Here is a basic step sequencer for creating rhythms. The sounds it plays are small MP3 files. Short sound clips of drum noises, blips and plings. So it’s more a sample player than a drum synth. But the main thing is that it loads and plays rhythm patterns and that the patterns can easily be edited. That’s what I wanted to do here.
It plays patterns in the same way FL Studio (FruityLoops) does. There’s twelve tracks that each play their own sound and each track has sixteen steps that can be switched on or off by clicking the step buttons.
Besides that the volume of each step can be changed by holding the “1” (one) key on the computer keyboard while dragging the mouse up or down on the step button. The blue rectangle on the step will change in height to indicate the volume level.
The image of the waveform on the left of each track can be clicked to play the sound. And there are four patterns. Switch between them with the ‘1’ to ‘4’ buttons at the bottom.
As a next project I want to make a simple drum machine. Several tracks that each play a different drum sample and each track has sixteen steps. Just like a Roland 808 or the main step sequencer in FruityLoops. It’s nothing complicated but it made me think about how to store the patterns the drum machine plays.
I’d like to put in an XML file all the information about how many patterns there are, how many tracks, which sound plays when and so on. But I want to keep the file as flexible as possible. It must be easy to later change the length of a pattern to 32 tracks, to add melodic patterns, notes of different length, filter settings… Whatever I need I want to be able to include in the file.
Then it occurred to me that such a file already exist: MIDI!
MIDI has been around since 1982, and today still is one of the main formats for music. Besides channels, notes and timing it can hold all kinds of extra settings as “Control Change” values or “System Exclusive” messages. Very general and flexible and just what I need. Only, MIDI is no XML. And Flash can’t read MIDI files. So I googled MIDI and XML and found I’m not the first to need MIDI in an XML form.
As far as I’ve been able to find two standards exist to format MIDI as XML:
Slicer with waveform display
This slicer works the same as the previous one. Only difference is that is uses eight slices instead of sixteen.
The new stuff is in the interface: Waveforms to see what’s playing. And nicer to look at as well.
To use this app you can click the buttons as before to select a slice to play, and now you will see the waveform change to represent the slice.
The second way to change the slices is to click and drag on the waveforms itself. Drag up for later slices and down for earlier ones. Drag all the way down for an empty slice and silence.
Part of the slicer
This is it! The first one: A slicer.
It loads and plays an MP3 file. But not only that. It divides the file in sixteen equal parts and you can change the order in which the parts are played.
When it’s loaded the slicer plays all the parts in the right order. Part 1, 2, 3 and up until part 16. With the buttons you can change the order and for example play part 4, then 3, 11, 8 and so on. With the right MP3 loop this will create all kinds of interesting rhythmic variations.
It works best if the MP3 file is a short piece of music with a steady rhythm that loops seamlessly (so the start and end of the sound fit together exactly). Then every slice has a little bit of rhythm in it. Just one or two sounds like a kick or snare drum.
Best to check it out for yourself. I will put the actual SWF files on the ‘Read more’ pages of the posts, so they won’t all be playing at the same time and use up each other’s CPU cycles.
Oh yes! Finally I have my own blog.
I’ve had this www.hisschemoller.com domain for years, but didn’t really have any use for it. I just got it because it was there – my own name as a dot com. Better get it before someone else does. Especially before one of those companies get it that buy domain names just to sell them for a profit. “Get some money out of those Hisschemöllers” they might think. That’s not how the internet should work.
Now I have a use: The past months I’ve been getting more and more into programming audio stuff with Flash. Small experiments with sound, music, samples etcetera. I’ll post those Flash files here. And the sources as well if I come up with anything good. I learned a lot from other blogs, so now I can do something in return.