Screenshot

The past weeks I’ve been learning PureMVC. It had been on my mind for some time, and now that a project (a pianoroll type MIDI editor) became too complicated and the lack of structure took the fun out of programming I figured it was time for a nice new clean framework.

I read the ‘Framework Overview with UML’, the ‘Implementation Idioms and Best Practices’ and memorized the famous conceptual diagram with it’s interconnected cloud of meaningfull data blobs (see picture). I looked up some basic tutorials, ‘Minimalist MVC example using the PureMVC Framework’, ‘Building a Flash site using PureMVC’ and found this useful page: ‘PureMVC in a week’. And then felt ready for some PureMVC action of my own.

The PureMVC conceptual diagram

And I must say: It’s not as complicated as I feared it would be. The first hurdle is a bit high with a lot of concepts and rules to remember, but then it quickly becomes clearer what part of the code goes where. My previous framework experience was with Cairngorm which is MVC based as well, so that definitely helped me to get my head round PureMVC.

Here is my first PureMVC test project, done to check how the audio programming stuff would integrate with the framework: It’s a small synthesizer that plays sine waves. Only sine waves. You can use keys on the computer keyboard to play the synth as a piano keyboard (see picture for the keys) and use Z and X to switch octaves and C and V to change the volume. This keyboard layout was taken from Ableton Live. I like the way it works there. So thank you Ableton.

The source files are available again : Download the source files here.

Take note however that there is no FLA file. This is an ActionScript only project made in FDT. But if you want to compile it in Flash, it will probably work if you set App.as as the document root.

Use the computer keyboard to play

Part of the sequencer

Here is a basic step sequencer for creating rhythms. The sounds it plays are small MP3 files. Short sound clips of drum noises, blips and plings. So it’s more a sample player than a drum synth. But the main thing is that it loads and plays rhythm patterns and that the patterns can easily be edited. That’s what I wanted to do here.

It plays patterns in the same way FL Studio (FruityLoops) does. There’s twelve tracks that each play their own sound and each track has sixteen steps that can be switched on or off by clicking the step buttons.

Besides that the volume of each step can be changed by holding the “1″ (one) key on the computer keyboard while dragging the mouse up or down on the step button. The blue rectangle on the step will change in height to indicate the volume level.

The image of the waveform on the left of each track can be clicked to play the sound. And there are four patterns. Switch between them with the ‘1′ to ‘4′ buttons at the bottom.

Try it out below. This time I added play and stop buttons because in the previous two players I didn’t and that started to annoy me. Things on web pages should only make noise when they’re specifically told so.

If you’re interested in any of the ActionScript code then download the full source of this project here. The ZIP includes all ActionScript classes, the samples and everything needed to compile and run the project.

A few notes on the source:

The project was made in FDT 3. There is no FLA file as a base to compile the SWF from. Instead you’ll need to set the class AppController (flash/classes/com/hisschemoller/AppController.as) as the main class in the project’s Run/Debug settings.

All visual elements of the projects were made in the Flash file flash/assets/assets.fla. That’s where all the button graphics are. This file is compiled to the SWC file assets.swc and that must be added to the class path in FDT. (Right click on the SWC file in FDT’s Flash explorer and choose ‘Add to Classpath’.) Then the scripts in FDT can find the MovieClips in the Flash library that were given a class name.

There might be all kinds of weird hidden Mac files in the ZIP. As well as the FDT launch and project setting files. I’m quite new to working on a Mac and I’m not shure what it adds that I don’t see. Sorry for that.

MIDI

I’d like to put in an XML file all the information about how many patterns there are, how many tracks, which sound plays when and so on. But I want to keep the file as flexible as possible. It must be easy to later change the length of a pattern to 32 tracks, to add melodic patterns, notes of different length, filter settings… Whatever I need I want to be able to include in the file.

Then it occurred to me that such a file already exist: MIDI!

MIDI has been around since 1982, and today still is one of the main formats for music. Besides channels, notes and timing it can hold all kinds of extra settings as “Control Change” values or “System Exclusive” messages. Very general and flexible and just what I need. Only, MIDI is no XML. And Flash can’t read MIDI files. So I googled MIDI and XML and found I’m not the first to need MIDI in an XML form.

As far as I’ve been able to find two standards exist to format MIDI as XML:

MIDI XML

The first one is MIDI XML. The good thing is that it’s been developed by the MIDI Manufacturers Organization which is the organization that made the original MIDI specification. But the bad thing is that it seems to be incomplete and their only page about MIDI XML was last updated in July 2003.

A few DTD pages exist to show how the XML should be formatted, but that’s about it. Not a lot of information. The most interesting DTD file is MIDIEvents10.dtd, which shows how to represent MIDI events. This is what the XML should look like according to the file:

1
2
3
4

<NoteOn Channel="1" Note="60" Velocity="100" />
<NoteOn Channel="1" Note="62" Velocity="127" />
<ControlChange Channel="16" Control="7" Value="50" />
<ProgramChange Channel="1" Number="2" />

Clear enough, but this is only useful for a direct stream of MIDI events. Not to represent a MIDI file with a complete song. If only because there is no time information. You’d never know how much time it takes from one note to the next.

MusicXML

The second one is MusicXML. A much more complete standard made by Recordare, a music and software company I had never heard of before. It seems to build on the MIDI XML standard, but goes much further. This is how an event would be represented in MusicXML:

1
2
3
4

<Event>
  <Delta>0</Delta>
 <NoteOn Channel="1" Note="60" Velocity="100" />
</Event>

The main new thing is the <Delta> tag. It’s value is the time it takes from this event to the next, measured in “Ticks Per Beat”. How many ticks per beat there are can be found in the first section of the XML file. This is from the XML file I use for my drum machine:

1
2
3
4
5
6
7
8
9

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE MIDIFile PUBLIC
  "-//Recordare//DTD MusicXML 0.9 MIDI//EN"
  "http://www.musicxml.org/dtds/midixml.dtd">
<MIDIFile>
  <Format>0</Format>
  <TrackCount>1</TrackCount>
  <TicksPerBeat>4</TicksPerBeat>
  <TimestampType>Delta</TimestampType>

This means:

• <Format> holds the MIDI file type (0 or 1. 2 is not supported by MusicXML)
• <TrackCount> the number of tracks (which is one because a type zero MIDI file can only have one track)
• <TicksPerBeat> is the finest timing precision of the file. I used four because my drum machine has sixteen steps for a pattern of one bar, which is four steps for one beat. I don’t need more precision here.
• <TimeStampType> is the way time is notated. It can be Delta or Absolute. Delta means  time is the amount of time between the current event and the next event, Absolute means the total time between the start of the song and the current event. I used Delta because real Standard MIDI files (.MID) only use Delta time.

Much more information is available at the Recordare MusicXML pages, including DTD and XSD files, FAQ, tutorials and alphabetical index.

Another very helpful page is http://staff.dasdeck.de/valentin/midi. It’s a collection of PHP scripts to work with MIDI. Especially useful is Midi2Xml where you can upload a standard MIDI file which it translates into MusicXML format. It helped me a lot to discover how MusicXML works.

Slicer with waveform display

This slicer works the same as the previous one. Only difference is that is uses eight slices instead of sixteen.

The new stuff is in the interface: Waveforms to see what’s playing. And nicer to look at as well.

To use this app you can click the buttons as before to select a slice to play, and now you will see the waveform change to represent the slice.

The second way to change the slices is to click and drag on the waveforms itself. Drag up for later slices and down for earlier ones. Drag all the way down for an empty slice and silence.

Here it is! The music is from a track by Flying Lotus. I don’t know which one exactly: It’s track nr. 23 from a collection called “38 Cartoons”, but I couldn’t find any information about it online.

How to get an image from MP3 sound data

The way I did it is first to calculate how many samples of a slice represent one pixel of the image, then get the biggest amplitude of those samples. That will be a number between 0 and 1. Then multiply it by half of the images height to get the size of the line to draw. And finally draw the line to a BitMapData object.

It generates a simplified waveform: The positive and negative amplitudes are the same and it’s only the left channel. But it’s good enough for this purpose. It’s a good impression of the waveform.

Here is the main part of the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

var mp3NumSamples : Number = mMp3Loop.extract(new ByteArray(), (41000 * 60 * 60)) - mMp3StartOffset;
mMp3NumSamplesPerSlice = Math.round(mp3NumSamples / Model.NUM_SLICES);
mBPM = TimeConversion.numSamplesToBPM(mp3NumSamples, 2);
 
var sampleIndex : int = mMp3StartOffset;
var imageWidth : int = Model.BUTTON_WIDTH;
var samplesPerPixel : int = mMp3NumSamplesPerSlice / imageWidth;
 
// loop through the slices
for(var i : int = 0;i < Model.NUM_SLICES; ++i)
{
	var byteArray : ByteArray = new ByteArray();
	var numSamples : int = mMp3Loop.extract(byteArray, mMp3NumSamplesPerSlice, sampleIndex);
	var position : int = 0;
	var lineMiddle : int = (Model.DISPLAY_HEIGHT / 2);
 
	byteArray.position = 0;
 
	mMp3WaveformData[i] = new BitmapData(Model.BUTTON_WIDTH, Model.DISPLAY_HEIGHT, true, 0xFF0000);
	var bitmapData : BitmapData = mMp3WaveformData[i];
 
	// loop through the pixels of a waveform image's width
	for(var j : int = 0;j < imageWidth; ++j)
	{
		var waveHeight : Number = 0.0;
 
		// get maximum of all samples within pixel
		for(var k : int = 0; k < samplesPerPixel ; ++k)
		{
			if(position < numSamples)
			{
				var l : Number = byteArray.readFloat();
				byteArray.readFloat();
			}
			else
			{
				l = 0;
			}
 
			waveHeight = Math.max(waveHeight, Math.abs(l));
			position ++;
		}
 
		waveHeight *= lineMiddle;
		var start : int = lineMiddle - waveHeight;
		var end : int = lineMiddle + waveHeight;
 
		// draw to bitmapdata
		for(k = start; k < end ; ++k)
		{
			bitmapData.setPixel32(j, k, 0xFFFFFFCC);
		}
	}
 
	sampleIndex += mMp3NumSamplesPerSlice;
}

Part of the slicer

This is it! The first one: A slicer.

It loads and plays an MP3 file. But not only that. It divides the file in sixteen equal parts and you can change the order in which the parts are played.

When it’s loaded the slicer plays all the parts in the right order. Part 1, 2, 3 and up until part 16. With the buttons you can change the order and for example play part 4, then 3, 11, 8 and so on. With the right MP3 loop this will create all kinds of interesting rhythmic variations.

It works best if the MP3 file is a short piece of music with a steady rhythm that loops seamlessly (so the start and end of the sound fit together exactly). Then every slice has a little bit of rhythm in it. Just one or two sounds like a kick or snare drum.

Best to check it out for yourself. I will put the actual SWF files on the ‘Read more’ pages of the posts, so they won’t all be playing at the same time and use up each other’s CPU cycles.

The interface

It’s a matrix! Sixteen buttons wide for the time slots, sixteen high for the slices. So the bottom left button selects slice one to be played first, the top left button plays slice sixteen at position one. Just click around and you’ll get it.

The code

The main thing in this application is the Sound.extract() method. Once the MP3 is loaded the total amount of samples in the file is divided by sixteen. Then it can store the duration in samples of a slice and at which sample exactly each slice starts. So the file isn’t actually split into sixteen parts. It only stores the starting point of each slice within the loop.

One thing I didn’t know but found out trying is how to get the total amount of samples in an MP3 file: Use extract() and request a length longer than you know the MP3 file is. For example one hour. Then extract() will get as many samples as it can. Which are all the samples in the file, and it returns that number:

var oneHour : uint = 44100 * 60 * 60;
var numSamples : uint = mp3File.extract(new ByteArray(), oneHour);

If you recognized the loop but thought it was somehow different: You’re right! It’s Mr Oizo’s “Bobby Can’t Dance” from his album “Analog Worms Attack” (1999). But speeded up to 120 BPM. I liked it a bit faster. (The speeding up wasn’t done with this slicer by the way. No advanced ActionScript 3.0 timestretching algorithm yet I’m afraid.)

I’ve had this www.hisschemoller.com domain for years, but didn’t really have any use for it. I just got it because it was there – my own name as a dot com. Better get it before someone else does. Especially before one of those companies get it that buy domain names just to sell them for a profit. “Get some money out of those Hisschemöllers” they might think. That’s not how the internet should work.

Now I have a use: The past months I’ve been getting more and more into programming audio stuff with Flash. Small experiments with sound, music, samples etcetera. I’ll post those Flash files here. And the sources as well if I come up with anything good. I learned a lot from other blogs, so now I can do something in return.