Sunday 23 January 2011

Introducing RMidi

In short, RMidi provides a way to work with MIDI from within R. The ambition is to open up the rich R language to applications in music informatics. There already exists R packages for working with audio, and I believe that the polygamous wedding of audio, MIDI and the strong data processing capabilities of R could become a powerful tool.

The basic data structure in RMidi is the RMidi matrix. This is a five column matrix with the following column headers: tick, command, channel, byte1 and byte2. The tick column sets the tick time at which the event will occur; the command column specifies the type of event (e.g. noteon, noteoff, etc.); channel is self-explanatory; byte1 and byte2 can differ depending on the command column, e.g. in noteon and noteoff messages, byte1 is the pitch and byte2 is the velocity.

If you are familiar with the MIDI standard you will notice that the RMidi matrix format is very similar to the way the actual MIDI protocol is laid out. I found this to be one of those rare occasions where minimising the level of abstraction actually simplifies the interface. Staying true to MIDI means that I am able to represent noteon, noteoff, control, pitch bend, etc., using the same data structure.

As an example, the sequence of notes C4, E4, G4 could look like this:

tick command channel byte1 byte2
[1,] 0 1 0 60 127
[2,] 92 1 0 64 127
[3,] 184 1 0 67 127
[4,] 92 2 0 60 0
[5,] 184 2 0 64 0
[6,] 276 2 0 67 0

In RMidi, all commands are represented by integers. The noteon command is represented by 1 and noteoff is represented by 2. In the matrix above, noteon for C4 is triggered at time 0, and noteoff for C4 at time 92. As you notice, the ordering of rows in the matrix is irrelevant.

In this first iteration of the software, RMidi supports only noteon and noteoff messages. This will of course change in the near future.

To create RMidi matrices you can either do it by hand, or you can use convenience functions. I recommend the latter approach. Using the midi.note function, creating the matrix above is as simple as

> midi.note(0:2 * 92, 92, c(midi.notes.c,
+ midi.notes.e, midi.notes.g) + 60)

The first parameter sets the start times of the notes, the second parameter the durations, and the third parameter is the actual pitches. See ?midi.note for full documentation.

Outputting this matrix as MIDI information is done using midi.play:

> m <- midi.note(0:2 * 92, 92, c(midi.notes.c,
+ midi.notes.e, midi.notes.g) + 60)
> midi.play(m)

Now is probably a good time to mention that RMidi uses the ALSA MIDI sequencer API, and is therefore (currently) Linux-only.

In order to route the output of RMidi to an actual audio module, I use qjackctl. Click the Connect button which opens the Connections window. In the ALSA tab you should see "rmidi" on your left hand side.

To set the tempo and pulses (ticks) per quarter note, use the midi.set.tempo and midi.set.ppq functions.

RMidi is able to read .mid files using the midi.read.file function. The set of supported messages is limited to whatever commands RMidi supports. In effect this means that while RMidi is able to parse an entire MIDI file, the resulting matrix may contain only a subset of the information in the file.

As a last touch to this initial version of RMidi, I've added the midi.scale function. midi.scale takes an arbitrary vector of numbers and "forces" the vector onto a musical scale. For example, if we have the following vector:

> v <- c(0.1, 1.2, 2.5, 4.3, 5.7, 8.9,
+ 10.6, 11.1, 15.5, 16.3)

We can apply a major scale to v like this:

> midi.scale(v, midi.scales.major)
[1] 0 0 2 4 5 7 9 11 14 16

Note that midi.scale always rounds down.

I've bundled around 300 scales with RMidi, all prepended with midi.scales. and all starting from C. More info at ?midi.scale.

RMidi is hosted on Google Code here on Github. If you spot any issues or have any feature requests, please file a bug report or just send me a message.

Sunday 16 January 2011

7x7

http://7x7.loft-dev.com/ Firefox (>=3.6) only!

This is a little hack I did a while back during the 2010 London Musichackday. I've always found that computer metaphors for musical instrument user interfaces are very poor. A computer has a complex keyboard, a screen and a mouse, yet I find myself having to adapt my computer keyboard to piano keyboard emulations. Surely there must be some way to make, if not better, then at least different use of the input devices?

With this hack I wanted to see what could be done with a two-dimensional user interface. I play a bit of button accordion so I'm quite used to this idea myself, but I wanted to see if the matrix arrangement of notes could be useful when presented on a computer screen. Given the graphic nature of this interface, I decided that the mouse would be best suited as the input device. When you hold down the mouse button you "draw" a rectangle over the matrix. When you release the button all enclosed notes are played simultaneously, allowing for the creation of complex chord structures.



I took care in designing matrices that, just like on the chromatic button accordion, can be directly transposed just by moving the cursor. This resulted in three different layouts: A simple C-major layout and two systems based on fixed semitonal increments. The C-major layout makes it easy to create "well-sounding" chords which also go well together, like Cmaj9, Fmaj9, Dm7, G9, etc. The other two layouts enable the creation of interesting dissonances.

So what does it actually sound like? Well, very harmonious. Relaxing. Like whale singing. Bland, basically. I did try my best to mess up the waveform, but the envelope had to stay the same. This was not really by choice, but more a technical constraint. The notes are generated on the fly by a PHP script and returned as raw .wav files to be played by the Audio element. What I found during my initial experiments was that the Audio element wasn't able to deal with timely sequences of notes, and I had to resort to an instrument where exact timing didn't really matter.

I'm not sure the end result is good or bad, but at least it's different, which sort of was the whole purpose from the beginning. My presentation at musichackday was a full-scale disaster, but I did manage to get a little mention on cnet. I've seen people play with it, and it seems to make some intuitive sense, even to musical non-snobs.

Drop me a line if you're interested in the source code.