So finally we present here our first release of WarpMap, an small application that we're using to play videos with advanced warping capabilities in our projection mapping projects. This first release it's just the binaries on Mac (10.5.7) and Windows (XP) systems,once we've cleaned the code and test it on some hardwares we'll publish the entire project.

WarpMap is basically a video player application that let's you distort the video texture with a quad-warping and a fine internal warping, with it you could match your videos on the projection mapping surface. The configuration of which videos and which images the software uses is done trough a XML file present on the /data/xml folder. 

normal quad-warping	internal fine warping

Give it a try and let'us know if it runs on your system ! This e-mail address is being protected from spambots. You need JavaScript enabled to view it

Finally we made it ! We've been some months behind the scenes preparing this show with the Telenoika crew. It's a facade audiovisual mapping made for the Ingravid Festival in Figueres, Girona. From Playmodes we developped the video-warper-player which just played a 20 minutes video on the facade. Also some of the 2D and 3D moments of this show were created by us with compositing and 3D rendering software. The video-warper-player was made in Openframeworks.cc 0.05. We're cleaning the code and we'll put it online soon, as well as a report of the process we followed to create this audiovisual performance. Enjoy !l  

Credits : Omar Alvarez, Miki Arregui, Miguel Gozalbo, Sergi Casero, Eloi Maduell, Santi Vilanova from Telenoika.NET audiovisual community.

Thanks to : Telenoika, Mas Queló, Festival Ingravid, LABfabrica.NET, BAF G.C

Here we'll post all the source code for the video-warper-player we developped for the Telenoika audiovisual mapping in Figueres, 2009.

Playmodes is an audiovisual tool for the sincronic manipulation of live audiovisual flows. It uses as a raw material the audiovisual stream coming from any live av source (cameras/microphones, AV mixers, internet) and it deconstructs this flow following some rules which are defined by the different play modes.

The tool works similar to an audio live-sampler / looper, with the difference that it manipulates not just audio but AV as a unique information. Once stored the buffer, it is possible to manipulate this content with a series of AV effects, linked to a BPM:

RANDOM.
Random movement of the playhead to stocastic positions of the movie. Thie frequency of the movement can be linked to a bpm, and quantized to the beat, so the jumps are only made to significant parts of a rhythm.

MULTIX
N copies of the AV flow are superposed and delayed by n miliseconds (or bpm divisor), resulting in an effect with a very special plasticity.

REVERSE
The AV signal is reversed. Using some audio envelope following techniques, this effect can hear when a word or a phrase is ended, and immediately reverse it. 

RECLOOP
AV overdubbing. Once a loop length is selected, the recording starts in a loop mode, permanently overwritting (in a non destructive mode) the stored content. Some features of this mode can also be triggered by audio events, and recording can be activated when the audio rises a definde treshold.

DELAY
The AV signal is delayd by n milisconds, or any bpm divisor.

L8P
AV loop recorder. 8 different loops can be stored, manipulated and played polyphonically.

REPEAT
A beat repeat effect, linked to a bpm. When activated, the last n bpm divisor is repeated.

SLITSCAN
A slitscan effect, as used by the great filmaker Zbigniew Ridcinski.

 Playmodes also implements Live AV Timestretching and Pitchsifting, so all of this effects can be applyed using this as modifiers. At the moment it consists in a C++ coded application (which is already downloadable) for the video effects, and a reaktor enseble which does the same effects in audio. Both applications communicate whith each other via the OSC protocol.Here you can see the control GUI:

Playmodes process will be presented during the IDN festival [Imatge Dansa i Nous mitjans] organized by NU2s iin El Mercat de les Flors, Barcelona, from 9th to 11th January 2008. We'll be introducing the new software Playmodes and presenting our results from the CorpusMedia workshops we've been taking part during last part of 2008. See you there !

Here's the first public release of the new Playmodes architecture. Playmodes is developed with openframeworks.

The new Playmodes is based in the openframeworks event system, that is still under development and for what this software is being also a test platform as it has diferent kinds of events and is also threaded.

You will need a modified openframeworks version that you can find in the links at the end of this post. This openframeworks version is not oficial and will be changed for sure for 0.06 so don't rely on it for any development. Also expect some changes in the event system in Playmodes. Apart from that, the public interface should remain more or less unchanged in the future.

Here's a brief explanation of the object model and it's use... + download

The new grabber - buffer - header - renderer architecture, allows great flexibility both in time control and rendering. Here's a video demoing some 3d effects with 40 delayed copies distributed in 3d and some perlin noise affecting the rotation of the frames.

---

This is the GUI for the new AV system we are developing. The main logic system is implemented in Reaktor, and communicates with the video software coded in OpenFrameworks through OSC protocol. As you may see, this instrument contains an audio buffer which gets audio from a live input (microphone, line-in, etc) and can be freezed at anytime. Then we have 8 different loop designers that can select a small portion of the whole audiobuffer and reproduce them triggered by MIDI notes. All of the 8 loop designers have 4 parameters: POS, LEN, PITCH and GATE.
 
With the POS (position) parameter we can set an in point for the loop to be triggered. This POS parameter can be also configured to move freely through the buffer or in a quantized flavour, which can be usefull if we stored a rhythm in the buffer and we want to travel to "relevant" parts of the rhythm. This Quantization is done entering the number of steps we want to subdivide the buffer by in the numeric field below the POS wheel.
 
The LEN (lenght) wheel defines the lenght (the outpoint) of the loop, and is defined in number of beats, this way we are able to achieve rhythmic results also from arhythmic sources.
 
The PITCH parameters defines the pitch of the loop, from -12 to 12 semitones. There are two ways of driving this effect: As long as this pitch parameter is a pitchbending (time reduces when pitch rises and viceversa) and not a pitchshifting (pitch changes but duration remains the same) we added a PITCH>LENGHT option that can switch between a mode in which the lenght of the grain remains the same when pitch is modified (and therefore adding more audio information from the buffer when pitching up, and loosing audio information when pitching down) and a mode in which the lenght of the loop grows or reduces proportionally to the pitch, without any loss of audio information.
 
The GATE parameter can be used to silence the grain before it has reached the Outpoint, and adds a stroboscopic feeling to the AV flow, making a stronger feeling of the ryhthmic sequences. 

The 8th loop designer is entitled as 8_stretch. The logics of this loop designer is a little different from the others, as it allows us to do a timestretching process over the signal (live signal or stored buffer). It is basically the same process than in the other 7 loopers, with the difference that we have a RAMP wheel instead of a POS wheel. With the lenght of this sawtooth ramp (defined also in beats) we define how long will last the way of the loop from the beggining of the buffer to the end. With this little tweaking, we achieved a granular flavour timestretching! We also added a switch between a live or offline mode (which just inverts the sense of the ramp).

This is the very first class diagram that Arturo Castro has developped for the new Playmodes. We start with this new design to develop the core.The idea is to have generalize modules which produce or receive audio or video stream  so then we'll be able to plug them ones to the others. Within the developpment process we'll evolve this with implementation changes to suit our needs.

One of the heaviest modes: Loop. With loop you can record a AV fragment and define an in point and the lenght of the loop you want to create. It can also be set up to act as a BeatRepeat effect, as seen in Ableton Live.

Loop can work in two diferentiated ways: Rec and Repeat

In the Rec mode, when the Rec button is pressed an audiovisual flow is being captured, by unfreezing the Core block. 8 different AV samples can be recorded and stored in individual places in memory. When the Rec button is released (the buffer in the Core block is freezed) and the av sample captured, user can define the in point (Playhead Pos) and lenght of the loop to be played (bpm/ms freq), both in ms or BPM divisors. In fact, the lenght of the loop is determined by the ticks received by the Clock module, which move the playhead to the in point at a defined rate. User can also configure the module in such a way that when the Rec button is released, the sample starts playing automatically from the beggining.

In the Repeat mode, which we understand is a particular case of loop, we have a Start button which starts repeating. This Start action is being quantized by the Qgrid clock, in order to correct the offset of the BPM grid. We can also define what portion of the last played AV will be repeated, both in ms or BPM, through the length parameter. The N parameter defines the number of repetitions before the buffer is actualized again and a new slice of AV is repeated. Note that the Repeat can be set up both in Pre or Post mode. If set up in Pre, the buffer will be freezed instantly and the last slice of recorded AV will be played. Otherwise, if Repeat is set up in Post, the buffer will be freezed once the length of the slice to be repeated has finished.

It is important to note that this Loop mode needs, as well as Reverse, a Playhead Ramp generator, in order to move the playhead through the buffer. This playRamps can be set up to move at any velocity and also to perform timestretching-like behaviors. These topic of the playhead ramps possibilities will be discussed later...

We are planning to implement a preset storing system, to save specific states of the parameters in the modes. We would also like this presets to be recalled from the GUI using buttons, to make easier the realtime performing of beatrepeats at different rates, or to define different in points and loop lengths.

The Reverse mode reverses the AV signal from the present time until a point in the past defined by the user. Let's see some of its particularities:

In the ENV mode, reverse gets the smoothed audio envelope information retrieved by the Core block. When the Audio Envelope returns a value higher than a Threshold defined by the user, the buffer in the Core block is unfreezed (it is freezed by default in this ENV mode) and starts recording for as long as the user defined with the Decay parameter(in ms). It has to be noted that this decay time begins its count when the Audio Envelope matches a value smaller than the Treshold. When the signal has gone below the threshold value for as long as defined in the decay time, the reverse module generates a PlayHead Position ramp that is inverted (from present to past), and that lasts until it matches the point in where the audio envelope was higher than the treshold.

In the BPM Mode, the signal is played from present to past for as long as the user defined (in BPM divisor or ms). When Start button is pressed, the buffer is freezed (it is unfreezed by default in this BPM mode) and the playhead ramp reversed. It has to be noted that the Start action (which begins reversing the signal) is quantized using the information received from the Clock module.

So, this is the first PlayMode we are really describing. It basically retards the realtime AV for the time defined by the user.

The delay mode gets the BPM info, and delays the AV signal for the time the user has defined as a BPM Divisor (4 beats, 2 beats, 1 beat,3/4, 1/2....). The mode can be switched between BPM or ms, so the user can also define the delay time in ms.

This delay time over the signal is performed moving the PlayHead position in the Core module as ms in the past as the user defined. In this mode there is no need to use the clock events (just the general BPM), as the calculation for the PlayHead offset is performed internally.