• Source:The classes that inherit from AudioSource or VideoSource are meant to be data providers for other objects in the pipeline. A source can provide data through event notification to other objects or through method call from other object that needs data with it's own timing.

event notification: a buffer receives frames from a grabber whenever there's a new frame available so the grabber notifies a new frame and the buffer stores it.

To add and remove listeners to an specific source you can call:

    void addListener(VideoFrameListener * listener)

    void removeListener(VideoFrameListener * listener)

Also most of the objects that receive data in Playmodes have can be created by passing a source in it's constructor, so most of the times you won't need this methods.

Sources have an event named newFrameEvent so if you implement you're own sources you will need to notify this method whenever new data is available:

    newFrameEvent.notify(this, frame);

This way every listener attached to a source will receive the new data.

method call: a renderer needs a new frame in the draw call so it calls it source getNextFrame method to get the frame and draw it.

   VideoFrame * getNextVideoFrame()

Depending on the kind of source this method will return a frame. Most of the times, it returns the last frame available, except for headers were it provide the frame at the position the header is pointing.

    int getFps()

This method returns the frames per second the source is using. Theoretically different sources can have different fps but in the current implementation every object in the pipeline gets its fps from the previous one so the fps for all of them is that of the grabber and the physical device providing the data.

 

• Sink : The classes inheriting from this one, are that ones that receive data for storing it as buffers, processing it as filters or rendering it as renderers. Every sink is a frame listener so it's able to be notified of a newFrame event so the only method that a sink needs to implement is:

Although it's not mandatory, most of the sinks also implement a constructor like:

    VideoSink(VideoSource* source)

So it can register as a listener for the newFrame event from it's creation and it can receive new frames when they are available.

• VideoGrabber: this class extends ofxAdvVideo to be able to provide video frames from both physical devices or video files. ofxAdvVideo is just a wrapper over VideoGrabber and VideoPlayer, in the case of linux also Pierre Proske's ofxDVGrabber. Depending on the initialization call it will work as one of them. This class will be substituted at some point when openframeworks provide some object hierarchy for video classes

• AudioGrabber: this class inherits from openframeworks' ofAudioListener and registers to ofEvents::audioReceived to be able to receive audio data and provide it to the rest of the objecs in Playmodes in a more playmodish way. Audio is divided in audio frames (you can call them grains). By now the audio frame size is that of the soundcard buffer used in the openframeworks audio initialization with the ofSoundStreamSetup call. This will change in the future to a continous data stream so audiograin size can be adjusted when recovering frames from a buffer.

    VideoFrame * getVideoFrame(int position);          // frame number in the buffer

    VideoFrame * getVideoFrame(ofTimeDiff time);   // frame at n microseconds from the end of the buffer

    VideoFrame * getVideoFrame(float pct);             // % of the buffer

    VideoFrame * getNextVideoFrame();                  // the last video frame in the buffer

    void stop();                                    // stop receiving new frames
    void resume();                               // continue receiving new frames 

    VideoHeader(VideoBuffer *buffer)

    void draw()

    VideoFrame * getNextVideoFrame()

    int getNextPosition()

    float speed;

    int delay;

    float in, out;

    bool loopMode; 

    void setPct(float & pct)

    void incrPct(float & incr)

• VideoRenderer: There's a simple renderer that receives a video source in it's constructor and renders the current frame from that source on the opengl draw cycle. It has also some methods and properties to set the draw position in 3D, this allows to set all this variables by using different methods (like the perlin noise generator used in the example app) and allows for different emergent behaivours:

    void draw()

Draws the current frame affected by the following properties:

    float scale

    float rotationZ, rotationX, rotationY

    float x, y, z

    float anchorX, anchorY      // to set the rotation origin of the renderer

    int alpha

    int tintR, tintG, tintB

    bool minmaxBlend          // to set the blending method to min or max instead of add

    void setDraw(bool isDrawing)  // allows to set the renderer not to draw 
                                              // when it's draw method is called

• MultixRenderer: This class implements the previous Multix mode with some additional 3D position and time controls. It receives a buffer and an initial number of headers and create that number of headers and the same number of renderers. You can access all that headers and renderers from outside the class to modify they're properties further than the multix renderer controls allow.

The controls provide the same properties as a simple renderer, when you set some of those properties, in the next update cycle, the first renderer is set with that property and the rest of the properties copy the property of the next from the last one giving a kind of trailing effect.

It also provides some offsets in the rotation, delay, speed.. from the first renderer to the next.

• RecLoopRenderer: This class implements the previous RecLoop mode. Combining it with a header and a simple renderer you'll get all the new 3D and timing effects provided by them.

You need to pass a buffered video source (usually a header) and a live input (usually a grabber) and it will modify the original frames in the buffer adding to them the live input using a max or min blend equation depending on how you set the mimaxBlend property.

    VideoFrame * getNextVideoFrame()

    void newVideoFrame(VideoFrame & frame)

• FreeFrameFilter: As an example filter implementation you will find a Playmodes wrapper for FreeFrame 1.0. This filter receives a video source in it's constructor and register to it's new frame event in it's creation. Then you can call it's loadPlugin and setParameter methods to configure it's behaivour. You can use any FF 1.0 plugin, that requires only one input.

• VideoFrame: It receives a byte array with the video frame data and the height and width in it's constructor. It stores that data in memory and also in an opengl texture. It's prepared to avoid the problems with opengl unsafeness by uploading the data to the texture only in the openframeworks update event. Whenever a new VideoFrame is created, it registers itself to the update event and only then upload the data to an ofTexture.

It's also prepared to be deleted only when no object is using it anymore. For that it extends ofxObjCPointer. This super useful openframeworks addon from Memo Atken provides a simple way of controlling when a class is not used anymore and only then delete it. For that it provides retain and release method. In Playmodes, buffers automatically retain frames when you ask for them so if you directly read them from a buffer or a header you don't need to retain them, only call they're release method when you don't use them anymore.

Also to be thread safe, the VideoFrame has to delete it's texture on deletion so instead of deleting the object just when the use counter arrives to 0, it registers to the openframeworks update event and only then it deletes itself.

Frames has an static counter that is updated on every frame creation and deletion for debugging purposes. This allows to know how many frames exists currently in memory.

• AudioFrame: The audioFrame works exactly the same as the video one. The only specifics in the audio version, is it doesn't need to be thread safe (in the sense it doesn't contains an opengl texture). Also on creation it calculates the average value of the data it contains for wave visualization purposes.

• Envelopes: The Playmodes audio system is based on a granular sampling paradigm. Although currently it's pretty basic we think this way it will allow for some nice effects at the audio level. Currently it's only used for time stretching in sync with the video speed so whenever you send data to the soundcard you need to apply an envelope to the frame in order to reduce the noise in case the reproduction speed is not the original one. You can see how this works in the testApp audioRequested method in the example app provided with this release. The only envelope provided by now is a raised cosine bell.

        oscInterface->mapMessage("speed",&speed);

        oscInterface.mapMessage("speed",
                                              new Poco::Delegate<VideoHeader,float>
                                              (videoHeader,&VideoHeader::setSpeed));