Valkka uses the live555 streaming media library. This page attempts to make it easier for the beginner to get a grasp on live555 (and Valkka).

Live555 is based on event loops and callbacks (event loop and callback "paradigms" to put it in fancy terms). Events are registered to an event loop. For example, an event can send a message through TCP socket and set a callback that is executed once a reply has been obtained. The callbacks then register new events with new callbacks to the event loop and so forth.

An event can be the arrival of a streaming UDP packet. This triggers a callback that aggregates the packet content to a slice of H264 video, for example. After this, a new event, that waits for the next packet, is registered to the event loop.

Connecting to an ip camera and starting to stream media, would look (very) roughly like this:

[ Event loop "tick" number : execution ]

001 :

002 : Send RTSP DESCRIBE command through a TCP socket to the ip camera. Define a callback function ("continueAfterDESCRIBE") that is called once the response is obtained. Wait for TCP socket using select.

003 :

004 : Got response to RTSP DESCRIBE. Use the callback function ("continueAfterDESCRIBE"). It sends the RTSP SETUP message to the ip camera and defines a callback function that is called once we get response to RTSP SETUP ("continueAfterSETUP"). Wait for TCP socket using select.

005 : Got response to RTSP SETUP. Use the callback function .. etc.

[Go through the whole RTSP negotiation this way]

...

040 : Got a packet of H264 slice from an UDP socket. Aggregate it to the current H264 slice under construction. Register an event waiting for the next UDP packet. Wait for the UDP socket using select.

042 : Got a packet of H264 slice from an UDP socket. Aggregate it to the current H264 slice under construction. Register an event waiting for the next UDP packet. Wait for the UDP socket using select.

...

050 : Got the last packet of a H264 slice from an UDP socket. Aggregate it to the current H264 slice under construction. Call afterGettingFrame (of the subclassed MediaSink object) with the H264 slice. Start a new H264 slice. Register an event waiting for the next UDP packet. Wait for the UDP socket using select.

...

The example above is a rough approximation to what live555 does. It also considers only one stream coming from a single ip camera. In practice, live555 "multiplexes" several streams coming from several ip cameras simultaneously and this way achieves concurrency - no threads required !

Libraries using live555 typically "hook up" to the afterGettingFrame callback, by implementing their own MediaSink that receives the composed frame. The complete frame is then passed on for further processing, decoding and visualization.

Valkka "isolates" live555 library into a separately running thread. Inside this thread, live555 runs happily and does its magic. In the thread, we also register a a periodic callback into the live555 event loop (periodic = a callback that re-registers itself periodically). This callback checks every second, from within the live555 event loop, any incoming commands to the thread via a mutex/condition variable protected message variable. By sending messages, we can instantiate, shut-down, etc. new RTSP connections inside the live555 event loop. This approach is completely thread-safe.

In Valkka, once the composed frame is obtained, it is passed through series of "frame filters" in a callback cascade. Each filter does something to the frame, say, corrects the presentation timestamp in some way or does further composition, etc. and then passes the frame to the next filter. The frame filter callback chain typically ends into a frame filter that inserts the frame into a mutex-protected fifo queue for inter-thread communication (see Library architecture).

It is important to remember that while such callback chains are executed, the live555 event loop is paralyzed (see step "050" of the event loop above) ! The callback cascades end typically on a thread or process "border".