From RTSP stream to X-window.

Let's recall a typical filtergraph from Library architecture

(1.LiveThread) --> {2.FifoFrameFilter} --> [3.FrameFifo] -->> (4.AVThread) --> {5.FifoFrameFilter} --> [6.OpenGLFrameFifo] -->> (7.OpenGLThread)

In detail, frames are transported from (1) to (7) like this:

1 LiveThread

Frame::frametype set to FrameType::h264
Frame::payload has the raw data

4 AVThread

Uses data in Frame::payload
Sets Frame::frametype to FrameType::avframe

6 OpenGLFrameFifo

Has internal stacks of YUVPBO objects (with direct memory access to GPU)
A frame from one of the stacks (corresponding to the Frame resolution) is reserved
OpenGLFrameFifo::prepareAVFrame uploads data to GPU

7 OpenGLThread

Handles all X11 and GLX calls
Reads OpenGLFrameFifo, presents frames according to their timestamps
Once a Frame has been presented, it is recycled back to OpenGLFrameFifo

OpenGLThread uses the following classes:

SlotContext == Unique data related to a stream. Textures.
RenderGroup == Corresponds to a unique X window. Has a list of RenderContext instances.
RenderContext == How to render a texture in OpenGL: vertex array object (VAO), vertex buffer object (VBO), transformation matrix, etc. Has a reference to an activated SlotContext. This is the render target for your RTSP stream!

Internal data structures of OpenGLThread:

OpenGLThread::slots_ == A vector. A SlotContext for each slot. Warehouses SlotContext instances

OpenGLThread::render_groups == Mapping. Window_id => RenderGroup mapping. Warehouses the RenderGroup instances (each RenderGroup instance warehouses RenderContext instances)

OpenGLThread::render_lists == A vector. There is one element for each slot. Each element is a list of references to RenderGroup instances. Only RenderGroup(s) that have some RenderContext(es) should be in this list.

So, OpenGLThread::run reads a new frame from OpenGLThread::infifo. It is passed to the presentation queue, i.e. to OpenGLThread::presfifo. OpenGLThread::handleFifo is eventually called.

OpenGLThread::handleFifo inspects Frame::n_slot => takes a SlotContext from OpenGLThread::slots_. Preload the textures to SlotContext::yuvtex. So, textures are now loaded and ready to go. Now we have to find all render contexes that use these textures (there might be many of them).

Using Frame::n_slot, pick a list from OpenGLThread::render_lists (there is one list for each slot)

Run through that list: each element is a RenderGroup (remember that each RenderGroup has a list of RenderContexes)

RenderGroup has method RenderGroup::render. That method fixes the current X window for manipulation. So, X window has been chosen. Let's draw into it.

Then RenderGroup runs through all its RenderContex(s). For each RenderContext, RenderContext::render is called

RenderContext has vertex array, vertex buffer objects, transformation matrix, etc., everything needed for rendering a YUV image in any way you wish (straight rectangle, twisted, whatever).

RenderContext also has a reference to a valid SlotContext: RenderContext::slot_context. From there we get the Shader and YUVTEX (pre-loaded textures) which are now used together with the vertex array, vertex buffer objects, transformation matrix, etc.

Finally, render the texture with glDrawElements in RenderContext::bindVertexArray.

Recycle frame back to OpenGLFrameFifo