Valkka  1.6.1
OpenSource Video Management

Definitions:

A Decoder class decodes a frame

A DecoderThread class uses a Decoder and takes care of queueing the frames, mutexes, synchronization, etc.

Decoder subclassing as currently practised:

Decoder (virtual)
    BasicFrame input_frame
    output_frame: defined in subclasses as type depends on decoder
    flush()
    virtual Frame *output() = 0 : returns a reference to output_frame
    virtual bool pull() = 0 # decode!
    isOK() # is the decoder still working?
    
    AVDecoder : Decoder (virtual)
        Audio or video decoding using libav
        AVCodecID, AVPAcket, AVCodecContext, etc. libav paraphernalia initiated at the ctor

        VideoDecoder : AVDecoder
            Video decoding using libav
            AVBitmapFrame out_frame
            implements Frame *output() that returns out_frame
            implements pull(): AVPacket's etc. filled with in_frame, results into out_frame

    AVHwDecoder : Decoder (virtual)
        Audio or video decoding using libav with hardware acceleration
        Takes as an input, the libav AVHWDeviceType
        Like AVDecoder, but creates also AVBufferRef hardware context:
        AVCodecID, AVPAcket, AVCodecContext, AVBufferRef etc. libav paraphernalia initiated at the ctor
        NOTE: most of the ctor code copy-pasted from AVDecoder
        
        HWVideoDecoder : AVHwDecoder
            Hw accelerated video decoding using libav
            Takes as an input, the libav AVHWDeviceType
            Identical (a copy-paste) to VideoDecoder

NOTE: if all the contexes, etc. were not created at ctor but in another method (say, "init"), we could create saner subclass structure. --> then the DecoderThread::chooseVideoDecoder should call that "init" method just after "new".

About Decoder::pull internals

- We'd like to fill the actual target frame data structures (out_frame.av_frame) directly during
  decoding, i.e. without any intermediate copies/transforms
- However, we only know if the pix format is the desired AV_PIX_FMT_YUV420P after-the-fact, i.e. after decoding

We use this scheme in Decoder::pull:

1. Assume initially current_pixel_format == AV_PIX_FMT_YUV420P

2. If current_pixel_format == AV_PIX_FMT_YUV420P, encode data directly to out_frame.av_frame 
   Otherwise encode data into auxiliary frame (aux_av_frame) [from which it will eventually be transformed into out_frame.av_frame]

3. Observe the pixel format (new_pixel_format) returned by the encoder

4. If not AV_PIX_FMT_YUV420P, then setup infra for performing pixel format transformation from
   new_pixel_format --> AV_PIX_FMT_YUV420P and replace current_pixel_format := new_pixel_format
   Also copy data that was (erroneosly) placed to out_frame.av_frame, into aux_av_frame
   or
   just return from function (we lose the current frame)

Step (4) in a more generalized form:

4. If pix fmt has changed (i.e. new_pixel_format != current_pixel_format), (re)setup infra for
   transforming new_pixel_format --> AV_PIX_FMT_YUV420P and replace current_pixel_format := new_pixel_format
   return from function (we lose the current frame)

5. If transformation infra is in place, then sws_scale from current_pixel_format into AV_PIX_FMT_YUV420P

6. For next frame, go to (2)

DecoderThread classes use the Decoder classes and run the queues, semaphores, etc.

DecoderThread

    virtual Decoder* chooseVideoDecoder(AVCodecID codec_id);
        returns a Decoder instance or NULL of none avail for the codec_id

    virtual Decoder* fallbackVideoDecoder(AVCodecID codec_id);
        returns a second option Decoder instance or NULL of none avail for the codec_id

    run
        - DecoderThread receives SetupFrame(s) and BasicFrames(s)
        - On receiving a SetupFrame, it tries to get a new Decoder by
          calling chooseVideoDecoder
        - It regularly calls Decoder::isOk() -> if that returns false,
          it calls fallbackVideoDecoder