|
Valkka
1.6.1
OpenSource Video Management
|
Definitions:
A Decoder class decodes a frame
A DecoderThread class uses a Decoder and takes care of queueing the frames, mutexes, synchronization, etc.
Decoder subclassing as currently practised:
Decoder (virtual)
BasicFrame input_frame
output_frame: defined in subclasses as type depends on decoder
flush()
virtual Frame *output() = 0 : returns a reference to output_frame
virtual bool pull() = 0 # decode!
isOK() # is the decoder still working?
AVDecoder : Decoder (virtual)
Audio or video decoding using libav
AVCodecID, AVPAcket, AVCodecContext, etc. libav paraphernalia initiated at the ctor
VideoDecoder : AVDecoder
Video decoding using libav
AVBitmapFrame out_frame
implements Frame *output() that returns out_frame
implements pull(): AVPacket's etc. filled with in_frame, results into out_frame
AVHwDecoder : Decoder (virtual)
Audio or video decoding using libav with hardware acceleration
Takes as an input, the libav AVHWDeviceType
Like AVDecoder, but creates also AVBufferRef hardware context:
AVCodecID, AVPAcket, AVCodecContext, AVBufferRef etc. libav paraphernalia initiated at the ctor
NOTE: most of the ctor code copy-pasted from AVDecoder
HWVideoDecoder : AVHwDecoder
Hw accelerated video decoding using libav
Takes as an input, the libav AVHWDeviceType
Identical (a copy-paste) to VideoDecoderNOTE: if all the contexes, etc. were not created at ctor but in another method (say, "init"), we could create saner subclass structure. --> then the DecoderThread::chooseVideoDecoder should call that "init" method just after "new".
About Decoder::pull internals
- We'd like to fill the actual target frame data structures (out_frame.av_frame) directly during decoding, i.e. without any intermediate copies/transforms - However, we only know if the pix format is the desired AV_PIX_FMT_YUV420P after-the-fact, i.e. after decoding We use this scheme in Decoder::pull: 1. Assume initially current_pixel_format == AV_PIX_FMT_YUV420P 2. If current_pixel_format == AV_PIX_FMT_YUV420P, encode data directly to out_frame.av_frame Otherwise encode data into auxiliary frame (aux_av_frame) [from which it will eventually be transformed into out_frame.av_frame] 3. Observe the pixel format (new_pixel_format) returned by the encoder 4. If not AV_PIX_FMT_YUV420P, then setup infra for performing pixel format transformation from new_pixel_format --> AV_PIX_FMT_YUV420P and replace current_pixel_format := new_pixel_format Also copy data that was (erroneosly) placed to out_frame.av_frame, into aux_av_frame or just return from function (we lose the current frame) Step (4) in a more generalized form: 4. If pix fmt has changed (i.e. new_pixel_format != current_pixel_format), (re)setup infra for transforming new_pixel_format --> AV_PIX_FMT_YUV420P and replace current_pixel_format := new_pixel_format return from function (we lose the current frame) 5. If transformation infra is in place, then sws_scale from current_pixel_format into AV_PIX_FMT_YUV420P 6. For next frame, go to (2)
DecoderThread classes use the Decoder classes and run the queues, semaphores, etc.
DecoderThread
virtual Decoder* chooseVideoDecoder(AVCodecID codec_id);
returns a Decoder instance or NULL of none avail for the codec_id
virtual Decoder* fallbackVideoDecoder(AVCodecID codec_id);
returns a second option Decoder instance or NULL of none avail for the codec_id
run
- DecoderThread receives SetupFrame(s) and BasicFrames(s)
- On receiving a SetupFrame, it tries to get a new Decoder by
calling chooseVideoDecoder
- It regularly calls Decoder::isOk() -> if that returns false,
it calls fallbackVideoDecoder