Valkka
1.6.1
OpenSource Video Management
|
Definitions:
A Decoder class decodes a frame
A DecoderThread class uses a Decoder and takes care of queueing the frames, mutexes, synchronization, etc.
Decoder subclassing as currently practised:
Decoder (virtual) BasicFrame input_frame output_frame: defined in subclasses as type depends on decoder flush() virtual Frame *output() = 0 : returns a reference to output_frame virtual bool pull() = 0 # decode! isOK() # is the decoder still working? AVDecoder : Decoder (virtual) Audio or video decoding using libav AVCodecID, AVPAcket, AVCodecContext, etc. libav paraphernalia initiated at the ctor VideoDecoder : AVDecoder Video decoding using libav AVBitmapFrame out_frame implements Frame *output() that returns out_frame implements pull(): AVPacket's etc. filled with in_frame, results into out_frame AVHwDecoder : Decoder (virtual) Audio or video decoding using libav with hardware acceleration Takes as an input, the libav AVHWDeviceType Like AVDecoder, but creates also AVBufferRef hardware context: AVCodecID, AVPAcket, AVCodecContext, AVBufferRef etc. libav paraphernalia initiated at the ctor NOTE: most of the ctor code copy-pasted from AVDecoder HWVideoDecoder : AVHwDecoder Hw accelerated video decoding using libav Takes as an input, the libav AVHWDeviceType Identical (a copy-paste) to VideoDecoder
NOTE: if all the contexes, etc. were not created at ctor but in another method (say, "init"), we could create saner subclass structure. --> then the DecoderThread::chooseVideoDecoder should call that "init" method just after "new".
About Decoder::pull internals
- We'd like to fill the actual target frame data structures (out_frame.av_frame) directly during decoding, i.e. without any intermediate copies/transforms - However, we only know if the pix format is the desired AV_PIX_FMT_YUV420P after-the-fact, i.e. after decoding We use this scheme in Decoder::pull: 1. Assume initially current_pixel_format == AV_PIX_FMT_YUV420P 2. If current_pixel_format == AV_PIX_FMT_YUV420P, encode data directly to out_frame.av_frame Otherwise encode data into auxiliary frame (aux_av_frame) [from which it will eventually be transformed into out_frame.av_frame] 3. Observe the pixel format (new_pixel_format) returned by the encoder 4. If not AV_PIX_FMT_YUV420P, then setup infra for performing pixel format transformation from new_pixel_format --> AV_PIX_FMT_YUV420P and replace current_pixel_format := new_pixel_format Also copy data that was (erroneosly) placed to out_frame.av_frame, into aux_av_frame or just return from function (we lose the current frame) Step (4) in a more generalized form: 4. If pix fmt has changed (i.e. new_pixel_format != current_pixel_format), (re)setup infra for transforming new_pixel_format --> AV_PIX_FMT_YUV420P and replace current_pixel_format := new_pixel_format return from function (we lose the current frame) 5. If transformation infra is in place, then sws_scale from current_pixel_format into AV_PIX_FMT_YUV420P 6. For next frame, go to (2)
DecoderThread classes use the Decoder classes and run the queues, semaphores, etc.
DecoderThread virtual Decoder* chooseVideoDecoder(AVCodecID codec_id); returns a Decoder instance or NULL of none avail for the codec_id virtual Decoder* fallbackVideoDecoder(AVCodecID codec_id); returns a second option Decoder instance or NULL of none avail for the codec_id run - DecoderThread receives SetupFrame(s) and BasicFrames(s) - On receiving a SetupFrame, it tries to get a new Decoder by calling chooseVideoDecoder - It regularly calls Decoder::isOk() -> if that returns false, it calls fallbackVideoDecoder