Framefilters

Framefilters are the building blocks of pipelines. The base class Limef::ff::FrameFilter defines the go() / cc() / pass() interface. Use Limef::ff::SimpleFrameFilter for one-to-one filters and Limef::ff::SplitFrameFilter for one-to-many forks.

namespace ff

Framefilters.

Typedefs

using json = nlohmann::json

Enums

enum class ChannelOrder

Output channel order for colour conversions.

Values:

enumerator RGB

Red-Green-Blue (PyTorch / PIL convention)

enumerator BGR

Blue-Green-Red (OpenCV convention)

Functions

static size_t serialize_frame_wrapper(uint8_t *buffer, size_t buffer_size, Frame *frame)

struct AudioTranscodeContext

#include <audiotranscode.h>

A Context variable that defines the audio codecs and their specific requirements.

NOTE: Only supports planar audio channel formats.

Public Functions

inline AudioTranscodeContext(AVCodecID codec, int bitrate, AVSampleFormat sample_format, int sample_size, const std::vector<int> &flags, const std::map<std::string, std::string> &opts = {})

Constructor with options.

Parameters:

• codec – Audio codec ID

• bitrate – Target bitrate

• sample_format – Target sample format (AVSampleFormat). NOTE: must be a planar format.

• calc – Sample calculation method

• flags – Codec flags

• opts – Dictionary options for encoder

Public Members

AVCodecID codec_id = {AV_CODEC_ID_AAC}

AV_CODEC_ID of the audio codec. Use to define the target codec.

int bit_rate = {44100}

Default / good bitrate for the codec. Feel free to change.

AVSampleFormat sample_format = {AV_SAMPLE_FMT_FLTP}

The default planar format for AAC.

int sample_size = {1024}

Sample size for the encoder.

const std::vector<int> codec_flags

Just store raw FFmpeg flags.

std::map<std::string, std::string> options

New field: dictionary options for the encoder.

Public Static Functions

static inline AudioTranscodeContext AAC()

Advanced Audio Codec (AAC)

static inline AudioTranscodeContext AAC(const std::map<std::string, std::string> &opts)

AAC with custom options.

Parameters:

opts – Dictionary options to override defaults

Returns:

AudioTranscodeContext

class AudioTranscodeFrameFilter : public Limef::ff::SimpleFrameFilter

#include <audiotranscode.h>

FrameFilter that transcodes audio to a different codec.

Takes a CodecFrame first to get stream information, then accepts audio PacketFrames, transcodes them and outputs new PacketFrames with the desired audio codec (target codec)

If the incoming codec matches the target codec, then passthrough is enabled and audio packets are passed downstream as-is

Param name:

Name for this framefilter

Param ctx:

A Context that defines the audio codec and it’s requirements

Param next:

(optional) next framefilter in the chain

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class CodecCacheFrameFilter : public Limef::ff::SimpleFrameFilter

#include <codec_cache.h>

FrameFilter that caches and can resend CodecFrames.

Acts as a passthrough filter for all frames, but when it encounters a CodecFrame, it makes an internal copy that can be resent later. This is useful when a new client/component needs to receive codec parameters mid-stream without waiting for a new CodecFrame to arrive naturally.

Thread-safe: can be called from multiple threads.

Example usage:

// In setup code
CodecCacheFrameFilter codec_cache("codec_cache");
source.getOutput().cc(codec_cache).cc(next_filter);

// ... later when a new component joins
codec_cache.reSend(); // Resends cached CodecFrame to next_filter

Public Functions

inline explicit CodecCacheFrameFilter(std::string name = "CodecCacheFrameFilter")

Construct a new Codec Cache Frame Filter.

Parameters:

• name – Unique name for this filter

• next – Optional pointer to the next filter in chain

~CodecCacheFrameFilter() override = default

Destroy the Codec Cache Frame Filter.

Cleans up any cached codec frame.

inline virtual void go(const Limef::frame::Frame *frame) override

Process incoming frame.

Passes all frames downstream. For CodecFrames, also maintains an internal copy.

Parameters:

frame – The frame to process

inline bool reSend()

Resend the cached CodecFrame downstream.

If a CodecFrame has been cached, sends a copy downstream. Thread-safe: can be called from any thread.

Returns:

true if a CodecFrame was sent

Returns:

false if no CodecFrame is available to send

inline void reset()

Reset the filter state.

Clears any cached CodecFrame. After calling this method, hasCodecFrame() will return false and reSend() will not send anything until a new CodecFrame is received.

Thread-safe: can be called from any thread.

inline bool hasCodecFrame() const

Check if a CodecFrame has been cached.

Thread-safe method to check if a CodecFrame is available.

Returns:

true if a CodecFrame has been cached

Returns:

false if no CodecFrame is cached

class CodecMessageFrameFilter : public Limef::ff::SimpleFrameFilter

#include <codec_message.h>

FrameFilter that sends a JSON message through a pipe when it encounters a CodecFrame.

Passes all frames through to the next filter unchanged, but when it encounters a CodecFrame, it sends a JSON message through the provided pipe for external monitoring.

Thread-safe: can be used from multiple threads.

Example usage:

// Create a pipe for receiving messages
auto [sender_pipe, receiver_pipe] = Limef::comm::Pipe::create();

// Create filter
CodecMessageFrameFilter message_filter("codec_monitor", &sender_pipe);

// Connect to pipeline
source.getOutput().cc(message_filter).cc(next_filter);

// In another thread or context, monitor for codec messages
std::string json_str;
if (receiver_pipe.recv(json_str)) {
    auto msg = json::parse(json_str);
    std::cout << "New codec: video=" << msg["has_video"]
              << ", audio=" << msg["has_audio"] << std::endl;
}

Public Functions

inline explicit CodecMessageFrameFilter(std::string name = "CodecMessageFrameFilter", Limef::comm::Pipe *pipe = nullptr)

Construct a new Codec Message Frame Filter.

Parameters:

• name – Unique name for this filter

• pipe – Pipe to send codec messages through

• next – Optional pointer to the next filter in chain

inline void setPipe(Limef::comm::Pipe *pipe)

Set the pipe to send messages through.

Parameters:

pipe – Pipe for sending messages

inline virtual void go(const Limef::frame::Frame *frame) override

Process incoming frame.

Passes all frames downstream. For CodecFrames, sends a JSON message through the pipe.

Parameters:

frame – The frame to process

class ComposeFrameFilter : public Limef::ff::FrameFilter

#include <compose.h>

Base class for composite framechain.

Allows creating complex framefilters by composing simpler ones. Subclasses should create and chain their specific filters in their constructor, for example:

class FMP4FrameFilter : public ComposeFrameFilter {
public:
    FMP4FrameFilter(const AudioParams& params) : ComposeFrameFilter("FMP4") {
        // Chain them together
        this.chain = {
             new AudioDecoder(params),
             new Muxer()
         };
    }
};

NOTE: ComposeFrameFilter will delete the framefilters you have created in-place in the vector initialization at dtor time, so no need to worry about them.

Subclassed by Limef::ff::FMP4FrameFilter

Public Functions

inline virtual void go(const Limef::frame::Frame *frame) override

Go is called by the upstream framefilter.

In this case, simply forward to first internal framefilter

inline virtual FrameFilter &cc(FrameFilter &next_) override

In this case, chain internal last filter to the next_

inline virtual void clear() override

Call clear() of the internal last framefilter

class DecodedToTensorConverter

#include <decoded_to_tensor.h>

Abstract base for pixel-format converters used by DecodedToTensorFrameFilter.

Each subclass handles one (or a small group of closely related) AVPixelFormats. Converters are owned by the filter and never exposed to users.

The convert() call must be reentrant-safe for the same object (called once per frame, always from the filter’s driving thread).

Subclassed by Limef::ff::GBRPConverter, Limef::ff::GRAY8Converter

Public Functions

virtual bool accepts(AVPixelFormat fmt) const = 0

Return true if this converter handles fmt.

virtual bool convert(const frame::DecodedFrame &src, frame::TensorFrame &dst, ChannelOrder order) = 0

Convert src into dst.

dst is the filter’s internal TensorFrame (grow-only, reused across calls). After a successful call, dst.planes[0] has shape (c, h, w), dtype UInt8.

Parameters:

• src – Incoming video DecodedFrame (isVideo() guaranteed by caller).

• dst – Output TensorFrame to fill (grow-only).

• order – Desired output channel order.

Returns:

true on success; false to signal a conversion error (frame is dropped).

virtual const char *converterName() const = 0

Short name used in log messages.

class DecodedToTensorFrameFilter : public Limef::ff::SimpleFrameFilter

#include <decoded_to_tensor.h>

FrameFilter that converts DecodedFrame → TensorFrame (c, h, w).

Usage:

DumpFrameFilter dump("dump");
DecodedToTensorFrameFilter d2t("d2t", ChannelOrder::RGB);
d2t.cc(dump);
// … drive with decoded frames …

Public Functions

explicit DecodedToTensorFrameFilter(std::string name = "DecodedToTensorFrameFilter", ChannelOrder out_order = ChannelOrder::RGB)

Parameters:

• name – Identifier used in log messages.

• out_order – Desired output channel order (default: RGB).

virtual void go(const frame::Frame *f) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class DecodingFrameFilter : public Limef::ff::SimpleFrameFilter

#include <decoding.h>

A FrameFilter that decodes frames.

• Receives Limef::frame::CodecFrames and Limef::frame::PacketFrames

• Produces Limef::frame::DecodedFrames

• Uses internally the Limef::decode::Decoder class

This framefilter uses both Limef::frame::CodecFrames and Limef::frame::PacketFrames that pass through it: the former is used to set up decoders and the latter ones are passed on to the set’ted up decoders as-is.

Output from this framefilter is a Limef::frame::DecodedFrame.

Always when this framefilter receives a new PacketFrame, it feeds it to the decoder and after that drains the decoder, i.e. calls the decoder until all decoded frames are fetched from the decoder and sent downstream.

Supports multiple decoder backends via the factory pattern:

// Default: FFmpeg software decoding
DecodingFrameFilter decode;

// Custom FFmpeg params (more threads, etc.)
FFmpegDecoderParams params;
params.thread_count = 4;
DecodingFrameFilter decode("decode", params);

// V4L2 hardware decoding (when available)
V4L2DecoderParams v4l2_params;
v4l2_params.device = "/dev/video0";
DecodingFrameFilter decode("decode", v4l2_params);

Note

For hardware frame lifetime details (when using VAAPI/CUDA decoding), see the “Hardware Frame Lifecycle” section in Limef::decode::Decoder.

Public Functions

explicit DecodingFrameFilter(std::string name = "DecodingFrameFilter", const Limef::decode::VideoDecoderParams &params = Limef::decode::FFmpegDecoderParams{})

Construct a decoding framefilter.

Parameters:

• name – Identifier for logging

• params – Decoder parameters (FFmpegDecoderParams, V4L2DecoderParams, etc.)

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class DumpFrameFilter : public Limef::ff::SimpleFrameFilter

#include <dump.h>

Dump info about the Frame

Uses Frame’s dump method to print information about the frame to stdout

Public Functions

inline virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class DumpPacketFrameFilter : public Limef::ff::SimpleFrameFilter

#include <dump_packet.h>

A FrameFilter that dumps information about PacketFrames.

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class EmptyFrameFilter : public Limef::ff::SimpleFrameFilter

#include <empty.h>

Dummy FrameFilter

A FrameFilter

that does nothing.

A use case: some object’s internal framefilter that start a framefilter chain

Public Functions

inline virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class EncodeCheckFrameFilter : public Limef::ff::SimpleFrameFilter

#include <encode_check.h>

Validates stream codec types and controls downstream frame flow.

Waits for initial CodecFrame and checks if its codecs match allowed lists. If match, passes all frames downstream. If no match, blocks further frames and sends warning InfoFrame.

Public Functions

EncodeCheckFrameFilter(std::string name = "EncodeCheckFilter", const std::vector<AVCodecID> &allowed_video_codecs = {}, const std::vector<AVCodecID> &allowed_audio_codecs = {}, std::string warning_message = "Unsupported codec detected")

Construct filter with allowed codec lists.

Parameters:

• name – Filter name

• allowed_video_codecs – List of allowed video codec IDs

• allowed_audio_codecs – List of allowed audio codec IDs

• warning_message – Message for log and InfoFrame on codec mismatch

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class EncodingFrameFilter : public Limef::ff::SimpleFrameFilter

#include <encoding.h>

A FrameFilter that encodes frames.

• Receives Limef::frame::StreamFrames and Limef::frame::DecodedFrames

• Produces Limef::frame::CodecFrame (once, after encoder init) and Limef::frame::PacketFrames

• Uses internally the Limef::encode::Encoder class

This framefilter uses both Limef::frame::StreamFrames and Limef::frame::DecodedFrames that pass through it: the former is used to configure encoder timing (fps, timebase) and the latter ones are encoded into packets.

Output sequence:

1. When encoder initializes (after first DecodedFrame with dimensions):

   • Sends CodecFrame downstream with encoder’s codec parameters

   • This allows downstream decoders to initialize

2. For each encoded packet:

   • Sends PacketFrame downstream

Has separate encoders for video and audio - routes DecodedFrames to the correct encoder based on their MediaType.

Always when this framefilter receives a new DecodedFrame, it feeds it to the appropriate encoder and after that drains the encoder, i.e. calls the encoder until all encoded packets are fetched and sent downstream.

Type checking:

• If a CodecFrame (encoded stream info) is received instead of StreamFrame, a warning is logged - this likely indicates misrouted frames.

Public Functions

explicit EncodingFrameFilter(std::string name = "EncodingFrameFilter", const Limef::encode::VideoEncoderParams &video_params = Limef::encode::FFmpegEncoderParams{}, const Limef::encode::VideoEncoderParams *audio_params = nullptr)

Construct an encoding framefilter.

Example:

// Using FFmpeg encoder (default)
FFmpegEncoderParams ffmpeg_params;
ffmpeg_params.codec_id = AV_CODEC_ID_H264;
EncodingFrameFilter enc("encoder", ffmpeg_params);

// Using V4L2 encoder (when available)
V4L2EncoderParams v4l2_params;
v4l2_params.device = "/dev/video2";
EncodingFrameFilter enc("encoder", v4l2_params);

Parameters:

• name – Filter name for logging

• video_params – Encoding parameters for video (variant of FFmpegEncoderParams, V4L2EncoderParams)

• audio_params – Encoding parameters for audio (optional, nullptr to disable)

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class FMP4FrameFilter : public Limef::ff::ComposeFrameFilter

#include <fmp4.h>

Composite filter that transcodes audio to AAC and muxes to fragmented MP4.

Creates a chain of:

1. GateFrameFilter for toggling the stream on/off

2. AudioTranscodeFrameFilter configured for AAC

3. MuxerFrameFilter configured for fragmented MP4

4. MP4PartitionFrameFilter for reconstructing the MP4 stream into concise packets

This framefilter is typically connected to MediaFileThread that has also AudioTranscodeFrameFilter and a DTS packet reordering thread.

That additional AudioTranscodeFrameFilter does no harm here as it goes into passthrough mode.

Example usage:

FMP4FrameFilter fmp4("aac-fmp4");
// Connect to upstream filter
upstream_filter.cc(fmp4);
// Connect downstream
fmp4.cc(downstream_filter);
// remember to open, so that stream can pass through
fmp4.open();

Public Functions

inline explicit FMP4FrameFilter(std::string name = "FMP4FrameFilter")

Construct the composite filter chain.

Parameters:

• name – Name for logging

• next – Optional next filter in chain

inline void open()

Call internal Limef:ff::SwithFrameFilter’s open method.

inline void close()

Call internal Limef:ff::SwithFrameFilter’s close method.

template<typename T>
class FrameFifoFrameFilter : public Limef::ff::SimpleFrameFilter

#include <framefifo_.h>

Feeding FrameFifo

A FrameFilter that feeds a FrameFifo

Param FrameFifo:

: the FrameFifo this framefilter is feeding

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class FrameFilter

#include <framefilter.h>

Subclassed by Limef::ff::ComposeFrameFilter, Limef::ff::SimpleFrameFilter, Limef::ff::SplitFrameFilter

Public Functions

virtual void clear() = 0

Removes framefilter downstream connection.

virtual FrameFilter &cc(FrameFilter &next) = 0

Register next framefilter in the chain.

This method does any necessary initialization. Framefilter is not fully functional until this method has been called. Should call init.

inline const std::string &getName() const

Initialize framefilter.

Get the name of this framefilter

virtual void go(const Limef::frame::Frame *frame) = 0

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class GateFrameFilter : public Limef::ff::SimpleFrameFilter

#include <gate.h>

Frame filter that can Gate packet stream on/off with timestamp correction.

Handles CodecFrames and PacketFrames with special logic, passes all other frame types through:

• Caches received CodecFrame(s) for later use whether the gate is open or closed (always listening for CodecFrames)

• Doesn’t pass PacketFrames by default (closed state). All other packets are passed through.

• When opened:

  • If has a cached CodecFrame: sends it downstream and starts passing PacketFrames if conditions are met (see below)

  • If no CodecFrame yet: waits for it, then sends it and starts passing packets if conditions are met (see below)

• When video stream present:

  • Waits for keyframe before passing any packets

  • After keyframe, passes all packets

• When only audio stream:

  • Passes packets immediately after opening (no keyframe concept)

• When closed:

  • Stops passing packets

  • Maintains CodecFrame cache .. keeps listening/caching CodecFrame(s) and passing all other packets than PacketFrames downstream

Thread-safe: can be opened/closed from any thread.

Public Functions

explicit GateFrameFilter(std::string name = "GateFrameFilter")

Construct a new GateFrameFilter.

Parameters:

• name – Identifier for this filter

• next – Next filter in chain

void cleanup()

Resets state.

virtual void go(const Limef::frame::Frame *frame) override

Process incoming frames.

For CodecFrame:

• Cache for later use

For PacketFrame when open:

• Adjust timestamps to start from zero

• For video, wait for keyframe before passing

• For audio, pass immediately

All other frame types pass through.

Parameters:

frame – Input frame

void open()

Open the Gate and start passing packets.

If we have CodecFrame:

• Send it downstream

• Start passing PacketFrame(s) (after keyframe for video)

If no CodecFrame:

• Wait for CodecFframe to appear before taking any other action

• Meanwhile pass downstream all other than PacketFrame(s)

void close()

Close the Gate and stop passing packets.

Resets internal state:

• Stops passing packets

• Maintains CodecFrame cache

class GBRPConverter : public Limef::ff::DecodedToTensorConverter

#include <decoded_to_tensor_cpu.h>

Converts AV_PIX_FMT_GBRP (planar GBR, 8-bit) → TensorFrame (c, h, w) UInt8.

AVFrame plane order: data[0]=G, data[1]=B, data[2]=R.

Output channel order: ChannelOrder::RGB → tensor channels [R, G, B] ChannelOrder::BGR → tensor channels [B, G, R]

Copies row-by-row to handle linesize padding in the source AVFrame.

Public Functions

inline virtual bool accepts(AVPixelFormat fmt) const override

Return true if this converter handles fmt.

virtual bool convert(const frame::DecodedFrame &src, frame::TensorFrame &dst, ChannelOrder order) override

Convert src into dst.

dst is the filter’s internal TensorFrame (grow-only, reused across calls). After a successful call, dst.planes[0] has shape (c, h, w), dtype UInt8.

Parameters:

• src – Incoming video DecodedFrame (isVideo() guaranteed by caller).

• dst – Output TensorFrame to fill (grow-only).

• order – Desired output channel order.

Returns:

true on success; false to signal a conversion error (frame is dropped).

inline virtual const char *converterName() const override

Short name used in log messages.

class GRAY8Converter : public Limef::ff::DecodedToTensorConverter

#include <decoded_to_tensor_cpu.h>

Converts AV_PIX_FMT_GRAY8 → TensorFrame (1, h, w) UInt8.

Copies row-by-row to handle linesize padding. ChannelOrder is ignored.

Public Functions

inline virtual bool accepts(AVPixelFormat fmt) const override

Return true if this converter handles fmt.

virtual bool convert(const frame::DecodedFrame &src, frame::TensorFrame &dst, ChannelOrder order) override

Convert src into dst.

dst is the filter’s internal TensorFrame (grow-only, reused across calls). After a successful call, dst.planes[0] has shape (c, h, w), dtype UInt8.

Parameters:

• src – Incoming video DecodedFrame (isVideo() guaranteed by caller).

• dst – Output TensorFrame to fill (grow-only).

• order – Desired output channel order.

Returns:

true on success; false to signal a conversion error (frame is dropped).

inline virtual const char *converterName() const override

Short name used in log messages.

class InfoFrameFilter : public Limef::ff::SimpleFrameFilter

#include <info.h>

Captures InfoFrames and stores their messages.

Passes all other frames downstream but captures InfoFrames. Messages can be retrieved later using popMessage().

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

bool hasMessage() const

Check if there are messages available.

Returns:

true if messages in queue

std::string popMessage()

Get oldest message from queue.

Returns:

Message string or empty string if none available

class MP4PartitionFrameFilter : public Limef::ff::SimpleFrameFilter

#include <mp4part.h>

Repartitions fragmented MP4 stream into proper box-aligned chunks.

Input: Raw byte blobs from FFmpeg muxer that may contain:

• Partial MP4 boxes

• Multiple MP4 boxes

• Parts of multiple MP4 boxes

Output: Clean-cut MP4 boxes as individual frames

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class MP4ProbeFrameFilter : public Limef::ff::SimpleFrameFilter

#include <mp4_probe.h>

Analyzes and logs fragmented MP4 boxes as they pass through.

Outputs box information in format: <BOX_TYPE/size=N> for incomplete boxes <BOX_TYPE/size=N/complete> for complete boxes

Public Functions

inline virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class MuxerFrameFilter : public Limef::ff::SimpleFrameFilter

#include <muxer.h>

FrameFilter that muxes elementary streams into a container format.

Takes a CodecFrame first to get stream information, then accepts elementary PacketFrames (H.264, AAC, etc.) as input and outputs RawFrames containing muxed data in specified container format.

The first output will contain format headers required for initializing downstream demuxers.

Internals:

AVFormatContext refers to AVIOContext AVIOContext refers to a custom writeCallback

walkthrough: go -> processPacket -> av_interleaved_write_frame -> returns immediately on some instances, when av_interleaved_write_frame is called, writeCallback is activated internally by ffmpeg -> pass

Public Functions

void close()

flush the muxer

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

void setOption(const char *key, const char *value)

Set muxer options using FFmpeg AVDictionary key/value pairs

class MuxingContext

#include <muxer.h>

Parameters for muxing streams into containers.

Public Functions

inline MuxingContext(const std::string &format, const std::map<std::string, std::string> &custom_options)

Create context with custom options

Public Members

const std::string format_name

FFmpeg format name (e.g. “mp4”, “flv”, etc.)

std::map<std::string, std::string> options

Custom muxer options (key-value pairs for AVDictionary)

Public Static Functions

static inline MuxingContext MKV()

Ready-made context for Matroska

static inline MuxingContext FMP4()

Ready-made context for fragmented MP4 A separate fragment for each frame: more balanced fragmentation

static inline MuxingContext FMP4_KEYFRAG()

Ready-made context for fragmented MP4 One fragment per each keyframe - all other frames are included in that same macrofragment

class RTPMuxerFrameFilter : public Limef::ff::SimpleFrameFilter

#include <rtp.h>

FrameFilter that muxes elementary streams into RTP packets.

Takes a CodecFrame first to get stream information and generate SDP, then accepts PacketFrames and outputs RTPPacketFrames.

Flow:

CodecFrame → RTPMuxerFrameFilter
                ├── SDPFrame (once, downstream)
                └── (waits for packets)

PacketFrame → RTPMuxerFrameFilter
                └── RTPPacketFrame (per RTP packet)

Usage:

RTPMuxerFrameFilter rtp_muxer("rtp");
rtp_muxer.cc(downstream);

// Send CodecFrame first - SDPFrame will be passed downstream
rtp_muxer.go(&codec_frame);

// Then send packets - RTPPacketFrames will be passed downstream
rtp_muxer.go(&packet_frame);

The RTP muxer uses FFmpeg’s RTP format. Each track gets its own AVFormatContext because RTP only supports one stream per context. SDP is generated from all tracks using av_sdp_create().

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

struct RTPTrackContext

#include <rtp.h>

Context for a single RTP track (video or audio)

Each track has its own FFmpeg muxer context because RTP format supports only one stream per context.

Public Members

AVFormatContext *fmt_ctx = {nullptr}

RTP muxer for this track.

int track_index = {0}

Track index (0=video, 1=audio, etc.)

AVRational input_time_base = {1, 1000}

Original input timebase for rescaling.

int64_t last_input_dts = {AV_NOPTS_VALUE}

Last input DTS before offset (for discontinuity detection)

int64_t max_output_dts = {AV_NOPTS_VALUE}

Maximum output DTS ever sent (for offset calculation)

int64_t dts_offset = {0}

Cumulative offset for loops.

class ShmemServerFrameFilter : public Limef::ff::SimpleFrameFilter

#include <shmemserver.h>

Server-side framefilter that serializes frames to shared memory.

Accepts DecodedFrames and RawFrames and serializes them directly to shared memory ring buffer. Other frame types are ignored.

This is a terminal framefilter: frame flow stops here, cannot be connected to downstream filters.

The shared memory ring buffer is initialized in the constructor and cleaned up in the destructor automatically.

Usage:

EventFd event_fd;
Limef::shmem::ShmemRingbufferContext ctx("decoder_ring", 5, 8*1024*1024, 100, event_fd);
ShmemServerFrameFilter server("shmem-server", ctx);

// Chain in framefilter pipeline (terminal filter)
decoder.cc(server);

Subclassed by ShmemServerFrameFilterPy

Public Functions

ShmemServerFrameFilter(const std::string &name, const Limef::shmem::ShmemRingbufferContext &ctx)

Construct server framefilter.

Automatically initializes the shared memory ring buffer.

Parameters:

• name – Name for logging and identification

• ctx – Context defining shared memory parameters

• next – Should be nullptr - this is a terminal filter

Throws:

std::runtime_error – if shared memory setup fails

~ShmemServerFrameFilter() override

Destructor - automatically cleans up shared memory.

virtual void go(const Limef::frame::Frame *frame) override

Process incoming frames.

DecodedFrames and RawFrames are serialized to shared memory. All other frame types are ignored (logged and dropped).

After each successful write the ring buffer increments the eventfd counter (via setEventFd()), waking any client blocked in select/poll.

Parameters:

frame – Incoming frame to process

inline const Limef::shmem::ShmemRingbufferContext &getContext() const

Get the context used by this server.

Returns:

const Limef::shmem::ShmemRingbufferContext& Context reference

size_t getOverflowCount() const

Get number of overflow events detected.

Overflows occur when frames are too large for the ring buffer cells or when the ring buffer is full.

Returns:

size_t Overflow count since creation or last reset

void resetOverflowCount()

Reset the overflow counter to zero.

bool isActive() const

Check if the server is currently active.

Returns:

true if ring buffer is available for writing

virtual void setLogger(std::shared_ptr<spdlog::logger> logger)

Set the logger for this framefilter and its ring buffer.

This hides the base class implementation to also set the ring buffer’s logger.

Parameters:

logger – Logger instance to use

class SimpleFrameFilter : public Limef::ff::FrameFilter

#include <simple.h>

A base class for simple one-to-one framefilters.

A FrameFilter of this class can be connected to only one framefilter in the framefilter chain, i.e.

FrameFilter ff1();
FrameFilter ff2();
FrameFilter ff3();
FrameFilter ff4();
ff1.cc(ff2).cc(ff3).cc(ff4)

For subclasses, you should only implement the virtual method Limef::ff::SimpleFrameFilter::go

Subclassed by Limef::ff::FrameFifoFrameFilter< Limef::frame::DecodedFrame >, Limef::ff::FrameFifoFrameFilter< Limef::frame::TensorFrame >, Limef::ff::AudioTranscodeFrameFilter, Limef::ff::CodecCacheFrameFilter, Limef::ff::CodecMessageFrameFilter, Limef::ff::DecodedToTensorFrameFilter, Limef::ff::DecodingFrameFilter, Limef::ff::DumpFrameFilter, Limef::ff::DumpPacketFrameFilter, Limef::ff::EmptyFrameFilter, Limef::ff::EncodeCheckFrameFilter, Limef::ff::EncodingFrameFilter, Limef::ff::FrameFifoFrameFilter< T >, Limef::ff::GateFrameFilter, Limef::ff::InfoFrameFilter, Limef::ff::MP4PartitionFrameFilter, Limef::ff::MP4ProbeFrameFilter, Limef::ff::MuxerFrameFilter, Limef::ff::RTPMuxerFrameFilter, Limef::ff::ShmemServerFrameFilter, Limef::ff::SwScaleFrameFilter, Limef::ff::TensorToDecodedFrameFilter, Limef::ff::UploadGPUFrameFilter

Public Functions

inline virtual void pass(const Limef::frame::Frame *frame) final

Define how frames are passed downstream.

Used by method Limef::ff:FrameFilter::go Calls Limef::ff:FrameFilter::go method(s) of the downstream filter(s)

inline virtual FrameFilter &cc(FrameFilter &next_) override

Register next framefilter in the chain.

This method does any necessary initialization. Framefilter is not fully functional until this method has been called. Should call init.

inline virtual void clear() override

Removes framefilter downstream connection.

class SplitFrameFilter : public Limef::ff::FrameFilter

#include <split.h>

A base class for one-to-many framefilters.

A FrameFilter of this class can be connected to several downstream framefilters:

FrameFilter ff1();
FrameFilter ff2();
SplitFrameFilter ff3();
// connect many downstream framefilters:
ff1.cc(ff2).cc(ff3).cm(ff41,ff42,ff43)
// connect many downstream framefilters one-by-one
ff3.cc(ff41)
ff3.cc(ff42)
// disconnect:
ff3.dc(ff41);
ff3.dc(ff42)

Maintains an internal registry of connected framefilters

NOTE: this framefilter is thread safe, i.e. you can use methods cc, cm and dc after threads have been started on the filterchain

Public Functions

inline virtual void pass(const Limef::frame::Frame *frame) final

Define how frames are passed downstream.

Used by method Limef::ff:FrameFilter::go Calls Limef::ff:FrameFilter::go method(s) of the downstream filter(s)

inline virtual FrameFilter &cc(FrameFilter &next) override

Register next framefilter in the chain.

This method does any necessary initialization. Framefilter is not fully functional until this method has been called. Should call init.

inline virtual void clear() override

Removes framefilter downstream connection.

class SwScaleFrameFilter : public Limef::ff::SimpleFrameFilter

#include <swscale.h>

Converts bitmap frames to specified pixel format.

For DecodedVideoFrames:

• If already in target format, passes downstream as-is

• If different format, converts using swscale

For StreamFrames: makes a deep copy, updates the video stream’s pixel format to target_format, then passes the copy downstream. This keeps downstream consumers (encoders, shmem servers) informed of the actual output format.

All other frame types pass through unchanged.

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class TensorToDecodedFrameFilter : public Limef::ff::SimpleFrameFilter

#include <tensor_to_decoded.h>

Converts TensorFrame (c, h, w) UInt8 → DecodedFrame.

CPU planes → GBRP or GRAY8 DecodedFrame (via memcpy). GPU planes → AV_PIX_FMT_CUDA / NV12 DecodedFrame (via CUDA kernel, LIMEF_CUDA only). Also emits a StreamFrame when the stream starts, dimensions change, or CPU/GPU mode changes.

Public Functions

explicit TensorToDecodedFrameFilter(std::string name = "TensorToDecodedFrameFilter", ChannelOrder in_order = ChannelOrder::RGB)

Parameters:

• name – Identifier used in log messages.

• in_order – Channel order of the input tensor (default: RGB).

virtual void go(const frame::Frame *f) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class UploadGPUFrameFilter : public Limef::ff::SimpleFrameFilter

#include <uploadgpu.h>

Uploads CPU DecodedFrames to GPU memory.

Takes DecodedFrames with pixel data in system RAM and uploads them to GPU memory, setting hw_frames_ctx on the output frame.

Non-DecodedFrame frames are passed through unchanged. DecodedFrames already on GPU (hw_frames_ctx set) are passed through.

Public Functions

explicit UploadGPUFrameFilter(std::string name, const UploadGPUParams &params = UploadGPUParams())

Construct GPU upload filter.

Parameters:

• name – Filter name for logging

• params – GPU configuration (backend, device, pool size)

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

struct UploadGPUParams

#include <uploadgpu.h>

Configuration for GPU upload.

Public Members

HWAccel hw_accel = {HWAccel::VAAPI}

GPU backend (VAAPI, CUDA)

std::string hw_device = {""}

Device path (empty = auto-detect)

int pool_size = {20}

Number of GPU surfaces in pool.