Cloud Streaming

Here we describe how to stream live, low latency video from your IP cameras to cloud and how to visualize the live video in a web browser.

A good video container format for this is “fragmented MP4” (frag-MP4 aka FMP4). This is basically the same format you have in those .mp4 files of yours, but it is fragmented into smaller chunks (aka “boxes”), so that it can be sent, chunk-by-chunk, for low-latency live video streaming over your LAN or WAN.

LibValkka is able produce the fragmented MP4 “boxes” for you, while you can read them one-by-one, in your python code. How to do this, please refer to the tutorial.

After obtaining the MP4 boxes, just use your imagination. You can send them over the internet using websockets, gRPC, or any protocol of your choice. You can also dump them into an .mp4 file, and that file is understood by all media clients (just remember to cache and write the ftyp and moov packets in the beginning of the file). For creating a pipelines like that, please take a look here.

To play live video in your browser, use Media Source Extensions (MSEs). Receive the MP4 boxes through a websocket and push them into the MSE API to achieve low-latency live video.

This is a true cross-platform solution, that works in Linux, Windows and desktop Mac iOS.

As of September 2020, iPhone iOS is still lacking the MSEs, so that is the only device where this approach doesn’t work. In that case, you should use dynamically generated HLS playlists, while in this approach it is again convenient to use frag-MP4.

For more information about the frag-MP4 structure, see this stack overflow post.

For a full-blown streaming example with libValkka, see valkka-streamer.

MP4 is described extensively in this document.

Remember that not all codec + container format combinations are supported by the major browser. Most typical combination for video is H264 + MP4. For a list of supported codec + container formats, please see this link. Note that H265 support is still lacking behind.

For some more references on the subject, see in here and here.