The future of video

Currently, we are migrating our workstations from Windows to Linux. While the basic stuff works perfectly fine, we run into trouble when dealing with music and video.

Especially video.

Our video collection does not work under Linux.

While playing videos under Windows is not an easy feat, playing them under Linux provides insight into a lot of things. Unfortunately, videos are not among them.

Let's summarize:
Videos are stored in container files. To play them, codecs and decoders are required. Some, like MPEG2 and MPEG4 are straight forward, some, like DivX, XVid, H264 are embedded in AVI files.

Under Windows, you install codecs, the installer hooks them into the search path of any installed media player. OK, some honor these paths, some don't. Sometimes, helper applications like Explorer get confused and crash. But low and behold, it works pretty well.

Under Linux, you need graphics libraries, that have library plug-ins. If you ever tried to make ends meet with gstreamer, you know what I am talking about.

My suggestion

Here is a proposal of how video streams could be encoded in order to eliminate the codec problem and thus make video handling user friendly.

All video is encoded using an video and audio encoder. Mixtures are possible and exist.

A new file format could accommodate 2 parts: One, the decoder codec and two, the film itself. The codec would be extracted by a generic decoding engine and used to decrypt the film stream. The player would be a generic codec interpreter that allows for platform independent decoder plug-ins to be hooked in.

None of the above mentioned technologies is new. Platform independent plug-ins are reality in Mozilla driven XUL tools. Piggyback codecs can be attached to the video stream. Interpretation engines can both be Python and JavaScript.

This mechanism would even allow for commercial content, as the codecs may well be connected to payment systems.

I haven't given this a deeper thought. Maybe my tweaking with Linux will take me to deeper insights and eventually make me mad enough to write a prototype.