Making web browser play streaming video (such as MPEG-DASH, Smooth Streaming, HLS) with one player.
Modern browsers with HTML5 video finally support the common video codec H.264. Unfortunately everything is not so easy with the variable bitrate streaming formats. With MSE browsers can play such formats. At the moment there is no JS player that supports all streaming formats. I’d like to show the way how to avoid hustle with different players and create solution that will play any streaming format.
What is Adaptive Bitrate Streaming?
Adaptive Bitrate Streaming is a technology that allows video player based on the end user’s preferences, bandwidth, CPU, screen resolution etc. adjust quality of video and audio streams accordingly.
At the backend it requires the same video content to be prepared with different resolutions and bitrate quality. Based on the codec it could be single file or multiple files with video chinks. Then manifest file is generated. For most popular standards it is an XML file that includes information over streams available, streams characteristics, URL for the files. Manifest can also include DRM information and so on.
*The best OpenAPI (Seagger) app for Confluence Cloud.
Most popular adaptive bitrate streaming implementations are:
- MPEG-DASH — open standard developed by multiple companies including Google, Microsoft, Apple, Netflix and so on. The most prospective one at the moment.
- Smooth Streaming — proprietary standard developed and supported by Microsoft. As soon as Silverlight support for browsers was dropped Smooth Streaming is rarely used in the web. While it is still popular in STB and smart TVs.
- HLS — proprietary standard developed and supported by Apple. The only standard supported by iDevices: iPhone, iPad, Mac, Apple TV.
What options do we have?
Imagine in your application you want to support all 3 most used streaming formats with one player (for free obviously). Unfortunately it is not possible so far. There are several players on GitHub that can play one of the format but can’t support all of them.
All these players use MSE which is supported by latest Chrome, FF and Safari. Internet Explorer 11 and later as well as any EDGE browsers support MSE only on Windows 8.1+.
The problem is that each player has its own specific interface and typical implementation will require 3 different set of implementations. It will also cause a headache if you want to add an extra player.
How do we mix it together?
The answer is simple. Let’s use the Adapter design pattern.
So there is an adapter interface that is similar for any player and there are several individual adapters for each player that translate player’s interface into adapter interface.
Let’s play the video…
I’m going to use ECMAScript 6 (aka ECMAScript 2015, aka ES6) for better readability. In the project it will be compiled by Babel and packed by Webpack.
Below is the explanation of the adapter player idea. Project’s source code can be found on Github https://github.com/stepashka69/adapter-player . This is not a production code for now.
I start with creating an adapter for native HTML5 video element. We need a method to set up basic player properties, initialise a player. Adapter should also be able to tell if it can play the content.
Adapter for Dash.js player for MPEG-DASH playback will look something like this:
Adapter for Hasplayer.js:
Then we need something that decides which adapter is suitable for our content. Let’s create Adapter Factory. First we register all adapters available there and then make it decide which one to use.
By using simple code below we already can play .MP4, .MPD and /Manifest files.
Everything looks nice so far but there is a problem. Native HTML5 video element’s interface is not as sexy as you want it to be.
It lacks many useful features, is different in different browsers, different players can have it own unchangeable interface and so on.
Below I will show how to create control elements that can be reused independently on which player is used.
*Good Simple PlantUML app for Confluence Cloud.
Let’s listen the video…
In order to control video player we should know what happens to the video. We should be able to catch player’s events in our Adapters.
First we define list of events supported by adapter player.
Then we implement specific event names for each player’s adapter and functionality to register callback with the event in player.
Adapter player has its own list of event names which is independent from player currently used. Each adapter has list of internal player event names corresponding adapter’s event names.
You can notice that event listener interface is the same for native HTML5 video player and Hasplayer. So in the real life code it will be used to AbstractAdapter class that all adapters extend.
The simplest player control commands will be the same for all players I use. For this example:
But more advanced player API calls could be very different. They should be implemented separately in each adapter.
Finally let’s control the video from application…
Having implemented method for control and listen to the video players it is possible to create sample control elements for our abstract video player. Each control element should be able to do the following:
- know how to render itself
- know how to control player
The play button should change itself based on the player status. When player is playing it should look like ‘pause’ and when player is paused it should look like ‘play’. Obviously when clicked it should either pause or resume playback. Here is how we can achieve it for project that uses twitter bootstrap.
Same for the timer. It should listen to video player’s time update event, ask for the current time, format it and output.
After implementing control elements we can add them to our application.
So having the code above:
- you have a player playing mp4, MPEG-DASH and Smooth Streaming.
- the page would like the same for all type of content.
- it is easy to plug in new type of player into your project without any need to change the existent project code. Just implement new adapter.
I hope this example will make your life easier but…
There is still long way to go
The implementation shown above is very simple. In order to have proper implementation of the player you’d need:
- implement proper video type detection. Not all videos that can be played by native player have ‘.mp4’ extension as well as other format’s filenames can vary.
- videos have different type of video/audio codecs that can be not supported by browsers. Adding proper check and error reporting is not easy. Without that verification users could see for example video without sound or vice versa.
- browsers can not support some types of video or concrete players. For example IE on windows prior to 8.1 would not play any MPEG-DASH or SmoothStreaming.
- there are CORS issues if you want to play video from other servers.
- there are DRM protected videos that require extra set up and verifications before they can be played.
- and so on…
As soon as you read to this point I assume you have to deal with the video in the browser. So good luck to you!
PS: Project’s source code can be found on Github https://github.com/stepashka69/adapter-player .
