Hi everyone, my name is Ali. I am a computer
science professor and I have been working
on streaming media since my PhD years. The topic
for today is the very recent developments
in the streaming space, in particular the common
media client and media server data
specs coming out of the Consumer Technology
Association. These are the two specs that
might have a fundamental impact on media
distribution. I will give a brief overview of
where we are
today, how we got here, and where we go next.
Without a doubt, streaming media is a big part of
our lives. With more efficient media
codecs, faster internet access, streaming any
content we want is quite easy today. HTTP
played an important role in this too. Back in the
day, it all started with progressive
download which allowed us to play the media
before having to download the entire media
file. With sufficient bandwidth, this was a much
better experience over the download
and play approach. Following that, with some
indexing tricks and the use of byte range
requests, seeking an immediate timeline was made
possible, and servers also got a bit
smarter. Yet the most important leap came out
about 15 years ago with chunked streaming,
which enabled live streaming over HTTP and splicing
arbitrary content for things like
an insertion. The natural extension to chunked
streaming was to generate multiple versions
of the content and offer rate switching
capability, in other words, adaptation capability
to streaming
clients. Today, all major streaming platforms use
a flavor of this approach.
In a nutshell, we have a source, which is live or
on demand, that provides the mezzanine
content, which is supposedly the best and most
crisp quality that we can offer. The
mezzanine stream is fed to an encoder or a transcoder
that generates a number of representations
according to a preselected bitrate ladder, which
is a combination of resolution and bitrate
pairs. These representations are then packaged
into media segments and optionally encrypted,
which are then published to the origin server.
Through the help of the CDN caches, media
segments can be efficiently accessed by the
countless streaming clients.
Why do we use HTTP? Well, it has all these nice
features, works on almost any device,
and goes through all the nasty middle boxes
without a problem. The CDNs have a huge
replication
capacity and offer cheap and low latency access
to everybody. HTTP is an object request response
protocol, but with small media pieces, it can
mimic streaming very nicely. And the ultimate
goal is to make the viewers happy, at which HTTP
adaptive streaming does a pretty good
job most of the time. But adaptive streaming is
not perfect either, so bad things do happen
from time to time.
Here's an example showing streaming client
behavior. I have been showing this plot for
more than 10 years now, so you might have seen
this before. There are 10 clients streaming
from the same server over a 10 Mbps link. Ideally,
all the clients should consistently
stream from the same representation. In this case,
it is the 866 Kbps representation. This
way the network capacity will also be used as
much as possible. However, this is not
the case as you can see from the frequent upshifts
and downshifts for each client. In
a scenario like this, which is very common, we
get either unfairness among the clients,
instability for each client, or underutilization
in the network. HTTP streaming traffic, which
is in and off traffic, and uses a set of discrete
bit rates, cannot converge to the ideal point.
If you are interested in knowing more about this,
we studied this problem in detail in
our Nostra 2012 paper, which you can refer to for
more details.
Since these problems have been identified about
10 years ago, we and many other groups
worked on finding a solution. Several solutions
with different pros and cons have floated
around. Now, when it came to the solution types,
there were primarily three camps. The
first camp was arguing that the servers must
always stay in control, and the network must
support appropriate quality of service, otherwise
streaming video could never be on par with
the traditional cable or broadcast TV. So, their
solution was to make everything more
controlled with smarter and more expensive
infrastructure. However, it soon became quite
clear that the market has already moved on from
more control on everything.
The second camp was on the other extreme, argued
not doing anything that will put more
control on the infrastructure, but simply
spending the money to improve the access capacity.
In other words, the money should be spent to put
more fiber under the ground and improve
cellular coverage. As nice as this sounds, one
thing we know for sure is that the demand
for bandwidth is always more than the supply. The
last camp was more pragmatic. How could
we best use whatever the bandwidth or server
capacity we have to meet the demand? So, the
last camp brought up the idea of some sort of a
control plane that might enable information
sharing between the servers and clients. This was
certainly an interesting idea, but bringing
together the developers of all the client
implementations with the content or network
providers wouldn't be as easy.
But we were determined. In 2013, we stirred up
enough interest to have a joint IETF MPEG
workshop on this subject. Different proposals
were brought to the table, and the discussions
led to the development of the Server-End Network
Assisted DASH standard in MPEG. This
standard actually covered a lot of ground and
established the control plane framework
we always wanted. Despite all the academic
research behind it, due to various reasons,
this wasn't really picked up by the industry
until a number of companies wanted to do
something
about it in the CTA a few years later. That CTA
effort first focused on the metadata information
sharing from the clients toward the servers, and
the spec was published last year as CTA
5004, and that's called Common Media Client Data.
With this completed, the CTA started
working on the sister spec a few months ago,
which is for information sharing toward the
clients, and it's called Common Media Server Data,
and it's expected to be published later
this year or early next year.
So what has really changed the mind of the CDN
providers? They didn't get on the bandwagon
a few years ago, but now they want to have
something similar? Here's the deal. Whose
fault is it when a user's video keeps freezing or
is low quality? One can blame the device
or the app or the internet connection speed or
the CDN or the content provider. It's a
vicious cycle that mostly ends up without a
satisfactory answer, which makes us, the
users, quite upset. Fault isolation is very
important for keeping the paying customers,
and it requires analytics data from various
points along the delivery pipeline. There
are many proprietary end-to-end analytics
solutions today, but surprisingly, the CDNs
are quite blind to all of this. CDNs see the
requests and send the responses, but they
don't have the glue to connect all the pieces
together to get the full picture and identify
how well things are working or not. The solution
is quite straightforward though. If with each
request the client sends some identifiers and
status information, the CDNs suddenly
gain visibility into the media analytics. See
this new paper from friends of mine to
explore how powerful CMCD reporting could be.
Here's a list of things a client can send
information about. Even with a unique session
ID sent with each request, the CDNs can now group
the requests per playback session, which
is already a big improvement over today's CDNs'
capabilities. Sending a deadline might
allow for getting the response with a higher
priority, reducing the risk for video stalling.
For example, see our non-stop paper this year on
the subject. Another use case is hints
about the next request. The CDN may reduce the
fetch time for the clients by using that
information. On the other hand, the CMSD work is
only a few months old, and there are already
a number of use cases described by the working
group. As you can see from the list, there
are many ways the CMSD can help with, or many
problems the CMSD can avoid, which would
otherwise
be very difficult to resolve. The work is still
ongoing. If you're interested, please
consider participating in the effort.
Developing or writing a spec is usually the
easiest part. We hit the SEND standard, but
it was not widely adopted. To get better adoption
with the CMCD and CMSD, all of us working
in this industry need to work together. We all
agree that information exchange is useful
when the information is relevant and actionable.
The information also needs to be fresh. There
are already a number of CDNs and clients
supporting CMSD, so for example, grab the latest
HGS
code, start experimenting yourself, and come up
with how CMCD or CMSD can help in your
environment. Both have extension mechanisms, so
extending the functionality is pretty
straightforward.
And most importantly, join the effort and
contribute. Review the use cases, propose
new ones, and so on.
With this, I provide a slide here with some
useful links for your reference.
And thanks for listening, and I hope those of you
who are interested in streaming stuff
and other multimedia systems topics can attend ACM
Multimedia Systems Conference in person
or virtually at the end of September. Thank you.