Hello. So today I'm really excited to talk about
this joint work with Matt, who's now at Nafeli,
and Justine Sherry, who are my advisors at
Carnegie Mellon.
And my name is Ranysha Ware. So today I'm going
to talk to you about this work on congestion
control deployment.
So let me start by motivating why we care about
this and why you should too.
So we want to answer the seemingly simple
question, if you design a new congestion control
algorithm, TACO, how do you show that TACO is
reasonable to deploy in the Internet today?
Well, typically we try to say a new algorithm TACO
is reasonable to deploy in the Internet if it
fairly shares with widely deployed legacy
algorithms PAIR.
For example, here are a bunch of graphs and
tables all trying to use fairness to make the
argument that their algorithm won't be too
aggressive towards the status quo, which today is
typically regarded to be cubic.
But here's the thing. Everyone falls short of
actually achieving fair outcomes.
But everyone still has to try to make these
arguments that their algorithm is deployable.
So let's look at some examples of what these
arguments look like.
So I don't want to dig into the graphs here. I
just want to draw your attention to the arguments.
So in the cubic paper, they show results where
cubic is unfair to Reno, but argue that this is
outside of the TCP friendly region and that this
doesn't highly impact Reno's performance,
whatever that means.
And in early presentations on BBR, Google showed
that BBR v1 was unfair to cubic and shallow
buffer networks and basically said they'll fix it
in later versions of BBR by modeling those
shallow buffer situations.
PCC Vivace also showed it could be unfair to
cubic, but argued that as the number of cubic
senders increases, it achieves the best fairness
among new generation algorithms.
And COPA makes a similar argument that it's much
more fair than BBR and PCC, and it uses bandwidth
that cubic wasn't going to use anyway.
So I highlight all of these arguments here to
drive home this point that in practice, we rarely
can or even want to achieve fair outcomes,
because it turns out fairness is either too
restrictive or impractical.
Consequently, everyone ends up just sort of
having to make these excuses about why their
algorithm is still reasonable to deploy despite
having unfair outcomes.
And it's really unclear if these excuses are okay.
So the problem is we're basically left with no
real bar or threshold for what makes a new
algorithm safe or reasonable to deploy in the
internet today.
So in this paper, in our paper, and in this talk,
we argue for the need for a practical deployment
threshold.
That is a bound on how aggressive any new
algorithm, which I'll refer to as taco throughout
this talk, can be to any legacy status quo widely
deployed algorithm, which I'll refer to as pair
in this talk.
So the outline for this talk is as follows.
First, I want to describe for you the desirable
properties for a good deployment threshold.
Then I'm going to define a new deployment
threshold based on harm, which, unlike fairness,
has all of these desirable properties.
So let's jump right in and start by discussing
what are the desirable properties for a good
deployment threshold?
Well, in our paper, we identify five such
properties.
We say a good deployment threshold to be
practical, multi metric status quo bias, demand
aware and future proof.
So those are just a bunch of words that don't
mean anything to you yet.
So let's start by defining what we mean by
practical.
So we've already seen that a threshold based on
fairness just is not practical, right, because no
one can actually achieve fair outcomes.
Therefore, a good deployment threshold needs to
be practical in that it should actually be
feasible in practice for a new congestion control
algorithm to meet the threshold.
So that's what we mean by practical.
Next, we say a deployment threshold needs to be
multi metric.
So let's illustrate what we mean by multi metric
through an example.
So assume.
So consider the scenario where, let's say,
Beyonce is at home.
She's trying to talk to her daughter, Blue Ivy,
over Skype and over her home Wi-Fi access link.
And let's say her access to the Internet is
really slow.
So her access link is a slow bottleneck link here.
And let's say it's using some new algorithm pair.
Some, sorry, let's say it's using some old legacy
congestion control algorithm pair.
Now, let's say base connection using pair is able
to achieve about five megabit per second
throughput and a good low latency.
So this is when it's not sharing the bottleneck
here.
It's just alone.
But now let's say her husband Jay-Z comes home
and he wants to download the Windows operating
system for some reason.
So this is going to be a large, long running
download.
And the server he's downloading from is using
some brand new congestion control algorithm, Taco.
Now, let's say Taco is able to use the rest of
the available bandwidth, but that this comes at a
cost.
And base latency is going to go way up.
So maybe now her video quality is terrible.
So the impact on latency matters here as to
whether or not we would say it's acceptable to
deploy Taco in an Internet with a ton of latency
sensitive applications using pair.
Therefore, it's important that a deployment
threshold be able to consider a variety of
metrics beyond just throughput.
So metrics beyond just throughput are becoming
increasingly important in the Internet today when
we have more and more applications that care
about things like latency or jitter or loss rate.
However, we can never really talk about anything
other than throughput when we're stuck talking
about fairness.
So we're used to thinking about deploying
algorithms as if the network is a pie that we're
cutting up to share.
But that doesn't make sense for performance
metrics like latency, loss or jitter.
So that's what we mean by multi metric.
Now let's walk through an example to illustrate
what we mean by status quo biased.
Now, let's say Beyonce is downloading the Linux
operating system from a server using some popular
legacy algorithm pair.
So this algorithm, let's say it works well in Wi-Fi
networks.
So her download speed is just using all of the
available bandwidth.
Jay-Z comes home, he's still downloading Windows
and he's hit the server he's using is using some
new algorithm, Taco.
Now, let's say unfortunately, this time pair is
too aggressive.
And so Taco is only able to get one megabit per
second throughput while Beyonce still gets nine.
So in this case, though, there is imbalance, I
would actually say it's perfectly acceptable to
deploy Taco alongside pair.
Right. This is because the currently deployed
legacy algorithm pair is the one being too
aggressive towards the new algorithm Taco, not
the other way around.
So when it comes to deployability, we don't
actually care if the new algorithm is the one
being hurt.
Thus, a deployment threshold needs to be status
quo biased.
It should only be based on the impact that Taco
has on pair and not vice versa.
As an example, James Fairness Index is not status
quo biased, right?
James Fairness Index would assign the same score
if the new algorithm was one being harmed as if
the old algorithm is the one being harmed.
So it can't distinguish between these two
scenarios. So it's not status quo biased.
So next we say deployment threshold should be
demand aware. Let's illustrate demand awareness
through what? An example.
So now let's say again, Beyonce is trying to
download this large file.
And the server uses some legacy algorithm pair.
But now let's say pair really sucks at fully
utilizing all of the available bandwidth in a Wi-Fi
network.
So she's only getting a download speed of three
megabits per second.
So now Jay-Z comes along trying to download that
Windows OS still using some fancy new algorithm
Taco.
Now, let's say Taco is much better at utilizing
the available bandwidth in this Wi-Fi network.
So but it's able to do that without hurting Bay's
connection.
So it's able to just use the rest of the
available bandwidth and get seven megabits per
second.
So here again, it's perfectly reasonable to
deploy Taco because it's not hurting Beyonce's
connection at all.
Although they're not equally sharing, Taco is
only using bandwidth that pair wasn't using
anyway.
So this is what so a good deployment threshold
should not penalize Taco when pair already has
inherently poor performance.
This is what we mean by demand aware. For example,
maximum fairness, which accounts for the demand
of a flow is demand aware.
But equal rate fairness, which just says each
flow should get the same rate is not.
So let's lastly, we say deployment threshold to
be future proof.
So by future proof, we just mean that a good
deployment threshold should be useful on a future
Internet where none of today's current congestion
control algorithms are deployed.
So we care about this property because many
discussions around new TCP is considered
something called TCP friendliness.
So TCP friendliness focuses on behaving just like
Reno to be fair to Reno, even though very few
centers on the Internet probably even still use
Reno anymore.
So I'm going to give you a little silly toy
example that explains why it's really silly for
us to keep buying ourselves Reno.
So consider this example where in the past Skype
and a bunch of other services use some algorithm,
Taco, Tomato.
But let's say Tomato was terribly inefficient.
So today, Skype, along with many other services,
switched to using a much better algorithm, and
they're now using pair.
Now, when Taco comes along, does Taco need to be
nice to pair and Tomato or just pair?
Well, if no one uses Tomato anymore, new
algorithms only need to be nice to pair whatever
the current status quo is.
So this is what we mean by future proof, that a
future proof threshold would only require Taco be
nice to pair whatever the current status quo is.
So I say all this to make this point that
traditional notions of TCP friendliness are just
not future proof.
In a future where no one uses Reno at all, we
should not be relying on thresholds bound to Reno
or any other particular algorithms behavior.
So these are the five properties of a good
deployment threshold.
So next, I want to define a new deployment
threshold based on something we call harm.
So let's begin by motivating this.
So when we're trying to show deployability, we
typically run experiments where we have pair,
whatever the current status quo is, versus Taco,
our new algorithm, and we want to measure
performance.
So if we care about something like throughput,
that might look like this, right?
So we have pairs performance, we have Tacos
performance, and fairness would say compare these
two bars.
So you want to say, is this outcome fair or not?
But remember, because of status quo bias, we don't
actually care what happens to Tacos performance
when the two compete.
We only care about what happens to pairs
performance.
In particular, if this was pairs performance
alone, we care about how pairs performance has
changed now that it's competing, when it's
competing with Taco.
So comparing this red and green bar.
So this leads to the proposal in our work, which
is that a deployment threshold should be based on
how much harm Taco does to pair.
So let me illustrate what we mean by harm with an
example.
So again, here, Bay's trying to do her video
conference.
And let's say that pair is able to use all of the
available bandwidth and gets low latency when
pair is alone.
But now, when Jay-Z comes along with his Taco
flow, Beyoncé's throughput goes down and our
latency goes up.
So here we would say that Jay's connection has
harmed Beyoncé.
So let's be more sort of formal about our
definition for harm.
So harm is measured between 0 and 1, and like
Jane's fairness index, sorry, it's measured
between 0 and 1 like Jane's fairness index.
Where 0 is harmless and 1 is maximally harmful.
So our harm function takes two inputs.
The first is x, where x is pair's solo
performance.
So this is pair's demand.
And it takes y, where y is pair's performance
when it's competing with Taco.
Thus, to compute harm for more is better metrics
like throughput, harm is x minus y over x.
And for less is better metrics like latency, harm
is y minus x over y.
So in this example, then, Taco causes 0.5
throughput harm to pair and Taco causes 0.95
latency harm to pair.
So you should be able to see that harm is multimetric,
right?
I can compute throughput harm, latency harm, jitter
harm, loss rate harm, whatever other kind of
metric I want.
It's also status quo biased.
So the two numbers in our harm calculation are
only based on pair's performance.
I don't care what pair's performance is here.
And it's also demand aware.
So here we define the demand as pair's solo
performance.
So harm gives us a way to talk about Taco's
impact on pair's performance.
But I haven't said how much harm is okay.
So for this, we need a harm-based deployment
threshold.
And our key proposal in our work is that a harm-based
threshold should be the following.
Taco should not harm pair much more than pair
harms itself.
So that is, any new CCA shouldn't harm the status
quo more than it harms itself.
So what do we mean by much more than?
Well, so far we've computed the harm that Taco
does to pair.
And now we want to compare the harm Taco does to
pair to the harm pair does to itself.
And say, how do we compare these two things?
So in the paper, we discussed three possible ways
to compare these.
But today, I just want to discuss one possible
threshold, which we call equivalent bounded harm.
And equivalent bounded harm, just like its name,
says these two things should be equal.
So the harm Taco does to pair should be equal to
the harm pair does to itself.
So returning to our previous example, we looked
at what happens when Taco competes with pair and
can see a certain performance.
And we compare that to pair's solo performance to
compute the harm that Taco does to pair.
And now we have the same scenario, but with pair
versus pair, and we get certain performance
outcomes.
And we want to say, what's the harm that pair
does to pair?
So let's go back to our harm calculations to
compute this.
So before, this is what the harm that Taco did to
pair.
And now we want to add in what's the harm that
pair does to pair.
So here, you should see that the throughput harm
is the same, because I get the same throughput
for pair when pair competes with Taco, when pair
competes with itself.
So under equivalent bounded harm, Taco would be
okay.
But if we look at latency harm, it's clear that
Taco is much more harmful to pair than pair is to
itself.
So under equivalent bounded harm, we would not
consider Taco deployable.
So a harm-based threshold is practical.
We can already see that pair can achieve certain
performance when it's competing in practice with
pair.
So it should also be possible for Taco in
practice to get similar performance outcomes.
Certainly, if pair can do it, then so can Taco.
And it's future-proof, right?
There's no baggage here.
There's no tomato here that I'm trying to behave
like.
The only thing that matters is what is the status
quo and how does it compete with itself.
So if tomorrow the status quo is banana, then
that's what we would compare against.
So is equivalent bounded harm like the perfect
deployment threshold?
Well, it certainly meets all of our criteria, as
we previously discussed, while alternative
thresholds based on fairness or TUTP friendliness
certainly do not.
So it's better than what we had before.
But we do have some concerns about equivalent
bounded harm.
So let me show you a little example.
So here I'm illustrating an issue with equivalent
bounded harm where, let's say, when pair competes
with pair for two long-running downloads, one
flow gets 7 Mbps and the other only gets 3.
So there's significant imbalance here when pair
competes with itself.
And under equivalent bounded harm, any new Taco
algorithm couldn't improve this imbalance, which
seems problematic.
So this is a hotness paper, and it's an open
question what's exactly the right harm-based
threshold.
And in the paper, we define two other thresholds
that allow Taco to take a little bit more
bandwidth.
But we're not sure what exactly is the right
threshold.
And there are other open questions in the paper.
For example, when you run performance experiments,
there are, of course, going to be a distribution
of these results.
So should we care about the average worst case
results or something else?
Also, we can't possibly measure the performance
of pair versus Taco in every possible scenario.
So we ask, what are the right workloads and
networks we want to test for deployability?
And even if we have a threshold, can we really
even enforce it?
So while we haven't exactly settled on the
perfect threshold, here is what we do believe.
Fairness is wrong.
It's not working as a practical threshold.
And thus, we need to stop making excuses for why
our new algorithms are not reading an unrealistic
goal like fairness.
And lastly, reasoning about harm is the right way
forward.
And it's going to give algorithm designers a much
more realistic goal.
So that concludes this talk.
Please look at the description box for a link to
the paper.
And if you have any questions, please shoot me an
email or ask me a question on Twitter.
And thanks for listening.