Hi, my name is I. Kunze. In our work we have
investigated the four spinbit cousins L, Q, R and
T
and in this talk I would like to share with you
some insights on their performance regarding
packet loss measurements.
Before I go into more detail on the different
spinbit cousins, let me first give you a quick
intro why work on them is actually meaningful.
I think it is safe to say that network
measurements have always been important to get a
better understanding of what is going on inside
of the network.
However, measurement techniques have typically
been developed independently from protocols
and thus they oftentimes depend on externally
visible protocol semantics.
A prominent example are TCP sequence numbers and
acknowledgements, which can be used to compute
the roundtrip time of a connection.
Let me quickly illustrate that with a short
example.
What we have here are two hosts interconnected by
a network probe in the middle.
If the host on the left hand side now sends a
packet with a certain sequence number,
the network probe in the middle can store that
sequence number and then basically start a timer.
The packet is then forwarded as usual and the
host on the right hand side then answers with a
corresponding acknowledgement.
As soon as the acknowledgement then arrives at
the network probe, the network probe can
basically stop the timer and compute the right
hand side half of the roundtrip time.
Unfortunately, such techniques are no longer
possible in times of encrypted transport
protocols such as QUIC,
because there the protocol semantics are no
longer visible to an observer.
To still allow for meaningful measurements, the
QUIC standard features a special purpose bit and
that is the spinbit.
It is a dedicated bit in the QUIC short header
and visible to on-path observers.
While the spinbit allows for roundtrip time
measurements, there are also other important
network properties that one might want to measure.
In this context, there is an ongoing discussion
in the IPPM working group focusing around four
different proposals that are similar to the spinbit
but enable packet loss measurements.
Let me quickly explain how they work in this
simple example.
We again have a server on the left hand side, a
client on the right hand side and now a network
probe in the middle.
But this network probe now is only able to
capture the traffic on the downstream path, so
from the server to the client.
The first spinbit cousin is the so-called LBIT
and it builds upon end-host based loss detection
as is for example incorporated in QUIC.
In other words, the end-hosts detect loss and
then the LBIT reports that loss into the network.
Due to this design, in our setting, the server
will report the overall loss on the downstream
path.
The second spinbit cousin is the so-called QBIT
or squarebit.
As the name implies, it generates a constant
square wave signal.
Or in other words, it first transmits a certain
number of packets with a set QBIT and then a
certain number of packets with an unset QBIT.
The network probe can then simply count how many
packets have arrived in which phase and can thus
derive the packet loss that has occurred here on
the downstream one link.
The third approach is then called the RBIT or
reflection squarebit and builds upon the QBIT.
In essence, the end-hosts deploy QBIT observer
logic and then report the number of observed QBITs
back into the network.
Due to this design, the RBIT in our setting
covers the overall upstream path as well as
downstream one.
Finally, we then have the so-called TBIT, where
we basically have one train of packets, which is
reflected several times between the server and
the client.
Mapped to our setting, our observer will now only
be able to compute the packet loss that has
occurred on the overall link.
So from the time that the train has left the
observer in the one direction until it has
entered the observer again from the other
direction.
As we can already see here, the approaches differ
in which parts of the network they actually cover.
And this is already the first important decision
factor when we want to decide on which of these
approaches we want to choose.
The second aspect would then be the measurement
accuracy. So how accurate can we actually
determine the packet loss rates?
This directly leads us to our contribution
because what we have done is an experimental
evaluation of the four spinbit cousins using a
mini-net based testbed.
At this point, it is very important to have a
fair comparison between the different approaches.
And this is why we induce packet loss only here
on the downstream one link, because that is the
only link that all of the four approaches can
actually cover.
We've then investigated three different scenarios.
In the first setting, we have induced random
packet loss on the downstream one link.
In the second setting, we have induced burst
packet loss. And for that, we have used the
simple Gilbert model.
And then finally, we've also considered the
impact of different flow sizes on the measurement
accuracy.
In this talk, I will mainly focus on the first
setting. So let's get directly into it.
The main goal of our random loss scenario was to
find out how well the different approaches
perform under ideal circumstances.
For this, we use symmetric traffic and disable
congestion control so that there's a constant
flow of packets.
We then transmit roughly one million packets in
each of our experiments and perform 30 iterations
for each of our settings.
We then report the cumulative loss rates that the
different approaches have derived at the end of
the experiments.
In addition to the four approaches, we also check
the ground truth. So how many packets have really
been lost?
What you can see in this plot is on the x-axis,
the configured loss rates and on the y-axis, the
measured loss rates.
If we now look at the result of all the four
approaches, we can see that they are mostly very
close to the ground truth, especially for higher
loss rates.
The only thing that stands out here is that the t-bit
is not that accurate for low loss percentages, as
evidenced by larger confidence intervals here.
The main reason is that the t-bit includes two
pause phases and thus it doesn't actually cover
the whole traffic.
Next, we wanted to find out about the timely
behavior of the different approaches and thus
looked at individual measurement runs.
What you can see on this slide are the first 10
seconds of one handpicked measurement run where
we had configured a loss rate of 1%.
Looking at the L-bit, we can see that it closely
follows the ground truth, although there's always
a slight timely delay.
This is reasoned by the fact that the L-bit
depends on the end host based loss detection,
which takes a bit of time until it actually deems
a packet to be lost.
Next, looking at the qubit, we can see that it
also follows the ground truth quite closely.
We can also identify the longer algorithmic
intervals and the qubit actually represents an
interpolation of the ground truth.
Next, looking at the R-bit, we can see that it is
significantly higher than the others here in the
beginning.
And also it starts later than the other two
approaches.
This is due to the fact that it builds upon the
qubit and first has to wait until the first
iteration of the qubit has succeeded.
However, in the long run, it then also follows
the ground truth quite closely.
Finally, looking at the T-bit, the previously
mentioned longer algorithmic intervals are now
really visible because it is the only one to only
produce three readings in the first second.
Additionally, it also has the highest
fluctuations in the measurements and takes really
long time until it gets close to the ground truth.
So summarizing, it can be said that the L-bit is
the closest representation of the ground truth,
while the Q and R-bit are not far behind.
Only the T-bit struggles a bit and takes a long
time to get close to the ground truth.
Let us finally get back to the question stated in
our title.
So which spinbit cousin is here to stay?
Well, solely based on the measurement accuracy,
the L-bit seems to be the best choice as it
closely follows the ground truth.
However, it depends on the end host loss
detection and there's always this slight temporal
delay between the actual packet loss and its
reporting.
On the other hand, we've seen that there are
longer algorithmic intervals for Q, R and T, and
these come with two disadvantages.
First, there's a decrease in accuracy when they
are subject to burst loss, as is evidenced in our
second experimental setting.
What we did is configure increasing average burst
sizes, and as our results indicate, the Q, R and
T-bits struggle if the burst sizes increase.
The second disadvantage of those longer algorithmic
intervals is that they prolong the time until the
measurement stabilizes.
These results now stem from our third setting,
where we investigated different flow lengths.
As is evidenced in the plot, the different
algorithms start their measurements at different
times.
So the L-bit starts first, then the Q-bit joins,
afterwards the T-bit joins, and finally the R-bit
joins.
This observation is especially important if we
want to observe short flows or if we want to have
fast measurements.
However, in the long run, all the measurements
start to stabilize at the same time, so at
roughly 1 to 2 megabytes.
So overall, the measurement accuracy seems to be
suitable in all of the four cases, although there
are of course differences in them.
But which of those approaches now to choose?
I think that another property of those approaches
will eventually be the deciding factor when
choosing which of them to deploy.
And that is their path segmentation properties.
These eventually decide on how closely the
network operators will be able to localize loss,
and thus it actually depends on the needs of the
operators on how fine-grained they want to localize
the loss in their networks.
Because from a measurement accuracy perspective,
all of the approaches should provide reasonable
results for that.
Thank you very much!