All right, everyone. I hope the mic is working. It's great to be here. This is my first fast
time, by the way. And I'm very happy to talk to you all about current measurement and energy
attribution for processes and hardware devices in Linux. My name's Aditya, but you can call
me Adi. That's the first three letters. I'm a grad student. And yeah, that's my contact.
I'm always very happy to talk to people during, before, and after my talk. Please reach out.
Please. I would love to hear from you. So a bit of background. I'm a graduate student
at ETHEHA Zirik in Switzerland. And I do research at the intersection of computer architecture
and operating systems. I love this stuff very much. Great. What do we want to talk about?
Let's get a bit of a brief background to bring everyone on the same page. Now, when we talk
about energy sources and computing systems, you can have a bunch of options. You can have
direct inputs from DC, from USB. You can have battery power systems. And if you're really
exotic, you can even have energy harvesting devices. Okay? Now, we want to use the maximum.
No, I'm sorry. We want to use the minimum amount of energy to perform our task. Why do
we want to use the minimum amount of energy? Because energy consumption correlates with
battery capacity. And battery capacity is a major, a significant design constraint for
your consumers. Okay? All of us have cell phones. We have the recent buzz around Apple
Vision Pro, AR devices. These devices are significantly restricted by the amount of
battery capacity. So we want to minimize the energy that we use to get the job done.
Okay? Now, what is the problem here? What do we want to solve? Let's flesh it out. Energy
consumption is defined as power times latency. Power is determined by your hardware. Okay?
Latency is determined by your software. Okay? Now, how do we measure this? How do we get
this data? Programmers often measure latency using well-established tools. I'm guessing
many of you would be familiar with Linux Perf, or you would have timed your own software
using wall clock time using CPU clock cycles, right? Now, these are well-established metrics
and well-established tools to quantify your latency. What if I ask you, do you know of
any tools to calculate your application's energy? What comes to your mind when I pose
this question to you? How would you calculate your application's energy consumption? You
would say, okay, Ali, I know. This is very simple, right? Energy is power times latency,
right? We just talked about this. I'll get the power from the CPU. My CPU has this magical
interface called Rappel, which stands for running average power limit. I'm going to
get the value and they, oh, voila, my CPU says 15 watts right now. Great. Then I'll time
my application and my application turns out to be, let's say, five milliseconds. Okay?
And we put these values into this formula and great, we have 75 mJ of energy consumption.
Stop done. Let's go home. Unfortunately, this is too simplistic. Let's try to dive into
what we missed here. This does not reflect the ground reality, okay? And now I'm going
to deconstruct what happened here and what we missed. The first step, we saw the power
was 15 watts. And unfortunately, this model assumes a linear power draw over time. That
is not the case. If you actually look at the system, this is what it looks like. You
have these values and you have these peaks. And if you measure your power at the wrong
time, you will end up with a significantly different number than what you should have.
On the x-axis, you have time value. On the y-axis, you have the power for CPU. And power
consumption is not linear. So this assumption that we have a linear power draw is incorrect.
Second, we got the power value from Rappel. Remember running average power limit? It turns
out that Rappel is only available on Intel or sometimes on AMD. ARM, for example, has
a very, very different interface to report power. So I would love to share a story. I
was doing energy profiling on a server-class system back in university and I said, oh,
I've built this great infrastructure on my Intel platform, right? Let me just use it
to run on ARM and see what happens. And the moment I ran it on ARM, Linux Perf said, I'm
sorry, I don't recognize the CPU. I can't give you any numbers. And it just crashed,
okay? So all of these interfaces are really different and you need a significant amount
of engineering to make sense of it across different platforms. Second limitation is that
we do not have uniform interfaces or the formats to measure power reliably.
All right. Let's try to go deeper. Let's try to get more into the closer to the ground
truth. Our model got the power value from the CPU. What about the other devices? I'm right
now broadcasting from this device and oh my God. I'm sorry for this.
I hope not. Give me a sec.
Beautiful. Beautiful. Okay, okay. So back to the presentation. We were talking about
the impact of devices like the screen, the memory, the network cards, right? We don't
know how to quantify them. So we did a lot of experiments and it turns out that these
devices very often dominate your power consumption and our findings are also correlated from
some similar observations at Google. So what Google did was they were trying to optimize
their data centers and did a huge amount of profiling on their server class CPUs. Server
class CPUs are the heaviest CPUs that you can get in the market. And it turns out that
they observed that DRAM is dominating their power because DRAM is burning power all the
time. CPU turns on and off, but the DRAM you cannot turn off. Remember, it is volatile.
So you need to break out of this mindset that CPU is the end all be all. Okay. So let me
try to summarize everything. We are inaccurately calculating only a fraction of the system's
actual energy consumption. Okay. And I would love to put this in a take of a code for you.
This is not from me, but I like this very much. We cannot improve what we cannot measure.
So we first need to understand how to measure energy correctly. And that's what my project
is all about. That's what I love to do. What is the goal of what I'm trying to do here?
My goal is to develop a framework to accurately and reliably measure the energy consumption
of processes in the kernel. All right. All of us can get this data. What is the use for
this? Because data without use is it does not get used. Okay. Once we have this data,
we want to report it to the end users in an easy to understand format. Right. End users
should be able to make sense of the number. Right. What does this number mean for me?
We wanted to report it to the programmers which improve their action ability, which enable
them to move their code up and down to change their code to move the numbers. Right. And
we want to report it to system designers to enable them to iterate much faster over low
energy designs over low carbon designs. Okay. So let's try to dive deeper. What do we mean
by a framework? What are we trying to do? Let's flesh it out. A framework comprises models
and tools. Let's break down these two words. A power model is how we think about a device.
When I say that I want to measure power, a power model is the mental model that I have
that I will use to get the value. Okay. And it turns out that these power models are often
very poorly understood for a number of devices. For example, DRAM power models are often not
available to the public. They're not available to academia. They're, let's say, a proprietary
trade secret. Don't quote me on that. And okay, once we have these power models, we can build
tools which accurately calculate power based on these models. A tool that I would like
to mention would be the NVIDIA SMI utility. It allows you to calculate the power of a
GPU using this tool. It's a good tool. And okay, so let's pull it all in. What I would
like you to take away is that we need accurate models, first and foremost accurate models,
and second, reliable tools to calculate the energy consumption correctly. So we defined
our problem and we defined our goal post where we want to go. And now let's see how are we
going to get from point A to point B. Great. So before I dive into what is the mechanism,
I would like to bring to knowledge what has been done before. All of us have been here
for the entire day, right? We love energy and we love efficiency. If this is such an
important problem, why didn't people solve it before? People did. People did try to solve
it before and I'm going to describe to you right now what they did before and why that
is insufficient, why we need to do better. Okay, on the screen you can see a screenshot
from a tool from Intel that is known as PowerTop. And you can see the first column here which
reports power estimate. And on the right side you have the description of the particular
device, interrupt, process for which this power estimate is calculated. Now, what are
the challenges? Well, first of all, I believe in energy. It turns out that power is a discrete
time event. What do we mean by discrete time event? Let's try to break this down. If you
have a graph, a power is a single point on that graph. Energy is the area under that
graph, okay? We want to calculate energy because energy is what correlates to your battery
drain. Your battery supplies you energy. Power is just one particular instance in that time.
Second, PowerTop has a vendor-specific implementation. I hope that is clear. Third, what is the actionability?
So I just showed you this data. I just showed you the screenshot. It says, oh, my display
backlight is taking 350 milliwatts. Great. This particular process is consuming 292 milliwatts.
Okay, fine. The question that comes to mind is what is the use for me? What is the actionability
for the programmer for this data? How does the programmer change the code to move this
number? And I don't know. How do I fix that? How do I fix something that I don't know how
to fix? And that is a gap that I would like to bridge, right? So let me dive into the
guts of the system. This is a system design. On the screen you can see an elementary flow
chart which summarizes the system at a very high level. And this is a regression-based
system. A regression-based system has two inputs. You have the parameters and you have
the inputs to the parameters. First, we calculate the parameters and then we calculate the inputs
to the parameters. Great, we have time. I will go into details now. Please bear with
me. Okay, let's first look into the parameters. How do we determine the regression model's
parameters? There's an algorithm for this. First of all, we turn off everything. We turn
off everything that we can turn off in the system. We measure the baseline draw. This
is what we refer to as the minimizing the system load. Then we pick each device one
by one. We isolate the impact of the device on the baseline load. And we measure the
drain over multiple times. So we turn on a single time. Let's say that I turn off everything
and then I turn on just the screen. Okay? And I measure the difference between these
two values. The difference is the impact of the screen on my baseline. And then I also
do one thing. I sweep the screen. So I change the brightness of the screen from minimum
to the maximum because obviously the minimum brightness is going to have different power
than the maximum brightness, right? I hope this makes sense to me. Are you guys still
with me? Okay, so this was just an example. But what we're trying to do is we're trying
to quantify the impact of each device on the baseline. Now, I would love to give a metaphor
to help explain this better. Imagine that you have a water tank. And in this water tank
there's one single input and there are 10,000 tiny outputs. And the problem that you're
trying to solve is what is the rate for each of the output pipes? You cannot measure it
directly. So you have these 10,000 outputs which go on their own anytime. They can go
off and you don't have levers to control them. What you're trying to figure out is what is
the drain rate for each of the output pipes? That is essentially the problem that you're
trying to solve. So what you do is you turn off all the outputs, okay? You turn off all
the outputs and you turn on one single output and then you see the difference in the tank
level before and after turning it off, okay? And that is essentially what we call as an
isolation or well in academic terms it's also sometimes known as an ablation study. But
we try to isolate the device and measure the impact. Next, we repeat this process for all
the pipes in the system and we try to get a reasonable estimate of what is the impact
of each pipe. Great. So that was the first step, the device-specific measurements. The
second step would be the kernel process accounting step. This would be the inputs to the regression
model. So we have the parameters that we got from this step and now we need the inputs.
Now how do we determine these inputs? Sorry, did I hear a question? Okay, great. Right,
how do we determine the inputs? What we do is we isolate the impact of each process.
So we identify how much time the process used for the CPU. We identify how much was the
network activity, the screen wakeups, file handles, memory usage and we put all of these
numbers together into the model. And this is what gives us a predicted energy consumption
value for that process. Okay, so what are the challenges? This seems very simple. This
seems, okay, you've done this work but what did you not tell us? Here comes the part that
I did not tell you. First part, estimated value. This is not the reality. It is really
hard to find out the reality. And there's a very famous line in machine learning community.
It's known as all models are wrong but some are useful. So my goal here is to build a
useful model that I hope is less wrong. I would love to make it perfect but unfortunately
we cannot make it perfect. But yes, I would love to make a useful model first. Second,
there's a bit of a cash 22 situation here if you observe that. What is the cash 22?
I am running a measurement process. There's a process that is doing measurement on my
system. Okay, that is also going to create a load. So there's going to be a skew in the
values that I get because of my measurement. Okay, and the more accurate I want it to be,
the more skew it is going to create. So we want to understand what is the right amount
of accuracy that we can use to also be useful while also minimizing the bias. So this is
very challenging, right? Because this is different for every system. And that's a problem that
I'm almost struggling to solve. I would love to get your inputs if you have. Great, next
challenge. There are millions of devices out there and these millions of devices have billions
of ICs inside them. Very often we don't even have the data sheets for these ICs to correlate
the values that we see. The estimates that we get can range across two to three orders
of magnitude. One device can say, oh, I use one microjoule and the second one can say,
oh, I use 10 milliwatts and those numbers don't make sense. Those numbers really blow
you away. So how do we maintain our sanity in the face of the variance that we see here?
And one more challenge would be that, assume that you can say, oh, let the users supply
this data, let me get the data and then build a centralized farm of this data and then try
to make sense of it. Should the users share this data? Would the users share their device
users' data to you and allow you to put it on a centralized server? Who will own that
data? Because there's enormous value in it. So this is, I would love to get your inputs
on. One more challenge here would be the validation. So we got a value that we estimated. How do
we make sure this value is as close as possible to the ground truth? In an ideal world, I
would have infinite money and I would go to every computer in this world and take a probe
and put it next to their CPU and say, oh, this says 17.5 watts and my tool says 17.5
watts. Great job. Let's go. I cannot do that because I don't have that much time. Okay,
so we want to minimize the difference from the ground truth and what we see in the tool.
There's a significant challenge in making sure that what we see is what is the reality.
Right? Remember, there's accuracy and there's precision and there's correctness. And all
of these trifecta come together and make this a very difficult tool to get right. But still,
I believe it's going to be great. I'm very happy to work on it. Great. So once we have
the energy consumption, how do we link it to the carbon emissions? We just saw that
we can calculate energy consumption using power time flatancy. The carbon footprint
can be calculated by multiplying this number by the composition of the energy. Where did
the energy come from that you used to power the device that you were running? And this
composition depends on multiple factors. It can include the geography. It can include
the time of availability. It can include the cost of generation of that energy. Right?
So fortunately, there are good tools and libraries out there which can simplify this problem for
you. So energy composition is, let's say, something that I believe people will solve
faster than I can solve this one. That is why I would love to focus on this one. Great.
All done. Let's get back to the good stuff. How is this going to look like? How is this
going to make your life better? If you're an end user, I would love to ship to you an
application like this, an application which tells you how much energy your inkscape usage
consumed, how much energy your screen was dissipating. So as an end user, you can remember to turn
off inkscape when you're not using it. Or you can figure out, oh, I need to deliver a
presentation to so many people in five minutes. I'd better save my battery. Otherwise, I'll
be in deep trouble. So it's for those use cases when you want to maximize your battery
life as an end user. As a programmer, if you want to expose an API that enables programmers
to take action, if you want to indicate the devices and the code regions which consume
the maximum amount of power in the code and enable the programmers to change it, to modify
it, to fix it. So actionability is the primary concern for programmers. In an ideal world,
I would love to have direct suggestions in the IDE that tell the programmer, oh, this
code is not, this code is going to burn this much carbon. You'd better change it. And for
the system designers, what we want to do is we want to enable them to iterate our designs
faster. We want them to enable this, we want them, we want to enable system designers to
discover designs which are really low on energy, which are really high on performance, which
are really high on carbon efficiency. So there's typically a design space that designers explore.
And we want to enable them to explore the design space faster. That would be the end
goal from this tool. Great. So what is the takeaway from this talk? If there's two things
that I would love for you to take away, that if you forget everything else, okay, just
remember these two things and I'd be very happy. First, we cannot improve what we cannot
measure. We must measure correctly, okay, to improve things. And second, we need to break
out of the CPU mindset, okay, non-CPU system components can dominate your power. Please
remember that. Please remember these two things. And the next time I see you, please come say
hi and I'll, I'll buy you lunch. Okay. Great. Thank you very much for listening to me. It's
great to be here. It's great to talk to you. Please be in touch. Please reach out and oh,
boy, we're out of time, but I'm very, very happy to get your questions. Come talk to
me.
There's still like two minutes for questions. So if there are any questions, please.
Go for it.
There's one in the back. Yes.
So.
So hello and thank you for this presentation. I hope you're not going to hate me for this
question because I'm a primary infrastructure guy. And one thing I was always concerned
about is redundancy, like a scale twice. So if one dies, is this part of your thinking
and scope?
Or does the question make sense?
I'm really sorry. I don't fully understand what you mean by redundancy.
I mean, sure, I understand, but redundancy is trying to solve the problem of fault tolerance.
Okay. It's not trying to solve the problem of efficiency. I'm trying to solve the problem
of efficiency. So redundancy is an orthogonal concern to mine. Does that, does that mean
it makes sense? Yeah. Thank you. But thank you for the question. I really appreciate
questions. Yes.
Did you try to monitor the hover head of monitoring the energy consumption? Yes, that's a great
question. No, we did not. On one side, I'm afraid it's going to be huge. On one side,
I don't know. It's like an infinite recursion, you know, like how can I measure the impact
of my tool itself? Like the tool is what measures the impact. But how do I measure the impact
of the tool? I don't know. I hope, I hope that, I would love to believe that. That's
what I, yes, that's what I want to believe. Yes, please. Thank you very much. It was great
to be here.
Thank you.