Okay, awesome to see there's so many people here, really cool that there's a big interest
in the energy topic.
My name is Frederik Stool and I'm from Alliander which is a grid operator and I'll be talking
about the open staff today.
So first of all I put here in the graph this is a load profile, so the energy load somewhere
in the grid and well you can see how it fluctuates over time, sometimes it's positive, sometimes
it's negative, this means whether there's neto production or neto consumption.
Now the question is, or you could ask, if we are at the red line right now, what will
be the load in the future?
And that's what we want to predict and if you're interested in that then you can use
open staff because open staff means short-term energy forecasting.
Okay, let's zoom out a bit first and before I go into a bit more detail about what open
staff does, first I want to talk about, give a short introduction about why this is relevant.
I don't have hours so I have to keep it short but there's a lot of to talk about here.
But I want to start out with this picture and it's actually quite cool because I think
the last presentation talked about flexible energy that consumers can use and this is
one of the many things that are changing in the energy sector.
So consumers have flexible products that also start producing, consumers also have solar
panels, your local farmer might have a wind turbine somewhere, you have big wind parks
on the sea, so there's all kinds of developments going on right now that make it harder for
grid operators to forecast what's going to happen tomorrow or even the day after tomorrow.
And I'll put this picture because all the things that I mentioned you can see right
there and probably in the future it's only going to get harder and harder.
For now I want to focus on the renewable energy part because it's also quite impactful.
As you can see on this graph here this is for the Netherlands, the percentage of renewable
energy production and you can see that in just a couple of years like five years it
has more than doubled the electricity percentage that has been produced by renewable sources
and renewable sources don't produce at a constant load of course, they change all the
time depending on weather and this means it's harder to forecast and to put that into perspective
I have another slide here and this is a typical consumption profile.
If you have your local neighborhood then this is what the energy load will often look like.
So you have the five peaks which means it's a peak for every day and in the weekend it's
a bit lower, you can see the dips, these dips in the middle of the day that's because there's
a couple of solar panels on some roofs you know, it still looks easy to predict.
If you go to other places where there's way more renewable energy you can see these energy
profiles change dramatically.
So here you can see really a profile for a big solar park and you can see these huge
negative peaks which on some days are there and some days they're not probably that's
a cloudy day so there's no or less energy generated.
And this is an energy profile for a big wind farm which you can see is well, seems hard
to predict because there's seems no real, yeah so a negative energy means that the consumer
or the customer I mean is giving back to the grid so then it's negative for us.
If it's positive it means sorry, yeah exactly, so if it means a big negative peak it means
the customer is producing a lot of energy.
So it's just a convention you could also switch the sign but you have to choose one
convention.
Yeah power, yeah.
Yeah so it's not just one side it's more like a general profile for like a substation for
the grid operator but connected to it is a lot of solar.
Yeah exactly so on all of these there's load and production but yeah I just wanted to share
this feeling.
So this can be difficult and this also leads to problems and this is two maps of the Netherlands
and the colored areas are the areas where Alliander is currently active and on the left
is energy consumption and on the right is energy production and this map shows if you're
a new customer and you want to be connected to the grid if it's red it's probably difficult
because there's no more room.
According to the Dutch law the energy grid is full over there and you can see that this
is for huge areas in the Netherlands and also large areas on the consumption side.
And of course Alliander is doing everything they can to solve this by building new cables
and new substations but this takes time, a lot of time and we don't have the time as
you can see in the graph before.
So I don't have that one but I assume it's very similar because we're not the only one
who are having these issues.
So how can we solve this?
Well one important thing is that we need grid inside and therefore this also includes forecasts.
So transmission forecasts and these are important for all three parts in the electrical grid
so all three parties.
So for customers, for DSOs such as Alliander and for TSOs such as Tenet which control the
high voltage grid.
Using these forecasts operators can try to maintain grid safety and grid balance and
can give customers as much electricity as they want and as they need because the need
is high.
With these forecasts we can also enable smart solutions and I put here two brochure pictures
of those solutions, one of them is a pilot FlexPower which was in Amsterdam which was
about charging electrical vehicles and charging them faster if it's possible and not charging
them as fast if it's not possible.
We at Alliander supplied forecasts for this project and another platform is the GOPEX
platform which is like a trading platform for electricity where customers can trade
with operators to either consume or to produce energy flexibly and this is also being used
right now at Alliander and we also provide a forecast for that.
So it's no longer working so let's use it.
So now let's talk about Opelstaff again because that's why I'm here and I'm going to give
a short introduction to Opelstaff and then I'm just going to give a short demo about
Opelstaff how you can make a forecast and also want to talk a bit about using Opelstaff
in an operational setting.
So first of all, the primary thing Opelstaff can help you with is that it's just a complete
machine learning pipeline.
So I'm just going to give a short list of what it can do.
It handles input validation such as checking whether your data is complete.
It has feature engineering so it automatically calculates for you lag features or other features
that are based on input features.
So for example, if you input it with wind speed, it can calculate wind turbine power
output for you or the same for direct normal irradiance.
Next it is some kind of intelligent train validation split of the time series.
It has support for multiple type of regressors.
So right now we have, for example, HGBoost which is at Allende the most commonly used
but we also had a collaboration with Sonyo which added ProLove to Opelstaff and we also
have support for probabilistic forecasts.
So that means not just one line but quantiles.
And unless it has integrated the model and artifact storage using MLflow.
So what does this all mean then?
Let's go to an actual demo.
So I'm going to put this up here.
That's a low resolution.
Let's zoom out.
It's a bit too much.
Okay, so I'm just going to walk you through an example or how you could make a forecast.
So first we need to make some kind of config object that's just what you have to feed Opelstaff.
Let's close this.
So let's run this line.
Next I put some example input in this project.
So we can load it and we can visualize it.
So as you can see here, well, this is upon a stator frame and here we have the load and
we have a lot of predictors.
Well, some of these, well, the names should make sense.
So for example, the amount of variation predicted by the KMI or the temperature, well, all
these predictors are already in this example data.
So if we have this, I can also plot it for you so you can see, well, this is another
power profile.
Okay, so now imagine you have this and you want to know, well, what's next?
Then first we need to train a model.
So Opelstaff has a train model pipeline which basically does all those things I just mentioned.
So we can just call the pipeline and let's hope the live demo does not fill me.
It will take about 15 seconds I think to train a model and store it.
So you can see some info about what it's doing, well, and it's stored it.
So let's have a look and we'll flow comes with an interface so we can directly see that
we train a model.
So right here, this was the run.
Now let's hope this works.
I see that my internet is no longer working so apparently this, then this figure this
will work.
All right, I'm not showing it.
So, well, this is the MLflow interface and you can see that we just train a model.
You can also click on the model or on the train run, this is just MLflow and you can
see a bit more, well, information about what happened during the training.
The next, of course, we want to make a prediction.
So again, OpenStep has a pipeline for that so we can just say, okay, I want a prediction.
So it's loading the model and using data to create a prediction and then we can visualize
that as well.
And then we have a graph right there.
So this is the forecast that it made for the next, well, this was in some example data
in 2021 but about 48 hours of forecast so that's OpenStep in practice.
Let's go back to the presentation.
Do this slide.
Yeah, so, well, this flow has been a minimal flow but of course in reality, at least for
if you're a grid operator, you want to do this in an operational setting.
So this means that you want to do daily forecasts for a lot of different locations with all
kinds of configurations.
And OpenStep also comes with a so-called reference implementation about how you could do this.
So this is a picture of what you would have to do so we have OpenStep right here which
is basically, I just showed you the training and forecasting pipeline.
Then we have another package which is called OpenStep DBC, database connector which can
connect to a database.
And we use MySQL and Influx DB to store all the data required to run it operationally.
And we also have a Gafana dashboard built upon this database stack so we can also see
what's going on.
And again, as I already have shown, you can use MLflow to keep track of all the models
and all the runs that are being done to see what's going on.
So I want to show this dashboard as well.
So this dashboard is just example data so it's not our real dashboard.
But here you can for example see some load that was there on the system and you can also
see that for example this is not just one area but it has a sum of two systems which
is quite common in an electrical grid that you have a lot of measurement points that
you have to add together with different signs.
And you can see for example here's then a live forecast of this location as well.
You can also see plots of the feature importance that obtained during training of the model.
You can see on which data the model has been trained.
Over here these plots are really small but here you can see them.
So it's a dashboard where you can see everything that Oberstaff does for every location that
you are could be interested in.
You talk about forecast, are you within the forecast also taking other forecasts like
the weather forecast into account or are you forecasting that yourself?
No so we use all kinds of data and the weather forecast is, oh the question is whether we
are using other forecasted data or whether we forecast, do those forecasts as well?
Like whether we forecast the weather ourselves and the answer is that it depends a bit.
So in general we use the weather forecasts for multiple sources and also for example
price like the head pricing.
So we use those data but sometimes we also feed the prediction itself or we feed one
prediction into another prediction.
So I mean you can play around with that but you have to feed Oberstaff with all the predictors
that you wanted to know.
Okay let's move on, I have one last slide and that's basically
key information because that was my presentation so here I put all the info you might be interested
in on this slide and if there are any questions then feel free to ask.
So the information is really useful but what is the purpose for the net grid operator,
what use does a grid operator make of this information, is it for congestion management,
is it for some kind of load shedding, what is the role of this exercise?
So I think the question is why would the grid operator be interested in forecast I guess
is what you're asking.
So there are many reasons but I think you already mentioned congestion management is
indeed an important reason but also well grid insight.
So the more congestion management is going to be used for the grid the more important
it is also to maintain grid safety and grid safety is not just one operator, we are all
connected to multiple grid operators so everyone has to communicate what they are going to
do and what they expect the energy flow to be the next day so every operator can decide
to do what's necessary to maintain grid safety.
So that's what I mentioned before the transmission forecast, every operator has to communicate
to everyone who is connected to what they expect the load to be on the next day.
I see that my time is up so I'm afraid I have to answer the questions in the chat.
Yeah or in the hallway, I think for time management we have to learn from these talks and see
if we can manage to keep a couple of more minutes for questions, sorry.
Thank you very much for listening.