Hi everyone. Thank you for having some patience with me. Computers are not my strong suit,
although I am in IT. So my name is Sunita Rijder. I am the Community Manager of Open
Staff. And I work at Alieander. So let's get into a little bit of background. So Alieander
is a distributed grid operator. So we are responsible for the distribution of energy
in both electricity and gas in about a third of the Netherlands. So I think we all know
these kinds of gas. So this is energy consumption on some place in the Netherlands. However,
we have no idea what's going to happen in the future. Well, this is where Open Staff
comes in. Open Staff stands for Open Short-term Energy Forecast. So instead of our question
mark, we actually know what's going to happen. So after this very short introduction, let's
tell, let's talk about what I'm going to talk about today. So first of all, I'll start with
the challenges on the grid and why we actually need Open Staff. Then I'll talk about Open
Staff, of course. And finally, I really want to discuss our recent developments and collaborations.
So the challenges on the grid. So when everything was still good and easy on the electricity
grid, it looks like this. So on the left, you see one big producer, just one direction
energy flow, and then we have our consumers. Fairly easy. However, due to the energy transition,
I think you're all aware, it looks like this. So very chaotic. So on the production side,
we have a distributed production due to our solar and wind, both on the mid and low voltage,
but also at our consumers. And on the consumption side, we have the issue that our consumption
has exponentially increased. We had a lot about EV charging over here. Well, those electrical
vehicles need electricity through the grid. And this is where our capacity issues start.
So this is a map of the Netherlands. And I think you can all guess that red is bad. So
on the red parts, we actually have no capacity available. So let's say that you want to start
a company of one of these areas, we cannot connect you. So you get no power from us because
we just simply have none to give. But of course, we're all very smart and over the people.
So we have some solutions. So one of these solutions is actually to shave the peak. If
we expect grid limitations to be surpassed. So on this left image, you see a forecast
on the load on, for example, a transformer. We see a very clear peak. And this is where
our grid limitations are surpassed. So our solution is just to say shave the peak. So
for example, if this is production, we just ask one of our solar farms to just shut off
for a little while. Of course, they can money for this, but that's something else. And then
this is the result. So our grid limitations are not surpassed and nothing breaks. So great.
But then able to do this, we do need to know just left image. So we actually need accurate forecasts.
And this is where we have open step. So again, open step stands for open short term energy
forecasting. And let me a little bit explain a little bit more about it. So first of all,
what the hell is it? Well, it's a complete software stack to forecast the load on the
electricity grid. But it's energy forecast. So it could also do it for heat.
And it's automated machine learning pipelines. So it's a step by step process, which is automated
to make a forecast. So in these dark blue boxes is all everything that open stuff can do. And
I'll talk a little bit more about it. So what does the software look like? So first of all,
you need a database. This is one that you have to make yourself, of course. But we do have
open step DBC, open step database connector. And this is able to get all of your data from your
database. And then we get into open step already talked about pipelines. So of course, these are
in the software overview. And these are part of the tasks orchestration. Then we have data preprocessing,
which includes data validation. So for example, if you see a little flat line, as we're able to
cancel these out of your input data, and there was something very interesting, feature engineering.
So in this feature engineering, we're, for example, able to calculate the wind speed at the height of
a windmill from the wind speed on the ground. And we're also able to calculate the lag load
for one time stamp. And then of course, we're machine learning pipelines. So we have some
machine learning in there. So we're using open source models such as XG boost to make our machine
learning models. So we're able to train, optimize hyper parameters, of course, make a forecast.
And we're also able to make a split forecast to our Dazzles model. And finally, we are able to
evaluate our forecasts, store our model, and do some post processing. So let's look into the
methodology on a really high level. So on the left, we have our target load. This is where we
actually want to forecast. Then we have some external predictors. So we have our weather forecast,
market prices, and typical profiles of companies and households. From these external predictors,
we can actually calculate our derived features. So this is the feature engineering I just talked
about. So we're able to calculate lag loads for each time stamp, but also to have some derived
weather features, such as, for example, the wind speed at the height of a windmill. And for the
more calendar info, it really matters if you're are forecasting on a Sunday or Christmas compared
to a Monday. And then we can train a single model for all our lead times. So here you can see what
the data, for example, looks like. So if a daytime with increments of 15 minutes, our targets,
and external predictors, you can also see here that we have the Dutch energy prices in there.
So if you have multiple training horizons, we just simply do pick late our data and use this
for our training horizons. And then if there are questions about it, please ask me in the break,
but I don't have time to go into this in 15 minutes. We can with our trained model now actually make
this forecast. And of course, we want it to look nice. So we have this beautiful Grafana dashboard,
which actually summarizes all of the information that you need for your forecast. So let's look
into it. First and foremost, our forecast. So the red line on the left is actually the low that
has been historically measured. And then we see here the yellow lines is our forecast. Well,
now you see that there are a lot of yellow lines. What do those mean? Well, those are actually the
quantiles. So you have actually a certainty in your forecast. And this can be actually useful if we're
a certain location. You're quite sure what your forecast is going to be. You can go into
another quantile. Then if you have a location where you have a lot of factors that you actually
don't know anything about. And also very nice our feature importance plot. So here in the feature
importance plot, we can see our lag loads and some other features. And this is actually
nice. So you can see for every location, which features are important for your forecast.
So for example, here we see radiation. I don't think it's readable for you, but it says radiation.
So you know that there are quite some solar parks or solar panels behind, for example, your substation.
Wind speeds nowhere to be seen. So probably no windmills in that area.
So this was really short about open staff. Let me see how much time I have left.
Six minutes. Perfect. Okay. So community and upcoming events. One of the main things that has
really changed in open staff this last year is our community. So before it was just Alliander
who actually created it together with RTE, working on open staff. And now it looks like this. So let
me go over every company really quick. So Alliander, that's where I'm from, talked about that enough.
RT actually working on open staff for quite a while and they're actually ready to implement it very
soon. RT International just joined us this year. They have a very nice proof of concept and they're
going to work on it further. Fidel has actually been using open staff quite a long. I've heard
some terms, leeches this today. Well, that was a feed on up to actually a month ago. So we contacted
them and they were like, oh yeah, we found some bugs. We fixed it. We can implement this. So they
actually joined our community as of this year. Sigelman still working on a proof of concept
and seeing if they want to replace their own forecasting model with open staff. And Shell is
working on open staff DBC and seeing if they can use their method of data important.
Now, I hope everyone feels like they want to try open staff. Well, you're in luck because we are
organizing a workshop. So on the Friday, the first of March from two to four, we were organizing a
workshop. And I would like everyone who's interested to join. So you'll get a better
introduction to open staff and also a little bit more of the technical details.
It will be virtual. And you will get really a hands on experience. So you get some example
notebooks from us where you have to make your own exercises and you can actually make your own
forecast with open staff and see how easy it is. If you want to sign up, just scan the QR
card over here. And it will be very nice. I also have it on the next slide for people who are too
slow. So want to know more about open staff, maybe even before you sign up for the workshop,
we of course have our GitHub website documentation, etc. You're only one command away from using open
staff. And if there's anything you want to ask or give some comments or anything, you can just
send me an email or send me a message on LinkedIn. So thank you for your time and I welcome any
questions.
Who's running the microphone?
I'll try to do my best. Please feel free to guess to find the best path.
Hello. First of all, thank you so much. This was very interesting. And I have no experience,
I have never heard of open staff before reading on the FOSTA website. I have one question about
the data collection. Do you provide like some examples or standards on how and where to fetch
data because the data source is very, I tried, I looked.
So very good question, I think this is something that a community indeed struggles with. So for the
Netherlands, we actually do have those sources because we are using them ourselves for other
countries who are working on it to see if we can find some open data for everyone.
But if you're interested, you can always send me an email and I'll see what we have. Yeah, great.
Hi, it's Miné. I'm from Red Hat. So obviously I will ask the question about scaling this, right? How
will you standardize and scale this because it's a project. It sounds super interesting. But how
are we going to scale this to 49,000 substations or millions of smart meters at home?
Very good question. This is actually something we're working on right now. So we are actually
employing our open step stack on Dexter probably anytime soon and seeing if you can actually scale
from that. Currently we have it scaled up at I think 100 substations. And if you're curious how we
have a reference implementation on our GitHub and you can see all the information there on how we
deploy this. Thanks.
Yeah, yeah, sure.
I have a question about the data sources. Is there any thought given to adding geographical
information systems data into the system for forecasting models? Because especially stuff
like wind and solar radiation also not just depend on the time of day and the wind speeds,
but the location itself.
Great question. Yeah, actually for our system, it just connects to the closest K and MI. So that's
the Royal Dutch Weather Organization. So it's able to find the closest station to where you actually
want to forecast. So it definitely takes a location to account. We have a prediction job
class where you can put in all of the information for your forecast and in there you also put the
latitude and longitude of your location. So it does take that into account. Question over there.
Thanks for the question about the geographic data because I was thinking about an approach of
just using cheap raspberry weather stations in Austria and distributing them across some locations
to fetch the data because I have the Google Weather API and the Open Weather API or whatever as
comparison values. And for the geographic thing, thanks for the question. How would you connect
that? Like is this a plan of open stuff? Did I miss this?
Yeah, thanks for the kind of difficult question because I don't know the answer.
So I'll ask my colleagues who actually made this part of the open stuff and I'll get back to you
if we connect afterwards. So then you'll know. But it's very interesting to do with the Raspberry
Price things. Thanks.