[00:00.000 --> 00:12.680]  Okay, awesome to see there's so many people here, really cool that there's a big interest
[00:12.680 --> 00:15.800]  in the energy topic.
[00:15.800 --> 00:24.200]  My name is Frederik Stool and I'm from Alliander which is a grid operator and I'll be talking
[00:24.200 --> 00:27.200]  about the open staff today.
[00:27.200 --> 00:37.440]  So first of all I put here in the graph this is a load profile, so the energy load somewhere
[00:37.440 --> 00:45.040]  in the grid and well you can see how it fluctuates over time, sometimes it's positive, sometimes
[00:45.040 --> 00:52.040]  it's negative, this means whether there's neto production or neto consumption.
[00:52.040 --> 00:58.920]  Now the question is, or you could ask, if we are at the red line right now, what will
[00:58.920 --> 01:03.920]  be the load in the future?
[01:03.920 --> 01:09.360]  And that's what we want to predict and if you're interested in that then you can use
[01:09.360 --> 01:14.720]  open staff because open staff means short-term energy forecasting.
[01:14.720 --> 01:22.000]  Okay, let's zoom out a bit first and before I go into a bit more detail about what open
[01:22.000 --> 01:29.920]  staff does, first I want to talk about, give a short introduction about why this is relevant.
[01:29.920 --> 01:37.680]  I don't have hours so I have to keep it short but there's a lot of to talk about here.
[01:37.680 --> 01:42.680]  But I want to start out with this picture and it's actually quite cool because I think
[01:42.680 --> 01:51.760]  the last presentation talked about flexible energy that consumers can use and this is
[01:51.760 --> 01:56.400]  one of the many things that are changing in the energy sector.
[01:56.400 --> 02:03.120]  So consumers have flexible products that also start producing, consumers also have solar
[02:03.120 --> 02:09.040]  panels, your local farmer might have a wind turbine somewhere, you have big wind parks
[02:09.040 --> 02:15.680]  on the sea, so there's all kinds of developments going on right now that make it harder for
[02:15.680 --> 02:24.880]  grid operators to forecast what's going to happen tomorrow or even the day after tomorrow.
[02:24.880 --> 02:32.040]  And I'll put this picture because all the things that I mentioned you can see right
[02:32.040 --> 02:39.480]  there and probably in the future it's only going to get harder and harder.
[02:39.480 --> 02:47.480]  For now I want to focus on the renewable energy part because it's also quite impactful.
[02:47.480 --> 02:53.440]  As you can see on this graph here this is for the Netherlands, the percentage of renewable
[02:53.440 --> 02:58.480]  energy production and you can see that in just a couple of years like five years it
[02:58.480 --> 03:05.200]  has more than doubled the electricity percentage that has been produced by renewable sources
[03:05.200 --> 03:10.040]  and renewable sources don't produce at a constant load of course, they change all the
[03:10.040 --> 03:19.280]  time depending on weather and this means it's harder to forecast and to put that into perspective
[03:19.280 --> 03:24.080]  I have another slide here and this is a typical consumption profile.
[03:24.080 --> 03:29.880]  If you have your local neighborhood then this is what the energy load will often look like.
[03:29.880 --> 03:35.080]  So you have the five peaks which means it's a peak for every day and in the weekend it's
[03:35.080 --> 03:41.880]  a bit lower, you can see the dips, these dips in the middle of the day that's because there's
[03:41.880 --> 03:49.000]  a couple of solar panels on some roofs you know, it still looks easy to predict.
[03:49.000 --> 03:57.520]  If you go to other places where there's way more renewable energy you can see these energy
[03:57.520 --> 03:59.800]  profiles change dramatically.
[03:59.800 --> 04:08.040]  So here you can see really a profile for a big solar park and you can see these huge
[04:08.040 --> 04:12.720]  negative peaks which on some days are there and some days they're not probably that's
[04:12.720 --> 04:17.800]  a cloudy day so there's no or less energy generated.
[04:17.800 --> 04:25.680]  And this is an energy profile for a big wind farm which you can see is well, seems hard
[04:25.680 --> 04:38.160]  to predict because there's seems no real, yeah so a negative energy means that the consumer
[04:38.160 --> 04:46.320]  or the customer I mean is giving back to the grid so then it's negative for us.
[04:46.320 --> 04:55.840]  If it's positive it means sorry, yeah exactly, so if it means a big negative peak it means
[04:55.840 --> 04:59.000]  the customer is producing a lot of energy.
[04:59.000 --> 05:03.120]  So it's just a convention you could also switch the sign but you have to choose one
[05:03.120 --> 05:04.120]  convention.
[05:04.120 --> 05:05.120]  Yeah power, yeah.
[05:05.120 --> 05:25.240]  Yeah so it's not just one side it's more like a general profile for like a substation for
[05:25.240 --> 05:29.200]  the grid operator but connected to it is a lot of solar.
[05:29.200 --> 05:42.080]  Yeah exactly so on all of these there's load and production but yeah I just wanted to share
[05:42.080 --> 05:45.320]  this feeling.
[05:45.320 --> 05:56.800]  So this can be difficult and this also leads to problems and this is two maps of the Netherlands
[05:56.800 --> 06:03.000]  and the colored areas are the areas where Alliander is currently active and on the left
[06:03.000 --> 06:08.280]  is energy consumption and on the right is energy production and this map shows if you're
[06:08.280 --> 06:15.240]  a new customer and you want to be connected to the grid if it's red it's probably difficult
[06:15.240 --> 06:17.240]  because there's no more room.
[06:17.240 --> 06:26.080]  According to the Dutch law the energy grid is full over there and you can see that this
[06:26.080 --> 06:33.120]  is for huge areas in the Netherlands and also large areas on the consumption side.
[06:33.120 --> 06:38.280]  And of course Alliander is doing everything they can to solve this by building new cables
[06:38.280 --> 06:43.840]  and new substations but this takes time, a lot of time and we don't have the time as
[06:43.840 --> 06:47.120]  you can see in the graph before.
[06:47.120 --> 07:00.120]  So I don't have that one but I assume it's very similar because we're not the only one
[07:00.120 --> 07:04.720]  who are having these issues.
[07:04.720 --> 07:06.040]  So how can we solve this?
[07:06.040 --> 07:16.760]  Well one important thing is that we need grid inside and therefore this also includes forecasts.
[07:16.760 --> 07:25.280]  So transmission forecasts and these are important for all three parts in the electrical grid
[07:25.280 --> 07:27.520]  so all three parties.
[07:27.520 --> 07:38.680]  So for customers, for DSOs such as Alliander and for TSOs such as Tenet which control the
[07:38.680 --> 07:43.960]  high voltage grid.
[07:43.960 --> 07:51.040]  Using these forecasts operators can try to maintain grid safety and grid balance and
[07:51.040 --> 07:59.040]  can give customers as much electricity as they want and as they need because the need
[07:59.040 --> 08:03.200]  is high.
[08:03.200 --> 08:12.280]  With these forecasts we can also enable smart solutions and I put here two brochure pictures
[08:12.280 --> 08:22.400]  of those solutions, one of them is a pilot FlexPower which was in Amsterdam which was
[08:22.400 --> 08:30.080]  about charging electrical vehicles and charging them faster if it's possible and not charging
[08:30.080 --> 08:36.240]  them as fast if it's not possible.
[08:36.240 --> 08:43.080]  We at Alliander supplied forecasts for this project and another platform is the GOPEX
[08:43.080 --> 08:48.960]  platform which is like a trading platform for electricity where customers can trade
[08:48.960 --> 08:58.400]  with operators to either consume or to produce energy flexibly and this is also being used
[08:58.400 --> 09:02.400]  right now at Alliander and we also provide a forecast for that.
[09:02.400 --> 09:17.440]  So it's no longer working so let's use it.
[09:17.440 --> 09:22.800]  So now let's talk about Opelstaff again because that's why I'm here and I'm going to give
[09:22.800 --> 09:28.440]  a short introduction to Opelstaff and then I'm just going to give a short demo about
[09:28.440 --> 09:34.000]  Opelstaff how you can make a forecast and also want to talk a bit about using Opelstaff
[09:34.000 --> 09:38.200]  in an operational setting.
[09:38.200 --> 09:44.440]  So first of all, the primary thing Opelstaff can help you with is that it's just a complete
[09:44.440 --> 09:46.680]  machine learning pipeline.
[09:46.680 --> 09:53.200]  So I'm just going to give a short list of what it can do.
[09:53.200 --> 10:00.760]  It handles input validation such as checking whether your data is complete.
[10:00.760 --> 10:10.720]  It has feature engineering so it automatically calculates for you lag features or other features
[10:10.720 --> 10:13.080]  that are based on input features.
[10:13.080 --> 10:20.120]  So for example, if you input it with wind speed, it can calculate wind turbine power
[10:20.120 --> 10:30.000]  output for you or the same for direct normal irradiance.
[10:30.000 --> 10:39.720]  Next it is some kind of intelligent train validation split of the time series.
[10:39.720 --> 10:43.560]  It has support for multiple type of regressors.
[10:43.560 --> 10:50.240]  So right now we have, for example, HGBoost which is at Allende the most commonly used
[10:50.240 --> 10:56.840]  but we also had a collaboration with Sonyo which added ProLove to Opelstaff and we also
[10:56.840 --> 10:58.920]  have support for probabilistic forecasts.
[10:58.920 --> 11:05.560]  So that means not just one line but quantiles.
[11:05.560 --> 11:12.880]  And unless it has integrated the model and artifact storage using MLflow.
[11:12.880 --> 11:14.920]  So what does this all mean then?
[11:14.920 --> 11:19.520]  Let's go to an actual demo.
[11:19.520 --> 11:24.840]  So I'm going to put this up here.
[11:24.840 --> 11:31.240]  That's a low resolution.
[11:31.240 --> 11:32.240]  Let's zoom out.
[11:32.240 --> 11:36.960]  It's a bit too much.
[11:36.960 --> 11:44.520]  Okay, so I'm just going to walk you through an example or how you could make a forecast.
[11:44.520 --> 11:51.080]  So first we need to make some kind of config object that's just what you have to feed Opelstaff.
[11:51.080 --> 11:53.800]  Let's close this.
[11:53.800 --> 11:57.160]  So let's run this line.
[11:57.160 --> 12:02.680]  Next I put some example input in this project.
[12:02.680 --> 12:11.480]  So we can load it and we can visualize it.
[12:11.480 --> 12:18.760]  So as you can see here, well, this is upon a stator frame and here we have the load and
[12:18.760 --> 12:20.960]  we have a lot of predictors.
[12:20.960 --> 12:26.280]  Well, some of these, well, the names should make sense.
[12:26.280 --> 12:34.120]  So for example, the amount of variation predicted by the KMI or the temperature, well, all
[12:34.120 --> 12:38.920]  these predictors are already in this example data.
[12:38.920 --> 12:50.280]  So if we have this, I can also plot it for you so you can see, well, this is another
[12:50.280 --> 12:52.480]  power profile.
[12:52.480 --> 12:59.080]  Okay, so now imagine you have this and you want to know, well, what's next?
[12:59.080 --> 13:02.560]  Then first we need to train a model.
[13:02.560 --> 13:08.720]  So Opelstaff has a train model pipeline which basically does all those things I just mentioned.
[13:08.720 --> 13:16.320]  So we can just call the pipeline and let's hope the live demo does not fill me.
[13:16.320 --> 13:22.760]  It will take about 15 seconds I think to train a model and store it.
[13:22.760 --> 13:31.800]  So you can see some info about what it's doing, well, and it's stored it.
[13:31.800 --> 13:44.840]  So let's have a look and we'll flow comes with an interface so we can directly see that
[13:44.840 --> 13:48.880]  we train a model.
[13:48.880 --> 13:53.360]  So right here, this was the run.
[13:53.360 --> 13:56.000]  Now let's hope this works.
[13:56.000 --> 14:01.800]  I see that my internet is no longer working so apparently this, then this figure this
[14:01.800 --> 14:02.800]  will work.
[14:02.800 --> 14:10.880]  All right, I'm not showing it.
[14:10.880 --> 14:20.080]  So, well, this is the MLflow interface and you can see that we just train a model.
[14:20.080 --> 14:25.440]  You can also click on the model or on the train run, this is just MLflow and you can
[14:25.440 --> 14:33.280]  see a bit more, well, information about what happened during the training.
[14:33.280 --> 14:38.280]  The next, of course, we want to make a prediction.
[14:38.280 --> 14:46.120]  So again, OpenStep has a pipeline for that so we can just say, okay, I want a prediction.
[14:46.120 --> 14:53.120]  So it's loading the model and using data to create a prediction and then we can visualize
[14:53.120 --> 14:56.400]  that as well.
[14:56.400 --> 14:58.560]  And then we have a graph right there.
[14:58.560 --> 15:06.320]  So this is the forecast that it made for the next, well, this was in some example data
[15:06.320 --> 15:14.600]  in 2021 but about 48 hours of forecast so that's OpenStep in practice.
[15:14.600 --> 15:20.800]  Let's go back to the presentation.
[15:20.800 --> 15:23.000]  Do this slide.
[15:23.000 --> 15:32.680]  Yeah, so, well, this flow has been a minimal flow but of course in reality, at least for
[15:32.680 --> 15:37.120]  if you're a grid operator, you want to do this in an operational setting.
[15:37.120 --> 15:43.120]  So this means that you want to do daily forecasts for a lot of different locations with all
[15:43.120 --> 15:45.080]  kinds of configurations.
[15:45.080 --> 15:53.520]  And OpenStep also comes with a so-called reference implementation about how you could do this.
[15:53.520 --> 16:00.400]  So this is a picture of what you would have to do so we have OpenStep right here which
[16:00.400 --> 16:04.560]  is basically, I just showed you the training and forecasting pipeline.
[16:04.560 --> 16:11.920]  Then we have another package which is called OpenStep DBC, database connector which can
[16:11.920 --> 16:15.680]  connect to a database.
[16:15.680 --> 16:26.480]  And we use MySQL and Influx DB to store all the data required to run it operationally.
[16:26.480 --> 16:32.720]  And we also have a Gafana dashboard built upon this database stack so we can also see
[16:32.720 --> 16:35.000]  what's going on.
[16:35.000 --> 16:39.760]  And again, as I already have shown, you can use MLflow to keep track of all the models
[16:39.760 --> 16:44.120]  and all the runs that are being done to see what's going on.
[16:44.120 --> 16:53.760]  So I want to show this dashboard as well.
[16:53.760 --> 17:07.840]  So this dashboard is just example data so it's not our real dashboard.
[17:07.840 --> 17:17.120]  But here you can for example see some load that was there on the system and you can also
[17:17.120 --> 17:24.960]  see that for example this is not just one area but it has a sum of two systems which
[17:24.960 --> 17:30.920]  is quite common in an electrical grid that you have a lot of measurement points that
[17:30.920 --> 17:35.400]  you have to add together with different signs.
[17:35.400 --> 17:46.360]  And you can see for example here's then a live forecast of this location as well.
[17:46.360 --> 17:55.800]  You can also see plots of the feature importance that obtained during training of the model.
[17:55.800 --> 18:00.080]  You can see on which data the model has been trained.
[18:00.080 --> 18:06.040]  Over here these plots are really small but here you can see them.
[18:06.040 --> 18:12.240]  So it's a dashboard where you can see everything that Oberstaff does for every location that
[18:12.240 --> 18:14.240]  you are could be interested in.
[18:14.240 --> 18:19.120]  You talk about forecast, are you within the forecast also taking other forecasts like
[18:19.120 --> 18:23.040]  the weather forecast into account or are you forecasting that yourself?
[18:23.040 --> 18:30.760]  No so we use all kinds of data and the weather forecast is, oh the question is whether we
[18:30.760 --> 18:38.360]  are using other forecasted data or whether we forecast, do those forecasts as well?
[18:38.360 --> 18:44.280]  Like whether we forecast the weather ourselves and the answer is that it depends a bit.
[18:44.280 --> 18:52.000]  So in general we use the weather forecasts for multiple sources and also for example
[18:52.000 --> 18:58.280]  price like the head pricing.
[18:58.280 --> 19:07.400]  So we use those data but sometimes we also feed the prediction itself or we feed one
[19:07.400 --> 19:12.480]  prediction into another prediction.
[19:12.480 --> 19:19.960]  So I mean you can play around with that but you have to feed Oberstaff with all the predictors
[19:19.960 --> 19:21.960]  that you wanted to know.
[19:21.960 --> 19:36.680]  Okay let's move on, I have one last slide and that's basically
[19:36.680 --> 19:49.760]  key information because that was my presentation so here I put all the info you might be interested
[19:49.760 --> 20:03.880]  in on this slide and if there are any questions then feel free to ask.
[20:03.880 --> 20:14.560]  So the information is really useful but what is the purpose for the net grid operator,
[20:14.560 --> 20:21.880]  what use does a grid operator make of this information, is it for congestion management,
[20:21.880 --> 20:28.240]  is it for some kind of load shedding, what is the role of this exercise?
[20:28.240 --> 20:39.920]  So I think the question is why would the grid operator be interested in forecast I guess
[20:39.920 --> 20:41.400]  is what you're asking.
[20:41.400 --> 20:47.600]  So there are many reasons but I think you already mentioned congestion management is
[20:47.600 --> 20:53.560]  indeed an important reason but also well grid insight.
[20:53.560 --> 21:03.720]  So the more congestion management is going to be used for the grid the more important
[21:03.720 --> 21:10.840]  it is also to maintain grid safety and grid safety is not just one operator, we are all
[21:10.840 --> 21:17.680]  connected to multiple grid operators so everyone has to communicate what they are going to
[21:17.680 --> 21:27.560]  do and what they expect the energy flow to be the next day so every operator can decide
[21:27.560 --> 21:29.840]  to do what's necessary to maintain grid safety.
[21:29.840 --> 21:37.160]  So that's what I mentioned before the transmission forecast, every operator has to communicate
[21:37.160 --> 21:45.040]  to everyone who is connected to what they expect the load to be on the next day.
[21:45.040 --> 21:49.240]  I see that my time is up so I'm afraid I have to answer the questions in the chat.
[21:49.240 --> 21:57.360]  Yeah or in the hallway, I think for time management we have to learn from these talks and see
[21:57.360 --> 22:01.320]  if we can manage to keep a couple of more minutes for questions, sorry.
[22:01.320 --> 22:16.400]  Thank you very much for listening.