[00:00.000 --> 00:02.060] you [00:30.000 --> 00:46.560] Hello everyone, I am Nizamuddin and I am with Ankush so we are here to cover a little [00:46.560 --> 00:54.000] about operating step from the step dashboard past present and future since we couldn't give [00:54.000 --> 00:58.280] the presentation live we are recording this separately so this is a two-part presentation [00:58.280 --> 01:07.400] the second part will be covered by Ankush so the part one will include the introduction [01:07.400 --> 01:13.960] why and who needs a dashboard the history the architecture of the dashboard the key features [01:13.960 --> 01:19.920] that we delivered on QnC then the road map to the reef release and I'll try to cover a short [01:19.920 --> 01:27.160] demo as well so just a brief intro on myself I started my career at Red Hat as an intent [01:27.160 --> 01:32.520] at the time I was working with the Rook and OCS operator later on I became an associate [01:32.520 --> 01:36.600] software engineer and got assigned with the step dashboard I've been ever since in the [01:36.600 --> 01:40.200] step dashboard team and last year I became the component lead of the step dashboard [01:40.200 --> 01:44.720] recently we got moved over to IBM as part of an internal transition and I am now and I am [01:44.720 --> 01:52.360] now working as a software engineer at IBM so we all know what a dashboard is right and what I [01:52.360 --> 01:57.520] want to cover is what I or what I want to reiterate over here is like why step dashboard is not [01:57.520 --> 02:05.720] just a dashboard for safe because step dashboard is not just a tool that is limited to monitoring [02:05.720 --> 02:14.480] only step dashboard is a fully-fledged management and monitoring web UI tool it has all the [02:14.480 --> 02:20.960] functionalities of management like you know you can manage host OSDs or you know poor management [02:20.960 --> 02:27.160] step affairs, IGW, NFS all these kinds of management functionalities are available in the [02:27.160 --> 02:32.640] step dashboard then why we didn't change it like why we didn't change the name to step dash from [02:32.640 --> 02:37.120] step dashboard to something else right because when it was fine it was initially introduced [02:37.120 --> 02:43.160] that's like a long time ago it was just a dashboard we can only do monitoring activities in it [02:43.160 --> 02:48.840] then later on with the introduction of the dashboard version 2 things started to change [02:48.840 --> 02:55.480] but it was deemed too heavy to replace all the code and documentation references so hence we [02:55.480 --> 03:04.880] just kept with the step dashboard name all along so the distributed systems are always complex it's [03:04.880 --> 03:11.400] it's not just a single piece of software right it's comprised of a big software ecosystem so if [03:11.400 --> 03:16.920] you take the example of step is it's comprised of all these you know software softwares like [03:16.920 --> 03:26.160] the rados kernel rbd rgw it's all it's it's it's a compliant system of all these components so [03:26.160 --> 03:31.240] to install this you have to scale out and this causes infrastructure and configuration management [03:31.240 --> 03:38.280] and also monitoring becomes you know challenging the configuration of step involves around 2000 [03:38.280 --> 03:44.320] settings which gets incremented on each releases and then the operating on each of these pieces [03:44.320 --> 03:50.600] it's like you know it's each of these components has its own CLIs APIs or workflows differences of [03:50.600 --> 03:56.000] things right and maintaining the whole cluster again it's a challenge if you want to maintain if [03:56.000 --> 04:01.840] you want to do maintenance activity on like one host it can affect the rest of the cluster so a [04:01.840 --> 04:08.520] proper maintenance should be holistic and not be restricted to per component monitoring troubleshooting [04:08.520 --> 04:14.280] analyzing the issues a request going through or inspecting multiple architectural levels like [04:14.280 --> 04:24.120] the hardware networking logs etc also the CF CLI doesn't provide a unified user experience the [04:24.120 --> 04:32.440] safe the all these different components has its own CLI safe has the CF CLI rados has the [04:32.440 --> 04:41.400] rados CLI rgw has the rados gateway admin rbd then this FATM don't get me wrong CLI is great if [04:41.400 --> 04:46.200] you are experienced with CLI then most of the things will be easier right which is a few scripts [04:46.200 --> 04:52.200] and you can do wonders with it with you but when you are new into these ecosystems like me it gets [04:52.200 --> 04:58.280] complicated like you have to consider many different things and you have to understand many [04:58.280 --> 05:07.000] different you know CLI ways so for me it for me or people like me it is easier to go through a UI [05:07.000 --> 05:12.160] where you get a unified experience you can do anything like you can do you can create an [05:12.160 --> 05:25.320] rgw and rbt in a similar fashion to forms so it's get easier with the UI and we have the whole SSH [05:25.320 --> 05:35.440] versus HTTPS discussion the comparison so HTTPS is more ease of use and it's more easily configurable [05:35.440 --> 05:42.760] and the access control is better with the HTTPS CLI versus REST REST is more standardized and [05:42.760 --> 05:49.040] it follows an open API specification then the whole text versus graphics discussion you can [05:49.040 --> 05:59.440] choose it which which is better so where do we come from surface board on 2006 and later on it's [05:59.440 --> 06:09.120] mostly the CLI and 2013 the calamari or the ink tank or the open attic version of the dashboard [06:09.120 --> 06:17.840] was released then the VSM then the safe dash and in 2016 the safe manager got introduced and along [06:17.840 --> 06:24.640] with that the first version of the safe dashboard is also introduced to the community then 2018 [06:24.640 --> 06:32.920] safe dashboard version 2 2019 with the introduction of safe for orchestrators like the safe ADM the [06:32.920 --> 06:43.720] the management activities or the management in the UI got overall increased or improved and 2020 [06:43.720 --> 06:51.640] we introduced the workflows in UI so workflows like the expand cluster or the OST creation wizard [06:51.640 --> 07:00.640] which allows a more simpler you know cluster expansion process or the OST creation processes [07:00.640 --> 07:07.080] then we have the I think right more we mostly focus on the service abilities which includes [07:07.080 --> 07:14.680] which includes the introduction of the features like the safe centralized logging [07:14.680 --> 07:22.520] so these are all the usual suspects in the safe dashboard not the safe dashboard I mean the [07:22.520 --> 07:32.360] safe GUIs as you can see the safe dashboard is mostly supported by the safe community itself so [07:32.360 --> 07:41.000] it's been here since 2018 and it has a very advanced set of features and it mostly utilizes the [07:41.000 --> 07:49.960] safe manager APIs which is which makes the you know communication between safe easier and along [07:49.960 --> 07:55.400] with that we also use the technology stacks like Python and Angular for the front end [07:55.400 --> 08:03.360] then we also text text use of the Grafana to populates the all the Prometheus metrics [08:03.360 --> 08:13.520] these are some of the examples of the safe early safe GUIs the Calamari one the open [08:13.520 --> 08:25.320] attic one the VSM by Intel ink scope the safe dash which is a community project not not [08:25.320 --> 08:30.480] safe I mean it's an open source project then we have the safe dashboard version one [08:30.480 --> 08:40.760] the safe dashboard version two which is which is the one you can see in the current step [08:40.760 --> 08:49.480] this is the safe dashboard version three which is which is a which is a working progress feature [08:49.480 --> 08:57.320] or a working progress UI and we are trying to push this on to the to the reef release so it's [08:57.320 --> 09:08.680] more or less there so this is as you can see from the main cluster or the main branch then [09:08.680 --> 09:15.000] how safe users monitor the safe this is like a survey done by the community sometime ago [09:15.000 --> 09:21.680] as you can see from here Prometheus is predominantly used for monitoring the safe but it is closely [09:21.680 --> 09:30.760] followed by the safe dashboard around 50% or 49% of this 49% of the users I use safe to [09:30.760 --> 09:37.240] monitor the safe the safe dashboard to monitor the safe since luminous safe offers what's needed [09:37.240 --> 09:43.520] to develop a fully featured fully featured and smoothly integrated dashboard safe manager provides [09:43.520 --> 09:49.840] and in memory cache for all core safe data it also provide a highly efficient interface to safe [09:49.840 --> 09:56.320] based on python cores safe more provides a highly available back source persistence for small [09:56.320 --> 10:04.280] datasets this is the architecture of the safe dashboard we have the safe cluster at the beginning [10:04.280 --> 10:12.520] and then we have the in the client side we have the front end which is which is an angular [10:12.520 --> 10:21.160] application so when the angular request some something from the REST API you know REST API [10:21.160 --> 10:27.040] gets in touch with the safe manager module APIs which you know talks with the safe cluster and [10:27.040 --> 10:33.640] retrieves information in the cache status and given back to the front end. Manager module [10:33.640 --> 10:39.920] APIs also exports the metrics through a Prometheus exporter which is given to Prometheus and through [10:39.920 --> 10:47.800] Grafana we get a nicely viewable graphs and these graphs are also shown in the front end [10:47.800 --> 10:55.400] in the safe dashboard using an iframe component so these are all the features that are available [10:55.400 --> 11:02.600] in the QnC release as you can see from this chart we are all we are almost achieved in some of the [11:02.600 --> 11:10.000] components like the pools or the configurations or the OSDs so we almost achieved a parity with [11:10.000 --> 11:17.200] the CLI we support most of the features that are available from the CLI and we are also we [11:17.200 --> 11:23.400] also provide some extra features like the some of the important features are highlighted at the [11:23.400 --> 11:32.080] bottom like the cluster expansion wizard the internationalization then we have the increased [11:32.080 --> 11:40.120] security the safe logs or the role-based access control the built-in grafanas alerting SSOs the [11:40.120 --> 11:49.360] REST APIs so these are all some of the features of safe dashboard and these are all you know what [11:49.360 --> 11:56.560] we try to achieve on the reef release so far so in reef we release some of some new features and [11:56.560 --> 12:04.760] we again increased the you know parity with the CLI so we introduced a new OSD cluster OSD [12:04.760 --> 12:12.160] creation wizard from from the dashboard which which increased the usability which increased [12:12.160 --> 12:19.600] the which simplified the OSD creation from you know if you are using safe dashboard then on the [12:19.600 --> 12:28.480] RBD site we introduced the RBD snapshot mirroring or we tried we started supporting the RBD [12:28.480 --> 12:35.440] snapshot mirroring from the dashboard with the RGW we introduced the RGW server-side encryption [12:35.440 --> 12:42.400] in the dashboard and also with the reef release we are trying to introduce more features for [12:42.400 --> 12:51.720] more RGW features like the the multi-site and the some users or role role policies so those [12:51.720 --> 12:59.560] stuffs are still in working progress we also have a cool new feature called centralized logging so [12:59.560 --> 13:07.960] with just one view you will be able to see all the safe logs or container logs like the manager [13:07.960 --> 13:17.200] mones or whatever you know component or container you have set up also we also tried to achieve [13:17.200 --> 13:28.800] WCAG level AA standardization standardization with the accessibility so these are some of [13:28.800 --> 13:34.240] the goals that are beyond a reef so we are trying to we are constantly trying to improve the [13:34.240 --> 13:40.480] usability experience so that's like a that's like a constant efforts from the dashboard side so we [13:40.480 --> 13:46.120] also something called the the low code initiative which we will talk about later on in the end of [13:46.120 --> 13:52.240] the session so we are trying to replace the Grafana with some built-in charts you know so [13:52.240 --> 13:57.960] instead of you know consuming instead of populating the charts with the Grafana using the iframe [13:57.960 --> 14:08.080] components we are trying to directly populate the charts using some charts as like frameworks we [14:08.080 --> 14:16.280] also have a plan to improve or improve the surface integration in the dashboard currently we have [14:16.280 --> 14:23.480] all the monitoring pieces settled in the dashboard but we still lack some of the management [14:23.480 --> 14:30.960] activities on this FFS area so we will focus on that later on and then also the multi cluster [14:30.960 --> 14:35.680] monitoring and management that's like a that's also an important feature that we are planning to [14:35.680 --> 14:43.960] deliver from the dashboard and also trying to achieve the feature parity with the CLIs so I will [14:43.960 --> 14:53.360] try to show a quick safe dashboard demo now so here is the login page of the safe dashboard and if [14:53.360 --> 15:01.080] you log in you will be greeted with the landing page or the dashboard page where you can see or [15:01.080 --> 15:08.080] view all the status or the information regarding your cluster you have the cluster status which [15:08.080 --> 15:14.440] is showing a warning for me you can click that and see why the cluster is in a one state and [15:14.440 --> 15:24.760] all other information regarding the cluster can be seen here I have like two hw's daemon's setup [15:24.760 --> 15:34.320] and all other informations then if you go to the cluster host you can do all the host management [15:34.320 --> 15:42.560] activities then we have the inventory or the physical disks part also you can see you know all [15:42.560 --> 15:49.640] the OSD's attached to the each of these devices you can identify the device by blinking the LEDs [15:49.640 --> 15:58.000] the monitors the all the services that are available right now in the cluster you can create more [15:58.000 --> 16:07.120] services and edit or delete these services also you can you can start or restart the daemon's if [16:07.120 --> 16:15.520] you go into the each of the services details there is also service events the daemon events are [16:15.520 --> 16:23.080] also there in the OSD's you can create you can create OSD's or you can you know we also have [16:23.080 --> 16:31.400] this new OSD creation wizard which has like some pre-optimized deployment options there or you [16:31.400 --> 16:37.400] can also go with the advanced mode and create advanced you know create a tribe group specification [16:37.400 --> 16:46.040] of your own also there are different set of activities that can be performed on which of [16:46.040 --> 16:53.240] these OSD's like you know same thing with flags or deep scrubbing marking an OSD out parsing or [16:53.240 --> 17:00.400] eventually destroying the OSD's then the cluster byte configuration like you know you can set flags [17:00.400 --> 17:08.400] out through to all the OSD's at once if you select one of the options here the recovery [17:08.400 --> 17:13.920] priority pg scrub and if you go to the overall performance you can see the overall performance [17:13.920 --> 17:21.880] of the OSD's which is you know code from the graphana itself so this is a graphana chat which [17:21.880 --> 17:27.440] is integrated into the dashboard using an iframe component and we have all the configuration listed [17:27.440 --> 17:38.280] here the crush map viewer for seeing the crush map tree then the manager modules we can start or [17:38.280 --> 17:44.920] restart you know you can enable or disable a manager module from here um this is the cfx [17:44.920 --> 17:53.200] auth users management so right now we have the we support listing the users and you know creating [17:53.200 --> 18:02.040] a cf user in the logs we have the cluster logs and the audit logs then we have the daemon logs [18:02.040 --> 18:08.240] which is the centralized logging that I was talking about before so in the centralized logging bus [18:08.240 --> 18:16.800] you can browse through the logs and if you click on one of the log here and if you click on one [18:16.800 --> 18:23.520] of the container here let's say I want to see the manager module manager logs and if I put show logs [18:23.520 --> 18:34.160] it will display the logs from the manager of the safe node 00 you can also put this in a live [18:34.160 --> 18:46.800] live okay so you can also put this live and you can see the logs populating in a live logs [18:46.800 --> 18:56.920] populating lively so right now it's paused and we have the monitoring this is kept you know this [18:56.920 --> 19:03.600] is directly consumed from the prometheus and the alert manager so if there are any alerts in the [19:03.600 --> 19:08.840] cluster it will show up here and it will show some indicator in the navigation pane itself [19:08.840 --> 19:16.280] then we have the pool management again all the all kinds of pool management like pool creation [19:16.280 --> 19:22.400] edit delete the overall performance you can see all the pool performance in the overall performance [19:22.400 --> 19:27.360] and if you want to see individual performance as well you can see the performance details of each [19:27.360 --> 19:34.600] of the pool individually then we have the block the RBD section so if you have no RBD [19:34.600 --> 19:38.640] pools available it will show this message and if you want to create an RBD pool you can go [19:38.640 --> 19:47.960] and create an RBD pool by selecting the RBD application from here then we have the RBD [19:47.960 --> 19:53.040] mirroring right now it's not configured on my cluster so if I want to configure it you can [19:53.040 --> 20:00.720] click on this configure RBD mirroring and this will configure the the RBD mirroring and the [20:00.720 --> 20:08.240] iskasi section as well the NFS management so you can go and create an NFS export so if you [20:08.240 --> 20:14.800] don't have an NFS cluster created so it will show this info to create it or add an add a new NFS [20:14.800 --> 20:24.960] service we have the file systems again I don't have a file systems setup right now so we have [20:24.960 --> 20:30.840] an AGW section as well so if you go to the AGW daemons you can see all the daemons that are [20:30.840 --> 20:36.760] available in the cluster the overall performance of each of the daemon you can create and delete [20:36.760 --> 20:47.000] users and edit the users the buckets creation or the bucket management section over here dashboard [20:47.000 --> 20:54.120] also has the REST API which you can see if you click the API you will get redirected to the [20:54.120 --> 21:01.320] CIFREST API then we have the whole notification system here you can see all the notifications [21:01.320 --> 21:08.400] coming from different components of the CIFREST at one place you can also report an issue from [21:08.400 --> 21:14.760] the CIFREST dashboard so it will directly go to the tracker.CIFREST.com so if you you have to [21:14.760 --> 21:23.800] enable the feedback module you can go and enable it from here so once you do that you and you [21:23.800 --> 21:31.320] provide this you are a CIFREST tracker API key and you give all this information then you submit [21:31.320 --> 21:40.120] you submit that it will create an issue in the in the tracker.CIFREST.com then you have the user [21:40.120 --> 21:47.440] management the telemetry configuration and yeah that's more or less about the CIFREST dashboard [21:47.440 --> 21:55.000] yeah so that's all from me from my part I think the second part will be covered by Ankush. [21:55.000 --> 22:07.040] Hi and hello everyone thanks for the wonderful demo and the part one presentation let me [22:07.040 --> 22:12.120] introduce myself my name is Ankush I will be working as an engineering manager in IBM team [22:12.120 --> 22:19.040] I have expertise of more than seven years in delivering the management and monitoring solutions [22:19.040 --> 22:26.400] for a software-defined storage on a containerized or a non-containerized environment in an open [22:26.400 --> 22:32.880] source system so today I'll be taking you to the part two of this discussion where we'll be [22:32.880 --> 22:39.360] discussing two major aspect first of them will be how you can contribute to the CIFREST dashboard [22:39.360 --> 22:49.240] and the second one will be the CIFREST dashboard community and how it looks like moving on I [22:49.240 --> 22:55.200] think discussing how you can contribute as a CIFREST in the CIFREST dashboard as a user we'll [22:55.200 --> 23:00.920] be talking about as a translator or a documenter and also as a developer we'll be discussing how [23:00.920 --> 23:08.880] you can contribute first as a user right CIFREST dashboard is by default enabled when you install [23:08.880 --> 23:16.280] CIFREST at the CIFADM and you can use it from the get go but if it is somehow not enabled or if [23:16.280 --> 23:23.520] you are facing some issues while enabling the CIFREST dashboard you can follow these steps [23:23.520 --> 23:29.760] that are mentioned over here or you can follow the documentation link once the dashboard is [23:29.760 --> 23:35.280] enabled you will see this is the first screen that you'll see as a user you need to login and [23:35.280 --> 23:40.040] once you are logged in you will be able to do a couple of things from the CIFREST dashboard [23:40.040 --> 23:47.880] from the management of the whole CIFREST ecosystem to monitoring from alerting logging and all those [23:47.880 --> 23:54.960] things you can do on the CIFREST dashboard but somehow if you if you if you see any issues or [23:54.960 --> 24:00.840] if you see if you have any suggestions for any of the things that you tried out or if you have [24:00.840 --> 24:08.560] any feature requests that you want to put in you can also go into a report report an issue in [24:08.560 --> 24:14.760] dashboard itself and you can report that issue and it will open a bug in a CIF tracker where we [24:14.760 --> 24:21.640] will be following up with you also you can directly share experience on the CIF users mailing list [24:21.640 --> 24:26.600] that is mentioned over there or at the IRC channels that is CIF dashboard you can reach out to us [24:26.600 --> 24:35.840] and we'll be happy to help you out so second part of that is how you can contribute to CIF dashboard [24:35.840 --> 24:42.800] as a documentor so this is the documentation that link that we have and if you see any issues or [24:42.800 --> 24:48.200] if you have any and think that you want to suggest you can directly go and report a documentation [24:48.200 --> 24:54.880] bug from the documentation link and or if you want to do a edit yourself and submit a pull [24:54.880 --> 25:00.520] request we'll be happy to help you out on that you can click on this edit on GitHub link directly [25:00.520 --> 25:05.720] and then you can submit a pull request and the team will get back to you on the reviews on that [25:05.720 --> 25:14.800] pull request as a translator like what you can how you can contribute to CIF dashboard is you can [25:14.800 --> 25:20.080] follow you can look into this link where we already have a lot of translation done in the CIF [25:20.080 --> 25:26.120] dashboard side I think more than 10 plus languages are already there but we still have some gaps [25:26.120 --> 25:33.360] that can be filled so you can look into that link and help us out on the localization and [25:33.360 --> 25:42.280] internationalization of the CIF dashboard moving on as a developer how you can contribute to the [25:42.280 --> 25:48.600] CIF dashboard so firstly what you can do is you can subscribe to this dev at the CIF.io mailing [25:48.600 --> 25:55.880] list second is the IRC channel that we already talked about which is CIF dashboard we also have [25:55.880 --> 26:03.160] some documentation link very generic to CIF but also to the dashboard you can follow to set up [26:03.160 --> 26:12.320] your dev environment and play around with the CIF dashboard moving on and let's discuss how the [26:12.320 --> 26:18.400] code looks like right so we'll talk about two major things over here one is the backend one is [26:18.400 --> 26:25.960] front end on the backend side we use pythons 3.6 and on the front end side we use Angular 12 with [26:25.960 --> 26:32.480] TypeScript plus bootstrap 4 but we are planning to upgrade it to bootstrap 5 and also we are [26:32.480 --> 26:42.480] planning to upgrade the Angular version to adopt the new features as a developer right there is [26:42.480 --> 26:48.360] another initiative that is taken by the CIF dashboard team where we understand that not [26:48.360 --> 26:58.000] everybody knows how to code in the front end languages like HTML JavaScript TypeScript CSS [26:58.000 --> 27:06.080] and all of that ecosystem and but what we have done is that even if you don't know that you can [27:06.080 --> 27:13.240] still contribute to CIF dashboard using a low code initiative where you can write your descriptors [27:13.240 --> 27:19.800] and this JSON kind of a format with that will create a route for some page and then in a Python [27:19.800 --> 27:26.680] code you can write how your page should look like and then both together can work and generate a [27:26.680 --> 27:33.120] page something like this for you where you will have a UI generated from a Python code itself I [27:33.120 --> 27:38.600] think we have the first of our kind feature that is already written and this is a CIF [27:38.600 --> 27:44.080] auth management but I think in the next release we have couple more features coming in which is [27:44.080 --> 27:51.840] following the low code initiative and it is being and it is being discussed in the in the community [27:51.840 --> 28:01.640] right now I think the second part of this part to presentation was dashboard and and the community [28:01.640 --> 28:09.760] around it so we'll let's talk about dashboard in number of pool request we have 2700 pool [28:09.760 --> 28:15.680] requests that are there in dashboard we have 4200 comments and these many lines of code most of [28:15.680 --> 28:22.480] them are written in TypeScript but second prominent pre-dominant language is Python and the last [28:22.480 --> 28:32.840] but not the least is html we have behind the CIF dashboard there are a lot of great minds from [28:32.840 --> 28:37.920] different continent different countries that contribute to CIF dashboard and these are all [28:37.920 --> 28:43.840] majorly I think this called all the people that are working from different countries and continents [28:43.840 --> 28:54.800] and helping us grow moving on as you have seen that we have a we have a big community spread [28:54.800 --> 29:00.560] across the world time zone and all that stuff so we have tried to come up with something that is [29:00.560 --> 29:06.880] that works for everybody but can be difficult for somebody at some time so we have a daily [29:06.880 --> 29:14.240] stand-up at 11 am CET time or upstream wide audience think that happens or Tuesday for [29:14.240 --> 29:24.040] tonight at 2 pm CET and the face-to-face used to happen pre-pandemic times for almost three [29:24.040 --> 29:32.480] four days at different locations it has happened not happened after after the covid pandemic now [29:32.480 --> 29:39.600] we'll see some glimpse of how our face-to-face look like this is the first one that happened in [29:39.600 --> 29:46.240] Newenburg and and all the community members are present most of the community members are present [29:46.240 --> 29:54.280] over here and these are some of the images from that there was another step dashboard of orchestrator [29:54.280 --> 30:02.520] face-to-face that happened in Berlin coral surface and these are images from there and the second [30:02.520 --> 30:10.760] safe dashboard specific face-to-face that happened in June 2019 in Fulda and this these are images [30:10.760 --> 30:18.440] from there the upstream safe dashboard community is doing some team building activities and and [30:18.440 --> 30:24.920] enjoying work together I think that's all for today from my side for the part two presentation [30:25.720 --> 30:31.880] we'll be happy to take any questions if you have otherwise thank you everyone for joining this [30:31.880 --> 30:49.160] meeting and also this presentation