you
Hello everyone, I am Nizamuddin and I am with Ankush so we are here to cover a little
about operating step from the step dashboard past present and future since we couldn't give
the presentation live we are recording this separately so this is a two-part presentation
the second part will be covered by Ankush so the part one will include the introduction
why and who needs a dashboard the history the architecture of the dashboard the key features
that we delivered on QnC then the road map to the reef release and I'll try to cover a short
demo as well so just a brief intro on myself I started my career at Red Hat as an intent
at the time I was working with the Rook and OCS operator later on I became an associate
software engineer and got assigned with the step dashboard I've been ever since in the
step dashboard team and last year I became the component lead of the step dashboard
recently we got moved over to IBM as part of an internal transition and I am now and I am
now working as a software engineer at IBM so we all know what a dashboard is right and what I
want to cover is what I or what I want to reiterate over here is like why step dashboard is not
just a dashboard for safe because step dashboard is not just a tool that is limited to monitoring
only step dashboard is a fully-fledged management and monitoring web UI tool it has all the
functionalities of management like you know you can manage host OSDs or you know poor management
step affairs, IGW, NFS all these kinds of management functionalities are available in the
step dashboard then why we didn't change it like why we didn't change the name to step dash from
step dashboard to something else right because when it was fine it was initially introduced
that's like a long time ago it was just a dashboard we can only do monitoring activities in it
then later on with the introduction of the dashboard version 2 things started to change
but it was deemed too heavy to replace all the code and documentation references so hence we
just kept with the step dashboard name all along so the distributed systems are always complex it's
it's not just a single piece of software right it's comprised of a big software ecosystem so if
you take the example of step is it's comprised of all these you know software softwares like
the rados kernel rbd rgw it's all it's it's it's a compliant system of all these components so
to install this you have to scale out and this causes infrastructure and configuration management
and also monitoring becomes you know challenging the configuration of step involves around 2000
settings which gets incremented on each releases and then the operating on each of these pieces
it's like you know it's each of these components has its own CLIs APIs or workflows differences of
things right and maintaining the whole cluster again it's a challenge if you want to maintain if
you want to do maintenance activity on like one host it can affect the rest of the cluster so a
proper maintenance should be holistic and not be restricted to per component monitoring troubleshooting
analyzing the issues a request going through or inspecting multiple architectural levels like
the hardware networking logs etc also the CF CLI doesn't provide a unified user experience the
safe the all these different components has its own CLI safe has the CF CLI rados has the
rados CLI rgw has the rados gateway admin rbd then this FATM don't get me wrong CLI is great if
you are experienced with CLI then most of the things will be easier right which is a few scripts
and you can do wonders with it with you but when you are new into these ecosystems like me it gets
complicated like you have to consider many different things and you have to understand many
different you know CLI ways so for me it for me or people like me it is easier to go through a UI
where you get a unified experience you can do anything like you can do you can create an
rgw and rbt in a similar fashion to forms so it's get easier with the UI and we have the whole SSH
versus HTTPS discussion the comparison so HTTPS is more ease of use and it's more easily configurable
and the access control is better with the HTTPS CLI versus REST REST is more standardized and
it follows an open API specification then the whole text versus graphics discussion you can
choose it which which is better so where do we come from surface board on 2006 and later on it's
mostly the CLI and 2013 the calamari or the ink tank or the open attic version of the dashboard
was released then the VSM then the safe dash and in 2016 the safe manager got introduced and along
with that the first version of the safe dashboard is also introduced to the community then 2018
safe dashboard version 2 2019 with the introduction of safe for orchestrators like the safe ADM the
the management activities or the management in the UI got overall increased or improved and 2020
we introduced the workflows in UI so workflows like the expand cluster or the OST creation wizard
which allows a more simpler you know cluster expansion process or the OST creation processes
then we have the I think right more we mostly focus on the service abilities which includes
which includes the introduction of the features like the safe centralized logging
so these are all the usual suspects in the safe dashboard not the safe dashboard I mean the
safe GUIs as you can see the safe dashboard is mostly supported by the safe community itself so
it's been here since 2018 and it has a very advanced set of features and it mostly utilizes the
safe manager APIs which is which makes the you know communication between safe easier and along
with that we also use the technology stacks like Python and Angular for the front end
then we also text text use of the Grafana to populates the all the Prometheus metrics
these are some of the examples of the safe early safe GUIs the Calamari one the open
attic one the VSM by Intel ink scope the safe dash which is a community project not not
safe I mean it's an open source project then we have the safe dashboard version one
the safe dashboard version two which is which is the one you can see in the current step
this is the safe dashboard version three which is which is a which is a working progress feature
or a working progress UI and we are trying to push this on to the to the reef release so it's
more or less there so this is as you can see from the main cluster or the main branch then
how safe users monitor the safe this is like a survey done by the community sometime ago
as you can see from here Prometheus is predominantly used for monitoring the safe but it is closely
followed by the safe dashboard around 50% or 49% of this 49% of the users I use safe to
monitor the safe the safe dashboard to monitor the safe since luminous safe offers what's needed
to develop a fully featured fully featured and smoothly integrated dashboard safe manager provides
and in memory cache for all core safe data it also provide a highly efficient interface to safe
based on python cores safe more provides a highly available back source persistence for small
datasets this is the architecture of the safe dashboard we have the safe cluster at the beginning
and then we have the in the client side we have the front end which is which is an angular
application so when the angular request some something from the REST API you know REST API
gets in touch with the safe manager module APIs which you know talks with the safe cluster and
retrieves information in the cache status and given back to the front end. Manager module
APIs also exports the metrics through a Prometheus exporter which is given to Prometheus and through
Grafana we get a nicely viewable graphs and these graphs are also shown in the front end
in the safe dashboard using an iframe component so these are all the features that are available
in the QnC release as you can see from this chart we are all we are almost achieved in some of the
components like the pools or the configurations or the OSDs so we almost achieved a parity with
the CLI we support most of the features that are available from the CLI and we are also we
also provide some extra features like the some of the important features are highlighted at the
bottom like the cluster expansion wizard the internationalization then we have the increased
security the safe logs or the role-based access control the built-in grafanas alerting SSOs the
REST APIs so these are all some of the features of safe dashboard and these are all you know what
we try to achieve on the reef release so far so in reef we release some of some new features and
we again increased the you know parity with the CLI so we introduced a new OSD cluster OSD
creation wizard from from the dashboard which which increased the usability which increased
the which simplified the OSD creation from you know if you are using safe dashboard then on the
RBD site we introduced the RBD snapshot mirroring or we tried we started supporting the RBD
snapshot mirroring from the dashboard with the RGW we introduced the RGW server-side encryption
in the dashboard and also with the reef release we are trying to introduce more features for
more RGW features like the the multi-site and the some users or role role policies so those
stuffs are still in working progress we also have a cool new feature called centralized logging so
with just one view you will be able to see all the safe logs or container logs like the manager
mones or whatever you know component or container you have set up also we also tried to achieve
WCAG level AA standardization standardization with the accessibility so these are some of
the goals that are beyond a reef so we are trying to we are constantly trying to improve the
usability experience so that's like a that's like a constant efforts from the dashboard side so we
also something called the the low code initiative which we will talk about later on in the end of
the session so we are trying to replace the Grafana with some built-in charts you know so
instead of you know consuming instead of populating the charts with the Grafana using the iframe
components we are trying to directly populate the charts using some charts as like frameworks we
also have a plan to improve or improve the surface integration in the dashboard currently we have
all the monitoring pieces settled in the dashboard but we still lack some of the management
activities on this FFS area so we will focus on that later on and then also the multi cluster
monitoring and management that's like a that's also an important feature that we are planning to
deliver from the dashboard and also trying to achieve the feature parity with the CLIs so I will
try to show a quick safe dashboard demo now so here is the login page of the safe dashboard and if
you log in you will be greeted with the landing page or the dashboard page where you can see or
view all the status or the information regarding your cluster you have the cluster status which
is showing a warning for me you can click that and see why the cluster is in a one state and
all other information regarding the cluster can be seen here I have like two hw's daemon's setup
and all other informations then if you go to the cluster host you can do all the host management
activities then we have the inventory or the physical disks part also you can see you know all
the OSD's attached to the each of these devices you can identify the device by blinking the LEDs
the monitors the all the services that are available right now in the cluster you can create more
services and edit or delete these services also you can you can start or restart the daemon's if
you go into the each of the services details there is also service events the daemon events are
also there in the OSD's you can create you can create OSD's or you can you know we also have
this new OSD creation wizard which has like some pre-optimized deployment options there or you
can also go with the advanced mode and create advanced you know create a tribe group specification
of your own also there are different set of activities that can be performed on which of
these OSD's like you know same thing with flags or deep scrubbing marking an OSD out parsing or
eventually destroying the OSD's then the cluster byte configuration like you know you can set flags
out through to all the OSD's at once if you select one of the options here the recovery
priority pg scrub and if you go to the overall performance you can see the overall performance
of the OSD's which is you know code from the graphana itself so this is a graphana chat which
is integrated into the dashboard using an iframe component and we have all the configuration listed
here the crush map viewer for seeing the crush map tree then the manager modules we can start or
restart you know you can enable or disable a manager module from here um this is the cfx
auth users management so right now we have the we support listing the users and you know creating
a cf user in the logs we have the cluster logs and the audit logs then we have the daemon logs
which is the centralized logging that I was talking about before so in the centralized logging bus
you can browse through the logs and if you click on one of the log here and if you click on one
of the container here let's say I want to see the manager module manager logs and if I put show logs
it will display the logs from the manager of the safe node 00 you can also put this in a live
live okay so you can also put this live and you can see the logs populating in a live logs
populating lively so right now it's paused and we have the monitoring this is kept you know this
is directly consumed from the prometheus and the alert manager so if there are any alerts in the
cluster it will show up here and it will show some indicator in the navigation pane itself
then we have the pool management again all the all kinds of pool management like pool creation
edit delete the overall performance you can see all the pool performance in the overall performance
and if you want to see individual performance as well you can see the performance details of each
of the pool individually then we have the block the RBD section so if you have no RBD
pools available it will show this message and if you want to create an RBD pool you can go
and create an RBD pool by selecting the RBD application from here then we have the RBD
mirroring right now it's not configured on my cluster so if I want to configure it you can
click on this configure RBD mirroring and this will configure the the RBD mirroring and the
iskasi section as well the NFS management so you can go and create an NFS export so if you
don't have an NFS cluster created so it will show this info to create it or add an add a new NFS
service we have the file systems again I don't have a file systems setup right now so we have
an AGW section as well so if you go to the AGW daemons you can see all the daemons that are
available in the cluster the overall performance of each of the daemon you can create and delete
users and edit the users the buckets creation or the bucket management section over here dashboard
also has the REST API which you can see if you click the API you will get redirected to the
CIFREST API then we have the whole notification system here you can see all the notifications
coming from different components of the CIFREST at one place you can also report an issue from
the CIFREST dashboard so it will directly go to the tracker.CIFREST.com so if you you have to
enable the feedback module you can go and enable it from here so once you do that you and you
provide this you are a CIFREST tracker API key and you give all this information then you submit
you submit that it will create an issue in the in the tracker.CIFREST.com then you have the user
management the telemetry configuration and yeah that's more or less about the CIFREST dashboard
yeah so that's all from me from my part I think the second part will be covered by Ankush.
Hi and hello everyone thanks for the wonderful demo and the part one presentation let me
introduce myself my name is Ankush I will be working as an engineering manager in IBM team
I have expertise of more than seven years in delivering the management and monitoring solutions
for a software-defined storage on a containerized or a non-containerized environment in an open
source system so today I'll be taking you to the part two of this discussion where we'll be
discussing two major aspect first of them will be how you can contribute to the CIFREST dashboard
and the second one will be the CIFREST dashboard community and how it looks like moving on I
think discussing how you can contribute as a CIFREST in the CIFREST dashboard as a user we'll
be talking about as a translator or a documenter and also as a developer we'll be discussing how
you can contribute first as a user right CIFREST dashboard is by default enabled when you install
CIFREST at the CIFADM and you can use it from the get go but if it is somehow not enabled or if
you are facing some issues while enabling the CIFREST dashboard you can follow these steps
that are mentioned over here or you can follow the documentation link once the dashboard is
enabled you will see this is the first screen that you'll see as a user you need to login and
once you are logged in you will be able to do a couple of things from the CIFREST dashboard
from the management of the whole CIFREST ecosystem to monitoring from alerting logging and all those
things you can do on the CIFREST dashboard but somehow if you if you if you see any issues or
if you see if you have any suggestions for any of the things that you tried out or if you have
any feature requests that you want to put in you can also go into a report report an issue in
dashboard itself and you can report that issue and it will open a bug in a CIF tracker where we
will be following up with you also you can directly share experience on the CIF users mailing list
that is mentioned over there or at the IRC channels that is CIF dashboard you can reach out to us
and we'll be happy to help you out so second part of that is how you can contribute to CIF dashboard
as a documentor so this is the documentation that link that we have and if you see any issues or
if you have any and think that you want to suggest you can directly go and report a documentation
bug from the documentation link and or if you want to do a edit yourself and submit a pull
request we'll be happy to help you out on that you can click on this edit on GitHub link directly
and then you can submit a pull request and the team will get back to you on the reviews on that
pull request as a translator like what you can how you can contribute to CIF dashboard is you can
follow you can look into this link where we already have a lot of translation done in the CIF
dashboard side I think more than 10 plus languages are already there but we still have some gaps
that can be filled so you can look into that link and help us out on the localization and
internationalization of the CIF dashboard moving on as a developer how you can contribute to the
CIF dashboard so firstly what you can do is you can subscribe to this dev at the CIF.io mailing
list second is the IRC channel that we already talked about which is CIF dashboard we also have
some documentation link very generic to CIF but also to the dashboard you can follow to set up
your dev environment and play around with the CIF dashboard moving on and let's discuss how the
code looks like right so we'll talk about two major things over here one is the backend one is
front end on the backend side we use pythons 3.6 and on the front end side we use Angular 12 with
TypeScript plus bootstrap 4 but we are planning to upgrade it to bootstrap 5 and also we are
planning to upgrade the Angular version to adopt the new features as a developer right there is
another initiative that is taken by the CIF dashboard team where we understand that not
everybody knows how to code in the front end languages like HTML JavaScript TypeScript CSS
and all of that ecosystem and but what we have done is that even if you don't know that you can
still contribute to CIF dashboard using a low code initiative where you can write your descriptors
and this JSON kind of a format with that will create a route for some page and then in a Python
code you can write how your page should look like and then both together can work and generate a
page something like this for you where you will have a UI generated from a Python code itself I
think we have the first of our kind feature that is already written and this is a CIF
auth management but I think in the next release we have couple more features coming in which is
following the low code initiative and it is being and it is being discussed in the in the community
right now I think the second part of this part to presentation was dashboard and and the community
around it so we'll let's talk about dashboard in number of pool request we have 2700 pool
requests that are there in dashboard we have 4200 comments and these many lines of code most of
them are written in TypeScript but second prominent pre-dominant language is Python and the last
but not the least is html we have behind the CIF dashboard there are a lot of great minds from
different continent different countries that contribute to CIF dashboard and these are all
majorly I think this called all the people that are working from different countries and continents
and helping us grow moving on as you have seen that we have a we have a big community spread
across the world time zone and all that stuff so we have tried to come up with something that is
that works for everybody but can be difficult for somebody at some time so we have a daily
stand-up at 11 am CET time or upstream wide audience think that happens or Tuesday for
tonight at 2 pm CET and the face-to-face used to happen pre-pandemic times for almost three
four days at different locations it has happened not happened after after the covid pandemic now
we'll see some glimpse of how our face-to-face look like this is the first one that happened in
Newenburg and and all the community members are present most of the community members are present
over here and these are some of the images from that there was another step dashboard of orchestrator
face-to-face that happened in Berlin coral surface and these are images from there and the second
safe dashboard specific face-to-face that happened in June 2019 in Fulda and this these are images
from there the upstream safe dashboard community is doing some team building activities and and
enjoying work together I think that's all for today from my side for the part two presentation
we'll be happy to take any questions if you have otherwise thank you everyone for joining this
meeting and also this presentation