All right. Good morning, everyone. Welcome to the HPC Dev Room. Thanks for being here
so early in the morning, maybe not entirely sober. We'll let Nicolas get started with
opening a Dev Room with a talk on paraffin. Thank you, Nicolas.
Hello, everyone. Thanks to be here early in the morning to begin this HPC day. I trust
them. A big thanks to everyone who organized this room. That's really great to be here.
Thanks, Kenneth, on all your team. So, about me, I'm Nicolas Vieille. I'm a
C++ developer, and I have the chance to make my job about making first-code
contributions. I'm working at Kitware Europe, and I work mainly on ParaView,
so developing the software, but also interacting with the community. So, as my
may want to reach me later. So, ParaView, it's an end-user applications that work
for scientific data analysis and visualizations. We have an open community
on the GitLab for the code and discourse for discussions. It's supported by
Kitware, which is also behind VTK for visualization toolkits, and to make you
potentially already know about. So, what do we do with ParaView? It's for
displaying and analyzing scientific data sets. So, it's mainly a 3D visualizer, but
you've got also some charts and spreadsheets and so on. It's also intended for
data processing. So, we have a concept of filters to take an input and compute
some stuff on it and get your outputs. So, basically, you can extract the data
of interest of your raw data. We also have some other module like realistic
rendering. So, you can do your communications with directly your real
data sets or not with just some kind of fake one. So, here is some basic
screenshot of the applications. Who is using ParaView? We cover its generic
application. So, we cover a large range of domains like fluid dynamics. So, we can
compute streamlines, particle tracking, and so on. We have also volume rendering
that is real nice for our medical applications with you have some 3D scan
and you want to understand what happened inside. So, that's just, that's
non-finite list. We have a lot of domains that can be covered, but here is the more
well-known one. Oh, we do use ParaView. So, as I said, that's an application. So,
basically, the first way to learn is to use the GUI. So, you click on buttons, you
do some stuff, and you're happy. But you can also use the Python wrapping to
write some scripts. And so, you can run the script on that processing without
having to be behind the computer. It has a framework because you can code your
own extensions, your own derivative work from ParaView in the native
simplest language, but also some features can be done in Python code. And it's
all based on the visualization tool kit. I mean, all the hard work of processing
the data and do the rendering come from VTK. So, as I said, that's also supported
by Kitware. So, I do work sometimes on VTK to have some bug fix or some small new
features. Where we can run ParaView, on which hardware is it possible to use it?
Basically, on your small, classical, big stop, we have some official binaries you
can try to download and just run. It should be, it's cross-platform, I didn't say,
but you can run it on, as well, on proprietary software like Mac or Windows,
but it should be out of the box for Linux 2. You can also build it. We have a large
selection of build options, depending on what you want to do exactly. You can
enable or disable it. So, if you want to have Python data distribution,
parallelizations, or custom rendering, a lot of stuff. We have some documentation
about it, and we can help you on the discourse if you have to try to achieve
some specific build of it. And which kind of usage do we have of ParaView?
So, either a research and industry are using it. For instance, recently, there
were the Super... So, before winter, a supercomputing conference in the US, and
they organized a service contest where people are asked to upload some nice
videos made about their scientific analysis, and most of them are using
ParaView, either for just the data processing, but also sometimes for the
video generations, and animate their data. So, why all those people are using
ParaView? Because ParaView does some stuff efficiently to process the data
and their infrastructures. So, that's what I want to talk in the next part of
my... of my talk. So, what's ParaView used behind the hood to make it
possible? So, first, we have a client-server architecture. So, I always say
that you can do it from Python, but the degree on Python are just two
clients. So, you can do exactly the same stuff with one or the other. There's no
real limitation about using one or the other second. You can also run with
remote server, so, and you can run in a distributed environment your server. So,
in that case, you can connect your... you can either just run the server parts to
analysis with a script analysis, but you can also connect your local clients to
your distance server, and again, using the graphical interface to do the stuff
as if it was on your local machine, but instead, it's... yeah, the
supercomputer or the remote architecture. At the bottom, two other
modes that are available. If you... typically, if you have some graphic nodes on
your server, you can use them for just the rendering part and stay the data
management on the CPU nodes, and still, you can connect your client on it to see
what happened and to control from a graphical interface. And last mode, I will
go back on it later. We have an institute infrastructure, so, basically, your
simulation can call an IPI that's Fluid Paraview script analysis, and you can
even connect with a graphical client to see time step per time step what is
happening on your simulation. So, that's for the different mode of use for
Paraview. So, first, to run on HP infrastructure, we
implement data distributions for the analysis. So, basically, we rely on the
MPI standards. So, our readers are MPI aware, so they can distribute the data
over the rank early in the process when you read your data on the disk. Then,
most of the filters are okay to run just with their support of data, but some
other filters need to know about the neighborhood to execute correctly. So, for
that, we support the concept of ghost cells, where each rank knows a little bit
about the rank that is next to it. In that case, mainly, we split the data
geometrically. So, a subset is really a geometric subset of your data, and
different one can know and communicate with the other for specific tasks. At
least, we have some filters. So, what I call a filter is really something that
the user can instantiate from the client and ask to process. So, we can
ensure a load balancing by redistributing the data during the process. The
visualizations can also be distributed over several ranks. So, for that, we use
an inner library that call IST, that's also based on the MPI process to do that,
and parallel view support has a different kind of model of rendering. So, you
can, if you have dedicated rendering node, you can, as I said, create parallel
view server just for the rendering part and connect it to the data server. You
can have multiple GPUs per rank, yeah, multiple GPUs per rank to do the
rendering, but that's also possible locally if you have just one machine that
have multiple GPUs, you can ask to do a rendering on both simil-tune-y.
Concerning the performances now, so that distributions is not about
performance, it's just about running with too big data so you cannot just run on
your machine, that's a requirement when you have huge data to be able to
distribute it over your computer or your supercomputer. Now, we're talking a
little bit about performance because if you have big data, you also need to be
performant on how you analyze it, on how you are proceed with it. So, for that, we
have a thin layer for CPU parallelism, we call that a simple tool in our code
base. The goal is to parallelize, do code parallelizations for many for loop, and
main purpose is that you can choose at build time and then at run time, if you
enable the OpenMP or TBB backends, and if you don't want external live, you can
also use the C++ thread to do that. And so, as it just, for instance, to
parallelize a for loop or field operations, it's really widely used in a lot
of our, in our algorithm, and you have some environment variable that can
control the back end on some of the number of, of thread, the size of the
thread pools, or if you allow nested pools or so on, depending on the, on your
resources and back end. So, it has some documentation on it, and we made some
improvements last year about that. Still, still about performances, we also use as
an optional dependency, the VTKM, VTKM projects that stand for, yes, somewhat
some many core that is intended to be used on heterogeneous systems. So, basically,
when you want to have performance on supercomputer or even, you, you still need
to be aware of the current technology and the state of the art, and as we saw in
the past decades, a lot of new architectures emerging. We, we think about
using a dedicated library to, to be able to use this, this new architecture. So,
with VTKM, the goal, inside VTKM library, the goal is to split all
operations into really atomic operations, and then the, the, at runtime, it can
dispatch all, all that on the hardware you find on the back end that are
available. So, with VTKM, you can use CUDA, OpenMP, TBB also, to do the
computation. This time, with VTKM is not just accelerating some specific loop
inside an algorithm, it's more about VTKM is implementing some whole algorithm
like extracting ISO control or, or so. And then, we embed this into, into
Paraview with some kind of wrapper to, to communicate with all VTKM works. So,
that's optional, that's enabled by default in the binaries we, we provide.
Another point about performance, but that's really depending on the use case,
on, on the data you are using is the in-situ wall. So, basically, when you,
traditionally, when you have your simulation, it dumps every time step or
every end time step some data on the disk. And then, to analyze, you have to load
back to the data with post-processing tools. But that adds a cost of writing
and reading from your disk. And you, you should have the size on your disk, the
whole size of the disk. So, you should have big disk. And then, it's, it have
really a cost in term of time, when you should write a full, a full mesh or full
data on disk. And then, read back with another process. So, basically, the goal
of in-situ is to, to make the simulation communicate directly with the
processing tools. And then, the processing tools can wrap the memory in place and
analyze directly in, in the RAM without writing on the disk to save some
higher time. So, in the context of ParaView, we have a standalone API that's
called Catalysts. That was recently released, as we make big improvements into
Catalysts past years. And the goal of Catalysts is to have a really minimal API
and stable API. So, you can choose and run time the implementation you, you want.
And one other goal is to minimize the instrumentation you need to do in your
simulation code directly. So, it's really easy for simulation developer to
understand the few key places where they have to put a new code to call our
API. So, here is a really basic example from one on the tutorial we have. We need
to initialize, of course, and you need to call some method at each time step
where you want the processing to happen. And finalizations. Of course, the, you
still have to do a little layer to describe your, your data. For that, we
do some sort of a partial library to, to help us. So, ParaView, so Catalyst is a
standard, I say standalone, is not, is independent project, no, independent of
ParaView. But of course, the first real implementation is an implementation for
ParaView. So, we, sorry. So, yes, we, we implement Catalyst. So, the back end, so
you can run ParaView pipeline directly from your simulation. It's each time
step or when, when you call it. So, how does it, how does it work? Or do you, you
can, the idea is that you are, are called the communication between your
simulation and the, on Catalyst through the API. But then the actual script that
is executed, the actual pipeline and visualization, visualization you want to
produce. It's all scriptable thanks to the Python wrapping of, of ParaView. You
can even, you can even, sorry, load some representative data in the graphical
interface of ParaView. Do some analysis, export this as a Python script and use
the script to feed Catalyst. And then, when you run your simulation with
Catalyst enable, it will reuse the script you produce directly from the GUI. So,
people that are not at all developers still can do some stuff with, with
Catalyst. And last point is that, when you have a running simulation with the
Catalyst pipeline on your dedicated server, you also can use the GUI to
connect to this ParaView server and to see real-time get some screenshots of the
visualization on the analysis that is proceeding on the server. So, you can have
a feedback, a time step per test, a time step on what happened on the simulation.
So, if you see that something is diverging or going wrong, you can stop
your simulation directly and you don't waste all the time before seeing that
something went wrong and that you should tweak the parameter and start again.
So, yeah, I was quite faster than expected for me. So, in the conclusions of
to, to be able to run efficiently on the, on the supercomputer with ParaView, we
implemented a client-server mode. The server can be, is MPIO rare and can be run
on distributed environments. We are relying on old, on well-known libraries
such as implementation of MPI to the distributions, but we are also really
looking for, toward new, new library that can help us. Yeah, and we, we are able to
integrate new library to, to do some performance analysis on new library that
is aware of a new architecture of supercomputer or new technology. That's
okay with, for instance, with VTKM or, or others. And we have this API to do
institute that can save a lot of time and disk space. Yeah, just a slide to
summarize the organize. So, we have different kind of way to interact with
ParaView. Yeah, the grid, the Python scripting, the catalyst in city stuff. You
can also build some custom one. We have some web example of clients. And at the
bottom, we have a list, a non-finite list of library on which we will like to, to,
to do the effective work. So, basically, open GL, MPI, open, open MP. And so,
concerning roadmap, we have several improvements that are coming. First, I
talk about in, in situ, in the current implementation, you have each rank that
does the simulation, does also the, the co-processing work. So, that's not always
what is intended. Sometimes, you want to do the co-processing on the other rank.
Just because, for instance, you have dedicated the rank for visualization. So,
you want to do all the processing on the visualization nodes. That's for not
possible just with the in situ implementation, but we have an in-transit
implementation where the simulation can communicate with those different nodes.
And, and the analysis can happen on other ranks than the simulation. So, the
simulation can go forward directly. We use, we also use some new library,
recently used a library called DIY. That's here to do some wrapper for us. It's, we
take it as a wrapper around the MPI. So, DIY, I love to do some to, to cut the
data into different blocks. And then, the, at runtime DIY itself is a rare to do.
Okay, I should put three blocks on each rank. So, only one block. And, yeah, it's a,
yeah, just a new abstraction over cutting your, your data for distribution. We
are also looking for better VTK on always, yeah, better VTK integrations to, to be
able to, to run on a lot of hardware. And something very cool that is very new. It's
just in the development branch of VTK. So, absolutely not in Paraview yet. That was
merged, I think, one or two weeks ago. It's what we call implicit arrays. And,
basically, it's really cool for memory point of view because we, it's some kind
of views on memory. For now, in the Paraview process, your data is really an
array in the, in the memory, in your memory. So, with the implicit array, we have
some views. So, you can implement an open, open pattern on it. For instance, when
you do an isocontrol of your data, you know that the, the, that, the resulting
data will all have the same values. So, if you want, if you, after the isocontrol,
you still have one million points, you will know that all the points will share
the same value. For now, it's one million times duplicate in your memory. So, that's
a not-efficient. With implicit array, you can sort only one time the value and say,
okay, this should, this should be an array of size one million. And the value you
should return is this one. But you can imagine as a, a compressed array in your
memory and have an on-the-fly, uncompressed algorithm to when you just want, just
when you want to, to read your data. So, it has a cost in terms of time of
computations. But if you run out of memory with too huge data, that's, that can be
really great. Okay, I still can have a lot of things to, to say, but that's what
is the, the end of what I, I put in the slides. So, thanks for attending these
songs to be here early in the morning. And if you have any questions, I think it
will, it will be the time. I put just a lot of resources at the end of the
slides so you can get it from the website of the phone. Thank you.
Thanks, everyone.
Thank you very much, Nikol. Do we have any questions? Thank you. In our group, we
are happy users of ParaView. One thing that maybe I could add to a wishlist or
some, well, maybe just for discussion is that we have quite some headache when
using ParaView on GitHub Actions for multiple platforms. So, like to set up
environments for Linux, Mac and Windows with the same version of ParaView
coupling with Python just, just to be ready to use it. It's a bit of a headache,
especially when you go to Windows and you need to download things, brew things, up
to get install things and then they not necessarily work all together. So, wishlist
thing, GitHub Actions, ParaView, set up a thing. Unless it doesn't, it exists
already but I haven't found it. And the truth is, if there are questions here?
The use of ParaView and GitHub Actions, so in like a continuous integration,
limited environment, I guess? Yeah, it's a wishlist. Yeah, well, we don't
choose GitHub directly. We have a, we have a, the GitLab where you can find a lot
of stuff with our CI and CDO. We produce, we produce nightly releases of
ParaView through the GitLab. So, I don't know if I, if it's some sort of part of the question, but.
So, what kind of stuff are you doing with ParaView and GitHub Actions? Is it
rendering or, rendering with Python? The fact is, I don't really know about GitHub
Actions because I don't choose GitHub anymore. So, I don't see what you can do
with that, that you should not able to do otherwise. Any other questions?
There's a question on the chat. Okay. Yeah, there's a question on the chat. If I want to put
Catalyst in my simulation, what is the first step? Oh, sorry. If you want to use
Catalyst. In your, yeah. What's the first step? We have some, I think we have some
tutorials on example in the code base of ParaView. We have some examples where
there are some dummy simulations with just a main, so you can
enter from it to, to see how it is organized. And, and yeah, the first, one
first thing is to be able to know what do you want, which data do you want to, to
send through, through Catalyst and where you can access it in your code. And then,
it's, and then so you, at this time you, you have located the entry points from
your simulation code and then you will be able to, to start writing the, the small
wrapper you need to wrap your data on the need to the actual API of ParaView.
Thanks. Okay. Any other burning questions? Maybe one last, yeah. Last question?
Thank you for the talk. A very naive question, because I, is it working? A very
naive question because I know almost nothing about, about ParaView. You had
many components there. One of them was the client that does the
visualizations. Yeah. Is it, would it be possible at some point in the future to
be like a web client where you just log into the website and it just displays
everything? Or is it just, due to the architecture is it like super complicated
to do it that way? We, so the question is, yeah, the question is, are we able to use
a web client for ParaView? Just, just for the part that does the visual
ization, if that could be like a, we are, we have some web client for ParaView
already. So we have a framework called Trame, T-R-R-M-E, that's intended to, to
connect to a ParaView server. And then you build your own front end for
these applications. So basically it's, we don't have a, yeah, you should build your
own. But it can be, okay, I have a server on, I open the, always this data and the
front end is just a 3D round of view. That's already possible quite easily, I
think, with the Trame framework. And Jupyter Notebooks also, right?
Jupyter Notebooks, I think I saw it on the user interface line.
Yeah. Well, we are, yeah, as we use intensively Python, we also make the
step to, to be supported from a Jupyter Notebook and we also have a plugin that
allows you to control a ParaView GUI. So you can do some stuff in the Notebook.
And if something goes wrong or you don't understand, you can launch a magic
command run ParaView that's open the ParaView client with all your Python,
Python, and you can introspect in the GUI. And then you can go back to your, to
your Notebook. Okay. Thank you very much, Nicholas.
Thanks.
Thank you very much.