Okay, welcome everyone, as you can see from Adam, typical notebooks are a very important
tool in each data scientist, but using graphs, refer to notebook as a challenge, for instance
visualization.
And so Bjornville talked about, I pie Zygma today, which is a tool to use ZygmaJas as
a component in a Jupyter notebook.
So I'm really looking forward to that and without further ado.
Where I'm from this time, I'm actually not Guillaume, I'm sick.
So I apologize in advance, because I'm not the creator of the tool and so I will do as
much as I can to present it, but Guillaume can answer that by email or by Twitter or any
other means if you have more questions than what I can actually answer myself.
And so I will just start by a brief remember of why we sometimes want to use graphs and
actually visualize them and not only do statistics on notebooks and actually visualize graphs.
And so why do we do visual network analysis?
It actually goes back very old to 1736 and the Bridges of Collector, which is a classical
mathematical problem that was solved thanks to visualizing the graph that it was showing.
Later on in France, Moreno did a social graph where he tried to visualize how a connective
where students in a classroom.
And recently, thanks to the community assisting tools, we can do those kind of visualizations
but with massive graphs and we can try to do a computed processing to try and automatically
specialize nodes on the map and on the plane and also identify clusters within it.
So that brings a different mean to actually analyze graphs and actually visualize this
helps a lot understanding.
And we are coming from the field of social sciences and we use a lot of graphs to interpret
social issues in general.
And we use them actually as maps.
So it's not maps in which coordinates make sense.
X and Y don't mean anything.
You can just take the map.
But basically what you see on the plane indicates information on the, I mean the localization
of each node that makes a sense compared to the other nodes.
So, but I guess most of them are not.
That's another example of a map that was made a long time ago.
So to do that, there has been over the past years a lot of tools that have been developed
including the first desktop ones.
So this tool is the direct heritage of this long lineage which started with Gezi.
I believe later today there will be a presentation of Gezi version 1 which finally will go out
soon after so many versions already.
So you probably already know Gezi.
But recently we could switch from the actual desktop analysis to actual web representations
thanks to a variety of libraries.
D3.js proposes to do it.
But there's also a site escape and a bunch of others.
But so our community works with Simba.
And Simba has been developed by people who are actually close to the people of Gezi.
I don't think Alexi is here today.
But Alexi Jacomi is the small brother of Mathieu Jacomi who speaks about Gezi.
He's the one who invented Simba and Guillaume is the co-host of Simba with Alexi.
So please take a look at Simba.
I will put the slides to the conference and you will find links to all the tools around.
And then thanks to Simba, we could build a lot of Gezi-like tools but for the web.
So that we could do all those interactions that we do directly in a web page.
There's been a long history at Miguelab and around of trying to build such tools.
Minivan was one of them.
There's also Nancy which is a very small, very publishing-oriented way of displaying a graph
with very few options so that you can just put your Gezi-like search file or GraphML file
and very easily do what you do on your Gezi.
Retina is the one developed by people at Westware right now and is very rich,
proposing a lot of features.
And soon I think Mathieu and Mathieu will talk about it briefly also in the later talk about Gezi version 1.
There's a Gezi-like version that's currently being developed and that should come in soon.
Which brings me to all of those tools are very nice.
We have all those that are interactive and you can visualize, explore, publish,
manipulate all those graphs but they all require pre-processed graphs.
You cannot just work with your graph while you're visualizing it.
You have to pre-code in your file, usually JSON or JXF or GraphML,
then you load it into the tool and then you can explore it.
But we would like to be able to do that at the same time.
And so that's where the idea of I by Sigma came from,
to try and put within Jupyter a notebook, a widget that would display the graph using SigmaJS.
So it's really easy to install as long as you have Jupyter.
You usually need a tool to work with graphs under Python.
There's two main ones that you might know already about,
I by Sigma is built to handle both formats of graphs from both networks and I by Sigma.
And so you just install I by Sigma in addition,
and then I just switch to the brief demo.
Maybe at the seat.
So we'll do two small explorations of graphs.
There's the first one that we're working on right now,
which is on the, what I call the open source,
I mean, actually larger than that, open access, open world.
It's like for them, but just in France and the French communities working on that.
And so we built this network of websites, links together of those French communities of free software.
And let's take a look at it as well.
So first, I will import the projects.
Then I'm reading the graph that I built already.
So that's all this first example.
So here we have a graph with 621 nodes and 7000 edges.
Let's look at the node.
So the first node, I don't see that information.
It's April.org.
I don't know if the French people are in the room,
but people should know that April is from France.
It's the main NGO in France about open source and just the graph.
So we have this whole page.
That's all the data that was collected while making the graph.
And then let's try to just visualize it by just loading i5 sigma,
importing sigma and applying it to the graph.
Here, just by the widget with the graph, which is randomly specialized.
We have metadata information.
So we can run for FATAS on it.
So very easy.
You see your specialized graph.
Just a few seconds, and then we can also apply some...
The graph is too dense for that.
And suddenly no effect.
So yeah, but right now it's just a graph
and we don't have much information.
It's very complicated readable.
So let's go down and try to add a few other options to the sigma code.
So we can set the outside standards.
Let's use the number of pages for.
So here I can see that for this graph use,
we put a lot of pages on some specific websites.
Let's put a little bit more and try to adjust the sizes of the nodes.
So we can adjust the range of the values for instance.
Here it's really readable.
Okay, so we got sizes.
Let's add some colors.
So iBuySigma proposes some internal metrics that you can compute on the fly.
So for instance, as a result, it generates clusters.
And we will apply colors to the map.
I mean we will apply those color communities as colors.
So here we get a set graph of colors.
Let's see that there are a lot of communities.
As knowing this network and knowing this community,
I can tell you that basically what this is.
Here we got the open data, open command community.
Here we got uphill and basically the NGOs working on the open source.
Here we got GIL and it's mostly a lot of softwares.
Fedora and all the Linux distributions.
Here we got FFDL, MaproductionDenets,
and all those activists working with the open internet.
And I guess here is more the...
Oh, it's also a mobilization.
It's a form of formigated old form of food.
I'll just speak a little bit.
Okay, so now that we got this, let's try to make it a little bit nicer.
We can add, for instance, some border colors.
So it just proposes to see a stronger border of colors.
Graphs are a little bit sexier.
We can also try to do like Gephi, like curled edges.
All of those are in options.
I guess I'll show you briefly later on a list of the different options.
Here we also put the recursive font to the level.
So basically you can do a lot of things.
But all of that so far is mostly like Gephi.
There's no real new thing.
But here's something that actually proposes something else.
So right now we can see one graph.
But let's try and see multiple ones.
So I buy similar properties, what we call a similar grid.
And so I will put the same graph, but it will trickle out.
And those will be common options that I set for all versions of the graph.
But then, within the grid, I will add three different versions of the graph
using different metrics for the size of the load.
So here's one on the left one.
And we see it's on the middle of the degree.
And the right one on the bottom.
Now I'm going to add this.
So here are the three graphs, which are all synchronized.
If I visualize it, it happens at the same time.
If I over-enode, I will see it on the three different versions.
And then if I zoom a little bit, I guess we can see that...
Wow.
What can we see?
We can see that PharmaSoft, for instance, is very connected most globally,
but especially it has a very strong in-degree and not so big out-degree.
Why is that?
PharmaSoft is such a reference in France for open-source tools
that it gets a lot of links from the whole community.
And all websites of the Free Software community point to it,
because it's like a resource.
Whereas, of course, they cannot point to the whole rest of the community.
On another note, I guess we could find...
I think there was Linux...
Linux-affair.org is the opposite.
It's a media that pretty much talks about anything that happens on open-source in France.
And, of course, they're the ones having the most outlinks.
All right, so that's just a small example.
Then I can show you maybe another notebook that will show other things.
So this one is a notebook that was built out of data
collected by Laura Miguel, which is a trainee at Media Lab right now.
And she scraped the first-day website, the agendas,
to try and get all speakers and rooms over the past 15 years.
So here we will have to build the graph progressively.
We just had a CSV that she scraped of the data of one speaker and one room.
Disclaimer, the speakers have been anonymized.
So you won't find a name that you know about,
but they represent actual people.
So let's take a look at, for instance, three examples of the data.
So those are the three first lines.
I mean, that's one line and two other lines that I picked specifically.
This one is one speaker, and she talked about within this track.
Here it was a stock that was shared between two speakers.
So sometimes we get speakers separated by a pipe.
And here is obviously someone that was still anonymized,
but that should be in my seat right now.
And we did many talks in the past, including in this room.
So we will build the graph using NetworkX.
So for those who know NetworkX, it's quite simple.
You just create a new graph, and then for each row in our CSV,
we will, if there's no speaker, we don't take it.
Then we take the track and the year.
We add a node for each track, and for each speaker inside the talk,
we add a node for the speaker.
And then we had an edge in between those two,
and we increment it as a count if it's the second time that we meet him, for instance.
And we also upgrade the year to get, for the edge,
the last year that was used.
So by doing that, I built a new graph that has 5,000 nodes and 6,000 links.
Let's take a look at my alternate speaker here.
It was supposed to be a speaker, and apparently,
so it's linked to, in year 2018, to two talks in the graph room.
Yeah, he spoke twice in the room, back then.
In JavaScript in 2019, and in 2020 in the Open Research Tools and Technology Room.
So let's take a look at this graph now.
Oh, it was broken.
Yes, there's a comma missing here.
Here it is.
Still, I tried to add this earlier, but I'm not expert enough with it, so I'll remove this.
So here it is.
So this time, it's a bipartite graph, since we got two kinds of nodes,
the tracks and the speakers.
So I decided that the node color will be attached to the part type.
And if I take a look at it, we should see all big dots in blue are the rooms at first then,
and all pink ones are actually speakers.
And so we can see that there are a lot of lightning talks, of course, every year,
but there are some rooms that have way more speakers than others,
probably also because they exist for way longer.
So maybe we can try and explore that, and that's the main idea.
So sorry, I don't remember what this one is.
Let's just run it briefly.
I guess it's the same.
Yeah, it's the same.
Sorry, it's a copy-paste.
So what we could do is try and apply other things.
So let's do a grid again.
But this time, we'll try and display for each node a gradient of color.
That will indicate the intensity of the node at this moment.
So to do that, we will, for instance, take a look at the year 2012 and the year 2022,
and use the strength of the ALO, depending on how many talks were associated
to this node for this specific year.
So both graphs should show the intensity of the talks during those two years.
So let's show it again.
And here we can see that in 2012, the main rooms that were filled were actually more
on desktops, Mozilla, Lightning Talks, and Embedded,
whereas in 2022, there are way more rooms that are actually filled and spoken.
Then what we could do is continue working on our graph
and continue exploring while working with it.
So at Medialab, we also have a tool called Pelot,
which allows us to do a bunch of metrics and calculation on a,
so it's already installed, it's going faster.
And for instance, it can do a monopartite projection out of a bipartite graph.
So I'm just running this, and then we can try and display it.
And here, just in a few lines in Python, I can just see the alternate graph
that is the monopartite version of the graph,
and just see the links between the rooms depending on when they are co-spoken by speakers.
Let's continue.
And the problem is that if I look at this graph, I can see there are a bunch of nodes isolated.
And so usually when I want to visualize a graph, those are a bit annoying
because they take a lot of space in the visualization, and I don't want to see that.
So let's just use Pelot's Crop to Largest Component function
that will keep only the biggest component of the graph.
So then I can re-displace this graph without all of those single nodes.
And that's a rough idea of what could be done.
Then we can work with the graph and just visualize on the fly.
And I guess I'll conclude by just showing inside the GitHub page of the tool.
There's all the visual variables that are available.
So I showed you already node color, but you can also play on the saturation of the nodes.
You can play on the size we saw, but you can play with the label size, of course, the label color.
You can adjust the border ratio, how big it is.
So basically all visual ways to better help you interpret your graph can be proposed.
You can also add pictograms, use shapes for each node.
You can use halos like I showed earlier.
And play also a lot of those applied to edges.
So you can play on the colors, the form of them, and so on and so on.
And I guess that will be it for me.
And I will take all of your questions.
Sorry, I'm just scrolling back to things that are nicer.
All of you.
Yes.
Can you preserve the layout between the different steps so you can execute the layout every time you go to a new cell and preserve it?
That's a good question.
I don't think it has been planned yet.
Can you repeat the question?
Yes, sorry.
So the question was can we maintain the layout from one cell to the other and not having to re-click to apply the layout every time?
I don't think so.
And what I know is that the layout, the way Forza class works, has some chaos.
But here it's always instantiated on the same seed.
So whenever you run it, it will always generate the same exact layout.
So that's something.
But it won't reuse the one from the previous cells.
No.
That could be something that could be an idea.
Yes?
Do you have any numbers on the upper limits of this system?
And the size of the graph that you're going to run here?
So can you run the values of noting this one or the limits?
So the question is about volume and amplitude and how big of a graph we can display with this.
So I believe the limit is actually the one of your browser.
So it will depend on your GPU and your CPU and your RAM.
But I know that SigmaJS properly endows graphs with, I would say, 100,000 of nodes and links.
I guess I know I already displayed one with a few million links and 100,000 of nodes.
It takes a bit more time, of course.
Do you support something like collapsing nodes and expanding them?
For instance, in these kind of power graphs where the communities could collapse if you want to put height once,
and they could be selectively expanded as well.
So the question is, can we aggregate and split nodes that have, for instance, the same group?
For me, it would be, I don't think it's built-in within Sigma for sure.
Maybe in Pelot, the library I was showing, like the monopartite projection is pretty much this kind of ID.
And I don't know, but it might be in Pelot.
Yeah.
You might try the GPU.
Yeah, SigmaJS, sorry.
So the question is, does this use the GPU to display the graph?
Yes.
SigmaJS is heavily relying on WebGL.
The previous version of SigmaJS was proposing to choose between Canvas and WebGL.
Right now, it's only WebGL, so it won't work with all browsers.
But nowadays, most browsers know to work with the GPU.
So, yes.
Thank you so much.
Thank you.