Hello, HPC Room, my name is Gabor Sarnas.
I work at CWI Amsterdam as a researcher, and today I'm here on behalf of the LDBC.
The LDBC stands for the Linked Data Benchmark Council.
We are a non-profit company founded in 2012, and we design graph benchmarks and govern
their use.
Additionally, we do research on graph schemas and modern graph queue languages, and everything
we do is available under the Apache V2 license.
Organizationally, LDBC consists of more than 20 companies.
These are companies interested in graph data management.
We have financial service providers, database vendors, cloud vendors, hardware vendors, and
consultancy companies, as well as individual contributors like me.
So we design benchmarks, the first one being the LDBC social network benchmark, which targets
database systems.
Let's go through this benchmark by a series of examples.
I will touch on datasets, queries, and updates that we use in this benchmark.
As the name social network benchmark suggests, we have a social network that consists of
person nodes who know each other via a distribution that mimics the Facebook career social network.
The content that these people create is messages.
These form little three-shaped subgraphs and are connected via author edges to the people.
On this graph, we can run queries like the following.
Let's have a given person enumerate their friends and their friends of friends, get
the messages that these people created, and then filter them based on some condition on
their dates.
So a potential substitution could be on this graph that we are interested in this query
for Bob and the date set on Saturday.
And if we evaluate this query, we start with Bob.
We traverse the nose edges to Ada and Carl, then continue to Finn, Eve, and then we move
along the author edges.
And then finally, we apply the filter condition, which will cut message three and will leave
us messages one, two, and four.
So obviously, a social network is not a static environment.
There are always changes.
For example, people become friends, even Gia may add each other as a friend.
That will result in a new nose edge.
That's simple enough.
Gia can decide to create a message.
This message will be replied to message M3.
So we add a new node and connect it to the existing graph via two edges.
The heavy hitting updates are the deletes.
A person may decide to delete their account, and that will result in a cascade of deletes.
For example, if we remove the node Eve, that will result in the removal of their direct
edges, all the messages they created.
And in some social network, this will even trigger the deletion of all the message trees
and, of course, all the edges that point to those messages.
So this is quite a hard operation for systems to execute.
It stresses their garbage collectors, and this allows certain append-only data structures.
So if you want to weave these three components together, the data set, the queries, and
the updates, we need a benchmark driver that schedules the operations to be executable.
It runs the updates and the queries concurrently, and, of course, it collects the results.
The system under test that we run the benchmark on is provided by our members who are the
database vendors, and we go to great lengths to allow as many candidate systems as possible,
so graph databases, triple stores, and relational databases can all compete on this benchmark.
Speaking of relational databases, some of you may think is SQL sufficient to express
these queries, and the answer is that in most cases it is.
So the query that we have just seen can be formulated in a reasonably simple SQL query.
It is a bit unwieldy, but it is certainly doable, and the performance will be okay.
However, this being a graph benchmark, it lends itself quite naturally to other query
languages.
There are two new query languages that are going to be coming out, and both of them adopted
a visual graph syntax inspired by Neo4j's Cypher language.
The first one is called SQL-PGQ, where PGQ stands for property graph queries.
This will be released this summer, and as you can see, it's an extension to SQL, so
you can use select and from, but it adds the graph table construct, and the query can
be formulated in a very concise and readable manner.
There is GQL, the graph query language, which is a standalone language that is going to
be released next year, and it shares the same pattern matching language as SQL-PGQ.
So the social network benchmark has multiple workloads to cover the diverse challenges
that are created by graph workloads.
The first one, the older one, is the social network benchmark interactive workload.
This is transactional in nature, and it has queries like the one I have shown before.
So these queries typically start in one or two person nodes.
They are not very heavy hitting.
They only touch on a limited amount of data.
They have concurrent reads and updates, and systems are competing on achieving high throughputs.
So this benchmark has been around for a few years, and we have seen actually very good
results.
In the last three years, we witnessed an exponential increase in throughput, starting from a little
above 5,000 operations per second to almost 17,000 operations per second this year.
Our newer benchmark is the social network benchmark business intelligence workload.
This is analytical in nature, and it has queries that touch on large portions of the data.
For example, the query on this slide enumerates all triangles of friendships in a given country
which can potentially reach billions of edges, and is a very difficult computational problem.
Systems here are allowed to do either a bulk or a concurrent update approach, but they
should strive to get both a high throughput and low individual query runtimes.
This benchmark being relatively new, we only have a single result, so it's a bit difficult
to put it into context.
But it allows me to highlight one thing.
Many of our benchmarks use different CPUs.
We actually have quite a healthy diversity in the CPUs.
We have results with the AMD Epic Genoa, like this one achieved by TigerGraph.
We have results using Intel Xeon Ice Lakes and the ETN 710s, which use an ARM architecture.
We have more and larger scale results expected this year, and we are also quite interested
in some graph and machine learning accelerators that are going to be released soon.
So our benchmark process is quite involved.
For each workload, we release a specification.
We have an academic paper that motivates the benchmark.
We have data generators, pre-generated data sets, as well as a benchmark driver and at
least two reference implementations.
We do this because we have an auditing process that allows the vendors to implement this benchmark
to actually go through a rigorous test, and if they do so, they can claim that they have
an official benchmark result.
So we trademark the term ADBC such that the vendors have to go through these hoops of
auditing, and we still allow researchers and developers to do unofficial benchmarks,
but they have to say that this is not unofficial ADBC benchmark result.
Another benchmark I would like to touch upon briefly is the Graph Analytics benchmark.
This casts a wider net, so it targets graph databases, graph processing framework, embedded
graph libraries like NetworkX and so on.
This uses untyped, unattributed graphs, so it's only the person knows person graphs
of the social network benchmark or other well-known graphs like Graph 500.
We have six algorithms.
Many of these are textbook algorithms like BFS, which just traverses the graph from a
given source node, or we have PageRank, which selects the most important nodes in the network.
We also have clustering coefficient, community detection, connected components, and shortest
paths.
This benchmark is a bit simpler to implement.
We have a leaderboard that we update periodically.
The next one is going to come out in Spring 2023, so talk to us if you're interested.
So wrapping up, you should consider becoming an ADBC member because members can participate
in the benchmark design and have a say in where we go, they can commission audits of
their benchmarks, and they can also gain early access to the ISO standard drafts, SQL, PGQ,
and GQ that I have shown.
It's free for individuals and has a yearly fee for companies.
So to sum up, these are our three main benchmarks.
We have other benchmarks and many future ideas.
If you're interested, please reach out.
Again we have time for one question.
Any questions for Gabor?
This is a newbie question.
I'm not into graphs.
Apart from advertisement, optimization, mass surveillance, and perhaps content distribution,
which I don't know if they're the major applications, but it's just what my naive minds come with.
What other applications are those benchmarks meant to optimize?
So the big one this year is supply chain optimization, like strengthening supply chains, ensuring
that they are ethical, ensuring that they are not passing conflict zones.
It's something that is very important these days.
You can also track CO2 emissions and other aspects of labor and manufacturing.
So that's certainly a big one, and that's something that we have seen.
And there are, of course, all the graphic problems like power grid, a lot of e-commerce
programs, and the financial fraud detection, which is going to be part of our financial
benchmark this year.