Next lightning talk is Jan Verschijl talking about multiple double arithmetic on GPUs.
Thank you very much, the organizers for allowing me to speak here.
So I will hope to talk about computations that I've been doing with multiple doubles.
So the multiple doubles go back actually from the time when people, when the hardware was
not yet supporting doubles.
So this was the late 60s.
So this is actually a similar idea.
So you use the hardware arithmetic to extend your precision.
It has a lot of advantages.
So if you're used to working with complex arithmetic, then double-double arithmetic has about the
same intensity.
So speaking of intensities, relative to the previous talk where we were working with graphs,
so in the previous talk I had the impression that everything was about graphs and there
was a memory bound.
My problems are compute bound.
So I get really good arithmetic intensities.
There are some disadvantages, of course.
If you want to work with, say, 17 decimal places, you can't.
Also if you want to work with truly infinite decimals, well, you can't either because
you're still having your 11 bits of the exponent.
Disadvantage might also be that you can still do numerical analysis.
So this might be an advantage or disadvantage.
I got into this by power series arithmetic.
So this is about the EXP and the EPS.
So when I started working with power series, I was using 11111111.
And I know the binomial theorem.
Well, I only knew it when I saw the numbers blowing up on me.
So you know it when you don't know it.
So here is a table.
The exponential has a very nice development, nicely decaying.
And if you multiply these exponentials, you don't have any blow-up.
However, the last coefficient, if you want to represent that, and you have to think about
GPUs.
GPUs are actually quite happy if they can do things in groups of 32.
So actually a 32 power series, an order 32 power series, is actually still very small
for GPUs.
But there you have already to use quad doubles.
Otherwise, your last coefficients, you can't represent it anymore.
OK.
So I started working with the QD library.
And then we were doing multi-core.
Me and my student, Gennady Jofi, and we looked at each other, should we do this on the GPU?
Should we write the entire library on the GPU?
My student didn't really want to do it, and I didn't want to do it.
But then we discovered GQD, and we used GQD.
And the recent package that we are using is Compari.
It's actually the only software I know that is named after a beverage.
I don't know if that's a good sign or not.
In my supermarket store in Chicago, I once saw Compari, but it's not my drink.
So I didn't want to ruin the taste of using Compari.
So I stayed off this.
Compari is actually quite good.
So because it allowed me to go to quad double, and now also octo double.
The numbers in this table are kind of good, because I want to have really performance.
But it also comes somehow misleading, because as soon as you're using complex double-double,
everything becomes compute bound.
And the problems that you have with memory transfer and all, you do a lot of arithmetic
operations on a relatively small amount of data.
I also like to do quality up.
If you can afford the time for, say, a double precision calculation, well, you will see
that everything is not really right.
But then you can allow the same amount of time, and you quadruple the precision.
So the 439 there, think about 1 gigaflop, 2 gigaflop, and then you go to teraflop.
So the 439 is kind of, if you have teraflop performance, it's like as if you would be
doing this on a single core.
So I mentioned the funding agencies at the very slight.
I would like to have a hopper.
But so for now, I have to deal with Pascal and Volta.
And the last one is a gaming laptop, which is also actually quite a powerful GPU.
My first teraflop card was Kepler.
And this last list of GPU actually gets there.
Okay.
If you think of a double-double, there is a double-two.
And then for a quad-double, there is the double-four.
So that was what the GQD was using.
And that's very good for memory coalescing.
But we actually got into trouble with the complex quad-double because there was no double-eight.
So instead of working, if you work with a vector of quad-doubles, a vector of arrays
of four length, you actually better use four vectors.
The first one with the highest double, second double, third double, fourth double.
So it's a little bit similar like working with power series.
So power series is invertible if the leading coefficient is not zero.
You can work with matrices of power series.
But actually, that's not good.
You should actually work with a series where the coefficients are matrices.
Same idea here.
QDLIP is a very good library still.
It's quite complete.
So I have extended the square root, for example, to octodouble precision.
OK.
So here is then my beginning.
So I mentioned, so you saw this eight.
So if you take a vector of random complex numbers, 64, then the norm is eight.
Should be eight.
So that's a really nice test property.
If you work with GPUs, you actually define kernels, and kernels, the name says it itself,
it should be small.
So think small.
And actually, this problem is a small problem, mathematically speaking, but it has all the
richness and the complexity of all the problems that you will run into.
You will have to study the prefix sum algorithm, for example.
So that is needed.
You also have to tune your software for large vectors or for small vectors.
You can only have one block of threads that is active.
The square root works with staggered.
So you apply a Newton method.
And then actually, this is where the dot comes in.
So the nice thing about double doubles, quad doubles, is that everything fits into registers.
So it's also very good if you do multi-core.
So you don't have to use the heap ever.
But of course, when you get to complex quad doubles, you have these eight arrays.
If you do octodoubles, so it doubles and doubles and doubles.
So I have with my old graphics cards, they can no longer even compile the octodoubles
if you inline too much.
So it's still very interesting that, actually, you have to tailor your kernels towards the
precision levels.
So here is my last slide.
I did more than just norms.
So we have teraflop performance when we evaluate polynomials and differentiate them.
The QR, the blocked householder QR, is also wonderful.
You get already teraflop performance with 1,000 in complex double-double.
And then the last paper is where you try to combine these things by computing Taylor series
for solutions of solution developments for polynomial systems.
Newton's method is actually a quite nice operator.
You start with a multivariate system where all the variables are linked to each other.
And what Newton actually does, it spits out power series for each component.
So actually, it untangles all the linearities, all the nonlinearities.
So I listed the archive.
So the IEEE puts things in a paywall, behind the paywall.
So you have the archive versions there.
And you're more than welcome to the bottom line of this slide.
I mean, the conclusion, actually, is that all the software is free and open source.
I'd have the GitHub handle there.