All right, good morning everybody.
Welcome to the session on SNAP.
My name is Jens, and with me is my colleague and friend Yadka.
We're here to present SNAP, build your own blocks.
But it's not just Yadka and me who are working on SNAP.
We're just the ones presenting it today.
There's a larger community and more people involved.
There's Benat Ramagosa down there.
There's Simon Walters from the SNAP community here.
There's John Maloney, Turgut Gunezu.
The whole blocks gang is here.
And all of those have been more or less also involved
in the development of SNAP.
I've brought also Olanzah, who's very involved in SNAP,
our mascot.
I keep saying this for people who might not
have heard it before.
Every once or so often we get somebody who says,
take down SNAP.
It's a copy of Scratch.
And so I just want to point out, yes,
you know this is also a mascot on Scratch.
It's called Gobo in Scratch.
And we're friends with the Scratch team.
And we're allowed to use Gobo.
And our friend and collaborator, co-author Brian Harvey,
mutated Gobo with this funny haircut that, of course,
everybody in this room knows about.
It's a lambda.
So SNAP is Scratch on lambda.
This is what it's about.
It's about building your own blocks.
And so what Yatka and I are planning for today
is kind of not so much a talk as a demo.
It's a walk.
It's a lab visit.
And we'd like to invite you to a visit of our lab.
So when we say SNAP, the title is Build Your Own Blocks,
it used to be called BYOB for Build Your Own Blocks,
until some parents in the US who didn't
have a sense of humor decided that it also
means something else, like bring your own bottle.
Usually there's alcohol in it.
And the implication that there is alcohol
might entice children to try it.
So we had to give it a different name.
So this is why it's called SNAP.
We're still calling it Build Your Own Blocks.
But to us, and what we'd like to show you today,
it is as much about learning to code
or learning about technology than it
is about building one's own mind,
about learning something about society,
about the surroundings, about the environment.
And we'd like to kind of give you
a little overview of not just our technology,
but also our pedagogy.
So just very quickly, this is SNAP.
How many of you know SNAP or have used SNAP?
OK, about one third.
So just very quickly, I'm just going to show you so in SNAP.
One thing that we have, when you open it,
this is kind of pure vanilla SNAP.
You can, here's a block, you can move something.
You can also kind of stick blocks together
like these puzzle pieces.
There are different kind of categories
with control structures.
You have kind of these control structures.
This is a repetition.
So when you have kind of four times this,
and you click on this, everything is live.
It moves.
You can see what it does.
You can, you have a drawing pen.
You can get the pen down, pen up.
So now you can draw a square.
And always the question is how you get rid of it.
So there's a clear block.
You can click on these things.
You don't really have to think that much ahead to do something.
And it's called build your own blocks,
because you can build your own blocks.
So you can make a new block.
I want to make a block in the motion category.
It's a square block, and it has as input a size.
It's really hard to build a block with one block.
It's really hard to build a block with one hand.
And so I'm getting this editor.
I can say the size should be an input.
It should be kind of a number.
So now I can say, oh, this part is this thing that makes a square.
I can drag this in here, and the size is an input.
And I can say the size is how much it should move.
So now I've made my own block.
Wait, did I make my own block?
Or did it?
Oh, look at it.
It didn't work.
OK, see, this is the fun of doing it with one hand.
OK, you guys, this is a collaborative coding environment.
Let's try this again.
Oh, I need to even make size and input again.
So as we were just hanging out with a bunch of teachers
is that the repetition is the mother of all learning.
So now I have the square block.
Let's try to make a square of 100 again.
It works.
And then we can do things like, now we can build around this.
We can say, OK, let's do, let's clear this,
and let's maybe say, OK, i times 10.
And we also want to turn a little bit, like maybe 36.
And we can kind of build things.
This is kind of the technology part that isn't really new.
It's great.
We love it.
But the thing is that we want to build blocks,
but we also want to build ourselves.
We want to interact with our surroundings.
And so we want to look at data.
We want to look at the world.
And through programming, as I think Mitchell says it,
learn to program and program to learn.
But not just learn about technology,
but learn about something else.
So here's a little example of that a while ago.
John and I were also working on and we're
thinking about how we could express things that
are really important.
And here's a little puzzle.
You can make an educator.
You can kind of make a project and give it to your students.
And in this project, I've downloaded a bunch of data
from the internet, from GitHub.
This is the population data of 195 countries
over a span from the year 1800 to today.
It's the population development.
It's a lot of data.
I've also downloaded the life expectancy data.
And for some periods, there isn't one.
But it's the life expectancy in each of these countries.
And also, the gross domestic product adjusted for inflation
and broken down per person living in that country.
And that might tell us something about life conditions
and how life conditions in countries
have evolved over time.
And some of you who can read it probably already know this.
There's the GapMinder project by Hans Rosling.
Who knows this?
GapMinder.
And so we thought, it's great to work with this.
It's great to look at it.
And we thought it's also really great to use coding,
not just to use it, but to interact with the stator
yourself.
And so what we like to give people
is an imperfect project that has something like here.
It's something that already has a slider built in.
And you can configure that slider,
kind of give it some numbers.
And then it emits a broadcast, like the slider changed.
And now we could say, OK, in this project,
like when this receives the slider changed,
me, we might want to hide it.
And we might want to go to some coordinate,
like go a little bit to the left and down.
And we might want to set a color to something light
and write the value of the slider.
So we're going to query the value of the slider.
But it should be big.
Let's make it big, probably at some point,
before we also want to clear it.
So now, as I move the slider, I'm
getting a readout of the numbers.
But I want it to be the numbers from 1800 until now.
So I probably should have some kind of formula in here
where I say it's not the value of the slider,
but the value of the slider.
Plus 1799.
That's about as much math as I'm allowed to do.
OK, so now I have a sort of a timeline, an interactive timeline.
And now I want to kind of map all of this data
and sort of scatter it in there.
So I'm going to make a new, I'm going to draw a little bit
of data.
So I'm going to draw actually a new object.
I'm going to use a vector pen and draw something like a,
oops.
Sorry.
Draw just a little dot, which is about my drawing things.
I'm going to call this a country.
And a country is going to, I'm going to add another variable.
That's just a name.
So it's sort of a, OK.
And actually, I don't need to show that.
And so when the green flag is pressed,
I can just set the name to nothing.
And actually, what I want to do is now I want to do something
for all the countries.
So I can take out, so if I look at all the countries,
they're all in the first column.
So I could say, OK, this is the first item of the columns
of this table.
And as you can see, it's always live.
So these are all the countries.
So I can loop through them.
We could say for all of these, I want to make a new clone.
So I'm telling a new clone.
I just want to assign the name to whatever is in that list.
And I want to do this really fast.
I don't want to wait for it to take long.
And when I'm done, I want to do something.
I want to broadcast that something has changed.
So I'm going to broadcast the slider change.
And now what I want to do is that when each of these clones
receives the slider change, it should align itself to the data.
And here's something that we're saying, build your own block.
So here's a block that we made that you can already use.
So we give this block that arranges a record.
So the record here is indicated by the name.
And we're selecting in these records,
we're selecting the year indicated,
which is dependent on our slider.
It's the value of the slider.
And so we want to map the wealth, kind of, you know, the money.
We want to map that to the x-axis on a logarithmic scale.
The life expectancy, we want to map to the y-axis.
And the size of the bubble, we want to map to the population.
And let's actually try this.
Do you think this could work?
OK.
Wait, I wanted to do one more thing.
I wanted to make them a little transparent.
So we could set the ghost effect to something like 60.
So we see the ones that are underneath.
Let's try this again.
Yeah.
OK.
So here's a map of the world of 200 countries in the year 1800.
And now we can see some interesting things.
As we move the slider, we can see how the countries develop.
We can see how life expectancy rises,
how kind of things are distributed.
Now, it would be fun to see which country is which.
So we could also add some other interactive elements,
like we could say, when I am, mouse entered,
we could say, you know, say my name.
Say my name.
Say my name.
OK.
Say my name.
When I'm, again, when the mouse goes out,
like when the mouse is departed, just stop saying whatever it's saying.
Say nothing.
Does it work?
Yeah.
We don't have to restart it.
So this is China, the big thing.
This is the US.
And now we can do interesting things.
Like now we know this big blob is China.
We can see what happened to China.
Whoops.
Here there's a famine in the early 60s.
Or we can see, we can look for other spectacular things
like right at the beginning, like here in 1880.
And there's a problem in Tunisia, the last case of the Black Plague.
And with all this data, it's already interesting now to use this
and to use Google and to find out what happened at certain periods of time.
For example, we can go, let's go to the early 20th century here.
1904, the first genocide committed by Germans in Namibia of the 20th century.
We can see how, oops, here you probably know this.
It's not World War I.
It's a drop in life expectancy almost everywhere except in Denmark.
Nobody knew that before COVID.
Nobody knew that before COVID.
Now everybody knows it.
It's right.
That's the Spanish flu, which wasn't Spanish, but it's the flu epidemic.
And so this is what we're talking about.
It is as much about looking at data, finding out how to work with tables,
how to model things, but also about really discovering things
that are fun to discover with a computer.
This is much more fun to discover with a computer than with a textbook.
And it's also way more interactive.
And even the learning can be more self-directed because now I want to know
what happens at this bubble at this time.
And so it's about building blocks, but also building knowledge very much
in the constructivist, constructionist way.
And so one thing that's really cooking that we're working on right now
for the next version is, so with these abstractions,
we're building up.
We're sticking together blocks to build up abstractions that let us do
more awesome things with less blocks.
Now the question always comes, especially conferences like this,
but at the core, at the bottom, there's got to be some real language.
And the real language has to be text-based because everything is text-based.
And even artificial intelligence uses large language models,
so obviously there must be something to textual language that is very powerful.
And it is.
But here's something that we're working on.
And we've had several projects.
John and I were working on a project where we tried to make a,
and we actually went pretty far to build a block-space language
that was completely written in itself.
Snap is not completely written in itself, but we're trying this.
Look at this.
This is something that's in the development version.
Here's a new thing we're working on.
It says blocks all the way.
And if I click blocks all the way, now I can look at all these blocks
that are called primitives that are actually written in JavaScript,
but I can edit them.
I can edit them and I see blocks, how they could be written in Snap itself.
And here's a block that is a primitive block,
which means we're actually, we're calling up a native JavaScript function.
But we could turn this off, and then it would run this,
and it would totally do the right thing.
And if you actually look at this code, you might be astonished
because this is probably not what you were expecting.
It has sort of a NumPy APL-ish way to deal with coordinates
in a way that uses vectors.
And it's really fun to check out how things work.
Like, for example, here's the glide block.
So there's glide written in Snap itself.
There is, of course, if an edge bounds, and this is way more complicated.
But we can even go to other things, and like, obviously,
the control category is very interesting.
Now, we can look how forever loop is done.
If you look at forever loop, it's implemented recursively.
It's a higher order function that calls the function it is given,
and it calls it recursively.
But, or here's how repeat until would be,
and repeat until is interesting,
because it also uses if.
Now, what about if?
So here's how if is done, or things like not.
And if you look at if, if again uses if else.
So it's almost sort of like a not really infinite,
but kind of it goes not all the way down to the metal,
but it goes down some more subterranean stories.
And what we're hoping to achieve with that is to let kids and learners explore the system,
and find out it's not about one language versus the other,
but it's about how to express your ideas and how to do these.
So let's actually try to really find something,
which is we're at a hackers conference.
So what if we said, you know, move?
Let's break move and say, okay, I don't want to go to something,
I don't want to use the primitive.
Actually, I want to see how it moves.
I want to glide something like not so long.
And so what I'm going to do is I'm going to run the glide block.
I'm going to leave the coordinates empty, which turns it turns them into implicit parameters.
I'm going to put in this vector in here.
So now I've really fine move.
And now if I run this, I can see that I changed the way how move works.
And I've now made it such that I'm using glide instead of going to.
And this is going to be fun because it sort of gives you agency to even change the way
snap works because all of this is now editable.
All of this is really a system that is malleable.
So this is what it's about.
Build your own blocks.
I'm now going to hand over to Yatka to tell you some more about our kind of the pedagogy
and the kinds of things we're working on.
Yeah, thanks.
So we do this thing together with people at UC Berkeley.
And it's also a lot about education.
So it's used at universities around the globe and also in a lot of schools by now.
And if you have a school that you want to use, snap, feel free to reach out.
We're always looking for more collaborators.
And as you already saw, you can program snap in different ways.
And that's also something that's really important to us that we can.
So there's not one way to the solution, but there's several ways.
And we also accommodate the boring ways, like using all the four loops that kids are required
to learn in school.
But we also want to elevate the mind to new ideas about programming, like the stuff that
Jens has just been showing you.
And I just wanted to give a short, can I close that?
Oh, I'll go over here.
Yeah, great.
So, no, not right now.
Thanks.
So I wanted to do a short, oh, have we tried whether we can record something on the thing?
No, but we'll try.
OK, we'll try.
I record a sound and see whether it works.
Hello.
OK, let's try to play it.
OK, it doesn't.
Good.
Ah, OK, it comes out of here.
Yeah, OK.
Hello.
So we can record sounds.
This is something that's also really important to us that we can extend projects beyond the
working with numbers.
So I personally am not a developer.
Like, I didn't study programming.
I am a developer now, but I studied biology.
And until I was 25, I thought programming is, can I say that shit and boring?
Then I tried it out and I used Snap for it and it is actually fun.
I mean, you guys all know that, but it's awesome.
So I'm not that much of a math person.
So I really love that you can extend Snap using media, using data.
And now I want to show something using this recording.
So you can access the samples of a sound file that I recorded by using this block from the
sound category.
So you can see we have different categories here with differently colored blocks that
helps to structure the programs and also to read the code.
And if I click on that, it's also a list with 51,840 samples.
At the beginning, I didn't say anything, so it's all zeros.
But then when we move down, we see that it's negative values, but very small negative values.
So the samples are the amount that a membrane swings either to the left or the right.
In our ear, for example, or in a speaker or something like that.
And now I could try to modify that sound.
So let me grab one more recording that we can always play it.
And I could do it the traditional way using the for loop that I've just mentioned.
Because as Yen said, in German schools, kids are required to learn loops.
And this is really important.
So for I from, we start lists with one.
So let me start at one for I from one to the length of the samples of my recording.
I want to do something.
And the something I want to do is let's make it louder, maybe.
Let's try this.
So I want to create a new sound.
And I call this new sound.
And I set the new sound to an empty list.
Okay, now maybe I do want to.
So I set the sound to an empty list.
And now I want to add stuff to my sound.
And what I want to add is the value that I had before.
So item I of the samples of the recording.
And I want to multiply them with the factor.
So increasing the number that's in the in the sample makes the sound louder times.
So let's try times five maybe and hope I don't fry anything.
And then I want to add that to my new sound.
When I do it like that, it's pretty slow.
So as you see, this runs rather slowly.
So we have this, what we call the warp block that just speeds up things.
So let me wrap this around here.
And now I can have my new sound.
And I hope that it's louder than the one before.
So this was the one before.
Hello.
Can you even hear that?
Hello.
Okay, now let's try the louder one.
Hello.
Definitely louder.
So we can use that way to change media.
But we also want to support, as I mentioned, other ways of thinking.
So what we have down here, and this is where the lambda that Alonso has,
comes into play is the map block.
So this is a higher order function, a function that takes another function as an input.
And we represent this function with these gray rings.
This is like a lot of the rings, one ring to rule them all metaphor.
So this is what gives the power to one of the powers to snap.
And we can use data here in the second input slot.
So what I could use is again, let me just duplicate that, the samples of my sound.
And now I can add another function down here.
Let's do the apps function maybe.
So this gives back the absolute value.
And now I can play that sound as well.
Hello.
So you can hear that I sound more like when I don't have air through my nose.
So you can make sound effects like that.
And then the last way is the one that Jens already mentioned, and
I didn't come up with the third effect.
So I'm just going to use apps again.
You can just drop lists like vectors into functions directly.
So I could just drop this one in here to create the exact same effect.
You could use floor.
I could use floor.
What?
Let's try that.
Beautiful.
Okay, so this is just, I think this is just negative ones, zeros and ones.
So with only three different values, you can still kind of understand the sound.
Because what I said, because the pattern is represented by the way the samples are arranged.
And not by the actual values in the samples.
Okay, and you can also do that with more complex data.
So we can also access the webcam from Snap.
And I can try the same thing with the camera.
Maybe I can try the same thing with the camera.
Maybe we'll just unplug this real quick and I take a picture.
Okay, we broke the webcam.
So let's leave it at that.
So let's go over and I tell you you can do it with graphic effects.
And if you want, we have a workshop later come by and I'll show you how it works with
graphic effects or photos.
Okay, and then stuff that we are currently working on is AI.
Since this is the big thing and the schools wanted, universities wanted.
So we had to come up with resources that they could use to teach artificial intelligence.
And I wanted to show you one that we've developed last year and
it's available in German and English.
You can download it from the internet.
And it's a detailed walkthrough guide on how you can use that in a classroom setting.
And how to program the whole thing, some ideas on what you could do with it.
And we call it grant gestures.
So it's a simple gesture recognizer program based on the $1 gesture
recognizer.
I don't know whether you know that, but it's a prototyping gesture
recognizer for Unistro gestures, things that you can draw in one line.
And I already prepared something like a TV cook.
So I have this project here already and this is a simple drawing program.
When I click on the stage, so we call this window here the stage, then it'll
broadcast the word sketch.
And when I receive the word sketch here, I'm reacting to that and
I can actually draw something.
Yeah, since some of you are sitting really far away,
let's just increase the pen size a bit.
Yeah, so I can draw stuff here.
And what I already also prepared is I'm storing values in a variable.
What you can see here is I have the sketch variable that I can also show.
It's 164 points and it's the position that my sprite,
the object that I'm programming went to.
And now I also have this examples variable here where I already
stored a few examples and this is always a path and a word that's attached to
this path, so basically a label for that path.
And now I want to create a few more things and then we're going to animate them.
Can you hold the microphone again, please?
Thanks.
So to create an animation, let's start with the animation.
We gave you a block here that we call animate and here you can also see again
one of the awesome things in Snap.
You can make your own control structures.
So this is a C-shaped block like control structures, some control
structures look in Snap.
And this is a custom block that runs actions that you can put here into
the C-shaped input slot and we made this block.
And what it does is it takes what I've drawn and puts it on as a costume.
So the costume is an image that the sprite is wearing and then it does something.
So when I draw a heart, I want it to have a heartbeat.
So increase the size a bit and then decrease the size in one step.
So increasing the size, I do with the change size by 10 block and
I do that 10 times and then I reset the stage.
And since hearts are bumping like twice, I want to do this two times.
And if I put that into the animate block, you see that I can do that with the drawing
that I just made.
So I can draw something and it takes this actual drawing and does something with it.
And now I want to trigger this reaction whenever I receive the message heart.
Okay, this is what's supposed to happen, but now I need to identify this heart.
So we want to find out how these paths work and
to see that I can render what I've drawn.
So this is also a block we prepared and I can just put my sketch in here and
I can render that and I see that the points through my path are not very well
distributed.
So I draw really slowly at the beginning and the end and
then I was really fast here.
So to really make them comparable, it's important that we have, we normalize them.
So we have this resample block here and now I could resample my sketch to 64 points.
And this evenly distributes the path between my,
like the points on the path.
Yeah, so now I can use that to train my algorithm here a bit more on my
program a bit more.
So here this is what I've drawn was a heart.
So let me draw a new one and I can now add that to my examples.
Let's add another one.
Okay, and now I need to recognize this heart in all my examples.
We also prepared a block for that, but you could build it yourself if you wanted to.
This is the recognized block.
And this recognized block looks for the smallest difference between two shapes.
Like it's measuring the distance between the first point in the first path and
the first point in the second path.
And then adds up all the differences between the points and reports the one
that has like the label of the one that has the smallest difference.
And we can just use that as an input to the broadcast block.
So let me just show this real quick.
I'm recognizing my resampled sketch which is the heart that I've just drawn
in all my examples.
And since this is the heart that I've just stored,
it should report back the heart which it does.
I can try another thing.
So this is also the heart.
I also wrote down FOSSTEM.
So let me try to write FOSSTEM.
Writing is really hard on a touchpad.
So this reports FOSSTEM.
Great, seems to work.
And now I can just broadcast this thing.
And we want kids, people who use that to tell stories with it.
So this is supposed to be an interactive storytelling project.
And the story that I just came up with was, how did it work again?
Let me check.
Ah, yeah, okay.
The weather in Brussels is not really nice.
It's raining all day.
And I'm sitting in dark buildings all day.
But still, I love to be with you at FOSSTEM.
Okay.
So this is a resource that we've been working with again to also inspire people who might
not be the traditional audiences for programming.
But it's also pretty cool if you are a programmer and love math,
you can still do stuff with it.
And now I would hand over to Jens to tell you what we're venturing for next.
Thank you, Jette Gah.
I also love to be at FOSSTEM.
This is so cool.
So you might see, you know, this isn't really about an algorithm about using AI,
about using a large language model.
What we're trying to do is to sort of at least lift the lid a little bit to let you see a
little bit underneath the hood.
So for us, it's not about upskilling youth to be employable, but it's about bringing
across a sense of awe and wonder about what you can do and maybe letting you reflect about
things.
So now, since it's been mentioned for two years, generative AI is this big thing with
chat, GPT being everywhere.
And it really boils down to, as we've seen before, language, even textual language being
the basis for everything.
And we thought, well, yeah, that's nice, but it's, we love language, but we also love
structure.
And so one thing that we've tried to come up is with a activity that is more on the basis
of these language projects, which really is a next token prediction system.
And that might lead us up to experiencing and learning something about the really
generality of AI.
So it's all inspired by a wonderful little project.
I have to kind of give them credit by Michael Hilsche from Peha, Schweiz, who, this wonderful
project, Zykia GPT, you all have to look at it.
And so this is what we're trying to do is now build something like chat GPT ourselves
and on little data.
So we don't actually have to use chat GPT.
So here's something I scourged the internet for 30 fairy tales of the brother's
grim.
And here's the English version of these 30 fairy tales.
It's not a huge corpus, but 30 fairy tales.
And it's just a text.
And in order to work with this and turn this into an AI, I need to split these 30 fairy
tales into a list of words.
So now I've got a list of words like 58,000 words.
And so sevens, waybians were once together.
OK, so just a list of words doesn't give us a lot.
So in order to use this in a language AI, we have to do some statistical analysis.
And the way we do statistical analysis is by we're grouping these words by their sequences.
They're called like pairs or triples or bigrams, trigrams, tetragrams.
So, you know, it's build your own blocks.
So I'm going to, can you please hold again, I'm going to make a category.
Wow.
I'm going to make a category that's called, be called generative AI.
And I want to build this one function that I'm using.
And it's the, it's going to be a function.
It's going to be the n grams of a corpus.
And n is going to be an input.
It's going to be a number like two or three or five.
And the corpus is what's going in the language thing.
It's a list of words.
So what I want to do is I want to get the numbers from zero to the length of the corpus to go through all of this.
But I don't need the full length.
I can decrease it by the n that I'm looking for minus one.
Now I want to take this as an input to map.
So I'm taking these numbers and for every of these numbers.
So when I have a list, list of the numbers one to ten, item one is the number one.
But if I put in a list of, of, of items to check, like three or six, I'm getting a list of, of individual items in there.
So I can slice my input.
So what I want to do is I want to get the item, whichever the number is, the numbers, whichever the numbers is, two n minus one of my corpus.
This is what I want to do.
So this is the function, the n-gram function.
Let's actually try this.
Here's the n-grams, like let's, let's go to the bigrams of my 30 fairy tales.
Click on this.
See, oh, seven Swabians.
Swabians were, were once.
You get it?
It's kind of broken up.
I can also do this for four.
So now I'm getting seven.
Swabians were once.
Swabians were once together.
You get the idea, right?
So, so, so this breaks up the corpus into these little sequences of words.
Now I want to do a statistical language model.
So to do a model, I'm going to make a variable called model.
And what the model should do is the model should be, let's actually hide this, should be several variants of this, not just bigrams or trigrams, but let's go all the way to five.
So I'm going to say the model is going to be another map block.
Map the n-grams.
I'm going to leave this blank over the numbers from one to five.
So let's actually just run this once.
So now let's look at this model.
Now this model is a kind of weird looking table.
If I format this a little bit different, you can see it's a five item list.
The first item is like though.
Unigrams, bigrams, trigrams.
Five is great because tetragrams.
That sounds like diabolic pentagrams, right?
Pentagrams.
So it's a sort of a cascade of a model.
And I can also try this like the item one of the model, you know, is a list with one, the item three of the models list with three.
Okay.
So this is really the heart of a statistical language model.
Because now, for example, let's take a list.
Let's take a list of words.
Let's say the king's daughter.
So we want to see what's a good way to find out how to complete a sentence that starts with the king's daughter.
So we could look for, you know, in all these, in these four grams, whether anything starts with the first words like, oh, the king's daughter and then find out which words come.
Let's actually do this.
So this is keep in my model of four, something that is equal to the king's daughter.
And we want to find out the items with the numbers from one to four.
Okay.
Let's, let's try this.
One, two, three.
You see.
So we see, oh, there's a bunch of sentences.
The king's daughter came is began, love said again came.
So you see, there are several times sometimes the same thing in there.
So we could say, okay, we could just take, you know, the last item of a random element of these things that we get to complete that sentence.
So we can say the king's daughter came, the king's daughter laugh.
And this is basically finding next token, something that has been used in that context before.
So I've made a little block for that, which I'm going to import that does that.
It's the next token block.
It's literally just that.
And now we could build something like chat to PTRself.
So we could say, okay, when the green flag is clicked, ask.
Enter the beginning of a tail and wait.
And then when the user enters said, we're setting, oh, we need to make a new variable.
That is going to be a tail.
And we're going to set the tail to what the user entered, which is the answer.
And we're going to split that by the word.
And then what we want to do is we want to take the next token in that tail based on the model.
And we want to add that to the tail.
And then we want to say the thing, right?
So we want to say the text of the tail, that just so we don't see a list, but we see a nice text.
We want to say that.
And we could say, okay, how about we do this when I receive next.
And here we broadcast next.
And then whenever the user presses space key, we also broadcast next.
Okay.
So, well, let's try this.
Does it work?
Enter the beginning of a new tail.
Once upon, ah, once upon a time.
So now whenever I press space, it is continuing this and it creates fairy tales.
Sometimes it'll stick to one fairy tale pretty long, but since it only has a context of about five words,
it keeps forgetting which fairy tale it's in and then just finds something else that is plausible linguistically,
but maybe not from the story.
And so you all know, right?
This is not how chatGPT really works, but it's a statistical language model and the similarities are actually striking.
This is what they call a Markov text generator, Markov chain text generator.
And it has like a GPT model, a transformer model is just this except a little bit more complicated.
It has a longer context and it has some neural networks that makes itself aware,
so it kind of doesn't just take the last n words, but kind of knows, has a little more memory and distinguishes,
which things are more important, but it's literally just making up stuff.
It doesn't have any idea about the language it is written in.
So when, for example, I take 30 fairy tales in German and instead of these 30 fairy tales in English,
I'm doing the model in German and now, yeah, I'm just saying, you know, I don't care.
So now it's going to speak German, right?
Because it's been trained on these German fairy tales.
And so it's not about language, it's about statistics.
And so again, we could think that this is all about language, but at the core,
it's about finding things that correlate with other things.
So we thought, well, wouldn't it be nice if this were also a good pedagogical model for an AGI,
for something that is more general, not really super intense, but more general.
So there's lots of sequential data.
One thing that I think every hacker loves, I don't know, is music.
So here's, I transcribed 20 songs.
And these songs aren't just words, they're notes.
And so the notes are, you know, there's a pitch, a mini-pitch, and there's a duration, how many beats.
So what if instead of these words, we just took these notes and shopped them up into a bunch of n-grams
and, well, okay, so I prepared this little script that does that.
It's the music improvisation script.
So this takes the 20 songs, chops it up, uses the exact same blocks, like the n-grams, and the next token block.
The exact same thing. And remember, the data is differently dimensioned.
It's not a single list. It's a multi-dimensional list.
I'll try whether it does something.
So you sort of see, it's like me whistling, you know, ah, there's this, oh no, there's this other thing.
Let's kind of go and do some funny associations.
But it's already beyond language.
It's already generalizing the principle of finding the next meaningful token.
So we thought, okay, wow, this is nice.
So what about pictures?
Could it work with pictures?
I mean, pictures aren't sequences.
They're a plane with multi-dimensional.
But you know what's a sequence.
Sketching something is a sequence.
So we thought, why don't we take the sketching thing?
But instead of mapping coordinates, we remember for each line segment the direction it went to.
So let me again kind of write something like an honor of, so, hello.
So now I have the sketch in this model.
And let's try to find out if we do the exact same thing that we did with the music by now doing a sketch program,
training it on just this little data.
Oh, it tries to imitate my handwriting and does something that sort of looks like I'd written it.
And there's, it's going to be different kind of the next time.
And it's sort of fun.
It is finding some meaningful next tokens.
And at this time we were really, really having fun.
And we thought, well, what if we don't just write something?
What if we, like Bernat's idea was, what if we, for example, draw something that already kind of makes sense?
Like, let's draw something that looks like a tower, something that looks like a castle.
Whatever, a roof and a moat.
So here's a little thing that I drew.
Let's try to find out how we can do a skyline.
Isn't that cool?
And that's almost, you know, a glimpse of, it doesn't matter about language.
The secret is to seek when
you're in a building.
You can't get out of the building.
You can't get out of the building.
You can't get out of the building.
You can't get out of the building.
You can't get out of the building.
You can't get out of the building.
You can't get out of the building.
You can't get out of the building.
You can't get out of the building.
You can't get out of the building.
You can't get out of the building.
No, you can't get out of the building.
You can't get out of the building.
like you want to edit it.
How do you visualize what change?
This is a list of numbers.
What do you want to do?
Do you think the delta between the change of your graph and the numbers?
How do we do the software engineering spiel?
We don't teach children about version control. We teach children powerful ideas.
There is a version that is called SMURGE.
We are mixing up the graphics towards refining the code.
Do you have examples of the SNAP?
We have the gesture recognizer including all the materials you need for school.
We are working on the SNAP GPT thing that will be published in a few weeks.
If you want, you can search for projects like the grand gestures.
Here is the project.
Go to snap.berkeley.edu and use the search bar.
I am not aware of any other block space programming language offering the notion of procedure as data.
We have data. We are more like researchers.
We do not teach kids lambda. We use lambda to build blocks that kids can use.
In higher classes, there is a curriculum that is used as an advanced placement course.
We will use higher order functions totally.
We have a high school version of that. They have a middle school curriculum for seventh graders.
Thank you everybody. See you next year.