Thank you.
Well, welcome for this session and congratulations on waking up so early after yesterday evening.
It's always hard on Sunday morning to watch them.
And thank you for those who are watching online.
So who am I?
My name is Anis, as Mahmoud said.
You can follow me on social media, find my blog here.
I'm writing this gamegear emulator called Gears.
This is not the subject of this talk, but maybe I'll tell you a bit more about the gamegear
hardware so you can see how that helps writing an emulator.
I'm not an emulation expert.
I know there are a few here which are very well versed, but I'm hoping that helps gives
another perspective.
I also gave a presentation on the Z80 which was pre-recorded in the emulator dev room
two years ago.
You can watch the talk.
And yesterday on WebAssembly, putting this emulator to the web browser in the rest dev
room, we can also watch the recording when it's online.
So this is a small demo, what you can see here.
This is the emulator that's running in a native window.
Yeah, nothing very specific.
So first of all, I'll tell you why I'm giving you this presentation.
But before that, has anyone here ever written an emulator before?
Okay.
Oh, that's quite interesting.
Who here knows how to program, how to write code?
Oh, nice.
That's good because that's not the goal of this talk.
It's not to teaching you how to code, right?
You know how to program.
I'm hoping with those skills you'll be able to give you a few pointers to how to start,
like where to find documentation, things like that.
The goal of this talk is not to be exhaustive, otherwise it will be a full university course
over a semester or something.
And I want to also tell you why you should write an emulator.
That's something that should come from you.
And yes, so the focus of this talk would be on simpler platform because it's always easier
to start with something a bit simpler.
Yeah.
So what is an emulator first?
So a few definitions.
It's something I struggled a bit because they come in many shapes, but in general it's a
software, I would say, a software program that is used to run software from another
computer or another platform, whatever.
To give a few examples, here I showed a few screenshots of existing emulators.
You have Gameboy, an gameboy emulator named Semboy.
You have another BGB.
Some support weird devices like printer.
I showed an emulator running on the Android platform for the BBC Micro.
There's also an Android emulator itself.
So you want to emulate the computer that runs an Android OS.
And also put something in here which might be debatable, which is an analog pocket, which
is an emulator using FPGA.
So you write software defined hardware and use real-time thridges to run software from
other platforms.
An emulator can have a huge spectrum of, let's say, accuracy and emulation.
What does it emulate?
Accuracy is how faithful you will be to the original.
When you're emulating something, will it be running like one software?
If that's your goal, that's all right.
You just emulate enough of the platform to run one game with burning all the available
software.
Or maybe you want to do even more and be able to run any software that's on the target
platform identically as if it was running on real hardware.
We call it clock accurate, but there are us even in this spectrum.
Before we continue, I wanted to show you a crazy example I found a few weeks ago, a few
months ago of an emulator.
It's a Linux emulator.
It's a RISC-5 emulator of running Linux written in Scratch.
It's a Scratch programming language.
So we can't really see anything here on the screen.
So I'll describe.
You have a Linux terminal on Game Console.
It has already booted.
I wrote some commands.
And here I'm scrolling, and you can see the Scratch code of the RISC-5 core.
So yeah, emulators comes in all shapes and colors.
So you want to write an emulator.
Let's go with the first level.
What will it be?
Starting.
How do you start?
So if you want to start, the first thing you have to do is to pick a target.
And by target is the platform you're going, what I mean is the platform you want to emulate.
You have to pick this target.
You have to pick a host platform to start somewhere.
Even if your goal is to write something that's portable and running on everything, you have
to again start somewhere.
So you pick a host platform and make sure you have a bit of time.
If you want something that's complete, emulators are something that's hard to decide when it's
complete.
You can always have more features, more things.
So you don't have to have a lot of time in a short period, but maybe on a longer period
it works as well.
For example, for my emulator, I started two years ago.
I've been working on and off, so it's not something that's taking a lot of time every
day.
That's what I mean.
Where to start?
Okay.
Start simple.
So with the CPU, you pick one CPU instruction.
You write some code that will be able to disassemble it, which means you will take the binary form
of this instruction, this one instruction.
It will be a few bytes, one byte, I don't know, depends on your platform.
Can your code recognize this one instruction?
It might seem trivial to you.
Is this a few bytes?
It's how it starts, basically.
So you start with this, and then you start adding stuff on top of it.
You have your disassembler.
It's very useful to debug.
Then you add something, which is execution.
So you have the CPU.
How do you model its state?
Okay.
What's inside the CPU?
Go look for more information.
What's the CPU?
Build this state, change it, which is basically what an executing an instruction does, and
verify it has the state change as you expected.
So if you want to add something to a variable, you do an add operation in your language.
And then as you, let's say it's a good starter, and as you go, you keep learning new CPU concepts
and how a CPU works.
So yeah, this is helpful for starting.
So a CPU is a processor.
It's a half of, usually it's considered a half of consoles.
Nowadays it might be a GPU, but as I said, this is focused on 8-bit platforms.
As I told you, it has states.
A CPU is a processor.
So this state is basically what we call registers.
It has all the kind of states, but the stuff with registers.
I told you about an instruction.
An instruction is a minimum operation that a CPU can do.
It has assembly visualization, a text.
You probably have heard of assembly programming language.
That's how you visualize for human instructions.
And it has a binary version and encoding.
And this binary is, yeah, it's bytes you have to recognize.
It has other concepts that are interesting, interrupts.
These CPUs usually they can do execute instructions sequentially, and they can also be interrupted.
So when they have an event from the outside world, you can change the way it's executing
code.
Also interesting is how do you access memory?
I told you about states.
Usually as a programmer, when you write in code, you think about variables and things
like that.
And this hides that on a hardware.
State can be on registers or in memory.
And the way a CPU accesses memory is also quite interesting.
But the goal is not to teach you those concepts.
It's to give you pointers on how to learn.
So we've learned about how we start.
Let's talk about how do we structure an emulator?
So you've been writing a bit of CPU code.
How do you structure the whole emulator?
Because the CPU does not make a complete thing.
I'm giving you here an example of the emulator structure.
It's schematic by Rodrigo Copetti, which has been doing a very nice introductory documentation
on hardware platforms, various hardware platforms.
And here I took the master system one.
You can see here, as I told you, that the CPU has the central part.
So it's the square in the middle where it's written as Xilog ZAT.
Then you have other devices that are interesting.
I told you about the memory.
Here on this platform, you have two kinds of memory.
There's ROM and there's RAM.
You have IO control, which is how you plug a joystick or the time it was more controllers.
So this is plugged on this platform on an IO controller, and this is connected to the
CPU.
You have the game cartridges.
So those are specific type of memory with things like paging in order to access more
memory than the CPU can access.
It has also a way to generate sound.
Small device, a chip called PSG.
This device is from Texas Instruments.
It generates a very simple sound.
It has a video display processor, which would be the ancestors of today's GPUs.
And other things like something to...
Okay, it's a video display processor.
It's a bit specific here, but it has access to its own video RAM, which is a concept that
you have to think through if you want to emulate this platform.
And the video encoder is used for TV output.
So it depends again on the platform.
This is nothing very special.
Many platforms of the time had very similar architectures.
This is interesting because as you want to structure your code, your emulator code, you
will want to...
We probably will want to follow this structure.
You want to take those devices and maybe organize your code...
Take them as a code boundary and organize your code in modules.
I don't know whatever your language has, functions, objects, classes, namespaces, whatever is
on your programming language.
It's an interesting code boundary to know, okay, this device could be emulated like this.
There's another device like that.
Another trick I'd like to share is when you're writing an emulator, you don't have to think
about optimization too much, but you're allowed to optimize a bit.
Only, for example, you're writing a CPU.
It's a very simple thing.
You probably want it to be fast.
It's something that will have to be very fast.
You might want to, for example, not do allocations on the emulation path.
If you know what memory allocation is, it means that you want...
It's something that can be quite costly.
It's very useful, but when you're emulating, it's not something you want to do every other
instruction or every instruction.
You might want to use drum tables.
This one, on the case, is debatable.
Depending on your language, it might be automatic.
Quite a common advice we see when advising people to write an emulator is that you should
write a vertical slice.
What does that mean?
That means you have all those things.
You know, I told you about the CPU, there's the video display processors, the audio.
If you go on to writing an emulator, you probably want to see results quickly.
That means that you will write support for a few CPU instructions and then a bit of display
code so that very quickly you'll be able to have feedback and see on the screen that's
showing something like the Nintendo logo on Game Boy or Sega or whatever.
You can do that.
That's not what I did.
Do what works best for you.
For example, I gave you a talk two years ago on the ZAT.
It was a pre-recorded talk and at the time I had nothing else but a CPU.
It depends on what you want to do.
Do not hesitate if you have any questions.
No, maybe we'll take them at the end because they're recorded.
Sorry about that.
Another trick is that I told you a bit here about the disassembler before.
You should disassemble and write the text versions, assembly versions of instructions.
It will be very useful to have a debugger.
You might want to build debugging tooling to debug what's happening inside your emulator
early because you will have bugs.
You will have emulation bugs.
Build this tooling early.
Or you can use already existing tooling.
Here you have the emulation.
It's a great one.
It's not open source unfortunately but you should definitely check it out.
It's a multi-platform emulator.
I think we have the developer here in the room.
Definitely use the malicious.
I can't tell you how many platforms it emulates because I don't remember but it includes the
Game Gear and Master System.
It has great debugging toolkit.
You can see assembly.
You can see video devices.
You can see many things.
Always make sure you have debugging in whatever form works for you if it's tracing or logging.
It's nice but be able to inspect the state that's happening on the target emulated machine.
I told you about all of this but that's quite interesting.
How does one find information on where to start?
That's a very common question I've had.
Where do you find documentation on emulation, on how the hardware works?
Well, basically you look online.
There are many different communities.
If you want to emulate a Game Boy you probably want to go to GB Dev.
It has information on how to write Game Boy software but also how the hardware works.
In fact, I like reading documentation aimed at developers of the platform instead of emulators
developers because it tells you how you're supposed to develop for this platform.
It also means you'll be able to understand how to emulate it this way.
There's also the Geq's complete technical reference.
It's considered a definitive guide on the Game Boy.
If you want to emulate a Sega platform you probably want to go to SMS Power.
It's a community around the Sega Master System and other devices like the Game Gears, Sega
Mark 1, Mark 3, SG-1000 and the most recently announced the Sega AI which was a computer
from the 80s.
I'm sorry I don't have a screenshot here but that was very interesting.
It was an AI computer that Sega released in 1986.
I invite you to look for it online.
So here, SMS Power has documentation on how to develop software for the Sega Master System,
the Game Gear, it has documentation on how the Z80 works.
It has many links to other documentation on video, audio, etc.
One of the guides I used when writing my newsletter, there are three main ones.
The hardware reference manual for the Sega Game Gear console.
Some people in the community took this developer manual that Sega wrote for game developers
and they scanned it and OCRed it and made a great PDF version.
I don't know who did that but it was invaluable as a preservation effort and also used that
for developing my emulator.
It has some things that, well, a small caveat here is when you're describing stuff for developers,
you might not go into the details of how the hardware works and sometimes you'll have
edge cases that won't be explained to other developers but you need to emulate properly
if you want the emulation to be correct.
So yeah, in general it was very useful.
The CPU of the Master System as a Game Gear is fully documented by Xilog.
It has very complete manuals, the company still exists.
I suppose, for example, to the Game Boy where all the documentation is unofficial.
The Z80 CPU is well documented and even then there are other tricks and things that are
not documented in the official manual.
It's kind of part of the talk I gave two years ago.
You probably want to go read this undocumented Z80 documented and then afterwards watch my
talk for things that are not in this document.
So finding documentation on a very simple trick, you want to emulate something.
When you do research, use technical terms.
It might seem trivial like this but even I fell into this trap many times.
You're looking for something, how to get the more accurate thing.
An example is look online or what exact chip sets your target platform is using.
For example, for the audio, it's a chip from Texas Instruments and instead of searching
for how to do audio for X console, X computer, use more precise keywords.
It gives better results and I'm showing you here, on the left here, I Google for Game
Gear sound and you have audio videos, YouTube videos and things like that but nothing very
specific and on the right you almost find just SMS power and that's it.
So basically the link I gave you.
So let's get a bit more into practically what that means.
What would be, yeah, practically how does that work, how do devices work.
What I'm showing you here is an extract of the ZAT manual and how to do device IO.
It's very complex and you don't need to understand it.
It's basically electronics but it hides the fact that back then, using devices was quite
simple. It would be almost as simple as writing to a memory address and that's how you interact
with the device. Now on the ZAT CPU, there were dedicated instructions to do that but
it was then quite simple as opposed to modern platform where you have GPUs, memory mapping,
DMA, whatever is in a modern platform, it used to be much simpler and that's something
you can use when writing an emulator.
So in practice you want to write an emulator for a host platform.
Make sure you understand your host platform first. You want to write an emulator for windows.
Make sure you understand how to display pixel buffer on windows.
So you want, you know how to open a window, you know how to, I don't know, allocate a memory area
where you can write pixels, what is the pixel format, can you display something, a small image,
make an image, can you change it multiple times per second.
So yeah, make sure you understand your host platform and yeah, it's the same for audio.
Let's, you want to start emulating sound. Make sure you know how to do, play audio on your platform.
You have a buffer, can you generate, I don't know, a sine wave or square wave to make a beep.
Nothing about this is emulator specific but it's really something you have to do when you want to do interactive development
or game development more specifically.
So you want to, let's start with the graphics emulation, okay.
This is something you will need hardware understanding.
You will need to understand how the VDP works, for example, on the game share, how the PPU works on the Game Boy.
And so you will need to read the documents I pointed earlier.
I'm giving you an example here for the VDP, a few concepts that are interesting where you want to, when you want to display pixels,
you need to understand how developers were interacting with the device
and how they, for example, they accessed the video RAM, how did they use the registers of the VDP.
So, conceptually, I told you it's very simple.
You use specific instructions that are basically sending bytes one by one from the CPU to the VDP.
So that's how you send commands to it.
So you write your registers, you write to VRAM, so that's IO.
Internally, it has a display area.
Here, this is an extract of the game share documentation where the LCD display area, the small part of the screen,
will be part of a bigger buffer and then it's like a viewport and it can scroll on it.
It has an infinite scrolling.
The top and the bottom are connected.
The left and the right are connected, so it means that it's like a torus,
mathematically, the donut shape.
Other VDP interesting concepts, you have the sprites.
I told you about the background.
On the background, you display sprites.
Sprites are often used for game characters.
And they're very interesting because basically the VDP was like a sprite accelerator
because at the time, if you wanted to display things very fast, it was not simple
and the VDP helped with that.
The sprite also helped to do collision detection and things like that.
But you will need to understand how color encoding works, how sprite pixels are encoded
because it's not really a simple square buffer or whatever.
So everything has a specific encoding.
It's well documented.
Here, what I'm showing you is a tile map.
So it's a dump of the video RAM of the Sonic 1 game share game.
And this tile map has sprites on the bottom and background on the top.
It's not exactly the same as the display screen, but it shows how things are represented in memory
and then they can be mapped to the LCD display.
I won't go into details about that, but you probably want to have a synchronization strategy
between your CPU and your devices.
If you want to synchronize the VDP, for example, it's something that's easier to do line by line.
So you emulate a given number of instructions and then you emulate one line of the VDP.
So it allows doing this emulator single threaded because it's easier to think that way.
And it's an available thing strategy and one that can give accurate enough emulation.
Sound emulation.
Sound emulation is quite interesting.
Again, it is hardware understanding, so you will need to read the documentation.
I'll give you an example with the PSG.
So you write registers.
It has less registers.
It's much simpler.
It's a device that's conceptually quite simple.
It has four channels.
Three are tone generators.
Basically, the generate beeps with a given frequency.
And one is a noise generator and it generates noise, basically.
So you have multiple things that are interesting.
The tones are shown here on the top right.
There are square waves, at least in theory, because when you interact with hardware, life is analog,
and it's not perfectly square, so it might look a bit more like the wave on the bottom, just below it.
And what I'm showing here is a noise generator.
It's a very simple hardware device called Line Feedback Shift Register, or LFSR.
And it's used to generate noise by basically shifting a set of bits,
shifting them right or left.
Well, it's the same, but here it's right.
You start with one bit and you shift them, and then you output the bit that's on the right.
But if you were to do that without feedback, it would just shift the one,
and then you just put zero, and yeah, it's done.
Except this here has XOR function.
So we'll take two bits here, XOR them, and put them back as inputs.
And with this input, you're able to generate the random-looking noise.
It's not perfectly random, it's not cryptographically random, but it's good enough,
and yeah, that's how it used to work.
For sound emulation, again, you have to start simple.
You want to generate a square wave, as I told you.
It's a very good hello world for your platform, for sound.
But then you will need to add more things.
On the PSG, it varies on other platforms, on the master system and the megadra,
as the game is here.
You need to think about the tone channels having counters, and not the frequencies.
You need to think in terms of period and not frequency.
It's almost the same except when you're emulating, you will have edge cases that won't work well
if you think in terms of frequency.
A quick advice about thinking.
You can have multiple ways to think audio emulations with the CPU.
My advice would be to use CPU cycles.
So when you're emulating instructions, you will need to count the cycles.
Depending on the platform, one instruction can be a various number of cycles,
from, I don't know, from four cycles to 20 on the ZAT.
So you will need to count them accurately enough so that when you're playing audio,
it won't be distorted.
And in general, it's useful to count cycles properly even for display.
I wanted to give you an example about playing samples, but very quickly.
They also use a square wave, but it's quite similar.
They use amplitude variations.
So they play a wave that's always up.
So if you were to play it, it would be silent.
But they make the volume vary and they make it very, very, very fast, like 7,000 times per second.
And it generates an audio signal and that's how you have samples.
Samples are, when you hear, for example, Sega sound, something like you would play an audio file today.
They didn't support, well, this platform did not support playing a random audio file,
so developers had to get creative.
Testing, how does one test an emulator?
There are various strategies for that.
For example, for the ZAT CPU, there are unit tests you can use from other emulators.
For example, the Fuse test suite has very good unit tests that are not dependent on the Fuse emulator.
You can also use integrated tests.
For example, ZXOL and ZAT tests, these are programs that were written for the ZX Spectrum,
which was a computer from the 80s.
It has, they were generating instructions, lots of instructions, executing them,
and then dumping the CPU state and making a very small checksum.
And they run that on the real hardware and registered what was the checksum for each instruction set.
So these are bit long, ZAT tests that can take a few seconds up to minutes.
On real hardware, it was much longer, of course, and they are very useful.
Go test, go test with, even if it's another platform, you can reuse the CPU tests very simply
by doing a few bytes modification and it works on your platform.
How to test audio?
Well, this one I was not sure because I'm not so sure how to well test audio emulation.
Listen to the music. Does it look like the original one?
Yeah, you need a good ear for that.
You can use fast Fourier transforms as well, which are mathematical operations used to analyze an audio signal.
For example, you generate the square wave. Does it have the correct frequencies?
If you go through the emulation path.
And then can you hear the samples? I told you about the playing samples.
These are, I'd say, the hardest part of the audio emulation because it plugs into many accuracy things.
So yeah, can you hear them?
Other examples here for the games here. This is DGTest SMS test suite.
These are software developed by emulator developers for the platforms that will test various features.
Here for the games here is the Sega Master System.
For the Game Boy, you probably want to look at DMG AC test too, for example.
The Game Boy is a platform that's very well emulated. It's a good choice to start.
It has many tests.
Blogs, test suite, MoonEye test suite, many accuracy tests.
Yeah, you should look into it.
Another testing strategy is frame generation.
So you're emulating stuff. You're generating a display, pixel buffers.
You can very easily dump your buffers into an image and compare this image with a good emulator.
And you also can compare it with real hardware.
For example, if you use flashcards and you don't have all the original games, it can be useful.
In general, I would say test a lot of different software and look how it works.
For example, here you can see on the left side, this is my test directory for a few games where I'm basically using that as a regression test suite.
Does it always work?
So some images have a story like I had bugs that I had to fix.
And when it finally worked, I recorded that to make sure it kept working.
On the right, what you have is the same boy auto frame generation.
It's captured a very small part of a web page where they test all the Game Boy games, a Game Boy color, and other, and they make screenshots.
It's very interesting.
Other communities who are interesting in testing are the speedrun communities.
I'll let you look into that.
They also do frame testing, but they record the frames on real hardware.
So summary of everything we said here.
It was a bit fast. I'm sorry.
Pick platforms, host platform, a target platform, something you want to emulate.
First of all, always do something simple first and then make it grow.
Read a lot of documents.
That's probably something that's part of the emulator development.
So a lot of documentation.
Testing, because depending on how accurate you want to do, you probably want to test your software properly.
And don't forget if you ever go and write an emulator to write blog posts about it.
So people know about it and come here to first them in the emulator dev room and make a talk.
Please.
Thank you. Any questions?
Testing, testing.
Shall I do the question round?
We have a bunch of questions.
So I'm just going to run around and...
Thanks for the talk.
It was really good.
Two small questions.
Approximately how long did you spend on your first emulator?
It was like a few weeks, a few months before you got something running.
Do you have any recommendations from your experience?
I know you did some stuff in Rust.
Which languages did you use other languages?
How was your experience with Rust?
Is it good for emulators?
Does it make things harder?
Rust is very good.
I don't use it for anything.
So that's...
To be honest, it's a hobby project.
We have a question.
It's a hobby project that you didn't measure how much time it took to develop everything.
Before I had real feedback.
You asked me before I had real feedback how long it took.
Part of my strategy was different with what I usually recommended.
I developed the CPU first.
I wrote a talk about it.
The feedback here was does the test fit pass?
I used different tests and do they pass?
That's how you get early results without having a complete emulator.
About the programming language.
I intentionally did not go into details here in this talk
because I want people to be able to write in whatever language they feel comfortable.
Rust is great.
Go is great.
Use whatever language you want.
Especially if you're emulating an 8-bit platform.
You don't really need to care too much about performance.
You should be able to get very good results with whatever language you use.
Unless maybe it's...
I don't even have a good example.
So yeah.
Next question.
Thanks for the talk.
Regarding the audio emulation,
would it be possible to just record the waveform and compare them?
It can but...
I didn't want to go too much into details for that.
An example.
On the Game Gear and Master System,
the sound chip is generating audio at about 115 kHz.
On modern platforms, you will run at 44 kHz, 48 kHz.
You can have more on most laptops, but it's not what's usually used.
So what you will need is a down-sampling strategy.
So you will need to take the samples and down-sample them at your host platform sample rate.
And this will generate artifacts.
Yeah, okay.
Good.
Thanks.
Yeah.
Next question regarding the audio stuff.
Do you know if there's any ongoing work to emulate the original sounds of the Game Boy or Game Gear
because they have built-in speakers which will compress the sound and...
Yeah, it will have a specific sound which you can't hear these days.
Do you know if there's anything like that?
Like you could record the compression and all?
I think there is.
I think I found a website, I don't remember, on audio emulation specifically.
They were developing automatic filters to match the platforms as close as possible using machine learning
and with the target of you putting back into DSPs or FPGAs.
But I can't remember the name, I'm sorry.
But that's something I'm very interested in, especially because audio emulation is not that simple.
You need filtering.
If you want to go closer to the Game Gear, for example, which has speakers,
you will probably need filtering because it has a frequency response
which will not be the same as your modern speakers frequency response.
So yeah.
I'm sorry.
I'm sorry.
I'm sorry.
More questions?
Yeah, I almost forgot about you, I'm sorry.
So you mentioned tooling and debugging.
What?
Sorry.
Tooling and debugging of the emulator.
And you said there were two options like you can write your own tooling and debugging
so that you can inspect the state of your emulator.
But you also said you can use external existing debuggers.
How do they help?
How do they help for your emulator specifically?
They will help you understand if you have a bug with a given game.
They will help you with multiple things.
They will help you understand how the game works.
So you have a better view of how the game works, how the software is working.
And they'll help you understand what you're doing wrong.
So you're emulating the game, you have your own logging, your own debug tooling.
They'll help you understand what you're doing wrong in your emulation.
So it's more for a comparison type of thing.
Alright, we have time for a short question.
And also can you put your contacts to your first slide I think so people can find you after the talk?
Oh yeah.
I think that's a good idea.
It was a short question, right?
Or turn it into a short question.
Are there any worthwhile platforms left to emulate that are also approachable?
I would say yeah.
I gave the example of the Game Boy, it was very specific because it has so much documentation.
So yeah, it's a good platform to start.
Is there anything left?
It's okay if it has a lot of emulators.
If you want a platform that no one has written an emulator for it will be harder because you have to discover all this information by yourself.
So if you want an easy thing to start, it's not the same as exploring new stuff and reverse engineering and things like that.
Well, thank you very much.
Thank you, Anis.