We're here with Dennis Luxald to present annotated type hinge you can use at runtime.
Dennis is a Python developer working for the label on post-gres infrastructure and automation.
And he's been a long time free software hacker.
Thank you very much for being here with us, Dennis.
Thank you.
Thank you. So annotated type hinge you can use at runtime.
First of you want to learn me, I work at Delibos with a small French company doing post-gres service.
And they are do database infrastructure automation and also trying to contribute to process ecosystem.
Most notably recently the psychophagy database adapter in Python.
And last but not least, Python project at SciPy or Macro.
So why talking about annotated?
Perhaps we have seen this kind of code in the wild recently or less recently.
It's taken from Python documentation.
Python is a well-known and famous nowadays data modeling and validation library in Python.
And you can see from their documentation, especially recently since the version of library,
that's this annotated pattern kind of spread everywhere in the library.
Another example is here a code sample from Fast API, which is another famous library
useful for doing web API in Python.
And there is also this annotated pattern here.
When I stumbled upon this last year when doing a migration to Python in V2,
I was kind of disappointed because this syntax looks well in Python.
It's verbose, it's not really usual in Python.
So I wanted to talk about it and why.
First, let's see how it works because it's not very intuitive to me.
Then I wondered how can I use and define my annotation in order to use the annotated syntax
using my annotation to when I would define not just the one that would be provided by the library
and for which use cases.
So the line of the talk is threefold.
First I'll introduce typing.annotated, which is defined in a PEP, 593.
And then we'll walk through a few of these cases involving data-centric models
and doing validation, serialization, and user interface, all using annotated.
And first and third I'll discuss what the adoption in the community and ecosystem of this annotated construct.
So let's start with PEP 593.
So when it is defined in the typing module, it's in the standard library,
but in my opinion it's not really a typing similar to others.
It's more like an annotation.
Maybe more in the spirit of the initial idea of function annotation defined in a pre-old PEP now
in which you can attach an annotation to an identifier using the colon symbol.
And then it was used for typing, but really here we have the ability to annotate identifiers.
Identifiers are for example class attributes, function parameters, or anything in the name space like a module or something.
It's here since Python 3.9 or in typing instruction if you have a Python installation.
And the PEP is named flexible function and variable annotations.
What does it tell?
It tells that you can annotate a variable named veer with the type int annotated,
which takes at least two arguments.
The first one is a valid type, could be a built-in type or custom type you define.
And then it takes at least one metadata or annotation.
I would use the metadata or annotation in an interchangeable manner in the following.
And you need to pass only one annotation, but you can pass many of them.
The key idea of this is that this metadata can be used for static analysis and also at runtime,
which is pretty new in the typing ecosystem.
And it opens for pretty interesting use case in my opinion, although syntax is a bit well.
I think it's also interesting because it's designed to be composable, meaning I'm quoting the PEP here,
but basically it means that when a tool analyses the code base,
the data static analysis or at runtime, and it encounters an annotation it does not know,
it should ignore it.
And if it encounters an annotation it owns, typically because it defines it, it should handle it.
So it means that if you use annotated to combine many annotations from different source,
you can expect them to play well together.
How to consume annotations?
Because once you have defined annotated values you will need to consume them.
In the typing module, still in the standard library, you have a couple of utility functions I will introduce here.
The first one is getTypins, which can be used for any kind of object like a class,
even an instance of a class or functions, any three terms, a mapping from attribute names to their typings.
So if we have this point at a class with two attributes, the second one being annotated,
we can see that the int value, which is dictionary, returns x and y,
and x is simply the integer class, and y is the annotated construct.
You can also use annotations and gender attributes, but getTypins is more powerful in general.
From there, you need to inspect individual annotations,
and you have two functions, you have getOrigin and getAx, still from the typing module.
If you use getOrigin on the typings, it allows to discriminate between all the typing constructs that are in the typing module,
and especially in the defying in your case, which one are annotated types.
So you can compare these results with the ease operator, for instance.
Then you can extract the arguments of the annotated value,
which means here in our example, in our y example, getting the type and annotations as a list here.
So that's how you consume annotation.
And in general, thanks to the composability principle I exposed before,
once you have extracted all the annotations, you would ignore the ones that you are not interested in.
Here we only endow the label annotation which we own, and we ignore the others.
So we typically check this with an instant check.
To wrap this up, I'll introduce this simple helper.
It's not full proof, but it works for our examples.
Basically, it uses getTypings, getOrigin, and getArgs in order to work through the annotation of an object
and find the one matching the specified type here.
So we to illustrate it, if we use getAnnotation on the point class introduced before
and trying to match the label annotation, we get these results.
So the y attribute, the y name, the annotation, and the type bound to the attribute.
So I'll reuse this function here later on.
So that's all for the presentation of annotated from the PEP and how to consume them.
I will now introduce some use cases in order to illustrate why you would want to use annotated or why not, maybe.
I'll use this simple model which is a calendar event model.
It uses pyidentic as a base model.
So again, pyidentic is a famous library for doing data modeling and validation or stylization.
It's similar in scope to data class.
And here we have an event model with a few fields, a summary description, and two dates defining the duration of the event.
And the following will do three things.
The first one will do validation of the daytime fields.
We'll do this using pyidentic built-ins annotation.
So we'll see how to use third-party annotations.
Then we'll do high-calendar stylization.
High-calendar is a simple format for XMG calendar data, text format.
But we'll see how to implement our own annotation in order to perform stylization.
And the third one will be console rendering.
We'll build some kind of user interface in the console in order to print and display calendar events.
So again, this will illustrate how to define our own annotation.
So starting with daytime validation.
We define, we use here the built-in annotation from pyidentic.
And namely, it's the aftervalidator, which is some kind of annotation factory,
because it takes a function here tzware, which as its name suggests,
would validate that the daytime value is not naive.
Meaning it has a time zone defined.
So here, you can simply define tzware type by combining the daytime field
with the pyidentic shipped annotation construct and your own validation logic.
Here, I'm defining an event with a naive time zone.
And we can see that pyidentic produced a validation error,
which under the hood is triggered by our value error with the expecting a tzware daytime message.
So it works.
As a side step, I would like to mention that before using annotated in pyidentic,
in all the versions of pyidentic, the pattern for doing validation
was through using class methods and decorators,
namely the field validator decorator,
in which you have to define custom method in your model classes,
and you have to bind the attributes you want to validate to the method
using the field validator decorator.
So that's another way to define validation in pyidentic, not using the annotation.
And why, in my opinion, the annotated pattern was adopted in pyidentic
is for the following reason, I would say.
Because the validation method, the class method is loosely bound to the attributes.
You can see that there is no direct relationship between the start art and end art
field definition and the validation, whereas in the previous example,
with the annotation, you have inline combination of type and annotations.
Then if you have different classes using the same validation
because validating a non-native daytime is quite usual,
you would have to repeat the method in all your model classes.
And similar for all use cases, less like serialization.
So that's why I think the annotated patterns
has taken adoption in this kind of library.
Here we simply introduce an alias, tzdatetime,
which is the annotated daytime with or validator in order to make or code less variables.
So that's one idea of annotated, despite being verbose, you can use aliases.
And you can define your base model class just using the aliases.
Next, next you see is ICANN under serialization.
Here, obviously, I'm using, I'm adding another annotation.
These annotations are defined in the ICANN module, which I will introduce later.
And there are serializer instances.
An ICANN under serialization just takes a label name,
which would be used to serialize the data.
So here you can see that we have combined different annotations
in the same annotated construct,
because here our tzdatetime is already in an annotated value
and it's again wrapped in another annotated construct with another annotation.
All this is flattened at runtime.
In the ICANN module I mentioned earlier, we have this serializer class.
Here it's a data class. We have the label field.
And we have a simple serialization method,
which does some transformation between the value, which would be the field value,
and specify it.
So if it's a date, we need to remove the time zone and change to ETC
and use this kind of format.
Then we use this function, which takes the object events,
walks through the annotation using the get annotation function introduced later
and it calls the serial's value.
And we join the result using this.
So an example here, we still define our event model.
We get the date and we can print the serialization in the 9-calendar format.
I'm going to work up to define and consume an annotation.
The first thing you do is to define annotation, typically as classes,
the data class is quite handy for this.
You can define options and you would typically implement an underline method
in order to process the value.
Then you add and take your data type using annotated, obviously.
And then you consume your object using the get annotation patterns
I would annotate a bit earlier.
The third use case, here we are stacking another layer of annotations
from the UI module, the UI module I will introduce later.
We are adding UI annotation in order to define all the fields of our event model
will be serialized in the console.
We have a text annotation, a margithon field for the description
and we have data serializer for description.
Here we will be using the rich library, which is a pretty nice library
for doing console rendering and building terminal user interfaces.
The widgets, which are the annotations we have used in the previous example
are defined here using classes, following the pattern I introduced before.
And they basically delegate to rich to do the rendering of a field
and then do the running on the type of the field.
Here it's another way to process the annotation instead of introducing a custom function
for processing the object.
We introduce a mixing class, which follows the rich protocol.
It's defined in documentation.
You need to define a Dunder rich console method.
And there, again, we used our get annotation function
looking for the widget class, widget annotations.
And if we found some, we call the render method of our widgets, get the value
and we need the text value.
We use this as a mixing, so we extend our class events.
If we take another example, we added a description with some margithon formats.
The dates field here are the same.
Simply, they will be colored depending on whether the events are started or not.
So if you reach, print, or events, we get this.
We can see that we have margithon interpreted things in the description
and the start date is green.
So that's all for use cases.
I hope I've demonstrated why you would want to use annotated or why not, maybe.
Then I will discuss about the adoption of this pattern in the ecosystem and community.
First, we have adopters, like I mentioned before,
by the antique, fast API, or typo.
So you have this kind of code in the wild.
And you might want to get used to this because I think it's here to stay in this library.
I hope I've demystified the pattern so that you can understand what it does it mean
to have this kind of code in the documentation and if you copy-paste it, for example.
There's an interesting project in annotated type,
which provides reusable constraint type to be used with annotated
so that you don't have to define your quite classic annotation.
It's also adopted in a SQL alchemy,
although it's a bit more involved because you have to use the mapped type annotated file.
And obviously, in a project with less coupling with the typing system, there is less enthusiasm.
It's obvious.
This brings me to some skepticism I've seen in the community.
First of all, I've used in annotated is quite verbose.
The symbol is already in uncultured and if you stack different kind of annotation in the same type,
it's getting verbose so you have to take care about this.
It's us, readability.
Then it's kind of awkward because annotations are not necessarily typing,
although most consumers do use typing information.
But if you don't want to use typing, you cannot really use the annotation
as provided by annotated at the moment.
Also, consuming annotation is a bit tedious as we have seen.
You have to write some bullet-plate code.
And there is more coming.
Here, there is a pep I would like to mention, 724.27,
which would introduce a doc construct in the typing module,
which would be used through document fields.
So again, it's not typed, but it's in the typing module.
So this brings us to the typing topic, which as you may know,
in the Python community is quite divisive because there are some fans of the typing and some are not.
And this, I think Python is growing with its features because they bring user value.
The example I've shown in Fast API are a lot more expressive, in my opinion,
than the one that you would typically use in other metaprogramming patterns,
like decorators and so on.
And if you want to deep dive more in this topic,
I encourage you to read this LWNN article, which was published recently,
and it provides quite a nice overview of the typing issue or topic in the ecosystem and community.
Thank you for your attention.
I'm done.
And if you have some questions or some thoughts to share, but annotated, I'm happy to discuss.
Thank you very much, Tennis. Do we have any questions?
Let's see. Now is your chance to raise your hand.
I can see one question there.
Don't be shy. Raise high, please.
Hello.
Thanks for your talk. It was very cool.
Could you go into a little bit more detail of how I could use annotated on my own class?
Because like I saw it in Pydantic and I was like, oh, cool.
But I didn't quite get how I could use that in my own...
Sorry, can you repeat?
No, okay. Could you tell me how I could use that annotated trick on my own class
instead of it being like a Pydantic base model thing?
Yeah, that's what I illustrated.
So here you have this cellulizer, which is a class I just defined for the example.
And then you annotate your attribute with your cellulizer instances.
Here it takes an option, which is the label of the cellulization value.
And then you need to write this kind of function in order to consume the annotation
of the instance of your event class.
Yeah, but the event was inheriting from base model, which was a Pydantic thing.
So if I'm in my own project...
You don't need to do this Pydantic for this.
You can take a built-in class or a data class, or...
It's not related to Pydantic.
The Pydantic thing was just for the first example, validating the time.
Apart from that, you don't need Pydantic at all.
Okay, thank you very much.
Thank you. Do we have any more questions?
One more. One more.
The session is still being recorded.
So please be silent there.
How can we reach you?
I'm not sure if it's okay to ask that.
Is it possible to use that in Django, REST, and WorkSphere Realizer?
Have you tried or have you seen anyone try that using this one?
Is it Django, REST, and WorkSphere Realizer?
In Django?
Yes, Django, REST, and WorkSphere Realizer.
I don't know. I don't know Django, REST, and WorkSphere Realizer.
In fact, you can use it as long as you define your own way of consuming the annotation.
So if you want this kind of helper function, you can use it, definitely.
Okay.
It's not bound to a particular framework, in fact.
It's in the standard pattern.
Thank you very much.
Another round of applause for Dennis.