Okay. So, good afternoon. I'm Nilo Menezes, and I would like to share some scripts, and
after all these presentations we had this afternoon, quick hacks that I need to implement
while translating a NIN document, how Python developers could use NIN on the wiki. Okay,
so I will explain it better later on. So, translation documentations with cloud tools
and scripts. So, how it started, we had a document called NIN for Python programmers
that was written in English and Spanish by the same author. And this document is simply
a translation table from Python to NIN. So, if you already know how to write Python programs,
you can read this document and it will explain, okay, this is a structure you have to write
that way and so on and so on. But it was written by the same developer in English and Spanish
in a wiki format using the markdown, and it's the wiki of GitHub. So, in the Brazilian NIN
group, when people started to say, hey, it would be useful to get more users to the NIN
language, if we had this document, NIN for Python programmers, also translated in Brazilian
Portuguese. And then I said, okay, I did some translations before, I think I can help on
this. So, I started checking how I could do the contribution. So, I went to the GitHub
wiki, I saw that the source code was in markdown format. I just checked it out on Git, and
I started translating. Just after the table of contents, I said, this will not end very
well. I will have lots of problems later on because this document will be changed, it
will be updated, nobody will tell me that the original was updated. And in a document,
you can also move sections and do different things. So, I said, this is not a good start.
So, the idea was, how can I help translating but also building an initial infrastructure
to help translating this document to other languages, okay? So, how they were doing,
they just cloned the English page, and they started writing, overwriting the text in the
native language, okay? If you are the same author, it's fine. So, he wrote the English
version, he also translated to Spanish. Three or four languages, I think it's still workable,
but if you start to have as many languages as we saw in very big projects, you know
that it's impossible to keep up to date. So, I started to think about how can I update
this? And I had the previous experience of working with translations in PO files, using
PO files for Lin City, and also translations like December how to, and in very old document
formats where we really have to translate copy and translate. But the PO format started
to be very interesting, because I said, if I managed to convert this markdown to PO
and create a process around it, then we'll be back to the standard translation process
where we can use PO edit, for example, to do the translation and move on. So, I will
report what I did and the tools I selected for doing this job. As I said, it looked very
much like an old problem. So, how to translate software to another language? The PO and get
text combinations is very, very good. It's easy to use. Even if it's the first time you
are translating software to just tag the text you want to be translated using get text,
it's relatively easy. But I wasn't working with source code, I was working with markdown.
And the markdown, you also have some extra markers regarding the text formatting and very
specific items that I didn't want to spend time creating a converter from markdown to
the PO file format. Of course, I started searching, I found some tools. But not all tools support
the same kind of markdown as the GitHub markdown. And also, they create PO files with different
qualities. So, I spent some time tweaking. So, if you're never working with the PO file,
it looks like this. You have a message ID that is usually the original string and you
have the message string that is the translated one. And you create a new file for each language
you are working on. So, usually English is the base language. And then you create a PO
file for Portuguese, another for Spanish, and so on. Prerestanda, I think most of the
afternoon we heard about PO. So, I think it was a good idea to keep this format. So, this
was the process if we were translating standard source code. So, we have the source file, we
extract the PO file. Using the PO file, we use a translation tool or an editor to do
the translation manually, string by string. After that, we compile the ML file and the
executables or the script can use the ML file and present the text translated to the end
user. So, I just needed to adapt this for markdown. And there is also a very special
point regarding how the wiki is kept on GitHub that I will explain later. So, how to convert
markdown, this specific for GitHub markdown to the PO file. And also, I started to test
multiple packages because as I said, if you Google, you find many converters from markdown
to PO file, I think I tested two or three. I didn't rule down each one, but the final
one is MD-PO. There is a library, a series of Python scripts, so it would be, it is much
more easier for me to use packages in the same language because I could just write a
PyProject and put all these libraries in the same PyProject. The previous one was in Java,
JavaScript. And you know, if you want to run something in JavaScript, especially if you
are not using Linux, you need to install a lot of other software. So, this would enable
the translation from markdown to PO and vice versa because this is the hardest part. Once
you transform the markdown file into a PO file, you do the translation, but the main
objective is to get a translated markdown file back so you can use it in GitHub and
present the page in your native language with all the formatting that the original auto
did before. Okay. And I also started to think, okay, maybe I can write another script that
you manipulate the PO file and help me do the initial translation. Why? We have a series
of tools for automatic translation, but of course, this was something I just started.
I didn't have the integrations or anything like this, but I also didn't want to mess
with the PO file itself. So, I found a Python library called POlib that does exactly this.
I can open a PO file. I can, for example, do some filtering, like, okay, give me just
the strings that are not yet translated, and so it's very, very easy to build and manipulate
the PO files with it. Okay. So, this is the example problem, program. You simply open
the file, we start the translation, and with the help with AWS translator that I was using
because most of the time I work with AWS, I could easily send string by string to the
cloud and get an initial translation that I would just review later on. Okay. Because
you can never trust the automatic translation, especially if you are working phrase by phrase,
it's very easy to miss your target. But it's very good nowadays. So, I would say at least
80% of everything you do in the automatic translation, you can keep as it is, but you
still have to fix the 20%, and most of the times the 20% is quite embarrassing. So, you
really need to review and double check before you publish anything. And as it's a paid service,
as many translation services, and as our colleague, I didn't know the delivery translator that
was presented this afternoon, you don't want to pay every time you do this. So, I use the
previous script to create a list of the strings that were never translated, they were still
empty. So, I know that if I run it multiple times, I will not be bugging AWS translator
and paying for the translation of strings that are already translated or reviewed. Okay.
And as you can see, the script is very simple. You open the PO file, you send the text to
the cloud, in this case, AWS translate. You save, you replace the string and you save
it in the new PO file, it's done. Okay. So, the script is almost out of this. You have
it complete on GitHub. And this is the main job. Another advantage of these tools is that
you can create a list of words that should not be translated. And this helps a lot, especially
if you have a product or a document with a common name, that is, that you don't want
to be translated. So, you can pass these special lists, you can create some exceptions. But
as was explaining in the previous presentation, it's also a problem because as we, most of
the time, we select English as the main or the source translation language, you cannot
create exclusion lists using programming languages, keywords. So, in this document, Python, name
for Python programmers, of course, there are lots of source code and we're not translated
to Portuguese, for example, keywords like for, in, we're all translated to Portuguese.
So, by using the automatic translation engine, you have to review and revert this translation,
so the translated program will continue to be valid. And you have to pay very close attention,
because of course, you also translate variable names if you have source code in the document.
So, you need to pay attention that the output is still coherent, okay? And of course, as
you need to do this manual work, you can use a PO edit tool or any other tool that you
are used to use to work with PO files. So, here we have the English version and there
we have the Brazilian Portuguese version. I could just step item by item and reveal
these translations until I was satisfied with the result and then you can simply regenerate
the markdown file from the PO file, okay? So, we started with the markdown source code,
we tracked using AMD PO to create the initial PO file. I ran the script that sends the untranslated
strings to AWS translate, but you can use any provider you want. You review with PO edit
and you do the opposite conversion from PO file to markdown and publish the wiki, okay?
So, this is the workflow I tried to implement using my collection of scripts or hacks. It
is not really a tool, but with the intention to facilitate a single markdown file translation,
okay? So, the document looks like this. This is the English version. Yes, I put the English,
no, this is the Portuguese one, okay? So, in the end, I could publish this document in
GitHub. It's not yet fully integrated with the GitHub wiki because ideally, I should
put the GitHub wiki of this documentation as a sub module of my project. So, when I
updated it, I also get the newest version of the markdown and if I do this kind of integration
using GitHub, you'll be able to publish the markdown file also using GitHub. For this
initial version, I just went to the editor and I paste the markdown file, but it's possible
to do the integration. It's the next step that I still have to do. So, I published the scripts
in this GitHub. So, it's an info Python programmer. It's useful for any markdown file. You may
want to apply the same workflow. And the page is in beta because I asked all the translators,
all the people that can read Brazilian Portuguese to check if everything is fine. Because the
main goal to have a process is that usually, you never do the translation a single time.
The translation is something that you need to keep alive. As soon as the English version
is extended, translated, updated, you have to do the same thing in Portuguese. If you
don't have an automatic process able to license this document and present a subset of changes,
you'll be obliged to review a full document and this can be very, very cumbersome over
time if the document has 15, 20 pages. So, it's not ideal. And another advantage is that
the tool is smart enough to detect repetitions of the same stream. So, you also, you don't
have the boring work to re-translate the same text multiple times. This also saves a lot
of time. Yes. So, these are the main findings and the main problems I try to solve. And
that's it if you have any questions.
You can say. My phrase. Yes. So, it's much easier and especially if your text has source
code because the challenge was the source code. And sometimes you have to keep the indentation
and so on and so on. You don't want to pass the indentation mess to the translator. So,
if you use a tool like this, it will structure just the part of the program with text. And
if you keep the white space, which is very important in Python, so you can translate.
But you still have to pay attention because of the automatic translation to translate everything
to the target language. So, you have to revert the keywords. But at least the generation
is quite strong. So, when you have generated this, the program is still a correct program
in the end. So, it's good.
Okay. I think it's the last one. Okay. Thank you.