So, thank you so much for joining me.
My name's Jessica Rose.
And I work on the Common Voice team.
I am very, very annoying about at least four things.
And one of them is I'm really, really excited about human languages and the web.
So when I sat down and said, oh, I'd really like to submit to speak, immediately I thought,
I'd like to give a 45 minute talk about Common Voice.
And then I thought, it's not especially helpful for people.
So I thought I'd go ahead and pull that back slightly and talk a little bit more broadly
about where our data comes from when we talk about AI models and AI outputs.
Given that we're at a Fostum audience, I'm reasonably sure that folks have a general
vibe about what's going on with AI, that you have a set of data, models are applied
to it, and that some kind of machine generated output comes out of that.
The focus of this talk is going to be about where data sets are sourced.
We're not going to be talking a lot about models.
And I do fear that the older I get, the more my thinking and the more my speaking is about
the philosophy that we bring to our work.
It is a fantastic opportunity for those of you who are watching remotely, if you're not
super excited about consent and data models to escape, for those of you who are here in
the room, you can pointedly look at your watch and escape and you're safe.
AI is increasingly in the news, and you may have noticed that it's not a bubble this time.
It's definitely not hype that every single headline about AI is grounded and focused
and deadly serious.
But that a lot of the headlines we're seeing are both the people saying, the tool makers
behind AI saying, AI is absolutely going to save us, with their other hand saying AI,
which I'm making, which I'm selling, which I'd like some funding for, could also kill
us all.
Assumingly, kidding, Sam Aitman, the head of OpenAI said, AI will most probably lead
to the end of the world, but in the meantime, they'll be great companies.
Very relaxing.
Elon Musk generated a robust amount of AI hype.
I liked this quote best because it is equally true of bears.
There's some chance above zero that AI will kill us.
Very relaxing coming from someone working on AI who presumably likes some more money
for AI tooling.
But both of the quotes before, both of these gentlemen, are also signatories to the AI
safety statement on AI risk saying mitigating the risk of extinction from AI should be a
global priority alongside other societal scale risks such as pandemics and nuclear war.
How relaxing, how calming.
One thing that I genuinely enjoy about being me is I'm not as smart and not as confident
as these people who would both like to sell us a future where AI fixes everything and
a future where AI kills us all.
I'm very, very happy to scale down my concerns about AI.
This is very much not the company take.
This is my own personal take, which is that both the incredibly positive AI as our saviour
and the very scary AI like bears could eliminate us all.
This kind of hype distracts from something that's very, very real, which is the things
that are wrong with AI today and the things that are likely to become more calcified with
time.
One of my largest concerns around these personally is that AI from tool makers and AI from individual
projects are taking the sort of commons of what people have created and turning them
into individual company generators for money that you're living in a future, we're all
living in a future.
Oh, where sometimes things don't load.
That's fine and everything's going to be fine.
We're living in a future where these large AI models come from data.
If anybody's interested, this was a screenshot about where chat GPT's data comes from.
And I'm going to paraphrase this badly, please don't sue me.
If I recall correctly, they were very, very proud.
A lot of the AI projects are a little hand-wavy.
They say, oh, don't worry about it.
We find our data.
The Facebook AI Lama model said, our pre-training data is collected generally in accordance with
open sourcing practices.
It's a very long sentence.
The chat GPT said, hey, we find this data publicly on the internet.
This is all publicly available data.
That's fantastic.
We all know that if we find an image online, we can reuse this on our websites no matter
what.
If you accidentally pick the wrong cartoon mouse off the internet and reuse that, it
won't ruin your life.
The idea that AI models can scrape the internet at large, build data sets of our own work,
and give us a future where we've duplicated slides is very, very relaxing.
Fantastic.
Everything's fine.
Open AI's large language models, including chat GPT, cool, cool, cool, are developed
using three primary sources.
And I love this.
Information that is publicly available on the internet.
Have any of you put writing or art?
Y'all actually, who has not put your stuff on the internet?
Amazing.
Who is super happy about chat GPT taking your stuff?
Those two of you, it was like, yeah, the future.
Information that we license from third parties.
I'm a lot chiller about this.
And I think this one's really exciting.
Or stuff that people gave us.
How chill.
How cool.
You can just find stuff online or people.
And I like the human traders.
This is especially menacing.
And the really exciting, the really terrifying part of this, I think, for me is if you write
for a living, if you make art for a living, if you perform for a living, we're slowly
seeing a future where you're being asked to compete with models trained against your
past work.
That when we're looking at an environment where scarcity of work comes together, it's
including the opportunity to compete against your past selves.
The question of whether or not scraping is theft.
Is something I will leave to people who are, again, much more opinionated than I am.
This picture of someone stealing has nothing to do with anything.
But what are our options?
If we want to make AI now, what can we do that's not taking publicly available information?
What can we do that's not having our human traders give us what they want?
This question, non-rhetorical, I appreciate it.
I am naturally biased, but I do work with the common voice team.
And this is a open source, crowdsourced, multilingual voice speech data set.
We've taken, and this is a very, very 2001 way to say it, the most YOLO approach to licensing
possible.
I'm very old.
And what we've done is we've said that there are more than 7,000 languages in the world.
Right now, most voice assistants are very, very chill, about 20 languages.
If any of you have used common voice assistants, which are going to be some of the more common
speech AIs you're likely to encounter, you absolutely know that these work 100% of the
time, 100% perfectly, as long as you sound exactly like I do, and as long as you speak
English with a very, very standard cadence.
For those of you who don't, very chill, good luck with your other 19 languages of the current
time.
What we've got is we've got, this is a lie, we've got 118 languages right now.
And people donate their voices.
They read clips online, and we have about 28,000 recorded hours.
One of my favorite questions, and you can yell, nobody on the other thing will hear you,
can you guess what language we have the most of?
Like most clips, most data?
People, they all said the correct answer immediately, which is Katalon.
They're watching remotely, they get what they get.
So we're really, really excited about this.
We've asked people, can you donate your voice to us?
We'd be so excited, and they do.
And we release this under a CC0 license.
For those not familiar with the CC0 license, this is the most have a good day license you
can do it.
You can, we ask that you don't identify individual voices from the data set, because that's
incredibly creepy and weird, and y'all wouldn't.
But you can do whatever you want with it.
It's free, you can build stuff with it now.
We've had people do academic research, we've had people make weird art.
I love all of it, and the stuff I don't love, I think try not to think too much about.
There's a ton and ton of stuff available, but this is, you can tell, I'd go on forever.
This is not the only way to do things.
So our first model, ask people for their data, and give it out as unlimited as possible.
I asked four different people tightly connected to the Wikimedia Foundation for help with
this.
I'm still going to get this wrong.
Y'all can email me when I'm wrong.
The Wikimedia Foundation has an incredible data set they've built, which was not necessarily
aiming to be used in AI data, but is being picked up by many data sets with being publicly
available on the internet.
They have users generate original text data, which is being released at a Commons by attribution
license, where at the same time they've got Wikimedia Commons data, video, audio, photographs,
being ingested under a network of different licenses.
They've got tight internal regulations about what kinds of license data they can take in,
and the data that they give back continues to exist under this license.
It's CC by attribution and GFDL for their contributed, generated text and work for Wikimedia.
They do an incredibly complex, one of the experts I talked to said, our licensing system is
incredibly simple, and the other three said, no, no.
But they're governed internally by a network of internal policy guidance, legal guidance,
and volunteer support and community debate that keeps their licensing in check.
The two things we've done so far are ask people for their stuff and give it to anybody, or
ask people for their stuff under a licensing network and give it back out under that licensing
network.
Data trusts get complicated quickly.
This is saying we don't want to give out our data under set guidelines that are inflexible.
We want to have ideally humans, but we want to go ahead and in the current inception take
individually contributed data and have a board of directors or have a board of guidance give
access to this.
Some of this can be under specific open source guidelines.
Some of this can be for profit saying, hey, we only give this out under these contexts.
And unfortunately, right now, the only data trusts I've seen who are operating in large
data sets in meaningful ways tend to be operating in a individual contributor data comes in,
and that's handled as a large block.
One thing that I see again and again that I'm always excited for, that I'm always rooting
for is every three or four years there's a startup promising that they're going to bring
individual choice around data sales and data access to individuals through data trusts.
There was one two years ago, there was one five years ago, there will be one six months
from now.
I'm really, really excited to see if this is the future, but haven't seen this launch
yet.
While I'm very, very excited, both professionally and personally about open source access for
data and open source data sets, one option for AI models and AI companies for getting
your data is literally just to pay for it.
Right now, people's work is being scraped at large from the web, and we live in a reasonably
carefully constructed capitalist system.
We could, people working on AI, pay people instead of just finding it publicly available
on the web.
So far, and this is not the exhaustive list of where we could get consensually sourced
data, we're going to see a very, very easy, lazy similarity between all of these.
We can ask for contributions and pass that data back out freely.
We can ask for and offer contributions under a range of existing licenses.
We can create data trusts for controlling.
We can ask to buy their data, but really, all of these futures involve asking for consent
for people's data being ingested.
This doesn't really look like the direction we're heading in right now, though.
We need a couple different things for this to work.
For somebody looking to start a project around open source data or even closed source data
that's based around consent, looking at governance models and internal structures that build
the external policy, build the consent pipeline for asking for and disseminating data, and
policing that to make sure things go okay.
There was recently a data trust in the United Kingdom that handles health data where they
had promised again and again, this is only going to be for science.
This will only ever be for science.
Can we have your health data?
We will only ever use this for science.
Earlier last year, it turned out that science did actually include insurance companies in
a very relaxing and trusting way.
Looking at how do we set up our internal policies for data projects that increase trust?
Creating these external facing structures as well.
How are we going to do this?
Are we going to ask for data and give it under CC0?
Will we have an exhausting and delightful legal framework of licenses?
Or are we going to build a board?
Regulation and oversight is an incredibly spiky question when we look at where we're
going with AI in the futures.
A lot of the larger AI tool makers right now have said, you know what?
We absolutely welcome regulation, but you can't have a subject to copyright.
That would really hold us back.
We welcome regulation, but not any of the regulatory pathways that would hurt our businesses.
We welcome regulation, but none of that regulation.
Unfortunately, if we're hoping for a future where our data is only sourced consensually,
we would need some kind of framework where folks asking, folks seeking consent in our
data aren't in turn having their stuff scraped by folks not interested in consent.
Community enforcement as well.
I was so excited to come to speak with Tafastham about this today because I'd really, really
love to leave the folks who are going to be working on AI tooling, the folks who are interested
in AI tooling, and the folks who are going to be using AI tooling to be asking ourselves
as we pick it up, oh, this is so fun.
Oh, this does this.
Oh, where does the data for this come from so that we can start to make really, really
principled choices or, I can't tell you how to guide your ethical futures, well-considered,
somewhat principled choices that meet our needs, thinking about what we will and won't work
on, what we will and won't use, and what we will and won't pay for is something I'd
really like to leave everyone with.
It's very, very corny, but if we believe that an AI future where our stuff just gets ingested
and we can compete against it our own past selves, if we're lucky, isn't a very hopeful
future.
If we're going to try to do anything that's not lie down and wait, we need some level of
hope.
And hope is fantastic.
Hope seems so glimmery.
Hope seems so soft.
I think hope is also part of a righteous determination.
You all work incredibly hard on the things you build, on the things you write, and the
things you code.
I'd love for you to be just as determined as you are in your work with keeping the intellectual
rights to your work consensually given.
Human-source means, often that we give it to the world, but that gift, that giving has
to be optional.
Thank you so much.
Yes, coming.
Can you help me pass the mic please?
Thank you for the representation.
In the Common Voice Project, you have people submitting samples and people verifying samples.
I did some sample verification and I heard a lot of samples pronounced by the same persons.
And if I understand correctly, to make a good AI, you need samples from varied sources.
Have you ever had to worry about people with good intentions contributing too much, actually?
I love this question because we have, especially with some of our less commonly represented
languages, we've got a relatively small number of super contributors where we say, oh, wow,
this is a little bit less.
And all of the voice contributions are tagged by hash ID to identify the same speaker in
the training sets.
But we absolutely do know that we've got specific language communities where we've got
individual speakers overrepresented.
And both saying, cool, if you're doing this because you're a language nerd, if you're
doing this because you're studying, that's fantastic.
If you're doing this because you want to train models better and better, you can probably
take the summer off.
Our conversations we've absolutely had with people in the past.
Yeah.
But thank you, both for contributing and for the question.
Oh, I should add, and this is not a question but a plug, if anybody speaks a language that's
not currently on common voice, please come, I used to be a teacher.
I was like, please come and see me after class.
I'd love, love, love to see about getting new stuff on.
I'll ask you nine excited language questions.
Can you, oh yes, on the other side.
I'm at 5K steps.
I'm looking forward for 10.
Don't let me down, 10K.
We're doing this.
Thank you.
Thank you for the presentation.
I was wondering if there is a difference when you are trying to, for example, make sure
that the speech recognition is accurate for persons with a different accent, let's say,
because they come from maybe they're Swedish Iranians and they speak Swedish in a certain
way, or if let's say a person has a hearing loss and then they might speak, like people
have different types of hearing loss and therefore the accent, I believe, could be more varied
and I'm wondering if the approach would be different when you are trying to make it accurate
for these different types of groups.
Oh I could have just talked about speaker recognition this whole time.
I'm so excited.
So one thing we do is we allow people to include optional metadata and that includes accent.
And we've got a drop down of the most common accents we've seen for each language, but
that's a blank text field.
There's a researcher named Kathy Reed down at ANU who's doing research about language
identities and how people self describe.
So we have seen people saying, oh I speak Swedish with an Iranian accent.
We've seen people say a huge string of descriptors saying, oh I'm in my 20s and I have a little
bit of a Zoomer accent but I used to live in Massachusetts and I said okay.
It's very specific.
I like that.
But we do also have folks who are talking to us about say, oh I'm a bit hard of hearing
and this is the accent related to that.
So really, really free text accent descriptors has resulted in some incredibly interesting
metadata just in how people describe their own accents.
I do have a question myself now building on that.
Do we use that metadata in a way when I will go and approve or listen to someone to say
yes this is correct.
Would I get any of the inputs?
You know like yes we are targeting this with an accent or we are targeting as clear as
possible so I would be.
So if you're validating common voice clips, we won't show the accent data associated with
it.
The general guidance is if you can understand it, if they're saying what it's saying, I'm
generally pretty happy to accept it.
That's great.
If I give consent but want some attribution, is something like that possible?
So with common voice, I'm so sorry that it's CC0 so there's no attribution associated with
it.
One thing that is super exciting is the platform is all open source as well.
So we have seen language communities that said hey, we're not really comfortable with
the CC0 aspect.
We're going to go ahead and fork the platform and create parallel language corpora.
Plural corpus, I always flub it.
So they're parallel language corpora under different licenses or even we've seen individual
users create their own speech corpora based on what they wanted to do with their own voices.
So under common voice, no, I'm sorry.
Under other collection projects, often yes.
It's a hard decision where to go, which side to go.
Yeah, yeah, yeah.
This is open source.
Thank you.
So when you talk about the CC0, I also understood I remember that there was some issue of concern
because it could have been exploited by some other company.
What I wonder is if exist or if I've ever evaluated a license, let's say this is CC0
except from massive machine learning.
So on this very, very controversial question at Fostum as well.
I've heard other people say them like this.
It's not how I would describe them.
I've heard these licenses described as Franken licenses sometimes or open source alike licenses.
There are a ton of, so I think I want to separate out my personal opinions from the company
opinion because I don't know what that is.
As an individual speaking for myself, I think these are really exciting and interesting.
They're not true open source.
But for people to be evolving licenses and to be evolving how we think about permissions
as we evolve projects with data, even the ones that don't work are just such exciting
experiments.
Yeah, they don't always have to be something I use in my projects.
They don't always have to be pure open source.
But I love mess and they're all very exciting mess.
They're often not all.
So there's some really interesting ones out there.
There's an ethical, I can't recall what it's called.
There's a open source alike license based around, you can only use this for ethical
uses which immediately splits into 90 different, what is ethics?
What's a big company?
And it's just a very, very exciting street to go down.
Thank you so much.
Thank you.