> This extension is trying to provide some useful functionality in the hopes that Mozilla gets more data for https://commonvoice.mozilla.org

This is awesome! I love contributing to open source initiatives like this. I'm also a non-native speaker so hopefully I'll add some color to the voices recorded.

Very good point. Honestly I use my Echos for exactly two things: turning smart lights on and off and setting timers. I occasionally will ask it the weather or to play a song or a podcast. That’s about it. It seems like for my use cases it doesn’t need full on speech recognition and the million Alexa skills out there. Just a few simple phrases would suffice.

I'd go farther and say that I specifically don't want the million Alexa skills out there. A system that let me write my own intents with access to the top-level namespace would be ideal, but it would absolutely need to come with good hardware. That's where Alexa/Google have the upper hand currently, I think.

> if we want open, on-device voice recognition, we'll have to do the work and donate sample data.

Fair enough, but is any stopping Mozilla from _also_ selling your voice data to third parties including advertisers and commercial ML interests?

(Asking this because Firefox has started sharing our data with leanplum)

It still remains a risk well worth taking. With Mozilla, it is merely uncertain but for just about every other company, it is all but guaranteed. You can plainly see this philosophy in the industry's naming sense, where super-computers of yester-year are relegated to "edge" roles.

As of today, the open source and free software equivalents to machine learning and AI products are sorely lacking when compared to commercial offerings. Whether it is open-ended speech to text with good ergonomics, text to speech, intent recognition, speaker recognition, OCR for text, OCR in the wild, translation, object recognition, image segmentation, image to text or natural language processing, commercial offerings are leagues ahead of what free software can do.

If we look at one of the most impressive AI demonstrations in history, GPT-3, it is not apparent whether open source can even replicate it because with AI, unlike in the past, time and skill is no longer directly fungible with money. I would argue the concentration of such capabilities to Microsoft and Google servers is a threat to the ideals of free software as great as any it has seen before. Yet, relatively little attention is spent there because people are too focused on yesterday's problems.

This concentration is difficult to avoid because current algorithms require large amounts of data and computing ability, which only large corporations can marshal. Mozilla is far from perfect but despite their many stumbles, they're the only large organization seriously attempting to address this imbalance. As much as these algorithms are marketed as AI to users, ML is better thought of as libraries, in the line of ffmpeg, to programmers. Mozilla still do seem to care about creating a local-first offering. If everyone stops using them then what is gained exactly?

> it is not apparent whether open source can even replicate it because with AI, unlike in the past, time and skill is no longer directly fungible with money.

Indeed. The creator of LuaJIT is only a single, very skilled, person with a desktop computer. People like Fabrice Bellard can produce gigantic amounts of FLOSS source code. Yes, those are only examples, but people with their skills and motivations to build FLOSS software will need access to lots of money in order to be able to build ML models.

>With Mozilla, it is merely uncertain but for just about every other company, it is all but guaranteed

I disagree, most large companies view this data as a competitive advantage and won't sell it directly. They may sell the results but the data itself is their moat. Smaller companies on the other hand are more willing to sacrifice future profits for current money.

This is an open dataset. There's no point selling a dataset that's already free.

As far as recording things I didn't consent to, the likes of Google way more likely to do that[1].

1 - https://mashable.com/article/google-assistant-microphone-smo...

As long as they're upfront about what they collect and how to not send them data, am fine with it. Obviously opt-in is much proffered to opt-out, but from the Firefox Voice site, it clearly states sharing data is opt-in.

My guess would be there isn't much of a point selling voice data from an open dataset. Also, since the code is in the open, it would be relatively easy to spot if they were sending data to somewhere they do not or record when they shouldn't.

Right now you can only help the STT engine by contributing to commonvoice.mozilla.org, where the samples are published to the world. The add-on will now, if you opt into it, keep the data in only Mozilla's and Google's hands. Mozilla has an agreement that Google won't keep the data, but even if Google doesn't comply by the agreement, the number of parties with access to the data will be lower than "everyone".

You have wrong premise here. First of all, current Mozilla data is almost useless for training because it is carefully read speech. You do not need much of it, even accents. If you add 1000 hours of CV data to 1000 hours of random data the improvement in accuracy will be minimal. Same for the speech collected with Firefox Voice, it will be mostly a set of short commands, most likely it will not be very useful for generic transcription of random people speech.

Second, you can build models much better than Mozilla ones simply from public data, there is no need to collect user voices. We at Vosk https://alphacephei.com/vosk/ support 10+ languages for example without any user data. Everyone creates very good models from augmented text-to-speech data these days (Microsoft demonstrated in the last paper you can get almost as good as domain-specific data https://arxiv.org/abs/2007.15188).

Given that it surprises me that Mozilla continues to insist they need the voices of their userbase.

Nokia n800 was crushed by internal fighting.

> if we want open, on-device voice recognition, we'll have to do the work and donate sample data.

We absolutely will not. The only reason people believe this is that they've forgotten how to do speaker-dependent recognition (SDR), which is more accurate and more secure anyway. We were doing SDR in the 80s with 1/1000 the CPU power and 1/1000 the memory.

SDR does require an initial training session, but once that's done any modern computer or smartphone should be able to handle it locally with no cloud server environment.

You say “forgotten” as if we had great tools everyone just forgot about. Having actually used those systems I am rather skeptical of that claim - they really seemed to have hit a certain functional plateau below the level of modern systems.

Put another way, if this was off the shelf, why isn’t anyone marketing it?

One reason may be that since it doesn't require a cloud, there's no personal data to mine. Try getting VC without a recurring revenue stream. It's probably possible but it's more difficult. Same story for IoT: Cloudless home automation is trivial from a technical point of view, but cloudless home automation is a non-starter VC-wise.

This was a field with multiple products on the market. How much VC do you need to deliver benchmarks of shipping software?

Similarly, saying cloudless home automation is easy sounds like you’re leaving out a lot of experience other people gained about the challenges of getting consumer adoption with the need to take on 24x7 server maintenance, connectivity challenges blocking popular features, etc. which made that class of products less appealing to most customers.

Training a speaker-specific recogniser that improves over a generic recogniser requires a lot more data nowadays. First, generic systems are a lot better and trained on a lot more data nowadays. Second, speaker adaptation worked better for the Gaussian mixture models from the late nineties (don’t know about the eighties) than for neural networks.

Who's "we" in this context? Because just below you, HN has comments from willing donors.

My point being that while there may still be a market for SDR, there's a broader market for speaker-independent recognition (SIR) simply because people want the tech to just work rather than feel like they messed up training the device when the device can't recognize them.

Using someone else's voice assistant is also a legitimate use case, especially if it's used to control music, lights, blinds, AC, car functionality ... that absolutely requires solid SIR.

I think this can be viewed as a marketing and UX problem, sort of. It reminds me of the Wii Amiibo - people actually paid money to train their AI bots because of how Nintendo designed them. Not sure how many people, but a reasonable enough segment of the market that Nintendo thought it a worthwhile investment anyway