Okay, here's some brutal feedback, please take it with your best interests at heart. I am an English native speaker who has lived in Spain for 10 years, and has become fluent.
1) These speech-to-text models are poor when it comes to non-natives. This is unfortunate as the idea you had and the product you've designed could be incredible for language learning. However - it's a bit crap - sorry - I can speak Spanish well and was asked in the conversation if I wanted a medium sized cup of coffee, I replied "sí, mediano", the resulting text was outputted as "mariano", then in the role play the coffee shop worker then assumed my name was Mariano! Completely ludicrous and frustrating.... in real life the coffee shop worker is clearing expecting the word 'mediano' and will hear what I said and know that's what I was trying to say. The speech-to-text-model completely fails to get this.
Until speech-to-text models trained on non-natives are made readily available, products like this with so much promise will infuriate learners, which will stop them paying for it.
And this was ordering a coffee.... imagine an actually complicated conversation.
So my advice would be, right now the speech-to-text models aren't capable of doing what you're hoping they can do... but.... once you get a model that can, this will be insanely popular....
So hang in there, other than that it was a fun experience, and critically, people are scared of practising with real people, something like this would be insanely popular if it actually worked well. Good luck.
Eh I disagree. It's not perfect, but that's just about expectation management. Users don't expect voice-to-text to be perfect - in fact, the past 10 years shitty experiences have been the norm. I think it depends on the level of mistakes the transcribing makes, but that's only going to improve with time with advances in AI and as the product evolves (using the context of the conversation like you say would be a great start).
Even in its current state this is an awesome product. There are so so many people in the world learning a language, and one of the hardest parts is practicing after you stop learning (like you leaving Spain). People like that will really love something like this
In future especially to be able to cater to people who are still learning, it should be feasible to use a similar product to train and correct people's pronounciation.
Speech recognition is far from perfect but even then it's incredibly useful. It CAN be infuriating (or downright hilarious). Hell, it's infuriating when occasionally a (human!) waiter switches to English after I mangled a sound in Hungarian, even though I'm C1/advanced in the language.
Problem or opportunity? Like someone else pointed out, the limitations of speech-to-text can be turned into an opportunity to improve one's pronunciation. It's getting extremely good for native speakers. As foreign speakers it's a chance to improve.
In any case, I'm sure I can add a layer or two to the code to reduce misunderstandings. This is actually exciting!
p.s. as mentioned in the OP, feedback on pronunciation is planned (actually in the works).
It completely misunderstood me as well in spanish, it actually inputted english instead of spanish. I like the idea but this is not working at all for me, at least for now.
For the creator: Do not get discouraged, I hope you do get this working properly and see a lot of traction.
Exactly the level of quality we expect when the product idea starts out with "AI" and "scalable". And completely forgets (or doesn't bother think about) what beginning students actually need.
I also had difficulty getting it to understand me. Theres a couple solutions I can think of that may make this more usable:
1) Speech to text into an input field, allow the user to modify
2) I presume this is uses an LLM to generate the responses, submit the new text and give it the entire convo as context but initially ask it to "correct" the text to what would make sense in context based on similar sounding words.
Edit: Hah oh it's not too great right now at all. Tried it again and it ended up writing Cyrillic as my response despite me speaking Spanish.
Slightly off topic, but I could imagine that what you are alluding to regarding the expectation of certain words or phrases depending on the context of the conversation could be used to improve speech-to-text models. The speech could be parsed into multiple options which can ranked by a language model with the conversation context.
It does and that's indeed Whisper I'm currently using. I do have mixed feelings about it:
- On the one hand, it performs well in so many cases… and having multilingual support built-in is great!
- On the other hand: there's actually NO OPTION to Whisper to recognize just two languages (you either recognize ONE language or ANY language with it, which can cause issues depending on one's pronunciation and the language at hand.)
Will definitely turn OFF multilingual speech recognition by default, because the huge majority of negative reactions in this thread stem from this.
Yes, with German something similar happened and it misinterpreted "bitte" (please) as Peter and called me Peter from then on. I know my German is far from fluent, but I'm pretty sure what I said sounded more like bitte than Peter!
Great job! I just played with it and really liked it.
In terms of "getting it to take off", I'd suggest partnering with some language schools first. For example, there is an Alliance Francaise in my city and most decent-sized cities in the US. The difficulty, of course, is that a lot of language schools may have valid fears about AI replacing them, but I think it would be a nice tool to add as an adjunct to human-taught lessons. For example, could imagine a "teacher view" of this, where you let the teacher set up the original conversation prompt to mirror whatever individual lesson is happening in their class. Could even make this part of homework where the teacher could then ask students to role-play with it for homework and make individual student's responses available to the teacher.
Yes, the first of getting replaced makes sense. I think it's more about complementarity. There's not even speaking time in language classes and it's been like this for decades. (Language labs were supposed to make up for it, but I suspect they'll become AI-based… well, like most things )
> There's not even speaking time in language classes and it's been like this for decades.
That's not even the only issue. I took some strictly conversational classes for a while, but the problem is that it's one teacher with 5-10 students, so most of the speaking and conversing is done with people who suck as bad as you do at the language! The thing I really liked about your site (and just some of my own practicing that I've done with ChatGPT) is that I really get to converse with solely a dedicated "fluent" speaker.
That is true and I never really understood this 'babysitting' approach to language education (as in: language classes too often feel like they're designed to keep kids busy, while the goal should be to use the language.) Talking to a mirror would work better than a lot of language class activities I can think of.
Is there a way to disable or hide the suggested responses?
I've been a Duolingo French learner for 11months and I'm adept at reading French aloud or at least well enough to have it recognized by speech to text. However, my response synthesis skill is severely under-developed - I have a difficult time choosing how to respond when spoken to. I can see the suggested responses being useful to someone practicing their pronunciation, but for what I want to improve upon, they act as a crutch.
Additionally, is text-to-speech on the roadmap? It would be nice to have a fully audio conversation, even to the extent of disabling the text output of the agent or instituting touch-to-reveal. Like a lot of language learners, I am quite a bit more adept at reading and writing than I am at speaking and listening.
Finally, a small bug(?). When I've selected a language to practice, speech to text shouldn't generate text which is outside of the practice language. I purposefully got sloppy a few times and Portuguese and Arabic were generated from my speech. I understand this isn't a deal-breaker, but it broke the immersivity of the experience for me.
Thank you! I've had a few people request an option to disable suggestions but you're the first to present the use case, which is super useful to me!
I'll add an option to minimize them.
> is text-to-speech on the roadmap?
You mean an option to hide the text, right? Because there's already text-to-speech (with the teacher responding out loud, unless maybe if your phone is on silent.)
Will definitely implement this. People requested it on my previous product and I'm a fool for not having implemented it yet. Text shouldn't get in the way of listening.
> I purposefully got sloppy a few times and Portuguese and Arabic were generated from my speech
I'll add a layer of checks to get rid of this for good :)
> my response synthesis skill is severely under-developed
A million times this. I can read at pretty much an 18 year old level, but i can barely synthesize at a 14 year old level, and this is where i need pushing.
> speech to text shouldn't generate text which is outside of the practice language
Another take: the speech recognition is too good with my sloppy banlieu french. I need this to spot my mistakes and make me practice correcting myself. I said "chuis something-or-other" and it correctly parsed my intention to say "je suis", but i want that pointed out to me. also my wife corrected me from the other room on "toi" vs "tu" and Giglish just glossed over the mistake.
I am also working on an app for learning langues and I must say, you've built quite a lot in 6-7 months!
Gaining and retaining users is always a problem that startups face. With language learning apps, it's hard because the marketplace is already crowded and learners will stop when they've learned a sufficient amount or just give up.
Prioritizing marketing and promotion can be difficult because it's very easy to get sucked into a project improving the product or adding a new feature. I think you're probably in need of content that users will feel compelled to share on social media and other places. You've gotten your app on HN and ProductHunt, Good job.
Some feedback - I tried your app for Spanish. I am somewhere between A2 and B1 level. It was slow to send the recording then provide a response, the speed would make it impractical to use in a practice setting. I wanted to type in instead of speak because I knew it was going to take some time for it to process.
As a learner, I think that chat bots have a place but I much prefer to spend my time talking to a real person over an ai/chat bot. I'd rather it ask me what I wanted to talk about than having a really boring conversation (what's your name? what do you do? where are you from?, etc.)
What level would you say the majority of users that are using your app are?
> you're probably in need of content that users will feel compelled to share on social media and other places
That's a very good point.
> It was slow to send the recording then provide a response, the speed would make it impractical to use in a practice setting
Thank you for pointing that out. I haven't had any issues so far, and now realize the traffic PEAKED after this post. I should have scaled the server, my bad.
Super cool and I love that people are putting effort in this space, in 1-3 years language teachers might be a thing of the past.
I just tried this out (have tried some others as well) but currently the voice to text is just too finicky. My Spanish pronunciation is quite good, not great, but often the input just got completely messed up beyond recognition.
Thanks. Is it better with multilingual speech recognition turned off? If you give me the convo uuid I can have a look, too.
Hopefully language teachers won't be gone, but it definitely will change the industry. It's a bit like the music industry maybe? People don't buy CDs, but they go to concerts. Language teachers for their part might have better students, eager to talk and for human contact.
Greetings! I really like your idea. I have been building a similar open source/non-commercial tool. I am essentially trying to build a solution to get me ready for a standardized Korean language exam. The format and UX of Giglish are extremely good, but I think there are a few places where you need to fine tune your prompts, at least for Korean language learning. I also wish there was a drill mode that is less open ended (I’ve seen an alternative tool that has a feature called “phrase pump”). I have a hard time keeping a conversation with an AI but love the idea that it can feed me quick prompts over and over again. This is ideal for someone who is studying a language and does not have easy access to native speakers. If you ever want to hop on a call to talk about ideas or get feedback I would be happy to provide it. My contact details are pretty easy to find.
> I think there are a few places where you need to fine tune your prompts, at least for Korean language learning
Can you expand on this one, please?
> I also wish there was a drill mode that is less open ended
I have something similar to a drill mode in my previous product (to learn English) and people use it 1-3 hours a day. Want to incorporate it into Gliglish, just didn't get around to it yet but you make me realize I shouldn't wait!
Indeed it is! Very much a work-in-progress though, whenever my work schedule has a moment to spare.
> I have something similar to a drill mode in my previous product (to learn English) and people use it 1-3 hours a day. Want to incorporate it into Gliglish, just didn't get around to it yet but you make me realize I shouldn't wait!
I look forward to trying it in the future. I'm a fan of tools that let me just "plug in" and immerse for a solid hour via short drills. I wish there was a GPT-enhanced tool like using Anki, but for vocab drills and speaking practice. This was my main motivation for working on KoalaSRS
> Can you expand on this one, please?
Absolutely! When I tried the bakery scenario, I used the Korean word for "bread" but it got misinterpreted as "bell". It's not solely a transcription error (my pronunciation could use a little touch up), but I'm curious if you're using OpenAI's Whisper? If so, implementing a dynamic/context-aware prompt might improve the accuracy, and even overlook minor pronunciation glitches. I've done some tinkering in this area during my own experiments and saw some promising improvements. I'm not sure if you're using Whisper or if you've tweaked your Whisper prompts based on context, but I thought I'd mention that experience to see if it helps you tweak things a bit.
Thanks for your reply. If you ever feel like diving deeper, don't hesitate to reach out. I absolutely love discussing language learning software. I've shared some of my thoughts on my blog and in the KoalaSRS README as well, so feel free to take a peek if you're curious!
Very cool. I've been trying to learn Spanish off and on for years, and this was super easy to try. Love the fact that you can use it without even signing up.
I wonder if you could look into partnering with some existing language tools?
I might also look into telling schools about it directly, to help speed up the word of mouth.
Dag Fabian
Ben je Belg? I woonde 7 jaren in Den Haag en later dichtbij Amsterdam.
The work you have done & are doing sounds incredible.
I am a retired translator / interpreter in Denver, Colorado, USA. Over 10 years, I have single-handedly developed a quirky multilingual mega-glossary. I work on it every day, and it's now about 19,070 entries in 9 languages. EN-ES-BR-FR-IT-NL-DE-SV-FI
if you are interested, Fabian, I will send you a copy (Excel format). I resonate with what you wrote about most places forbidding self-promotion. bummer. My email address is easy to find using Google.
Sincerely,
André Fairchild
Multilingual Maverick
What are you using for the speech-to-text? One thing that I really liked about Whisper was that I could mix english and Spanish and it would just work.. like "what does "una palabra española" mean?" and it would just work. Doesn't seem to work with this though.
Hey Fabian
The work you have done & are doing sounds incredible.
I am a retired translator / interpreter in Denver, Colorado, USA. I can have single-handedly developed a quirky multilingual mega-glossary. I work on it every day, and it's now about 19,070 entries in 9 languages.
Very interesting, and free. EN-ES-BR-FR-IT-NL-DE-SV-FI
if you are interested, Fabian, I will send you a copy (Excel format). I agree with what you wrote about most places forbidding self-promotion. bummer. My email address is easy to find using Google.
Sincerely, André Fairchild
Have you thought about making more structured objectives? Shameless plug: I built a simple French/Spanish chat app and the #1 piece of feedback I got was that it's a bit aimless without any objectives.
I built out ~20 scenarios that would give users objectives like buying train tickets or ordering coffee. Now that's the most used part of the app (not the generic chat bot). As far as ads go, App Store ads are not bad and can't be ad-blocked, so that's an option.
I'll second this. Consider writing prompts for the LLM that push it towards asking questions of the user, evaluating their responses, and offering corrections and asking them to repeat themselves.
This is an interesting prompting problem, let me know if you want to collaborate. I'm not in the language space, but would gladly riff with you on the topic to improve Giglish and improve my own thinking on the topic.
- content marketing over a long period of time, to build up a following. eg, (a shameless plug) [1]
- twitter / instagram / facebook. I hear facebook groups are pretty effective
- check out indiehackers for more people like yourself
- find somebody with a bit of reach and partner with them in some win-win way. Eg, reach out to udemy language instructors? You'd be suprirised how welcoming many people are to cold emails that respect them for their expertise
Holy crap this is awesome. Talking for the first time after learning from books can be really intimidating, this is going to be a game-changing way for people to practice alone and build up to talking to other people.
signup (just notes from a battle hardened ecomm engineer):
- drop password, you already do the email confirmation loop, i almost noped out of the funnel when asked for yet another password
- transition anonymous sessions to signed up users cleanly
- drop the timezone question, too long, wtf, why even necessary
- add big signup button to login page, login is more visible than signup so add another signup cta on the login page
- see if you can get sessions to persist indefinitely and not even have to do signup
- see if you can get to email-only signup
- implement social auth so folks don't have to type their email, you are targeting mobile users who have tap (type) fatigue already
actual product:
- i need it to ask me questions, and attempt to determine my skill level on it's own. i found myself asking (in french, which this keyboard has a terrible time recognizing) "what's the deal with pension reform in France", which is just my pathology at work: I'll gladly cook up great questions to get my interlocutors talking, but i need more prompts to get my speech synthesis and auditory processing systems running better (which another poster alluded to as well).
totally tangentially, my wife corrected my French while i was playing with this toy and got a loving chiding. god bless the attentionally and impulse-inhibitorially challenged :eyeroll:
This is outstanding. And the limitations of its speech recognition are actually a feature - it forces me to really concentrate on pronunciation. Now do Mandarin and take my money please :)
Thank you . A friend's been asking for Mandarin for a while, time to deliver on my promise to add it! Will try to get it ready within 24 hours of posting this.
> And the limitations of its speech recognition are actually a feature - it forces me to really concentrate on pronunciation
I appreciate the positive comment! People complaining about the limitations of speech recognition make a valid point, but there's an ELEPHANT IN THE ROOM of language education that I feel needs to be addressed:
Language classes ignore speaking for the most part. Speaking time is ultra-limited. Phonetics are introduced super late (or never). And text is used as a form of baby-sitting to the detriment of listening (which is really –literally– backwards because the natural order of picking up a language is as follows: 1. listening 2. speaking 3. reading 4. writing.)
There's a cost to it: most people develop a fear of speaking and a confusing accent.
We can see the current limits with speech recognition and say that it and the products that use it are no good. But this feels like throwing the baby with the bathwater.
Or, we can seize it as an opportunity to improve one's accent. (Spent thousands of bucks on accent training, this is much cheaper! )
(I do plan to improve speech recognition as tech matures (fast), but there's an opportunity for improving one's accent here that would be too bad to miss.)
This is a very promising platform. However, it misses some very crucial things about learning languages.
Here goes:
Learning languages is mostly about getting tons of input. I tried using this for a few minutes, but all the input I got was some lines of text.
Also:
People say that learners don't get as much chances to output as they would like. That isn't mostly true. And even farther from the truth is to say that outputting more would lead to better language acquisition.
Hey, thanks for the input and happy to have a healthy debate!
I'm a bit confused though:
> all the input I got was some lines of text
Didn't you hear the audio? Or what else were you expecting? I've experimented with the length of answers a bit and people are put off by long answers.
·
I think people overcomplicate language acquisition. Or at least here's my two cents from learning four foreign languages to fluency and now seeing my kids grow up trilingual. There's a natural language to learning a language:
1. Listen
2. Speak
3. Read
4. Write
Foreign language education has been switching the first two with the last two and keeps wondering why the majority of learners have such thick accents. (Meanwhile, kids who grew up with YouTube sounds great.)
You need input to observe and 'pick up' the language. But you need experience to grow confident and fluent. No speaking practice means no confidence speaking. e.g. if you never spoke on the phone after years of learning you're likely to get cold sweats on your first phone call for work.
Time and motivation play a big role. You just cannot squeeze the equivalent of years of learning as a kid into a couple of months (usually before giving up) as an adult. Making language rewarding (which means seeing progress and having fun) is critical.
·
Now my observation is that people just don't talk much in most classes and that people should be able to learn foreign languages in the most natural way possible: conversations (listening and speaking.) It's somewhat ironic that ARTIFICIAL intelligence is now enabling this.
I'll put my money where my mouth is: I'm still working on the product and when it's ready will use it to learn a language FROM SCRATCH :)
I'm with you as far as input goes, but I don't understand this at all:
>Also: People say that learners don't get as much chances to output as they would like. That isn't mostly true. And even farther from the truth is to say that outputting more would lead to better language acquisition.
Isn't the only way (for most people) to actually learn a language to speak it as much as possible?
I recently released my own tool for language learners, bitextual[1]. I announced it on the language learning subreddit[2]. This kind of thing is allowed there, and it led to some useful feedback.
This is really cool. I like it. I think a few simple things would be:
-- a toggle to ensure that what you get back is automatically translated to english, and the translation is easier to read. In fact, making sure you can always see translations right away would help, even for suggestions.
-- Much simpler conversations to begin with, my Spanish is basically non-existent from when I took it in school, and the teacher input and the cafe prompt was already too far over my head.
Getting English translation is a good idea, but it should be "hold down 5 seconds" to reveal -- Allowing it to be shown instantly might actually slow language learning. Users should be allowed to struggle a few seconds first (struggle === learning) before translating.
Alternatively, it might be nice to be able to translate only individual words. If I understood 60% , inferred 35%, and was clueless about 5% of a foreign text, I think I'd end up learning more by using the smallest amount of translation possible and only translating the word I was stuck on.
It would also be nice to see support for more widely spoken languages. Arabic, Japanese, Chinese, and Hindi could be your killer apps -- If supporting non-latin text is a bottleneck, you could launch these languages in beta with some features disabled!
Overall, great product OP! This is definitely good enough to start charging for. Spending 5 minutes a day speaking Spanish is a very appealing idea for me. As silly as it sounds, I'd try selling door to door (in person) for the first few users. I think that could be much more effective for early stages. You should also consider adding a intermediate tier for people in lower income countries.
I will keep your site bookmarked -- I tested it for Spanish, but the language I really want to learn Arabic. If you can add Arabic language support, I will be the first person to sign up!
Hello, generally, Arabic is taught with two accents in language schools: Formal "Fus-ha" Arabic, and Egyptian dialect Arabic (most common by far). If you can add both here, I will go ahead and sign up for premium membership. :)
I would rather invest in a Marketing person. A lot of stuff in the language learning space is sold thru content marketing. There are a lot of influencers. And they are very open to test drive stuff like this.
@gorpomon and @jjkeddo199 Great talking points! I thought about this a lot, but still don't have a good answer: how easy should it be to translate?
- If you make it too easy, like jjkeddo199 pointed out you don't learn.
e.g. it's much more useful to watch a movie and understand say 60-70% of it than to have your work cutout for you and understand everything with subtitles.
- But this also depends on the current level you're at… And getting the difficulty just right is what gets people to feel good, learn and be "in the zone" (FLOW)! Such a good feeling!
- It's more beneficial to use THE FEW WORDS ONE KNOWS than to spread oneself too thin, seeing too many words and reading too much translated text. Time spent 100% in the language is super valuable. Story time: when I first lived in Hungary I talked like a 2-year old, but became fluent (in the sense of fast, no need to think) on ultra-limited topics in two months. This created a solid basis I was later able to build on. It's a method that works.
- At times, only translations will make a meaning crystal clear.
But I may also be at a bit of a disconnect. I learn languages by just pushing through, but not everyone has the same learning style (my wife is much more organized and does just as well.)
Sorry if this turned into a ramble…
Anyway… Would love to know what people's experience and feelings are with the whole shebang of translation.
If I could make one suggestion, add a way to input text using the keyboard. Yes, I want to practice talking, but the voice-to-text is still far from perfect, and it makes it awkward. It often mishears things, and sometimes thinks I speak other languages that I don't even know! Maybe also add a way to say that "this isn't what I meant to say", and give me the choice of the top 3 highest ranked interpretations.
Also, one feature I think would help a lot, is if the bot was a bit more active, like quiz me, comment on the errors I'm making, and build "lesson plans" so that I can focus on the topics, instead of having to think what questions I should ask.
Having said all that, what you already have is a really great start! I signed up and looking forward to check it again in a month or two.
Looking at this I've been trying to use GPT like this for a while which I'm assuming is all this is. That said, the experience leaves a lot to be desired.
1. As someone else mentioned, I am coming into this having no idea about the language. Some way to identify words would be nice. Or some actual teaching.
2. I don't like that I'm forced to talk to it. I wanted to be able to just chat with it.
1. Would having translations of the suggestions help? I obviously want to add them (along with their pronunciations), but have had to do other things first.
2. Like I said in the OP, I want to add an option to type and click.
Need to share something: I've taught English for decades and learned four foreign languages and the biggest mistake I've seen others and me do is to read too much and speak too little. Students who study a language for years before speaking develop apprehension and bad habits that take years to fix.
Wanna strike a balance between the need/encouragements to speak and the flexibility to use a tool however one wants to use it.
Love it! I built a similar product, Carlos App (carlos-app.com). Lots to take away.
re: tkaing off, Have you tried any 'growth hacking' stuff? One idea -- a "language fluency test" that people could share on social media that links back to your site. Something fun/shareable to go viral
I notice you disabled the dictionary to preserve screen real estate. Why not make the dictionary a hover-over effect per word? That's how Duolingo does it.
Really nice! I tried Russian. A few comments. (Also please add Hebrew and Arabic)
- The level was too high for me and I wasn't sure how to ask the teacher to ask me something simpler. She did say something about talking slower but my problem was with not understanding specific words. (OK, I see now that the text can be translated so that should help)
2. The suggestions assume you can read the langue but often it's not the case when one learns to speak. I don't see a translation option for the suggestions (adding pronunciation might help; can be English text)
3. It's a bit slow. I totally understand why but I think in order for users to have long conversations, speed is crucial.
4. I would be happy to pay for something like this. I think it could make sense to target kids. Many parents, me included, would be happy to pay for their kids to use something like this. I'm assuming Doulingo are going to add something similar but don't let that distract you. Great job!
1. Wanna go full-in on customization. Right now you can customize by telling the teacher what you want to talk about, but I can imagine in the near future the AI figuring where you stand and adapting as you go along. Matter of (coding) time.
2. You're right. I will add translations and audio to the suggestions. Something more urgent always came up but your feedback helps prioritize this.
3. Could you expand on this? Do you mean in terms of waiting for the answer? I've experience slowdowns when OpenAI is over capacity but generally it works well. Do you mind sharing where you used it from (country)?
yep if you could find a way to engage kids and keep them talking, even for ~5 minutes, this would be such a huge win for parents encouraging language-learning
As I'm really not fluent in the language I'm learning, it would be super useful to me to be able to click on any word in the dialog to see it's translation, no need for grammar or complete sentences, only the words I don't know!
Here is some feedback. I tried Spanish, but I don't speak it at all. The AI teacher starts speaking Spanish to me and I have no idea what is being said.
Exactly. If you're starting from zero (i.e. the learning state that the app promises to help you with), you're just learning to repeat gibberish. Which to some extent is what we're doing when we learn languages -- but hopefully no more than about 20 percent of the time.
It doesn't even tell us what to make of, gee, all those funny umlauts and stuff. I suppose I'll just have to guess.
Like most language apps - better than absolutely nothing, but on balance - pretty awful.
(Partial apologies to the creator -- I know you've worked hard on this. But still -- these are my impressions).
There's a translation button. Should I make it more prominent?
I'll also add voice and translations to the suggestions provided on every screen.
It's also a matter of learning style... I don't speak a word of Portuguese or Bosnian, but diving into those was a lot of fun! Language learning can be seen like a detective game and it's super exciting then.
In any case, will definitely try to accommodate more learning styles and personality types.
Thank you! A lot of posts seem to be focused on when things go wrong (which I DO need to hear of course), so it's good to get a simple reminder like this one.
I really love the idea -- learning a new language can be pretty embarrassing, and a chatbot is a wonderful way to practice (in theory). The site worked pretty well with German! It would be helpful to be able to click on words I'm not familiar with (or have forgotten - I haven't spoken in over a decade) and get a short definition.
Thank you, this is very specific feedback and helps me prioritize what to add next. I used to have a dictionary for English but wasn't happy with the UI. I'll try to figure something that would fit just this: looking up words (via a dictionary or otherwise – translator, explanation.)
Mexican Spanish didn't work at all. It asked me "Como te llamas" and I replied, "Me llamo Isaac", which got speech-recognized into some French sentence. I tried again saying "Si, me llamo Isaac" and it got recognized as some gibberish like "C. Mi anno E Sack" or something.
Did you try disabling multilingual speech recognition?
Speech recognition is multilingual by default, which means you can ask a question in any language. The tradeoff is that your pronunciation may be misunderstood at times.
e.g. English spoken in a thick French accent may be misinterpreted as French.
If you turn off multilingual speech recognition you won't have such issue. Maybe I should have made it the default?
A few thoughts:
- It means you can use it to improve your accent. The clearer you get, the better the transcription. But this may not be everyone's use case. (And this is something to build on.)
- I tried to keep things simple without asking users for details to get started. But I will start asking for the user's native and target language, then I'll add a layer to detect when things go wrong to avoid experiences like the one you've just had.
For what it's worth, I tried Spain Spanish afterwards and had no issues at all. My accent is pretty good (not native but grew up hearing and speaking Spanish as a kid, can sometimes pass for native if we don't talk enough that they hear my poor vocabulary).
In your Python course, I was able to run os.listdir() arbitrarily and explore your container and mounts and create files in /var/tmp.
I didn't try writing elsewhere or getting destructive, but one can imagine. I did not run a while true to exhaust your disk space, but somebody else could.
I started using it in my native language. The second sentence I spoke, it detected the first part correctly, but the second part it wrote in Chinese. The app told me sorry, I don't speak Chinese.
I think this could bridge the gap for many people stuck at A level and trying to cross to actual communication, which requires lots of interaction so that the words and sentences start popping up.
I've seen people terrified at the idea of speaking (hell, I experienced it myself decades ago.) It's not always even a matter of level. People with good grammar and a lot of vocabulary will have zero ease speaking for lack of practice.
On the other hand, at some point, my super limited Hungarian (of maybe 300 to 600 words?) was getting me praise just because I could use the little I knew without thinking.
I added Bosnian because of a trip I took a couple of months ago (big on using my own product to learn more languages – "eat your own dog food" kinda way .)
How do you feel about your pronunciation of Bosnian? Could you get your wife to try it? How
The elephant in the room with language education is that it's tough to learn clear pronunciation even though it's super useful. Things get a bit messed up with speech recognition:
1. Speech recognition is designed to understand no matter what (which is kinda stupid – in natural language when things are incomprehensible [because of noise, lack of context, mispronunciation, or whatever we just ASK people to repeat]
2. Speech recognition is trained on native speakers most of the time. But it would make sense to do two opposite things: 1) train it on non-native speakers as well (to improve performance), things are getting there… 2) train it to correct non-native speakers (some projects in that direction too, mostly for English.)
I'll add my own layer of checks to handle such cases. So that when speech recognition goes wrong (as it inevitably will – with HIGH variability between different contexts and speakers) the teacher does not get off track because of it.
Unfortunately speech recognition is very poor for Bosnian, and that seems to be the case here as well. I am not sure how usable it is for the described use case is, but given that it didn't understand 3/3 of my sentences at all, I wouldn't hold my breath yet.
incredible. As a non native speaker, I always want to improve my speaking skills. And I also thought of building something using AI. But you have nailed it. Kudos.
Multilingual speech recognition is proving both a curse and a blessing. I will:
1. Turn it off by default
2. Ask people for their native and target language so as to detect when things go wrong. In which case, I'd give a second shot at transcribing and, if nothing comes out of it, just ask you to repeat nicely. (As it happens in real conversations.)
Is it possible something more than the speed throttle for beginners? Could the AI use simpler language for beginners and more complex for advanced users?
Thank you! Turkish (along with Japanese and Chinese) has been on my mind for a while. If I know people will use it, I'll add it. Will try to add Turkish within 24 hours.
So here's some more balanced feedback, if you will (playing with a language I know quite well already):
On the positive side: the bot itself has excellent voice quality (it still sounds bot-like -- but people expect that) and makes very few pronunciation mistakes (I only noticed one). Its vocal recognition is, I would say ... not great, but good enough (it recognizes well enough if I say simple, standard things, but it easily confused if I use less common -- but also perfectly correct -- formulations). At least, vocal recognition as such is not its weakest spot.
You can ask it questions about fairly detailed topics and get reasonably correct answers most of the time (as with GPT, which I'm guessing is its backend). It can sometimes give you very nuanced feedback as to your diction and usage (it will notice when you're using a formulation that is correct but not standard, and give you a detailed explanation as to why).
However, what I don't like -- and what made me ultimately decide not to invest further time with the app at present:
(1) You can easily feel lost (in "Teacher" mode) because all it can do is answer questions -- it isn't guiding you anywhere, and you have to already know what questions to ask.
(2) Even so, I thought to myself -- "well I can see how a good Q+A bot can be useful -- let's see what we can do with it". And here's where I got most disappointed.
I told it I wanted to practice my pronunciation -- and first asked for it to come up with some sample sentences, and then thought of one of my own -- one of those sentences you would say to someone at the family dinner table to try to wow them with your language skills.
In both drills, I played a little trick on the poor bot -- I deliberately mispronounced the sentences, the way a stereotypical Ugly American would -- basically mangling every single syllable in some obvious way, hoping to get a long list of corrections from the bot.
And it totally failed at this task. It kept saying "You said that perfectly! Well done!" or some variant. When I knew for a fact that my pronunciation was an epic fail, and I would have greatly embarrassed myself at that family holiday dinner.
(3) But things got even weirder when I tried to tell the bot "Wait you're wrong, I actually made lots of mistakes." More than half the time it would get horribly confused, sometimes telling me -again- that I said that sentence perfectly, sometimes "correcting" its own sentence.
So it's quite scatterbrained (just like GPT-4 is, when you try to make it backtrack and acknowledge some kind of mistake). It also misunderstands questions about grammar if I ask it to explain something it just said, or which I said in a sentence previously.
Bottom line -- it's a very mixed bag. Partially quite impressive -- but far too often, extremely frustrating to work with. Once one gets past the "wow!" aspect of working with a bot that seems well-spoken and (quantitatively) seems to have a lot of knowledge about grammar and usage -- the frustration arising from its scatterbrained-ness, and more fundamentally, just never knowing if I can really trust its answers takes over.
On the whole it just grinds my gears ... like pretty much any other chat bot I've attempted to try to get useful information out of.
That and again -- it isn't guiding you anywhere. If it can't reliably keep track of what it just said (or what I just said), it's definitely not going to be able to assess my progress, or tell me how to improve (beyond the random piece of generic advice it doles when it doesn't have anything better to say).
Of course it won't. It's just a transformer bot, after all. And the technology is categorically nowhere capable of doing that.
Hey!
Thanks for the balanced feedback, all totally valid.
(1) I'm self-directed in my learning and always knew what I wanted my teachers to do, but everyone's different. Definitely going to add more choice of content (and not just roleplay.) I have a very structured and successful program to learn English and will try to adapt what works well in it to Gliglish.
(2) You're right! The speech recognition and LLM layers are separate. I actually do have data about the quality of pronunciation, I'm just not using it yet. It's a matter of adding a layer to handle it. Basically, all the building blocks are here and I need to get them to play nicely together.
(3) It's the same issue as with ChatGPT on which it's built. Part of the progress will come from OpenAI or similar endeavors themselves, part from further checks on my end as in (2).
I just don't share your pessimistic view about the future of such tech though (well, if I did I probably wouldn't have started the whole thing! ) Once you put ChatGPT into a corner, there's usually no hope of it getting back on track. BUT with clear context and prompts on things it does well it's just incredibly useful! Again, is the glass half empty or half full?
As for this use case, there's tremendous value in being able to practice. Just talking to oneself in the mirror or journaling in a foreign language is already super productive and this is it… on steroids.
Just talking to oneself in the mirror or journaling in a foreign language is already super productive and this is it… on steroids.
I agree that a working pronunciation "coach" could be highly useful. And a great distinguishing feature for the app, if done right. However right now the app objectively fails in that category. It is simply unable to tell me when I'm not saying things properly.
(Which I hope you will take positively. The main thing is -- you're doing it, putting it out there. Best of luck).
1) These speech-to-text models are poor when it comes to non-natives. This is unfortunate as the idea you had and the product you've designed could be incredible for language learning. However - it's a bit crap - sorry - I can speak Spanish well and was asked in the conversation if I wanted a medium sized cup of coffee, I replied "sí, mediano", the resulting text was outputted as "mariano", then in the role play the coffee shop worker then assumed my name was Mariano! Completely ludicrous and frustrating.... in real life the coffee shop worker is clearing expecting the word 'mediano' and will hear what I said and know that's what I was trying to say. The speech-to-text-model completely fails to get this.
Until speech-to-text models trained on non-natives are made readily available, products like this with so much promise will infuriate learners, which will stop them paying for it.
And this was ordering a coffee.... imagine an actually complicated conversation.
So my advice would be, right now the speech-to-text models aren't capable of doing what you're hoping they can do... but.... once you get a model that can, this will be insanely popular....
So hang in there, other than that it was a fun experience, and critically, people are scared of practising with real people, something like this would be insanely popular if it actually worked well. Good luck.
Even in its current state this is an awesome product. There are so so many people in the world learning a language, and one of the hardest parts is practicing after you stop learning (like you leaving Spain). People like that will really love something like this
In future especially to be able to cater to people who are still learning, it should be feasible to use a similar product to train and correct people's pronounciation.
Speech recognition is far from perfect but even then it's incredibly useful. It CAN be infuriating (or downright hilarious). Hell, it's infuriating when occasionally a (human!) waiter switches to English after I mangled a sound in Hungarian, even though I'm C1/advanced in the language.
Problem or opportunity? Like someone else pointed out, the limitations of speech-to-text can be turned into an opportunity to improve one's pronunciation. It's getting extremely good for native speakers. As foreign speakers it's a chance to improve.
In any case, I'm sure I can add a layer or two to the code to reduce misunderstandings. This is actually exciting!
p.s. as mentioned in the OP, feedback on pronunciation is planned (actually in the works).
For the creator: Do not get discouraged, I hope you do get this working properly and see a lot of traction.
Not sure what hating on 'AI' and 'scalable' (your word by the way, not mine) accomplishes.
1) Speech to text into an input field, allow the user to modify
2) I presume this is uses an LLM to generate the responses, submit the new text and give it the entire convo as context but initially ask it to "correct" the text to what would make sense in context based on similar sounding words.
Edit: Hah oh it's not too great right now at all. Tried it again and it ended up writing Cyrillic as my response despite me speaking Spanish.
- On the one hand, it performs well in so many cases… and having multilingual support built-in is great! - On the other hand: there's actually NO OPTION to Whisper to recognize just two languages (you either recognize ONE language or ANY language with it, which can cause issues depending on one's pronunciation and the language at hand.)
Will definitely turn OFF multilingual speech recognition by default, because the huge majority of negative reactions in this thread stem from this.
In terms of "getting it to take off", I'd suggest partnering with some language schools first. For example, there is an Alliance Francaise in my city and most decent-sized cities in the US. The difficulty, of course, is that a lot of language schools may have valid fears about AI replacing them, but I think it would be a nice tool to add as an adjunct to human-taught lessons. For example, could imagine a "teacher view" of this, where you let the teacher set up the original conversation prompt to mirror whatever individual lesson is happening in their class. Could even make this part of homework where the teacher could then ask students to role-play with it for homework and make individual student's responses available to the teacher.
That's not even the only issue. I took some strictly conversational classes for a while, but the problem is that it's one teacher with 5-10 students, so most of the speaking and conversing is done with people who suck as bad as you do at the language! The thing I really liked about your site (and just some of my own practicing that I've done with ChatGPT) is that I really get to converse with solely a dedicated "fluent" speaker.
Again, thanks and great job!
Will try to build on this positive, thank you!
I've been a Duolingo French learner for 11months and I'm adept at reading French aloud or at least well enough to have it recognized by speech to text. However, my response synthesis skill is severely under-developed - I have a difficult time choosing how to respond when spoken to. I can see the suggested responses being useful to someone practicing their pronunciation, but for what I want to improve upon, they act as a crutch.
Additionally, is text-to-speech on the roadmap? It would be nice to have a fully audio conversation, even to the extent of disabling the text output of the agent or instituting touch-to-reveal. Like a lot of language learners, I am quite a bit more adept at reading and writing than I am at speaking and listening.
Finally, a small bug(?). When I've selected a language to practice, speech to text shouldn't generate text which is outside of the practice language. I purposefully got sloppy a few times and Portuguese and Arabic were generated from my speech. I understand this isn't a deal-breaker, but it broke the immersivity of the experience for me.
I'll add an option to minimize them.
> is text-to-speech on the roadmap?
You mean an option to hide the text, right? Because there's already text-to-speech (with the teacher responding out loud, unless maybe if your phone is on silent.)
Will definitely implement this. People requested it on my previous product and I'm a fool for not having implemented it yet. Text shouldn't get in the way of listening.
> I purposefully got sloppy a few times and Portuguese and Arabic were generated from my speech
I'll add a layer of checks to get rid of this for good :)
A million times this. I can read at pretty much an 18 year old level, but i can barely synthesize at a 14 year old level, and this is where i need pushing.
> speech to text shouldn't generate text which is outside of the practice language
Another take: the speech recognition is too good with my sloppy banlieu french. I need this to spot my mistakes and make me practice correcting myself. I said "chuis something-or-other" and it correctly parsed my intention to say "je suis", but i want that pointed out to me. also my wife corrected me from the other room on "toi" vs "tu" and Giglish just glossed over the mistake.
Gaining and retaining users is always a problem that startups face. With language learning apps, it's hard because the marketplace is already crowded and learners will stop when they've learned a sufficient amount or just give up.
Prioritizing marketing and promotion can be difficult because it's very easy to get sucked into a project improving the product or adding a new feature. I think you're probably in need of content that users will feel compelled to share on social media and other places. You've gotten your app on HN and ProductHunt, Good job.
Some feedback - I tried your app for Spanish. I am somewhere between A2 and B1 level. It was slow to send the recording then provide a response, the speed would make it impractical to use in a practice setting. I wanted to type in instead of speak because I knew it was going to take some time for it to process.
As a learner, I think that chat bots have a place but I much prefer to spend my time talking to a real person over an ai/chat bot. I'd rather it ask me what I wanted to talk about than having a really boring conversation (what's your name? what do you do? where are you from?, etc.)
What level would you say the majority of users that are using your app are?
Good luck with your app, it's a huge effort.
> you're probably in need of content that users will feel compelled to share on social media and other places
That's a very good point.
> It was slow to send the recording then provide a response, the speed would make it impractical to use in a practice setting
Thank you for pointing that out. I haven't had any issues so far, and now realize the traffic PEAKED after this post. I should have scaled the server, my bad.
> I'd rather it ask me what I wanted to talk
Good point.
I just tried this out (have tried some others as well) but currently the voice to text is just too finicky. My Spanish pronunciation is quite good, not great, but often the input just got completely messed up beyond recognition.
Hopefully language teachers won't be gone, but it definitely will change the industry. It's a bit like the music industry maybe? People don't buy CDs, but they go to concerts. Language teachers for their part might have better students, eager to talk and for human contact.
> I think there are a few places where you need to fine tune your prompts, at least for Korean language learning
Can you expand on this one, please?
> I also wish there was a drill mode that is less open ended
I have something similar to a drill mode in my previous product (to learn English) and people use it 1-3 hours a day. Want to incorporate it into Gliglish, just didn't get around to it yet but you make me realize I shouldn't wait!
Is your project KoalaSRS?
Indeed it is! Very much a work-in-progress though, whenever my work schedule has a moment to spare.
> I have something similar to a drill mode in my previous product (to learn English) and people use it 1-3 hours a day. Want to incorporate it into Gliglish, just didn't get around to it yet but you make me realize I shouldn't wait!
I look forward to trying it in the future. I'm a fan of tools that let me just "plug in" and immerse for a solid hour via short drills. I wish there was a GPT-enhanced tool like using Anki, but for vocab drills and speaking practice. This was my main motivation for working on KoalaSRS
> Can you expand on this one, please?
Absolutely! When I tried the bakery scenario, I used the Korean word for "bread" but it got misinterpreted as "bell". It's not solely a transcription error (my pronunciation could use a little touch up), but I'm curious if you're using OpenAI's Whisper? If so, implementing a dynamic/context-aware prompt might improve the accuracy, and even overlook minor pronunciation glitches. I've done some tinkering in this area during my own experiments and saw some promising improvements. I'm not sure if you're using Whisper or if you've tweaked your Whisper prompts based on context, but I thought I'd mention that experience to see if it helps you tweak things a bit.
Thanks for your reply. If you ever feel like diving deeper, don't hesitate to reach out. I absolutely love discussing language learning software. I've shared some of my thoughts on my blog and in the KoalaSRS README as well, so feel free to take a peek if you're curious!
I wonder if you could look into partnering with some existing language tools? I might also look into telling schools about it directly, to help speed up the word of mouth.
Gotta look into language schools, you're right… Someone suggested doing a deal with the Canadian government because everything is bilingual there.
Big life decisions of working on the product vs. working to promote it. Balance is hard
I built out ~20 scenarios that would give users objectives like buying train tickets or ordering coffee. Now that's the most used part of the app (not the generic chat bot). As far as ads go, App Store ads are not bad and can't be ad-blocked, so that's an option.
Would love feedback! https://apps.apple.com/app/verbius-ai-language-learning/id64...
This is an interesting prompting problem, let me know if you want to collaborate. I'm not in the language space, but would gladly riff with you on the topic to improve Giglish and improve my own thinking on the topic.
- content marketing over a long period of time, to build up a following. eg, (a shameless plug) [1]
- twitter / instagram / facebook. I hear facebook groups are pretty effective
- check out indiehackers for more people like yourself
- find somebody with a bit of reach and partner with them in some win-win way. Eg, reach out to udemy language instructors? You'd be suprirised how welcoming many people are to cold emails that respect them for their expertise
- repeat launches on product hunt
[1] https://barbariangrunge.substack.com/
actual product: - i need it to ask me questions, and attempt to determine my skill level on it's own. i found myself asking (in french, which this keyboard has a terrible time recognizing) "what's the deal with pension reform in France", which is just my pathology at work: I'll gladly cook up great questions to get my interlocutors talking, but i need more prompts to get my speech synthesis and auditory processing systems running better (which another poster alluded to as well).
totally tangentially, my wife corrected my French while i was playing with this toy and got a loving chiding. god bless the attentionally and impulse-inhibitorially challenged :eyeroll:
> And the limitations of its speech recognition are actually a feature - it forces me to really concentrate on pronunciation
I appreciate the positive comment! People complaining about the limitations of speech recognition make a valid point, but there's an ELEPHANT IN THE ROOM of language education that I feel needs to be addressed:
Language classes ignore speaking for the most part. Speaking time is ultra-limited. Phonetics are introduced super late (or never). And text is used as a form of baby-sitting to the detriment of listening (which is really –literally– backwards because the natural order of picking up a language is as follows: 1. listening 2. speaking 3. reading 4. writing.)
There's a cost to it: most people develop a fear of speaking and a confusing accent.
We can see the current limits with speech recognition and say that it and the products that use it are no good. But this feels like throwing the baby with the bathwater.
Or, we can seize it as an opportunity to improve one's accent. (Spent thousands of bucks on accent training, this is much cheaper! )
(I do plan to improve speech recognition as tech matures (fast), but there's an opportunity for improving one's accent here that would be too bad to miss.)
Here goes: Learning languages is mostly about getting tons of input. I tried using this for a few minutes, but all the input I got was some lines of text.
Also: People say that learners don't get as much chances to output as they would like. That isn't mostly true. And even farther from the truth is to say that outputting more would lead to better language acquisition.
Nice try, though.
I'm a bit confused though:
> all the input I got was some lines of text
Didn't you hear the audio? Or what else were you expecting? I've experimented with the length of answers a bit and people are put off by long answers.
·
I think people overcomplicate language acquisition. Or at least here's my two cents from learning four foreign languages to fluency and now seeing my kids grow up trilingual. There's a natural language to learning a language:
1. Listen 2. Speak 3. Read 4. Write
Foreign language education has been switching the first two with the last two and keeps wondering why the majority of learners have such thick accents. (Meanwhile, kids who grew up with YouTube sounds great.)
You need input to observe and 'pick up' the language. But you need experience to grow confident and fluent. No speaking practice means no confidence speaking. e.g. if you never spoke on the phone after years of learning you're likely to get cold sweats on your first phone call for work.
Time and motivation play a big role. You just cannot squeeze the equivalent of years of learning as a kid into a couple of months (usually before giving up) as an adult. Making language rewarding (which means seeing progress and having fun) is critical.
·
Now my observation is that people just don't talk much in most classes and that people should be able to learn foreign languages in the most natural way possible: conversations (listening and speaking.) It's somewhat ironic that ARTIFICIAL intelligence is now enabling this.
I'll put my money where my mouth is: I'm still working on the product and when it's ready will use it to learn a language FROM SCRATCH :)
>Also: People say that learners don't get as much chances to output as they would like. That isn't mostly true. And even farther from the truth is to say that outputting more would lead to better language acquisition.
Isn't the only way (for most people) to actually learn a language to speak it as much as possible?
I recently released my own tool for language learners, bitextual[1]. I announced it on the language learning subreddit[2]. This kind of thing is allowed there, and it led to some useful feedback.
[1] https://bitextual.net [2] https://www.reddit.com/r/languagelearning/
r/languagelearning/ does not currently accept anything relating to AI. Here's the messages I got:
Automatically:
> Thank you for posting on r/languagelearning. Due to their frequency, posts relating to AI and chat bots have been banned from the sub.
And from the mods:
> Hello--it does look good. Unfortunately, we currently are severely curbing chatbot posts.
-- a toggle to ensure that what you get back is automatically translated to english, and the translation is easier to read. In fact, making sure you can always see translations right away would help, even for suggestions.
-- Much simpler conversations to begin with, my Spanish is basically non-existent from when I took it in school, and the teacher input and the cafe prompt was already too far over my head.
Alternatively, it might be nice to be able to translate only individual words. If I understood 60% , inferred 35%, and was clueless about 5% of a foreign text, I think I'd end up learning more by using the smallest amount of translation possible and only translating the word I was stuck on.
It would also be nice to see support for more widely spoken languages. Arabic, Japanese, Chinese, and Hindi could be your killer apps -- If supporting non-latin text is a bottleneck, you could launch these languages in beta with some features disabled!
Overall, great product OP! This is definitely good enough to start charging for. Spending 5 minutes a day speaking Spanish is a very appealing idea for me. As silly as it sounds, I'd try selling door to door (in person) for the first few users. I think that could be much more effective for early stages. You should also consider adding a intermediate tier for people in lower income countries.
I will keep your site bookmarked -- I tested it for Spanish, but the language I really want to learn Arabic. If you can add Arabic language support, I will be the first person to sign up!
Will add Japanese and Chinese momentarily.
Which variation of Arabic would you want to learn?
You make a good point. I might need to hire a salesperson or do a business deal for this.
- If you make it too easy, like jjkeddo199 pointed out you don't learn. e.g. it's much more useful to watch a movie and understand say 60-70% of it than to have your work cutout for you and understand everything with subtitles.
- But this also depends on the current level you're at… And getting the difficulty just right is what gets people to feel good, learn and be "in the zone" (FLOW)! Such a good feeling!
- It's more beneficial to use THE FEW WORDS ONE KNOWS than to spread oneself too thin, seeing too many words and reading too much translated text. Time spent 100% in the language is super valuable. Story time: when I first lived in Hungary I talked like a 2-year old, but became fluent (in the sense of fast, no need to think) on ultra-limited topics in two months. This created a solid basis I was later able to build on. It's a method that works.
- At times, only translations will make a meaning crystal clear.
But I may also be at a bit of a disconnect. I learn languages by just pushing through, but not everyone has the same learning style (my wife is much more organized and does just as well.)
Sorry if this turned into a ramble…
Anyway… Would love to know what people's experience and feelings are with the whole shebang of translation.
If I could make one suggestion, add a way to input text using the keyboard. Yes, I want to practice talking, but the voice-to-text is still far from perfect, and it makes it awkward. It often mishears things, and sometimes thinks I speak other languages that I don't even know! Maybe also add a way to say that "this isn't what I meant to say", and give me the choice of the top 3 highest ranked interpretations.
Also, one feature I think would help a lot, is if the bot was a bit more active, like quiz me, comment on the errors I'm making, and build "lesson plans" so that I can focus on the topics, instead of having to think what questions I should ask.
Having said all that, what you already have is a really great start! I signed up and looking forward to check it again in a month or two.
1. As someone else mentioned, I am coming into this having no idea about the language. Some way to identify words would be nice. Or some actual teaching. 2. I don't like that I'm forced to talk to it. I wanted to be able to just chat with it.
1. Would having translations of the suggestions help? I obviously want to add them (along with their pronunciations), but have had to do other things first.
2. Like I said in the OP, I want to add an option to type and click.
Need to share something: I've taught English for decades and learned four foreign languages and the biggest mistake I've seen others and me do is to read too much and speak too little. Students who study a language for years before speaking develop apprehension and bad habits that take years to fix.
Wanna strike a balance between the need/encouragements to speak and the flexibility to use a tool however one wants to use it.
I appreciate that! I do have a lot of trouble with that but I also tend to mostly be a text-based communicator if I can help it lol
re: tkaing off, Have you tried any 'growth hacking' stuff? One idea -- a "language fluency test" that people could share on social media that links back to your site. Something fun/shareable to go viral
I notice you disabled the dictionary to preserve screen real estate. Why not make the dictionary a hover-over effect per word? That's how Duolingo does it.
- The level was too high for me and I wasn't sure how to ask the teacher to ask me something simpler. She did say something about talking slower but my problem was with not understanding specific words. (OK, I see now that the text can be translated so that should help)
2. The suggestions assume you can read the langue but often it's not the case when one learns to speak. I don't see a translation option for the suggestions (adding pronunciation might help; can be English text)
3. It's a bit slow. I totally understand why but I think in order for users to have long conversations, speed is crucial.
4. I would be happy to pay for something like this. I think it could make sense to target kids. Many parents, me included, would be happy to pay for their kids to use something like this. I'm assuming Doulingo are going to add something similar but don't let that distract you. Great job!
1. Wanna go full-in on customization. Right now you can customize by telling the teacher what you want to talk about, but I can imagine in the near future the AI figuring where you stand and adapting as you go along. Matter of (coding) time.
2. You're right. I will add translations and audio to the suggestions. Something more urgent always came up but your feedback helps prioritize this.
3. Could you expand on this? Do you mean in terms of waiting for the answer? I've experience slowdowns when OpenAI is over capacity but generally it works well. Do you mind sharing where you used it from (country)?
4. Thank you, this is encouraging.
Super excited about all this!
As I'm really not fluent in the language I'm learning, it would be super useful to me to be able to click on any word in the dialog to see it's translation, no need for grammar or complete sentences, only the words I don't know!
Not sure how I can learn anything from this.
It doesn't even tell us what to make of, gee, all those funny umlauts and stuff. I suppose I'll just have to guess.
Like most language apps - better than absolutely nothing, but on balance - pretty awful.
(Partial apologies to the creator -- I know you've worked hard on this. But still -- these are my impressions).
I'll also add voice and translations to the suggestions provided on every screen.
It's also a matter of learning style... I don't speak a word of Portuguese or Bosnian, but diving into those was a lot of fun! Language learning can be seen like a detective game and it's super exciting then.
In any case, will definitely try to accommodate more learning styles and personality types.
Speech recognition is multilingual by default, which means you can ask a question in any language. The tradeoff is that your pronunciation may be misunderstood at times.
e.g. English spoken in a thick French accent may be misinterpreted as French.
If you turn off multilingual speech recognition you won't have such issue. Maybe I should have made it the default?
A few thoughts:
- It means you can use it to improve your accent. The clearer you get, the better the transcription. But this may not be everyone's use case. (And this is something to build on.) - I tried to keep things simple without asking users for details to get started. But I will start asking for the user's native and target language, then I'll add a layer to detect when things go wrong to avoid experiences like the one you've just had.
Thank you for sharing.
I didn't try writing elsewhere or getting destructive, but one can imagine. I did not run a while true to exhaust your disk space, but somebody else could.
On the other hand, at some point, my super limited Hungarian (of maybe 300 to 600 words?) was getting me praise just because I could use the little I knew without thinking.
”Practice, practice, practice!…”
Was it just easy to offer or is there more of a connection.
My wife is Bosnian and I have learned some over the years.
How do you feel about your pronunciation of Bosnian? Could you get your wife to try it? How
The elephant in the room with language education is that it's tough to learn clear pronunciation even though it's super useful. Things get a bit messed up with speech recognition:
1. Speech recognition is designed to understand no matter what (which is kinda stupid – in natural language when things are incomprehensible [because of noise, lack of context, mispronunciation, or whatever we just ASK people to repeat]
2. Speech recognition is trained on native speakers most of the time. But it would make sense to do two opposite things: 1) train it on non-native speakers as well (to improve performance), things are getting there… 2) train it to correct non-native speakers (some projects in that direction too, mostly for English.)
I'll add my own layer of checks to handle such cases. So that when speech recognition goes wrong (as it inevitably will – with HIGH variability between different contexts and speakers) the teacher does not get off track because of it.
Sorry this got so long!
Thank you for sharing your feedback :)
How do you feel about your accent? Did you try with multilingual speech recognition off?
Multilingual speech recognition is proving both a curse and a blessing. I will:
1. Turn it off by default 2. Ask people for their native and target language so as to detect when things go wrong. In which case, I'd give a second shot at transcribing and, if nothing comes out of it, just ask you to repeat nicely. (As it happens in real conversations.)
Is it possible something more than the speed throttle for beginners? Could the AI use simpler language for beginners and more complex for advanced users?
I love the idea though!
On the positive side: the bot itself has excellent voice quality (it still sounds bot-like -- but people expect that) and makes very few pronunciation mistakes (I only noticed one). Its vocal recognition is, I would say ... not great, but good enough (it recognizes well enough if I say simple, standard things, but it easily confused if I use less common -- but also perfectly correct -- formulations). At least, vocal recognition as such is not its weakest spot.
You can ask it questions about fairly detailed topics and get reasonably correct answers most of the time (as with GPT, which I'm guessing is its backend). It can sometimes give you very nuanced feedback as to your diction and usage (it will notice when you're using a formulation that is correct but not standard, and give you a detailed explanation as to why).
However, what I don't like -- and what made me ultimately decide not to invest further time with the app at present:
(1) You can easily feel lost (in "Teacher" mode) because all it can do is answer questions -- it isn't guiding you anywhere, and you have to already know what questions to ask.
(2) Even so, I thought to myself -- "well I can see how a good Q+A bot can be useful -- let's see what we can do with it". And here's where I got most disappointed.
I told it I wanted to practice my pronunciation -- and first asked for it to come up with some sample sentences, and then thought of one of my own -- one of those sentences you would say to someone at the family dinner table to try to wow them with your language skills.
In both drills, I played a little trick on the poor bot -- I deliberately mispronounced the sentences, the way a stereotypical Ugly American would -- basically mangling every single syllable in some obvious way, hoping to get a long list of corrections from the bot.
And it totally failed at this task. It kept saying "You said that perfectly! Well done!" or some variant. When I knew for a fact that my pronunciation was an epic fail, and I would have greatly embarrassed myself at that family holiday dinner.
(3) But things got even weirder when I tried to tell the bot "Wait you're wrong, I actually made lots of mistakes." More than half the time it would get horribly confused, sometimes telling me -again- that I said that sentence perfectly, sometimes "correcting" its own sentence.
So it's quite scatterbrained (just like GPT-4 is, when you try to make it backtrack and acknowledge some kind of mistake). It also misunderstands questions about grammar if I ask it to explain something it just said, or which I said in a sentence previously.
Bottom line -- it's a very mixed bag. Partially quite impressive -- but far too often, extremely frustrating to work with. Once one gets past the "wow!" aspect of working with a bot that seems well-spoken and (quantitatively) seems to have a lot of knowledge about grammar and usage -- the frustration arising from its scatterbrained-ness, and more fundamentally, just never knowing if I can really trust its answers takes over.
On the whole it just grinds my gears ... like pretty much any other chat bot I've attempted to try to get useful information out of.
That and again -- it isn't guiding you anywhere. If it can't reliably keep track of what it just said (or what I just said), it's definitely not going to be able to assess my progress, or tell me how to improve (beyond the random piece of generic advice it doles when it doesn't have anything better to say).
Of course it won't. It's just a transformer bot, after all. And the technology is categorically nowhere capable of doing that.
(1) I'm self-directed in my learning and always knew what I wanted my teachers to do, but everyone's different. Definitely going to add more choice of content (and not just roleplay.) I have a very structured and successful program to learn English and will try to adapt what works well in it to Gliglish.
(2) You're right! The speech recognition and LLM layers are separate. I actually do have data about the quality of pronunciation, I'm just not using it yet. It's a matter of adding a layer to handle it. Basically, all the building blocks are here and I need to get them to play nicely together.
(3) It's the same issue as with ChatGPT on which it's built. Part of the progress will come from OpenAI or similar endeavors themselves, part from further checks on my end as in (2).
I just don't share your pessimistic view about the future of such tech though (well, if I did I probably wouldn't have started the whole thing! ) Once you put ChatGPT into a corner, there's usually no hope of it getting back on track. BUT with clear context and prompts on things it does well it's just incredibly useful! Again, is the glass half empty or half full?
As for this use case, there's tremendous value in being able to practice. Just talking to oneself in the mirror or journaling in a foreign language is already super productive and this is it… on steroids.
I agree that a working pronunciation "coach" could be highly useful. And a great distinguishing feature for the app, if done right. However right now the app objectively fails in that category. It is simply unable to tell me when I'm not saying things properly.
(Which I hope you will take positively. The main thing is -- you're doing it, putting it out there. Best of luck).