Ask HN: Conversational AI to Learn a Language
Hi all,
I want to learn Dutch and by experience I know I learn better when talking with native speakers.
From your experience, is there a good AI I can converse with in Dutch? It would be even better if I could see the transcription in Dutch.
Gliglish (https://gliglish.com) looks good but is more for speaking than for learning. I'd like to be able to set a situation (negotiating a job offer, calling a supplier ...).
Hi! I have a WIP of this over at https://talktrainer.app/ -- I just added Dutch to it.
It uses OpenAI's realtime API to simulate either a tutoring session (the speaker will revert to English to help you) or a first date or business meeting (the speaker will always speak the target language)
You can see the AI's transcriptions but not your own, limitation of the current OpenAI API but definitely something I can fix.
The prompts are like this: https://gist.github.com/jc4p/d8b9d121425ec191d62602d8720eeed... and the rest of it is a Nextjs app wrapped around the WebRTC connection.
I'm not fully in love with the app so I'd love any feedback or hearing if it works well for you -- It doesn't have a lot of features yet (including saving context) and if you bump into the time limit just open it up in incognito to keep going.
This is really impressive! Great job.
One small piece of feedback… There were a couple times where I asked to learn something, and it asked me to repeat a phrase back, which was great. But when I repeated it back, I know I didn’t quite nail it (eg perhaps said “un” instead of “una”) and rather than correcting me, it actually told me I did it perfectly. Maybe there’s some tuning with the prompts that may help turn down the natural sycophancy of the model and make sure it’s a little more strict.
Keep up the great work!
I just tried it and it works perfectly. The color scheme and font size could be touched up to look better. Just out of curiosity, is $10/month enough to cover the (unlimited) API cost? Do you estimate how many percentage of your users will use more than $10 API fee each month?
Thanks so much for trying it out! The realtime API is actually very cheap especially for short connections, for each user who uses it 30 minutes a day every day in a month it costs me ~$5 and I assume the average user is going to use it way less than that (although i have 0 users right now haha)
This is great! Well done.
I've used the realtime API for something similar (also related to practicing speaking, though not for foreign languages). I just wanted to comment that the realtime API will definitely give you the user's transcriptions -- they come back as an `server.conversation.item.input_audio_transcription.completed` event. I use it in my app for exactly that purpose.
Thank you so much!! While the transcription is technically in the API it's not a native part of the model and runs through Whisper separately, in my testing with it I often end up with a transcription that's a different language than what the user is speaking and the current API has no way to force a language on the internal Whisper call.
If the language is correct, a lot of the times the exact text isn't 100% accurate, if that's 100% accurate, it comes in slower than the audio output and not in real time. All in all not what I would consider feature ready to release in my app.
What I've been thinking about is switching to a full audio in --> transcribe --> send to LLM --> TTS pipeline, in which case I would be able to show the exact input to the model, but that's way more work than just one single OpenAI API call.
Our Voice AI Agent makes it extremely simple and engaging to learn new language - create card dynamically, asks multiple-choice-questions, uses diagrams, does role play ...
Demo: https://www.youtube.com/watch?v=2iSIVnLR-nM Website: https://app.toughtongueai.com/
If you have a ChatGPT subscription, set up your own GPT with prompting around your level, how you want it to respond, how to correct mistakes etc. Then you can use it for anything - Generate tests based on words you know, roleplay like ordering in a restaurant, write stories and have it correct grammar.
This is what I have to supplement my Chinese and it is incredibly helpful.
Look at the comments already - Everyone is building a simple wrapper to do this very thing but charge you $20 per month for the privelege. These are souless, most likely vibe coded garbage. Avoid.
You don't even need a subscription, just start the conversation with something like "I'm trying to learn x, I'm at a beginner skill level. Can you have a conversation with me and correct my mistakes." and it works superbly.
The Duolingo CEO copped a lot of criticism for it, but I think he is right that LLMs play a huge role in the future of education, though he probably ragebaited everyone by overselling it and calling teachers babysitters. But ChatGPT as it is now is a better language learning tool than their hand crafted app is. Rather than clicking on word blocks, you can actually have a free form conversation and get feedback like "Yes your sentence is understandable but sounds unnatural, you could try ___"
You can also share them, e.g. someone made a specialised ChatGPT for korean:
https://chatgpt.com/g/g-erkTp2LNZ-learn-korean-with-gpt
I'm not sure how it works though. Just a canned prompt?
I want to practice speaking and last time I checked Advanced Voice Mode was not available in custom GPTs - is that still the case? Advanced Voice made a really big difference to conversational practice for me, but I do feel like I need to wrap it in a custom knowledge base/instructions to make it as useful as it could be for my language practice.
I'm learning Finnish. For the most part, Gemini and ChatGPT do a much better job at generating passable if sightly Internet-inflected Finnish than I'm likely to anyway within the next 5 years, so I just talk with those.
I would imagine Dutch to be in the same camp, unless a native Dutch speaker reviews some of your conversations and tells you otherwise. My native Finnish wife has given me the marginal all-clear with the vanilla models.
I'm trying to solve the problem of unlimited conversation practice for various languages with "CallAnnie", an iOS/android app with various kind of AI friends, powered by real time voice and avatars.
The interface is specifically made for advanced learners that want to simulate a conversation as close as possible to a real one (in terms of latency and without pushing any button). Learning to respond fast in a new language is important, so we're trying to keep a natural pace.
We support audio or video-calling the characters (with subtitle translations), guided conversations and we recently added mini games to learn vocabulary.
You can see a quick intro and demo video here: https://www.sixthtone.com/news/1013961
We don't have a Dutch speaking character in the default character list, but you can follow this link https://app.callannie.ai/a/mhLnHflAyf1Ygb0D0wm6 after installing the app to use a custom character speaking Dutch (or you can create a custom character).
If you'd like to try it, check out: callannie.ai . I would love to get your feedback (here or francesco a t callannie.ai) and suggestions - we're trying to solve this speaking practice use case.
I tried Spanish (I don't know Dutch at all) with Claude via duck.ai.
I immediately had to correct it. (My Spanish is limited to a half a semester 20 something years ago and season 1 of Community.) I don't think I'd be confident that I'd learn accurately from an AI.
I use ChatGPT's conversation mode to supplement my language learning (A2/B1-ish French and 1hr/wk 1-on-1 tutor). I tend to use it in the car, just asking about random facts or ideas or playing 20-questions.
The format forces me to just use my voice and listening skills - in other words, I'm forced to not touch my phone. It's also rather challenging because I'm doing two things at once and the hope is that I won't actually spend much brain power overthinking my responses - something I tend to do if I was talking to myself instead which typically turns into more of a rehearsal format.
I've been setting up Vapi.ai for this. Written and spoken are different skills. It comes down cheaper than an actual teacher, and it has the patience to correct you.
I added a Practice Mode in my speech-to-speech translator app 3PO. It allows you to practise talking with chatGPT and gets assessment score on your accuracy.
Topic is preset. (chatGPT plays the role of a travel guide to help you find places to go in Holland (for Dutch))
https://3po.evergreen-labs.org
I didn't try to learn Dutch. But I have used AI for learning Spanish. From my experience, they can guide and teach you the "School Spanish" perfectly. However, they cannot teach you slangs and conversational Spanish.
I’m making fluent.im, it has situations with goals for each conversation, and we’re about to ship lessons based on your favorite podcast content.
Im not sure in Dutch... But recommends starting with chatgpt until find a good product.