For family reasons, it would be helpful for me to learn some French. There are all the boring ways like textbooks, audiobooks, flashcards, and apps (Duolingo and its clones got really enshittified and focus more on pushing ads than language learning these days, another reason not to use them).
But we live in the LLM era, and language is one thing LLMs excel at, so it would be silly not to try that.
Written French
Languages differ quite a lot, and French is a bit of an outlier. For most "normal" languages like German, Spanish, or Polish, the written and spoken forms are very similar - the correspondence is never quite 1-to-1, but it's close enough you don't need to think about it much.
French is absolutely not like that. Written French is closer to the language people spoke back in the 1100s than to the French they speak now. In many ways, written French sits halfway between Vulgar Latin and Modern French.
The most egregious examples are verb conjugations: written French preserves all the 1100s endings, but in speech they've merged into just a few forms. So in writing there's "chante", "chantes", "chantent" (all pronounced /ʃɑ̃t/), "chanter", "chanté", "chantez" (all pronounced /ʃɑ̃ˈte/), and so on.
Fortunately this spelling isn't arbitrary way, it mostly just reflects a much older form of the language, so you can mostly figure out how a word is pronounced based on its spelling. It doesn't work the other way, a single spoken word can correspond to a lot of different written forms.
English suffers from a similar issue, but it's only about half as bad in terms of time passed and degree of sound change.
How to even learn French
Normally when spoken and written languages are far apart, it's easier to learn the spoken language. This is not the case at all here. If you know English, or any Romance language, you'll have fairly easy time recognizing a lot of words in written French, but when spoken you might just as well be hearing Korean.
And there's input hypothesis - the idea that the only meaningful language learning is by consuming a lot of input just above your current level, and output is pretty much a distraction.
Regardless of how accurate this is, reading comprehension is the easiest thing to practice with AI.
The Prompt
After a lot of experimentation, here's the prompt I settled on, followed by detailed explanations.
Do the following:
- write a short text in French of about 500 words total
- the text will be split into 10 paragraphs of about 50 words each
- the story needs to be at B1 level, only using grammar and vocabulary appropriate for that level
- write the first paragraph, then I translate it to English, then you let me know if it's correct or if I made any mistakes
- then continue with next paragraph until the story ends. Do not ask for confirmation before continuing.
- then print the scorecard
- do not nitpick English translation by suggesting a more natural one, as long as it translates French text correctly
- number each paragraphs
- use sidebar for French text
Topic for today: ...
Prompt explanations
Do the following:
- write a short text in French of about 500 words total
- the text will be split into 10 paragraphs of about 50 words each
First, we tell AI what we'll be doing. Translating a short 500 word French text to English is a good test that you understood the input.
A short text gives a lot more natural context than translating disconnected sentences, and is closer to natural input.
Dividing it into pieces makes it much easier to translate, and avoids having to scroll back and forth you'd need to with a single block of text. It has an additional advantage that if you encounter a difficult word early in the text that's repeated later, you'll get feedback on that word early.
- the story needs to be at B1 level, only using grammar and vocabulary appropriate for that level
It's good to give the AI some idea what level you're aiming for, but this is going to be very vibes based. The AIs don't really understand what's appropriate for each level, and you will see rare words - sometimes because they make sense for given topic, but often AI will throw a rare word even if a much more common word with similar meaning would work just as well.
I tried many variants of this instruction, and at least it generally sticks to appropriate grammar if you request a low level, without using rare tenses and such.
- write the first paragraph, then I translate it to English, then you let me know if it's correct or if I made any mistakes
This is the core idea. Technically "comprehensible input" doesn't require you to translate it, but translation ensures that you actually understand it, and gives instant feedback on any words or phrases you got wrong.
Another thing I tried was reading comprehension multiple choice test, but that's somehow more time consuming that translation (mostly due to scrolling required), and you can often answer multiple choice questions correctly even if you don't quite understand the text, so it wasn't all that useful.
- then continue with next paragraph until the story ends. Do not ask for confirmation before continuing.
Without this, AIs would sometimes proceed to the next paragraph, but sometimes they'd force you to say "continue" 10 times per story, which got really annoying. This solves the problem.
- then print the scorecard
The scorecard is not essential, and every time it will be in a different format anyway. If it was possible to do fine-grained adjustments to difficulty level, it would make sense to develop consistent scoring system, and then tweaks difficulty by +5% or -5% whenever it gets too easy or too hard, but you will get pretty wide range of difficulites with the same prompt, so there's not much point in doing this.
- do not nitpick English translation by suggesting a more natural one, as long as it translates French text correctly
Here's another big problem with current AIs. This is meant to be a reading comprehension practice, not translation practice. So ideally we'd like AI to pick up every error in our understanding, while not nitpicking about trying the most natural sounding translation.
Trying to fine tune the English text to sound great takes longer than just getting the meaning across, and it's a wasted time.
I tried many variants of this instruction, and this one causes mistakes both ways. Sometimes it ignores errors in understanding if I picked a vaguelly similar meaning (like translating "broderies" as "decorations"), and sometimes it nitpicks (nitpicking example: “small city” → better as “small town” (more natural for petite ville)), but it's balanced enough.
AIs also really love to say "good job translating this sentence: ...", which is basically noise, and perhaps it would be worth tweaking this prompt to reduce the noise.
- number each paragraphs
Most AIs do it anyway, but sometimes they don't. It's a bit more convenient to have paragraph numbers.
- use sidebar for French text
This is helpful to include for AIs like Claude that have sidebar interface. Don't include this instruction for other AIs, they'll either ignore it, or just make up some "sidebar" tags like <mui:sidebar> that don't actually do anything.
Topic for today: ...
And finally a really essential part - you need to give it a new topic every day, otherwise you'll see very similar stories about how a random textbook French person spent their day over and over. The topic can be one word like "cats", a phrase like "Bridezilla's wedding", or a short sentence like "competition to make the most ridiculous pizza", and it doesn't really matter what you put there, as long as it's different from the previous ones.
You could also ask AI to pick an interesting topic as well, which increases your odds of getting a non-textbooky text.
Which AIs to use
I tried it with a lot of AIs, free tiers only.
- ChatGPT - consistently best, but quite prone to going into censorship mode if it doesn't like your topic (especially on anything related to current events), something I never saw with any other AI with the topics I tried
- Claude - consistently very good, and sidebar interface for French text is very convenient
- Mistral Chat - somehow really good at it, biased towards slightly higher difficulty
- Grok - very good if you have access to it, but fairly slow message limit in free tier
- Qwen AI - I only tried it a few times, seems to be fine, but I'll pass judgement for now.
- Gemini - I have no idea why, but Gemini consistently disappoints me no matter what I try, not really suitable for this
- Meta AI - completely unsuitable for this
- Deepseek AI - completely unsuitable for this
All these AIs get regular upgardes, so your experience might be different from mine.
If you do this repeatedly (I aim at 2 stories per day), you can reuse existing chat and just say "next topic: ..." or even "pick a new topic". Some AIs have tendency to make the new text overly similar to the one it just did, so starting fresh conversation be better.
I don't think it's a good idea to use any kind of automated "memory" feature, as that will just cross-contaminate conversations.
What else didn't work
The prompt here is a result of a lot of experimenting. One notable thing all AIs are absolutely terrible at is IPA. Asking for IPA you'll get total vibe garbage - like IPA of completely unrelated words.
How you should treat the results
This is a reading comprehension exercise, not translation exercise, so don't worry if your score isn't perfect. It's not even supposed to be.
- typos - try to get better at typing, but you'll likely make a few typos here and there. AI will point them out, but they don't matter, just ignore them (if they were legitimate typos not mistakes).
- English text not sounding natural - it would be waste of time to try to polish it, so don't worry too much about it.
- comprehension errors - if you miss some words or gramatical structures, and they feel rare, that's totally fine. In real life you'll encounter small percentage of more difficult words too. If you miss common words, you're just getting very effective spaced repetition here, in proper context.
5 comprehension mistakes per 500 word text (so excluding typos and nitpicking) is a 99% understandig, and that's what you should be aiming at. If you make a lot more mistakes, adjust the vibe level. If standard CEFR levels are not enough, you can make up your own levels like "A2+", "B1.5", or whatever you want. Just remember the level you give the AI is not really CEFR level, and some AIs (notably Mistral Chat) have a notably different idea what each level means.
No comments:
Post a Comment