I’ve been speaking Esperanto for over 15 years and I still have this problem too. A few times it has caused some really awkward misunderstandings. It’s not always possible to distinguish with context and sometimes this tiny differnencedifference in sound is the only thing marking the difference between “and you are invited” and “I’m going on my own”. It’s a real flaw with the language.