I am a native speaker of an European language which is not an Indo-European one (Finnish). Had I not learnt in school English, German and Swedish, which all are Germanic languages, I would have had hard time learning Esperanto.
Coming from a non-Indo-European language most of the roots would have been Hebrew for me without prior exposure to those Germanic languages, especially English with its Romance-influenced vocabulary.
I have taken two courses of Esperanto with a foreign teacher (foreign in the sense that the teacher did not speak Finnish). Especially the first one made false assumptions that accusative and vowels ,especially diphthongs, would be hard. They are not: both are piece of cake for native Finnish speakers (c. 15 grammatical cases and 8 vowels with 46 vowel combinations).
Finnish has no articles, so the definite article is pretty much meaningless. And for the record, the rules in Esperanto are all but consistent (and English is even more incoherent in that regard).
Finnish has only a few prepositions, you can count them with the fingers of one hand (there are more postpositions but mostly grammatical cases are used instead), so without prior knowledge of prepositions in other languages the learning curve would have been even steeper. What still causes me problems is the fact that the prepositional expressions seem to get associated with the closest noun instead of the logical one (from my perspective). Let me illustrate with an example. Mi aĉetas en la vendejo means that I am in the store doing the buying, but since there is no indication that I leave the store, this creates a funny mental image where I keep wandering and picking things from the shelves. So I would say Mi aĉetas el la vendejo, because it indicates that after buying and I will leave the store. Seen from another point of view expressions with no movement get associated with the subject and with movement to the object.
The overwhelmingly most difficult part at least for me are the buzz sounds, c, ĉ, ĝ, ĵ and z. This is because of Finnish has just one s, which is slightly more buzzing than the Esperanto s. The combination sh does not natively exist, ts only in some dialects, and z is almost always pronounced as ts. So the ĉ, ĝ and ĵ sounds do not exist and I have from time to time extremely hard time to hear the difference (e.g. aĉa vs. aĝa), meaning that I need to memorise words with them by heart and, if I only hear them, infer from the context what is meant, making Esperanto all but as easy as touted.
And one more rant, in Finnish placement of comma quite often changes the meaning of a sentence, so the liberal comma rules of Esperanto are puzzling. Using the Finnish rules, Mi vidis alian, raran birdon means that only the later bird was rare vs. Mi vidis alian raran birdon where both birds were rare.
