Yes, it very much matters. The -j must come first. As far as affixes go, the order can very in relation to meaning.

Yes, it very much matters. The -j must come first. Therefore, you cannot say "libronj" rather "librojn". There may be some rationale besides pronunciation (the former has one extra syllable- "li-brau-ni" VS. "li-brauyn"), but I'm not aware of it.

As far as affixes go, the order can very in relation to meaning trying to be expressed.

Example:
Sano- healthy
malsano- unhealthy, sick
malsanulo- unhealthy, sick person
malsanulejo- Place for unhealthy people, hospital.