In my math book, it describes "the projection of b onto a." Esperanto translates "onto" as "en" + an o-word in the accusative. This effect on consonants is obvious because they end in -o when pronounced (b: "bon", c: "con", d: "don"). But what about vowels? Is a pronounced "an" or "a-on"? Is o pronounced "on" or "o-on"?
I think this section of PMEG mainly answers your question. So you can write la projekcio de b sur/en a-on. Like PMEG suggests, the o of consonant names are generally not considered to be an actual o-ending. Therefore, you'd write either b-on (say “bó-on”, not “bon”), b-oj, and b-ojn; or boon, booj, and boojn. Similarly, for vowels you'd get a-on or aon, oon/o-on etc.
Alternatively, you can leave out the ending entirely (PMEG gives examples like du B instead of du B-oj), but in certain mathematical contexts this is confusing: A entenas B. Is that A⊆B or B⊆A? So A entenas B-on is clear and standard (when pronounced bo-on).