The prefix is defined in Universala vortaro (part of Fundamento de Esperanto) by Zamenhof as
ge' les deux sexes réunis; ex. patr' père ― ge'patr'o'j les parents (père et mère) | of both sexes; e. g. patr' father ― ge'patr'o'j parents | beiderlei Geschlechtes; z. B. patr' Vater ― ge'patr'o'j Eltern; mastr' Wirth ― ge'mastr'o'j Wirth und Wirthin | обоего пола, напр. patr' отецъ ― ge'patr'o'j родители; mastr' хозяинъ ― ge'mastr'o'j хозяинъ съ хозяйкой | obojej płci, np. patr' ojciec ― ge'patr'o'j rodzice; mastr' gospodarz ― ge'mastr'o'j gospodarstwo (gospodarz i gospodyni).
There is also a clarification by Zamenhof in the form of Lingva respondo 6 where he states that the prefix relates to "both sexes". He continues that while so far he has used it only of persons of the same parental line, like gepatroj and gefratoj, there is no reason for not using it of any group of persons of both sexes. Thus he gives green light for expressions like gesinjoroj.
Since the definition and the clarification speak about "both sexes" and by the time of Zamenhof there was just two recognised sexes, the established meaning of the prefix has become "at least two persons with at least one male and one female".
The world has changed from the time of Zamenhof and some consider that there should be a way to express these changes of how we regard gender and sex. There are a lot of reform proposals for gender marking (I counted at some point eleven more or less thought-out proposals): some introduce small changes, some big, some create a parallel system of naming, some alter/reinterpret the meaning of current names for relatives.
One of the most widely spread proposals is just allowing singular ge- indicating a person of whatever gender. As Neil Roberts points out this could cause confusion. There is a longer argument in PMEG against singular ge-. Personally I think the problems are exaggerated.
Noteworthy is also that the need for ge- is shrinking. At the time of Zamenhof the sentence Mi renkontiĝos kun miaj amikoj meant meeting only male friends, nowadays many take it to mean meeting whatever gendered friends. Quite likely ge- will remain in use of the closest relatives only, while some for instance already use kuzo of any cousin, not just of a male one.
The affixes ge- and -in are incompatible, because what would a word with markings for "both sexes" (plural ge-) or "any gender" (singular ge-) and "female gender" (-in) mean.