Anyone has something like a glossary or the appear chapter, arc of the harem, I haven't read it in a while and can only remember half of the harem members.
You have five official by the current TL and one in the wings... plus androids. Elsie,Lindsey,Yae,Yumina(first official fiance),Rue. These five are current confirmed. Sue is in the wings. Elsie and Lindsey are sisters he met in the first town he arrived at. Yae is from Ishen and he met her traveling to the Royal Capital Yumina is the princess and he saved her father's life. Rue is the princess of a nearby Empire and he single handedly put down a rebellion in said Empire. Sue is Yumina's cousin and the daughter of the duke, he cured her mother's illness induced blindness.