Interesting question, it's hard to pinpoint imo. I guess if I had it my way it would be a weird sort of in-between. If I had it my way Themyscira would be part of a sort of fantastical divine realm, something like taking an old Greek world map with all the mythical tribes, races, monsters, and cities and expanding it even further. It's still an Island but part of a world map. Something akin to the map of Elden Ring or the current Zelda games (BotW and KotK). It would be this vast cosmic/mythic world that you could explore with Diana and the other wonders without having it bring them into a new city in our world every new run. You could even tie it to another head canon of this divine realm bleeding into the 'real world', like a pressure release system gone broken and now minotaurs are rampaging in Gateway City, for example. However, Themyscira also exists in our world, somewhere on the Mediterranean Sea, but the only way you're going to get there is by proxy i.e. the Gods allowing you to go there or Diana/Nubia being champions or Hippolyta being queen (this is still my head canon where she is) can grant you passage. Maybe a sort of medallion that opens a gateway or a ritual right. Obviously breaking in would be next to impossible via wards placed by the Patrons or even Amazon mages. Point is you can go there, but you're now a part of this fantasy world. It's not really heaven or an afterlife, but it's not the regular world either. The other city-states of the Amazons would work the same.
Edit: The teleport systems on Embassy Island would take you to Themyscira too. Embassy Island should have had more debate and continue to be a contentious point within amazon politics. Even have a short story on it. Because its easy access to their mainland on foreign waters in a world with governments that are wary of the amazons and filled with super-villains and terrorists, while Embassy Island has little defenses.
And to answer on Numenor, it's basically both tooLike its a place on a map you could theoretically go to it, but it's also close to basically elven heaven.