That was one of things that Marston never did refine or clarify. In early stories, Hippolyte's strength and that of her people were attributed to a magic girdle. Then their strength was attributed to Amazon training and regular mortal women are shown instantly achieving superstrength.
This one of the areas the TV show slightly improves upon the design. In that verse, all Amazons are uber. However no thank you to the notion that they are only uber if they remain on Paradise Island.