He has fallen from his own parameters of worthiness - it may be that he foresaw this when he hung from the tree and gave up his eye millennia ago and so crafted the enchantment in such a way that the Nine Realms would be safe from him.
(But then I haven't read Thor in years, so I'm just making stuff up )