Many of the costs of the Olympics are already set. Tokyo had to build a lot of stuff in order to get the Olympics.
A major problem with just waiting until 2024 is that it means many great talents will never be able to perform, because they missed their window.
I do suspect the odds are lower than you think, but I do respect you for sharing your answer.