I really hope he doesn't tear everything down. I hope it is just a test to see if he has earned parole, and if he stays put, he passes. I was thinking that Dwight could've built an explosive ankle bracelet for him or something to enforce some strict parole terms that at least allows him the freedom of house arrest, and then maybe time on the town.
Sadly, that is probably just wishful thinking. I'm gonna be sad as hell when Negan slaughters that fair or whatever terrible thing he decides to do. If he decides to go full villain, I was thinking he should go for Dwight. They made it clear that Dwight's command of the salt trade is pivotal to people, and that is seems to be getting sparce as is. If he took over an important outpost, he could force Rick to work with him to get his goods. Like how America is only in business with the Middle East because we need oil. Negan becomes a voyeuristic salt baron.