Quote Originally Posted by Jim Kelly View Post
It doesn't matter what we want, if corporations see a benefit in making sentient machines, they will, and then they'll manipulate us into thinking this is what we want, too.

There was a local TV ad about ten or fifteen years ago that promoted a city of the future where people would have the kind of communication tools we pretty much have now. To me this vision of the future was like some apocalyptic science fiction--like something from LOGAN'S RUN--but to the advertisers this vision of a future city was clearly their idea of paradise. Of course, it was prettied up, so you didn't see the downside of that techno-babble--but that Disneyfication of reality is itself horrifying.

If corporations have an easy way to track your actions and anticipate your needs, that has value because they can drive you toward their products. So a sentient machine, that becomes like a buddy that you interact with more than your friends and family, is a valuable resource for corporations to get you to do what they want when they want. If it can think on its own, then it doesn't have to await instructions from its corporate masters--it can immediately think up how to manipulate you to satisfy its programming.
It's not just corporations one needs to fear giving rise to sentient machines. There are all those wanna-be-Zuckerbergs out there who dismiss any cautionary voice as being unimaginative and unable to grasp the greatness of their vision.