Silva was nothing like the Joker besides being psychotic (which is something that quite a few Bond villains have been before anyway), and having a somewhat over elaborate plan (again, a staple of Bond films). He had a VERY specific motivations, unlike The Joker.
Also I liked how they brought Q and Moneypenny back, it was a nice twist on an old formula.
I could have been made ok with "Silva being connected to SPECTRE" if they had explained it better. Like say that Blofeld gave Silva some finances to start up his hacker network, or leak the existence of that jump drive with the names/faces of all the undercover Western agents to Silva, or was just providing some logistical support for Silva behind the scenes. Heck that last one could help address one of the issues that some people had with Skyfall. Namely that Silva seemed to think/plan a little TOO far ahead, and that it wasn't believable that he could pull this all off by himself.
Basically make the "connection" more indirect. It's still about Silva and his plan, and he's doing his own thing. It's just that Blofeld was helping BTS to a degree because Silva succeeding would also help Blofeld and SPECTRE further their agenda. But trying to imply that Silva was a flat-out member of SPECTRE, and that it was all Blofeld's plan, no that just didn't work and made no sense.
But then again, Spectre was just a poorly-written/thought out movie overall imo.