Uhh except he was the one who threatened the existence of humanity? The twins even found his plan to be so screwed up that they ended up betraying him. The twins couldn't do anything without Ultron's direction and knowledge of the Avengers. If they did, they wouldn't have stuck around him.
Someone clearly wasn't paying attention.
Ultron communicated many times how much he wanted to play God and throw a meteor at Earth, how you didn't think it was clear as day that the meteor idea was something he made up on his own is just reaching at this point. He was a global threat, and he had the means to cause human extinction. He lifted endangered an entire city and nearly caused human extinction. The Avengers would have been screwed if it weren't for SHIELD. Funny too, considering how in TWS, Cap was essentially against SHIELD and yet it's the "good ones" of the bunch that still ended up saving their butts in AOU anyway.
Most of the issues surrounding AOU had less to do with Ultron than anything else to be brutally honest (seriously, Hawkeye is the "glue" of the team? where the hell did that come from???)
Except this particular storyline was based on an alternate timeline Ultron? Ehh but who reads comics anyways right?
Personally if I were to choose a story/scheme for Ultron, I'd go with his assimilation tactic in
Rage Of Ultron (which is also a pretty good book for Hank/Ultron fans but that's beside the point). Literally achieves his "Peace in our time." directive and negates the issue of being deflected from using the nuclear warheads.