I also don't think people have much of problem with him teaming up with IM but how essential Iron Man is now to Spider-Man's character and his villains.
No problem with a team up, I wish it wasn't Stark. Totally expect Marvel / Downey to overuse / misuse Stark in the movie taking away from building the kid playing Spidey and the overall experience.