Ok, so I saw it again this weekend, and you guys are right. I paid close attention to what happened when and in what order.
1. Kirk changes his mind and decides to go after Khan BEFORE the core blows (which is actually opposite of the book, but not really surprising, and makes more sense).
2. Marcus knew the 72 Botany Bay'ers were in the torpedoes already.
3. Marcus rigged the core to blow from the get. So yeah, the plan was for Kirk to get to the border, send the torpedoes, the core blows, Khan and his people get blown up, the Klingons blow up the Enterprise, no one knows anything better and Marcus gets to start the war on his own terms. Nice, neat, and with a bow on it.
4. Marcus only took the Vengeance out once he learned Kirk had Khan on board the Enterprise.
So the only questions left revolve around Marcus' actions on the Vengeance. I suppose he decided to go out there because he was afraid Kirk would figure it all out and get a message out, or Khan would somehow escape and keep reigning chaos, or somehow the Enterprise would get out of it (after all, if Kirk had followed his orders, even if the Enterprise had gotten away, Kirk still would have been clueless about Khan and the whole thing could've still been written off).
So then why didn't he just blow up the Enterprise from the start instead of talking to Kirk about turning over Khan? Only thing I can think of is that he wanted to end Khan with his own two hands to make sure he was dead.
So no real plot hole, beyond just lukewarm reasoning behind his actions.
The only real plot holes, IMHO, at this point, are:
1. The fact that it only took one day to get to Kronos
2. The behavior of the Vengeance and Enterprise at warp (how the heck did the weapons work, for one?)
3. How was the Enterprise able to call New Vulcan for advice but couldn't call Earth, which was right outside the window, for help? Is it like long distance being on while local calls are off?
4. Why didn't Starfleet send one of the other ships in the sector (and there were a bunch, their senior staff was in the meeting) to check out the battle?
5. We've talked at length about this - but yeah, my second viewing confirmed that Marcus started an expanded exploration campaign, found Khan, woke him up, and had him help build the Vengeance all in one year. Whoa. Not a hole so much as kindof unbelievable (especially since we saw in Trek 09 that it took over three years to finish the much-smaller Enterprise). Though, I guess the ship might've already been almost done by the time Khan came on board and he just augmented the weapons...