In reality the gameplay and animations gets more complex, refined and expanded with every title.
The next gen titles of Unity and Syndicate are a long way above the others in that regard.
So it mostly comes down to story, setting and character. For me its:
Black Flag and Edward Kenway
I loved the open sea gameplay, the gorgeous West Indies islands, the sea shanty's, the true history of the Carribean pirates mixed in.
I've played that game multiple times just to be a pirate again
...and I have the McFarlane Kenway statue on my shelf.
They reused most of the mechanics for Rogue but I'd love to see an actual prequel set in the Caribbean again.
Black Flag covered the later period when Nassau was the pirate republic and the pirates were outlaws.
I'd love to see Assassins Creed: Privateers as a prequel that covers the English vs Spanish war when the pirates were actually working for their country as 'privateers'.