Yeah, it sits a bit too high on mine too. But it's not just "shaving down.
It's a double connector. Ball into the connector, connector into the head.
You can't hollow out the connector, due to his inside convex shape, you can't just trim it down on any of the pegs, cause you corrupt the ball, so...