Good point, but I also think he was supposed to be seen as a reluctant fighter, as he seemed more interested in understanding this "other" Loki. It's also not unreasonable to think that Sylvie is simply better at hand-to-hand combat. She's had to survive a much harder existence growing up compared to Loki.
Well again, inhibitor collar. Plus he was a rejected runt Frost Giant, so how super strong & durable is he really?
Enough to survive a Hulk beat down yes, but not invincible as demonstrated by IW Thanos.