Well there is no beating his charismaLoki was beaten by Thor alone in his first movie, and we still accepted him as the threat the whole Avengers needed to come together and stop next time he showed up.
Plus Loki is more brains over brawn. He works cause he’s a villain we kind of root for.
I can’t imagine how I would feel about Thanos if he was the main villain in the first avengers film and got beat