Nothing's wrong with free speech, but a lot of them were armed, too. The biggest problem with the whole Charlottesville incident was the lack of a sizable police presence. There were even Jews hiding in a Synagogue for fear of their lives, as people outside, were shouting "Sieg Heil", screaming threats, and staring at them from across the street. There's nothing "great" about that.
Free speech is Constitutionally protected, but threats of violence against others is not.
I'm honestly surprised they didn't shut that **** down, but they didn't even bother with Antifa whatsoever at any of their mayhem filled protests. I really don't know how these elected officials look at themselves in the mirror when they let their citizens get beaten into a coma.