I've always considered Cap to be boosted up to superhuman after the serum. His strength rivals Black Panther whom is enhanced and is even more knowledgeable in martial arts. He was able to chop Ultrons adamantium neck in half and shatter Iron Man's extremist armor. He's tackles enemies through walls and concrete. Dodges bullets and overpowers Beast. He beat an entire room of superhuman Asgardian trolls. He dodges cross bones energy attacks at point blank range. He clearly has superhuman durability when Warrior Woman punched him into the sky and he somersaulted onto a plane. He outran Atlaentiabs whom are far beyond any peak human. He withstood a fall of 200 feet easily. He jumped out if helicopters and left craters with his feet in the ground. Took a beating from Namor. He has pulled down a helicopter with two men inside with enough force that they came flying out the windsheild. Easily pushes a statue that weighs a ton by like it was nothing. These are just a few feats of his in a whole long list of others. He's superhuman as far as I'm concerned.
Out of curiosity what era were all those comics from? Post mid-90's? I gauge Cap's power levels based on his appearance in solo titles from the 60's to the 90's but I'm certainly no expert after about 1996 or so. So if they jacked him way up after that then I'd be ignorant to those events.