This was something I always heard before I experienced it but the truth is it wasn't so bad. You definitely weren't sitting in a full First Class seat (which were smaller in any case back then) but it was still perfectly comfortable especially for the relatively short duration.
And because the overall experience was so cool -- board directly from the lounge, the led display showing how fast you were going, seeing the curvature of the earth (sorry flat earthers), arriving before you left, etc. -- you never thought about the seat. I'm sure if you took it all the time you might care but most people it was awesome.
The Concorde was more or less what domestic business/first class is today which is pretty much what first class was internationally as well back when the Concorde was flying. (Maybe a bit more cramped--more like what's being called Premium Economy on an airline like United these days.)
The windows are so small because Concorde cruised at 60,000 feet. At that altitude the usual oxygen masks won't keep you conscious. The windows are small so that if one ever failed, Concorde could descend fast enough to an altitude where the masks would work before the pressure dropped too low.
When my dad was flying back and forth to Europe from the US East Coast a lot, he told me he got upgraded to the Concorde once for some reason. His reaction was that it was a neat experience but he'd just as soon fly first class in a 747.