It's not so much the proprietary blobs, as the complete lack of documentation and debuggability for the peripherals. The PC platform, and several other SBCs, are either well documented, or at least give you the possibility of obtaining hardware with documentation.
That, combined with general flakiness (eg, power delivery issues for peripherals on older pis), and you end up with users blaming the software for hardware issues.
