The innovation argument is particularly frustrating. By far the most innovation in computing this century has occurred in open source software, which by its nature is completely distributed. People who work on the Linux kernel or Postgres or Rust would be pretty shocked to learn that whatever they’re doing must not be innovative since they’re not hanging out in the hallway.
As a postgres dev, I do feel that the lack of in-person conferences over the last 1 1/2 years caused a perceptible slowdown in some areas. But I also can't stand working full in an office most of the time.
The quality and insight of the discussions at conference is usually way higher than at the office. Conferences allow specialists to come together around topics covered only by a few people in the company. Also, they allow you to look outside the box, how other approach similar problems or technologies. At work, I usually already know it most from the regular communication.