I didn't get that one, actually. Am I just being dumb or is there a backstory?

  10 PRINT "<rude message here>"
  20 GOTO 10

They used to sell computers in stores that you could walk up to and program. BASIC was the command prompt for some computers. The above was the inevitable result.

My favorite trick was to poke values into the sound registers, THEN goto!

yeah,

  10 BEEP
  20 GOTO 10

Actually I was thinking more along the lines of programming the SID to output a sawtooth waveform and modulate it for extra annoyance, but whatever floats your boat.

It's just a humorous way of saying they were stuck in the past (probably because they didn't integrate other language advances into basic).

reference to GOTO in basic