You don't even need a battery; even a 220uF capacitor gives you a couple of milliseconds at USB power levels[0].

0: USB is supposed to draw < 0.5A @ 5VDC, a resistance of >10Ω. 1F*1Ω=1s, so at 10Ω, each 100uF gets you one millisecond. You do need to worry about voltage dropping off over time though.