Been using camelcamelcamel.com for this. They have great FF and Chrome plugins that can display graphs of the price over time of products on amazon pages. Also can email alert you when price drops.
I'm a huge fan of HN and with the constant reminders here of people successfully doing something, I've (re)launched an app that tracks amazon price drops. Thanks hackers! Also special thank you to Joshua McKenty of http://www.piston.cc who built the first version of this.
Hmm... I just tried fresh FF4 and the settings window came up right away for me. Do you see "Buylater settings" in the Tools menu? If not can you email me your "Troubleshooting information" (under Help -> Troubleshooting information) to raph@buylatr.com? Sorry about that.
Scanning frequently for changes. We are aiming for 5 minute updates, but it depends on how many items are being tracked and if we saturate our Amazon API access.