Whenever I find myself at my desk not knowing what to fill the next few minutes with, I tend to call a more or less random customer - the ones who USE our products, not the ones who procure them - and ask them what they think suck about our current offerings.
Most educational, and it has resulted in a number of improvements to our product line (subsea handling equipment - used for deployment and maintenance of subsea wellheads, submarine communication&power cables, &c, &c.)
They all expect you to ask what feature they like best; they're always baffled when you rather ask where we've screwed up - and more than happy to help! :)
I used to work with this programmer who would get free beta testers by doing this.
Set up a meeting with endusers in one of our training labs and tell them they can complain about whatever they want. Then walk them through a 15-minute set of steps for whatever change we're going to roll out.
Not only do you get great feedback on the product as a whole, they'll inevitably find ways to break your new thing within five minutes. Literally a few hours of two analysts' time and you might avoid dozens if not hundreds of man hours spent fixing things. It was beautiful. And just like you said... the customers are amazed that you want them to complain. They love it.
In my business line, we have two kinds of users: the competent ones who have useful complaints but never seem to break anything and the incompetent ones who break everything and have no useful complaints. I try to get a mix when I want feedback on an upgrade or new tool or for more formal UAT. The former group shows me flaws in the data model and UI, the latter shows me the flaws in the business model.
I used to write software tools used by Amazon associates inside the warehouses (fulfillment centers). I loved meeting the users and asking them 'what sucks about this?'.
A few would be too freaked out by some manager-looking-type asking them any questions. Some would say the usual stuff that we knew but couldn't help. Some would point out a few small annoyances that we would try to improve. Rarely, one would tell you things that blew your mind, like holy shit why didn't we think of that? I kept going back to try to repeat that experience.
One thing we try to do is get non-customer service folks to sit on customer calls. It's amazing what can happen. Like one time, we had a CTO of an online dating service sit in on calls from customers for awhile. After one particularly grueling one, he goes "you guys need to give this woman's money back." We go "we can't, there's a policy against refunds." To which he responds "but she will never get a date (for a bunch of reasons not worth going into)." And we go "exactly...so maybe we should change the policy!" New policy the following week. Boom.
Most educational, and it has resulted in a number of improvements to our product line (subsea handling equipment - used for deployment and maintenance of subsea wellheads, submarine communication&power cables, &c, &c.)
They all expect you to ask what feature they like best; they're always baffled when you rather ask where we've screwed up - and more than happy to help! :)
This is very cheap and effective market research.