I tried feedbackarmy - from a purely ROI perspective I think the usability methodology is flawed.
It seems that the folks who give the feedback are actually trying to be a bit too helpful - to the extent that they think about questions as a challenge and try their best to answer them.
For example - If I pose a question that "What does this site do?" in a usbility experiment, I am expecting to figure out if the audience immidiately grasp the utility of a site.
However in this case it seems that the audience takes that as a challenge modifying the question to mean "Can you figure out what this side does?" and try out very hard to find the utility of a site and in the process totally defeating the purpose of a usability review.
It might be an artifact of the way I posed the question, but then I was just following the sites examples. I think using the site in the way it is setup (messaging, mturk) the concept does not give a good ROI w.r.t usability.
However, one thing that I did like was that the turkers gave some suggestions to improve the site - some of which was really valuable. So maybe if you change your positioning to something like an active feedback/testing tool it might work better.
It seems that the folks who give the feedback are actually trying to be a bit too helpful - to the extent that they think about questions as a challenge and try their best to answer them.
For example - If I pose a question that "What does this site do?" in a usbility experiment, I am expecting to figure out if the audience immidiately grasp the utility of a site.
However in this case it seems that the audience takes that as a challenge modifying the question to mean "Can you figure out what this side does?" and try out very hard to find the utility of a site and in the process totally defeating the purpose of a usability review.
It might be an artifact of the way I posed the question, but then I was just following the sites examples. I think using the site in the way it is setup (messaging, mturk) the concept does not give a good ROI w.r.t usability.
However, one thing that I did like was that the turkers gave some suggestions to improve the site - some of which was really valuable. So maybe if you change your positioning to something like an active feedback/testing tool it might work better.