* Was that test just on new users - or subscribers in general? If the latter you're going to self-select for folk who are interested enough in your stuff to read regardless, which will skew the results somewhat.
* Also - judging by the blog topic - you've got a pretty technical audience. One of the few groups that do have the knowledge to use tools and preferences to get around the problem. I'd be leary of generalising that to other markets.
* You're a/b testing on max-width setting - not line length. So a bunch of your users in both groups are going to get exactly the same experience if they're on smaller displays. This will make the improvement on larger displays seem less significant.
* Tests like this can be really skewed if you have audiences that shift device over time (e.g. getting popular link tweeted and retweeted can push sometimes push up the mobile portion of a site enough so that certain display types work better/worse.)
* The line length is something that has interactions with other typographic features like line spacing and font size. Optimising on one alone can lead to poor results.
I'm not saying that you're wrong - but those are things that occurred ;-)
> * Was that test just on new users - or subscribers in general? If the latter you're going to self-select for folk who are interested enough in your stuff to read regardless, which will skew the results somewhat.
I don't have 'subscribers' or 'new users'. Just people visiting the site. (I suppose there's people visiting from the RSS feed, but they're well under 1% of page views.)
> you've got a pretty technical audience...I'd be leary of generalising that to other markets.
OP's blog seems rather technical.
> This will make the improvement on larger displays seem less significant.
True, but this still wouldn't explain why the smaller max-widths - the max-widths that would hit the most viewers - would not win.
> * Tests like this can be really skewed if you have audiences that shift device over time
Certainly, but the 2 tests were separated by something like a year. Is it really likely the audience changed in both a/b tests in the same way?
> * The line length is something that has interactions with other typographic features like line spacing and font size. Optimising on one alone can lead to poor results.
Sure, but in the absence of any specific reason to suspect an interaction...
* Was that test just on new users - or subscribers in general? If the latter you're going to self-select for folk who are interested enough in your stuff to read regardless, which will skew the results somewhat.
* Also - judging by the blog topic - you've got a pretty technical audience. One of the few groups that do have the knowledge to use tools and preferences to get around the problem. I'd be leary of generalising that to other markets.
* You're a/b testing on max-width setting - not line length. So a bunch of your users in both groups are going to get exactly the same experience if they're on smaller displays. This will make the improvement on larger displays seem less significant.
* Tests like this can be really skewed if you have audiences that shift device over time (e.g. getting popular link tweeted and retweeted can push sometimes push up the mobile portion of a site enough so that certain display types work better/worse.)
* The line length is something that has interactions with other typographic features like line spacing and font size. Optimising on one alone can lead to poor results.
I'm not saying that you're wrong - but those are things that occurred ;-)
In general what the research done by others indicates is that users prefer shorter physical line lengths, but read faster on some longer line lengths (within limits obviously - backup http://www.usability.gov/articles/newsletter/pubs/082006news...)