You have to be careful with that analysis. There are at least three issues:
1. The time period over which he looks is extremely short. Although the change in data looks compelling, I'd really want to see more than just a few years of "normal" levels to be convinced there's really a major deviation from the norm.
2. The argument starts by asserting a correlation (rollout of smart phones, change in the data) but then moves smoothly to a different correlation (social media usage, change in the data).
3. The biggest problem: this is all social science. Someone took a close look at the studies Haidt is using to try and prove a link to social media specifically (vs other causes) and discovered they're of the sort of quality we've come to expect from the field, dominated by tiny unrepresentative samples, bad methodologies, bad statistics and so on.
To his credit I found this rebuttal on Haidt's Twitter feed and he promises a response. Unfortunately, judging from his tweets so far, that response might not be convincing:
Haidt: "Brown requires a standard of proof not appropriate for a public health crisis"
Reply from someone on Twitter: "Oh no, that's a bad argument I think! We shouldn't be calling for lower standards of evidence during a public health crisis, but the inverse"
Haidt: "Aaron has set his skepticism meter so high that he says none of the 300 studies in the google doc are valid. That is too high. No studies pass muster?"
I don't think Brown actually does assert all 300 are invalid, just that he checked quite a few and none of the ones he spot checked were valid.
But this is a way of thinking that crops up all the time in academic work. Back when I was reviewing COVID papers I was constantly hitting this kind type of assumption in the public health space, that quantity is some proxy/substitute for quality, or that you don't need to verify the quality of studies you're citing because they can't all be wrong, or (worst of all) that because a problem seems important the standard of evidence should be low.
It's totally possible that there is an abnormal increase in teen mental health issues lately but it's too early to consider it proven. More raw data is needed stretching back a lot further in time. Additionally the claim that it's driven by social media needs a smaller number of higher quality studies before it's taken seriously. Right now not only do the studies not convince, but it's not even clear what they're measuring. For example the growth in smartphones would also make it much easier to consume news around the clock, and social media often rebroadcasts news, so it could be hard to decorrelate these things to decide to what extent it's selfies vs headlines that drives the depression.
1. The time period over which he looks is extremely short. Although the change in data looks compelling, I'd really want to see more than just a few years of "normal" levels to be convinced there's really a major deviation from the norm.
2. The argument starts by asserting a correlation (rollout of smart phones, change in the data) but then moves smoothly to a different correlation (social media usage, change in the data).
3. The biggest problem: this is all social science. Someone took a close look at the studies Haidt is using to try and prove a link to social media specifically (vs other causes) and discovered they're of the sort of quality we've come to expect from the field, dominated by tiny unrepresentative samples, bad methodologies, bad statistics and so on.
https://reason.com/2023/03/29/the-statistically-flawed-evide...
To his credit I found this rebuttal on Haidt's Twitter feed and he promises a response. Unfortunately, judging from his tweets so far, that response might not be convincing:
Haidt: "Brown requires a standard of proof not appropriate for a public health crisis"
Reply from someone on Twitter: "Oh no, that's a bad argument I think! We shouldn't be calling for lower standards of evidence during a public health crisis, but the inverse"
Haidt: "Aaron has set his skepticism meter so high that he says none of the 300 studies in the google doc are valid. That is too high. No studies pass muster?"
I don't think Brown actually does assert all 300 are invalid, just that he checked quite a few and none of the ones he spot checked were valid.
But this is a way of thinking that crops up all the time in academic work. Back when I was reviewing COVID papers I was constantly hitting this kind type of assumption in the public health space, that quantity is some proxy/substitute for quality, or that you don't need to verify the quality of studies you're citing because they can't all be wrong, or (worst of all) that because a problem seems important the standard of evidence should be low.
It's totally possible that there is an abnormal increase in teen mental health issues lately but it's too early to consider it proven. More raw data is needed stretching back a lot further in time. Additionally the claim that it's driven by social media needs a smaller number of higher quality studies before it's taken seriously. Right now not only do the studies not convince, but it's not even clear what they're measuring. For example the growth in smartphones would also make it much easier to consume news around the clock, and social media often rebroadcasts news, so it could be hard to decorrelate these things to decide to what extent it's selfies vs headlines that drives the depression.