It does outside the English-speaking world. In many non-English-speaking countries—including Japan, where I live—English education is similar to mathematics education: All children have to study it and ability at school English is treated as an indicator of overall academic ability, but many children struggle with it and by adulthood most people have forgotten most of what they learned.

In Japan, school English education is also affected by problems similar to those mentioned in other comments on this page, including English teachers who themselves are not skilled at the language, educational policies that require that all children study the same material at the same age, and, sometimes, an overemphasis on rote memorization and teaching-to-the-test.

There’s a huge industry in Japan serving adults who have forgotten most of their school English—or didn’t learn much in the first place—and who now want to get better at it in order to advance their careers.