That's the point, that you can be successful and go broke doing it.
The reason fashion is a tough nut to crack is the short business cycle and associated inventory risk. You need to anticipate fashion trends, get product to market quickly, and if you screw up you have to dump your inventory at clearance sale prices and eat that loss. Big retailers are experts at finding ways to manage that risk, and that's why the Forever 21's of the world sell crap clothing made overseas.
Seriously, when Britney Spears popularized low-rise jeans, every teenage female in the USA had to go to the mall and change out her wardrobe. When Clark Gable appeared in a movie wearing a men's dress shirt with no undershirt, the market for men's undershirts tanked.
How does one plan for that sort of thing within the constraints of a VC-backed company?
Fashion is a pretty hard industry especially when you're targeting younger people with less disposable income. If you sell your brand to the mall type retailers, your brand image is pretty much dead. If you don't sell to them, they copy your designs and sell for less anyways.