Your definition of the libertarian philosophy (small-L) is an oversimplification and idealistic. We live in a world with finite resources; every action we make impacts others. At some point the line needs to be drawn as to what constitutes an 'impact' on other people. My understanding of small-L libertarianism was that this line shouldn't be drawn.