Before you reach a 1.0, you should be able to break backward compatibility for any and no reason.

Most likely the reason is that you want to make something that you yourself are proud of or at least satisfied with.

Unless you're being paid for the project, satisfying your psychological goals should be the prime point of pre-version 1.0 software.