The unfortunate thing is that Firefox could be the perfect platform for browser UI experimentation if more care were put into making the project easier to fork and reasonable to keep up to date with mainline.
A few months ago I played with forking it for my own tinkering but bailed because it seemed likely to turn into a rolling mass of merge conflicts if I were to make anything but minor changes.
Some forks are using a nice patch based system: see how the Zen browser is built for instance (https://github.com/zen-browser/desktop/). I think that's a better model than merging upstream updates into your own branch.
A few months ago I played with forking it for my own tinkering but bailed because it seemed likely to turn into a rolling mass of merge conflicts if I were to make anything but minor changes.