Git gets very slow with large files and isn't optimized for binaries. bup (https://github.com/bup/bup) is based on git but designed to solve this problem (has its own high-speed packwriter/readers).

Although still can't delete old data at the moment (though its being worked on).