Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> Can't be larger than 4 GB

How does that even happen in 2021?




4GB is the maximum value of a 32 bit unsigned int. If I had to guess that's the maximum size of the array/vector container in zig.


Zig has weird limits in a few places (1k compile-time branches without additional configuration, the width of any integer type needs to be less than 2^16, ...). Array/vector length's aren't the issue though; you can happily work with 64-bit lengths on a 64-bit system.

Skimming the source, there are places where the author explicitly chooses to represent lengths with 32-bit types (e.g., schema.zig/readByteArray()). I bet you could massage the code to work with larger data without many issues.


> I bet you could massage the code to work with larger data without many issues.

I'm sure the author would appreciate a pull request if it's that straight forward.


So it is that straightforward for a proof of concept (downgrade to an old version of Zig compatible with the project, patch an undefined variable bug the author introduced yesterday, s/32/64, add 4 bytes to main.zig->Header and its accesses).

Doing so makes the program slower though which might be a non-starter for a performance focused project. Plus you'd need a little more work to properly handle large archives on 32-bit and smaller systems (at least those which support >4GB files).


Offsets and lengths are stored as unsigned 32 bit integers. This saves space/memory, but means it won’t work for anything bigger than 4 GB

Maybe that’s overly conservative. Wouldn’t be hard to change


With most processors being 64-bit today, what would be the impact of using uint64 everywhere?


The answer is at the bottom of the page and all the offset/length data being uint32s.

2^32 = 4294967296

Not sure if this limitation is being enforced by upstream concerns, but this is why this code in particular is limited to that size.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: