> Why is your program so full of casts between pointer types that you have difficulty determining if you've avoided strict aliasing?
I write database storage engines. Most of the runtime address space is being dynamically paged to storage directly by user space. You can't use mmap() for this. Consequently, objects don't have a fixed address over their lifetime and what a pointer actually points to is not always knowable at compile-time. These are all things that have to be dynamically resolved at runtime with zero copies in every context the memory might be touched. Fairly standard high-performance database stuff. The intrinsic ambiguity about the contents of a memory address create many opportunities to inadvertently create strict aliasing violations.
I've been doing it a long time, so I know the correct incantation for virtually every difficult strict aliasing edge case. Most developers are ignorant of at least some of these incantations because they are surprisingly difficult to lookup, it took me years to figure out some of them. When developers don't know they tend to YOLO it and hope the compiler does the desired thing. Which mostly works in practice, until it doesn't.
Recent versions of C++ have added explicit helper functions, which is a big improvement. Most developers don't know the code incantation required to reliably achieve the same effect as std::start_lifetime_as and they shouldn't have to.
> These are all things that have to be dynamically resolved at runtime with zero copies in every context the memory might be touched. Fairly standard high-performance database stuff. The intrinsic ambiguity about the contents of a memory address create many opportunities to inadvertently create strict aliasing violations.
So how would do this in Rust, if at all? (That's the context of this subthread and the admonition not to play type punning games.)
Rust assumes noalias even for objects of the same type, and that's because the entire language is built on the foundational assumption that you cannot have both a mutable and immutable reference (or two mutable references) to the same object alive at the same time.
The main issue in C and C++ is that them aliasing rules are based on types, and the other aspects of the rules make type punning by casting pointers undefined behaviour no matter how valid either type is for a given region of memory. Rust actually does allow type punning through the obvious routes of std::mem::transmute or the same casting of raw pointers. It's wildly unsafe, and you need to have done your homework on whether the pointers represent valid states for the type you're punning to (right layout, alignment, and values), but it's defined behaviour if you're following those rules (one reason being that the lifetimes of the two references don't ever overlap in rust's model). In C and C++ you need to follow all those rules but also you need to jump through some extra hoops with the right incantations, just a pointer cast will be undefined behaviour even if you got everything else right.
(whats 100% not-OK in Rust is casting away constness. If you have a '&T' you must not touch, on pain of a thousand bugs)
The implementations of this type of thing in Rust I've seen just use "unsafe" everywhere. Even in strict C++ it is tricky to express the dynamic resolution of pointer type correctly. Type punning is a bad idea but there also weren't many alternatives so that was the idiom everyone used for a long time. Bugs do occur due to this, so the admonition against type punning has a purpose. It can be done without strict aliasing violations in C++, but the methods were a bit arcane until quite recently.
These scenarios cause other problems for Rust e.g. DMA hardware tacitly holds an invisible mutable reference to objects, but most developers never have to deal with cases like this. C++ provides some tools to annotate the code so that the compiler understands it cannot see all references to an object or that the lifetime is ambiguous.
This type of code is not common but high-performance storage engines are kind of a perfect storm of architectural requirements that break the core Rust invariants.
Rust's aliasing restrictions only apply to references. If you only stick to pointers and never at any point create a reference, you can alias to your heart's content.
For example, the equivalent[0] of the article's Offset Overlap example is perfectly valid according to Rust's abstract machine. What makes it hard is avoiding the creation of references. If I create a reference, then there's a good chance that the lifetimes don't get correctly linked up, and I accidentally have shared mutation/use after free/other UB.
Oh, wait, I just saw your name, I know who you are. But you are one of very few people on this planet writing C or C++ who get a pass on this kind of thing.
Almost nobody is using C or C++ to write super duper large data high performance databases. And even people who do work on databases don't need these breakneck levels of performance that you've dealt with.
In most cases people are breaking aliasing rules for no real performance advantage. These people should just stop, a large majority of code doesn't need to worry about aliasing rules because the vast majority of code written in C doesn't have these crazy performance requirements.
> The intrinsic ambiguity about the contents of a memory address create many opportunities to inadvertently create strict aliasing violations.
I don't get what you mean, at least the way you've explained it. Your memory might be volatile in the sense that it gets reused but if code is still operating on that memory then you don't have aliasing issues, you just have issues.
You can operate in terms of char * when it comes to your userspace paging implementation and your code which requested this paging (do you use use a segfault handler to implement this?) just operates in terms of whatever type it originally cast the void * value returned by your userspace mmap reimplementation. Am I misunderstanding something here?
> std::start_lifetime_as
I got into reading, since I don't know C++, I only know C, and this sounds like a relevant whitepaper:
By the sounds of it, this is a problem in C++ only, so it explains why I wasn't aware of such an issue. So you're telling me that in C++ you can't reliably implement a userspace mmap (or even use normal mmap) implementation before C++23 because without std::start_lifetime_as the C++ abstract machine doesn't provide a way of specifying when an object's lifetime starts?
This makes me wonder, what even is the incantation you're referring to?
> So you're telling me that in C++ you can't reliably implement a userspace mmap (or even use normal mmap) implementation before C++23 because without std::start_lifetime_as the C++ abstract machine doesn't provide a way of specifying when an object's lifetime starts?
std::start_lifetime_as is just a nice wrapper around an older incantation: do a no-op memmove and cast followed by a constant-folding barrier. C allows type punning with unions but I would assume the constant-folding issue would still exist. Compilers finally became clever enough about constant-folding to cause problems when you reinterpret the type at runtime.
I write database storage engines. Most of the runtime address space is being dynamically paged to storage directly by user space. You can't use mmap() for this. Consequently, objects don't have a fixed address over their lifetime and what a pointer actually points to is not always knowable at compile-time. These are all things that have to be dynamically resolved at runtime with zero copies in every context the memory might be touched. Fairly standard high-performance database stuff. The intrinsic ambiguity about the contents of a memory address create many opportunities to inadvertently create strict aliasing violations.
I've been doing it a long time, so I know the correct incantation for virtually every difficult strict aliasing edge case. Most developers are ignorant of at least some of these incantations because they are surprisingly difficult to lookup, it took me years to figure out some of them. When developers don't know they tend to YOLO it and hope the compiler does the desired thing. Which mostly works in practice, until it doesn't.
Recent versions of C++ have added explicit helper functions, which is a big improvement. Most developers don't know the code incantation required to reliably achieve the same effect as std::start_lifetime_as and they shouldn't have to.