I love osdev. I don't know what the end game is for me, but there's something really cool about being able to just create... whatever I want. It all sort of clicked for me when I was able to get some basic x86 interrupts and syscalls working. Once you get the syscalls going, the world is your oyster!
Seriously, I would highly recommend tinkering around with a hobby OS. I used it as an opportunity to learn Rust and I got more than I bargained for. Now, I feel somewhat comfortable in Rust AND I can throw more double and triple faults than most people in the world.
Unfortunately this book has the classic fault of osdev tutorials: lots of talking about x86 minutiae, very little osdev. X86 minutiae is not osdev. It is a vanishingly small portion of what even a basic OS has to deal with.
It would be great if there were more hobby OS resources targetting something other than X86. It is common and everything, but all the bootstrapping and device cruft involved isn't something you should need to get in your head, and 386 assembly by itself is terribly frustrating and obtuse to write by hand. I think most people might be better off starting out with a RISC microcontroller. There isn't nearly as much bullshit involved between getting a running kernel and talking some simple serial, and you can get an instruction set reference, assembler manual, and processor datasheet that will tell you everything you need to know. And it's easy to have fun hardware hacking that way too, making your own peripherals and such.
I think working with a ready-made Unix isn't as friendly to the idea of 'hobbying' an OS. For studying how Unix works, it's great. But if I wanted to make experimenting with my own bespoke system worth it, I would try to not have preconceived notions about how it should work, or copy an entire design. There are dime-a-dozen Unix-workalike hobby OSes out there, and in the shoes of an experimentalist there's a lot to weigh you down in the Unix ecosystem, and a lot of things worth trying that break from the way Unix does things.
If you really did want to work with an existing OS for actual research/experimentation purposes, and you're a fan of C, I would go with Plan 9/9front. There's a reason Bell Labs ditched Unix--because it wasn't a worthy research platform anymore.
>The old xv6 is x86-based and it's not officially maintained anymore.
Which is sensible. There's way too much legacy crap and ugliness that needs to be dealt with in that ISA to distract from the purpose of the project, which is to teach OS development.
- Get multitasking working in x86 since there are a ton of guides for that. Learn OSdev concepts.
- Read the ARMARM. It's many thousands of pages of dense technical material, nearly everything you need to get started. The only parts it doesn't cover are boot media/formats and peripherals.
There aren't good resources for ARM osdev because the environment is not standardized at all. ARM chips are used everywhere and you're often targeting specific boards, not chips. Writing the ARM-specific stuff is only half the battle.
True. I'm guessing it's the same reason that lexing and parsing are the focus of many langdev guides, despite being amongst the smallest/simplest parts of the compiler. I guess it makes sense, since those topics are the entry point into the field and at some point your learning is mostly self-directed.
People talk in hushed and revenant tones of SICP despite it having little to do with modern computing architecture. Perhaps x86 is still a great way to cut one's teeth.
The issue is not that x86 is mentioned. It's just that there is a lot more to an operating system than just getting code running on a CPU.
RPC, the input stack, the graphics stack, the network stack, audio, profiling, telemetry, scheduling, UI toolkits, security, service management, application management, etc.
If you want to learn these things you will need to read the documentation and source code for other OSs.
X86 also has little to do with modern computing architecture. That's just the ISA; it's not going to help you solve pipelining or cache issues inherent to the data flow.
Wishing x86 on students is sincerely such a sadistic act. Just pick an arbitrary isa that's easier to reason about (read: basically anything after 1990 not designed by intel) and stick to it.
x86 machines are cheap and ubiquitous. It is astoundingly easy to google for help/tips or ask random people on the net.
There are also really good virtual machines and emulators available.
Most of the weirdness came in in the 286 and can be more or less ignored. The parts that can't be ignored happen mostly during initialization.
If you want to baby step your way towards assembler and hw programming, DOSBox + an IDE/debugger setup from the late 80's/early 90's is really not a bad combo. That could be Turbo/Borland Pascal or C(++) with Turbo Debugger and Turbo Assembler, for example. You get a running environment, you have direct access to the hardware (DOS won't stop you), you can access it from Pascal/C, you can use inline assembler in both, and you can use external assembly files if you want. You can even successfully single-step and use breakpoints a lot of the time.
A Raspberry Pi or similar is a good alternative. I don't think any non-ARM platform is.
I don't find the x86 minutiae objectionable. If you want to build or even just understand an OS that runs on real hardware, you will have to deal with processor and hardware minutiae, and this gives one example of what you might encounter and how to deal with it.
For most OSes the amount of code that actually deals in processor minutiae is really quite small. Portable logic makes up the majority. Maybe an exception can be drawn for some very particular OSes like DOS or Windows 9x.
So you read a tutorial and it spends about 90% of its time talking about an area that really isn't that interesting compared to other areas. I just don't think that's very compelling at all.
I say this having my own hobby OS which I've ported to four architectures (m68k, amd64, aarch64, riscv). At the moment I'm working on a TCP/IP stack which has been really enjoyable and I've learnt a great deal in doing this already. Other fun areas have been virtual memory and page replacement, designing asychronous I/O facilities, and IPC mechanisms, as well as appropriate synchronisation for each (synchronisation alone is a very deep topic, and there's considerable scope for innovation with techniques like safe memory reclamation.) Others may differ, but in these I find a lot more of interest than x86 minutiae.
Without knowing much about the type of OS this book is, I imagine that there a lot of architectural aspects of x86 are considered "standard" to many of the x86 OSes out there. I hear a lot about code density in the x86, MIPS, and SH-2, compared to a lightweight RISC processor. Or, the book glazes over some of the more universal aspects of OSes, to address specific features of x86.
https://github.com/rswier/swieros is a cool project that uses it's own VM (with a C compiler included). I was surprised by all the capabilities it has.
My favorite OS book (less about development and more about how it works) is Operating Systems: Three Easy Pieces. Free in HTML and PDF form but they sell a printed book as well. https://pages.cs.wisc.edu/~remzi/OSTEP/
Thank you! This looks like a great resource on the topic.
I wish I still had the source code for the “OS” I made as a teenager. I got as far as writing an MBR boot loader, switching to protected mode, displaying characters on the screen, and keyboard input. I highly recommend it if you’re looking for a fun challenge.
Writing my own "OS" (read: mostly copying from the OG Bran's Kernel Development Tutorial [1]) was also a formative experience for me as a teenager. Really great way to learn systems programming and what goes on under the hood!
I didn't get as far as protected mode or keyboard input, but the MBR and characters on a screen were achievable by a teenager in the early 00s with the few online guides from osdev. I know, because I was one, and I achieved those things using nasm on a pentium and ran it on a 8086 in another room. it meant a lot of trips with a floppy disk and practising my swearing.
It was achievable long before that. The only hard part in the 80's was getting access to the datasheets, usually in the form of photocopies of photocopies.
Speaking of OS development, games make learning fun. I had the idea of making a game to teach operating systems while taking a journey through the history of computers. The player would play the part of the process scheduler and interrupt handler, starting on a single CPU system with very limited RAM, before growing to SMP systems then maybe getting to multi-system distributed computing platforms that we have today.
I have thought about making a game or a Fallout 3/4 mod in which the player needs to assembly pieces to repair a broken vintage computer before the nuclear war, and then search for some manuals and write programs to solve quests.
That sounds fun. I tried to design an assembly game once, but found I lack the creativity to design puzzles/goals that are not just "implement this common algorithm in assembly language". The idea of bootstrapping a PC from virtual firmware and writing an OS sounds nice, though.
> ...to teach operating systems while taking a journey through the history of computers. The player would play the part of the process scheduler and interrupt handler, starting on a single CPU system with very limited RAM...
Since everyone is plugging their favourite alternatives, I am still a big fan of Project Oberon, it may be not fit for purpose on 2025 computing, however it is a tiny graphical based OS, written in a memory safe systems language, with enough content to learn about OS development.
There are two Japanese operating system development books I wish have English translations because they take you as far as getting a graphical environment with windows working [0], one of the books is even a 30-days challenge of sort [1].
Fusion author here. I haven't abandoned the project, and I do plan to complete it, including a graphics subsystem and a network stack. I'm currently trying to crack a tough nut, which is efficient zero-copy IPC using message passing over channels with a shared heap. This is a fundamental piece that will be used to communicate between different components (both within the kernel and across kernel-user space). Once I have a proper implementation, I expect the pace to pick up.
And of course I will document everything as I make more progress.
Are you trying to do this message passing using Nim channels ?
For my use-cases, i always had to resort to just passing `pointers`, to prevent any copying, most of time i just divide the problem to use independent memory locations to write to avoid using locks, but that is not general pattern for sure. For read only purposes i just use `cursor` while casting to appropriate type. If you find a useful pattern, please share.
No. Nim channels are for inter-thread communication within the same process, and they perform a deep-copy of messages. Fusion channels are a different beast. They're kernel objects that require syscalls to create/open/send/recv/close. In order to achieve zero-copy, the channel heap needs to be in user-space, but managed by the kernel. It's kind of similar to shared memory on POSIX, but has message passing semantics, rather than arbitrary read/write from/to shared memory. And since Fusion is a single address space operating system, I have the luxury of passing pointers (to data on the channel heap) directly between tasks without resorting to (de)serialization. Protection is achieved through page table mappings, where e.g. the sender would have read/write access to the channel heap, but the receiver has read-only access to the same memory.
As for your case, I understand the need to avoid locks. In my case, the channel is implemented as a queue of messages, which is implicitly synchronized between tasks. Currently it's implemented using a blocking queue, since I want to put senders/receivers to sleep when the queue is full/empty, respectively. The API does support a no-wait option, but I'm not currently using it.
There will be a lot to write about once I'm past this point :)
Thanks for explaining it, given you are writing it from scratch it gives you a lot of control in modelling a particular feature!
I did bookmark this project a few months ago but couldn't spend time to understand more about it. I wasn't aware of documentation, which should now make it easy to start with. Thanks for putting a lot of work in documentation!
This took me a nostalgic dive back to 2000's when a website called planet-source-code.com was popular among the developer community. It had hundreds of mini operating systems written by members. Most of them were written in C/C++ and Assembly for the bootloader, compiled using Mingw32, and booting from a regular Floppy disk.
Seriously, I would highly recommend tinkering around with a hobby OS. I used it as an opportunity to learn Rust and I got more than I bargained for. Now, I feel somewhat comfortable in Rust AND I can throw more double and triple faults than most people in the world.