Author here. I stopped working on this a few years ago, so now's as good a time as any for an overly-long, rambly, unedited retrospective.
When I started this, I was working on GNOME's window manager full-time, and wanted to learn intricately how X11's drawing model worked, so over the course of a few weeks in a hotel room, I recreated large parts of X11's drawing model in a web browser, fixing artifacts as I went along, until I feel I had a really good grasp of it. My initial test scene was a traditional desktop-like approach with a taskbar and xeyes, both of which are still in the codebase today, but untested [0].
I didn't know what I wanted to do with it, until I settled upon using snippets of it to build a long-form article. I learned a lot about the difficulty of writing, of pedagogy, of that blurred line between being technically correct 100% of the time vs. telling a few small lies here and there to keep the flow consistent and help people see the broader picture.
At my day job, I had mostly moved onto Wayland, where some of the bits I picked up here really helped me design better protocols and systems. My goal with the series was to try to be as neutral as possible, and my original design was to have a giant caution sign around "Author Opinion Zones" where I would talk about how certain design features haven't held up well in practice. But quickly, people on Hacker News or Phoronix or Reddit seemed to skim the article, pick up a piece here or there, and go on straight to bashing Wayland, gleefully unaware that I was one of the people making it.
So, the end result was that I basically stopped working on Xplain basically after the second article. The COMPOSITE article was one I made after a colleague was having trouble understanding COMPOSITE, and I figured it was easier to write with my framework than explain in a chatroom, and maybe some others would appreciate it.
I have a deep passion for sharing my knowledge, and Xplain was the format I first really used to do it widely, so I tried to keep it exciting for me by changing it from "Xplain" to "Explanations", and opening up the topics from X11 to just about anything, but at some point I was just unhappy working on it.
The last thing I was working on was a continuation of my Basic 2D rasterization article, where I had a fun code editor you could use to make your own graphics [1] [2], but as fun as the technology was, I couldn't find a satisfying flow to the article, so I stopped it. Parts of it were later recycled for an article on the histories of 2D and 3D graphics. [3]
Around mid-2016, I had stopped working on Linux and open-source graphics entirely, and by 2017 I had exited the open-source industry completely and jumped ship to professional game development. I still have a deep love for graphics and a passion to explain things. I just released a new side project a few days ago for it, even.
Here's some stuff I'm working on these days. It's much cooler than X11/Wayland flamewars, in my opinion.
I want to thank you for this page! It really helped me understand how X11 worked when my job at the time required some rather esoteric (to me) X11 knowledge. I had to write a very specific purpose window manager (in JavaScript, which really made it interesting) that had to work around some odd client behavior, and I also had to submit a patch to X.org to fix a bug in Xephyr. Without your guide I would have spent far longer digging in the docs to figure things out.
I just watched the YouTube link, and it's amazing! I've been toying with making some of my own video content, and this is inspiring. There's only so many hours in the day though.
Thank you! Making good YouTube content has been on my bucket list for a while, so I'm really happy I did it. I will admit it took a lot out of me -- basically all my weekends and spare time for about a month. But the reception has been more than worth it.
If you ever do make some good YouTube content, do let me know! You've definitely been an inspiration for me in my graphics journey, and I imagine you have some incredible insights and perspective into the graphics world.
Blame your graphics drivers. The lack of working modern graphics on Linux is one of the reasons I moved away from it. It works fine for me locally, so it's an issue with your machine. Stare at chrome://gpu and https://webglreport.com/?v=2 for a bit, or maybe file a bug with your graphics driver.
Hey, that YouTube video got recommended to me a couple days ago on their front page and I really enjoyed watching it. One of the few instances where YouTube didn't recommend random crap or just videos of channels I'm already following.
Again, great job! That's exactly the kind of content I'm enjoying most on YouTube.
Looking over this stuff, I get an uneasy feeling. X was a brilliant solution to problems 35 years ago, where you barely had enough memory for a full frame buffer and the graphics model was primitive (no antialiasing or transparency). From what I've seen, Wayland is a good solution for the model maybe 15 years ago, where you're willing to accept the frame-and-a-half latency hit from having a compositor in the path.
Basically, my question is whether anybody is working on a solution for the hardware we have today, where there are hardware overlays, the GPU throughput is orders of magnitude higher than what's needed to composite a desktop, but we still care about memory bandwidth, largely because all of resolution, bit depth, and frame rate is still going up. And you really want to be able to schedule every pixel that gets presented with a high degree of timing accuracy.
I've become familiar with Windows, but feel like it's a bunch of hacks. They will promote a window to an overlay, but it's all a heuristic, and I'm pretty sure latency will stutter while it does the transition in and out. And the new flip-model stuff actually regresses the ability to do smooth window resizing.
I'm not even sure macOS is working on lowering latency. I haven't seen any mention from them of hardware overlays.
Most of the attention these days seems to be in the VR/AR space, where latency is obviously important, and there's more of a green field with respect to legacy compatibility, but I'd love to see this problem space actually solved on the desktop. Doing this right is not easy - it probably requires some form of soft real time discipline across the graphics stack (including GPU scheduling) so you can race the beam.
I have a lot of replies to this, but the back-and-forth format that HN has isn't the best for collaborative discussion. Imagine us sitting in a room with a whiteboard, rather than bickering at each other at loud volume. If you're ever in south bay area and want to meet up at a coffee shop, let's :)
> Basically, my question is whether anybody is working on a solution for the hardware we have today, where there are hardware overlays, the GPU throughput is orders of magnitude higher than what's needed to composite a desktop
Hardware overlays are... not that popular on desktop devices? You just stomp on the gas pedal with bandwidth and power consumption on consumer desktop gaming GPUs. Besides a couple YUV planes and a cursor overlay, you don't see that much in the way of CRTC capabilities on desktop. All of that's happening in the mobile space, because of power consumption reasons, where you basically have an SNES PPU in your CRTC with multi-plane compositing.
I remain unconvinced that compositors need to add a frame and a half of latency. 16ms is a lot of time to render a frame, and composition isn't too difficult of a task. It just comes down to buffer handoff, which can be smoothed out over time. Similarly, the new flip-model isn't the thing that makes smooth resizing difficult, buffer allocation and resizing has always been a pain and racy on Windows/X11. This is one of the things that Wayland fixes -- it was actually the thing that caused Kristian to start Wayland forever ago.
Presentation and timing is something I think should be possible, but I don't think it makes sense for every app to have to handle it. Multi-monitor scenarios are what make global timing difficult, and having per-monitor surfaces is just a pain to deal with from an API standpoint; if we did the work to have per-monitor timing information and split surface rendering, nobody wants to go through the work to set up two devices and present different ones at the right time.
You might protest that all monitor timing is roughly the same these days, but someone hooking up their laptop to a projector, you get radically different timing.
In many respects, I think graphics is in a local minima right now: it works well enough and we understand it well enough that nobody's really interested in pushing us to the next thing. VR/AR research might, but that's been relatively unimportant as a target platform, commercially.
> If you're ever in south bay area and want to meet up at a coffee shop, let's :)
For what it's worth, I would be very interested in chatting about this too, as this is my area right now. I'm working on what will hopefully become the Firefox OS compositor integration library, with an eye toward optimum power consumption, and am knee-deep in details of cross-platform composition. It's a very interesting topic.
This topic is also of interest to me. I work on a compositor integration protocol called virtio-wayland that is intended to allow Wayland apps running inside of VMs to communicate with a Wayland compositor on the host, shared memory and all. It's used for the Linux apps support for Chrome OS.
I don't get to the south bay often, but sounds like it might be good to have a little compositor meetup. I can certainly vouch for pcwalton's interest and knowledge in the topic :) Follow up by email?
Wayland's latency was improved a few years ago to about half a frame, there's probably more room for improvement, but I think just getting a 240hz monitor is better investment of resources: https://ppaalanen.blogspot.com/2015/02/weston-repaint-schedu...
The only applications strongly sensitive to latency are video games or similar, and they take a special low-latency path through the graphics stack, exclusive full-screen mode. Monitor, graphics card, and game engine developers are all doing interesting things with scheduling there, but the main function of the OS / compositor is to get out of the way.
Beyond that there's video playback, but 60fps video is rare, videos have (relatively) huge buffers of decoded video, and dropped frames are easy to overlook, so I don't think it's much of a priority for the OS. Nobody will notice the window manager adding an extra 10ms delay to the video.
Businesses don't care about visual fidelity at all, the most common monitor size is still 1366 x 768. So there's no big push commercially, besides the aforementioned VR/AR.
You can still paint with 200ms of latency, it just isn't very pleasant. Atom adds an average of 50ms of latency, people still use it: https://pavelfatin.com/typing-with-pleasure/.
It still seems like it's mostly about having the bandwidth to drive a 240hz or higher display though. The ongoing computation required for painting is minimal. Unless Wayland or X are doing a lot of IPC on every frame there's no reason to write a new graphics stack.
I would like to see someone reimplement NeWS now. Hardware is way faster and we have more choices of embedded interpreter. Not that PostScript was, and is, not genius for that purpose, but there choices.
Also, Don Hopkins hangs out on HN, where he's already weighed in a few times with some ideas for future work if you grok through his comments.
On the contrary, PostScript was a brilliant choice for scripting your window system at the time (≈1990). The modern crop of dynamic languages hadn't matured (JS, Guile, Python, Perl, Ruby, even Tcl), so the only real alternative would have been a Lisp, and PostScript already had a truly great graphics API — an improvement in many ways over many of its successors, including X11 and Windows GDI. Gosling certainly could have implemented something like Emacs Lisp, and it would have been easier to program in than PostScript, but it also would have been painfully slow, and he would have had to do the graphics API design work — and probably fucked it up, as the Win16 and X11 teams did. Even today, PostScript interpreters are quite a bit faster than most of the language implementations I described above, though modern JS, LuaJIT, Self, and of course HotSpot Java do beat them.
I agree, and I've written a bunch of stuff in Dercuano about this problem, notes/real-time-windowing.html and notes/wercam-scriptable-windows.html for example. I'm interested to hear what approaches you think are likely to help with the problem.
> So, a lot of the design of X11 is giving clients and apps a large set of tools to design and implement their app, without any real opinionated design about how an app should be made. An X client can use this tool or that tool to battle this tradeoff or that tradeoffs, and neither is fundamentally a wrong choice.
That is something that can not be said about Wayland. Besides the "every frame is perfect" mantra Wayland has no design philosophy at all. It is also a weak argument to introduce a new protocol. With modern drivers it is rare to experience tearing on X11 and on some occasions it even makes sense to disable vsync (e.g. competitive gaming).
The introduction of Wayland was a mistake because X11 has a huge ecosystem with lot of little programs and a large diverse users base. Almost every Unix OS supported X11 at some point. There is a reason why Intel still stuffs the i368 instruction set in their processors, Microsoft Windows still can execute programs from 1995 and IBM is still able to make money with vastly overpriced mainframes. Successful systems have the tendency to stay and accumulate cruft as a result. The abandonment of X11 will be a heavy blow to the FOSS community (and maybe that is exactly the intend).
One of my many mistakes that burned me out on the whole project was trying to present X11 in a neutral way. I personally believe there's a lot broken with the architecture and the ecosystem, but wanted to let X11 sell its own ideas, slowly letting you figure out they were the wrong ones. But that never happened. I should have been more up-front by pointing out brokenness when I see it. The whole window tree was the wrong idea.
I worked closely on Wayland for a few years of my professional life, and helped design important parts. I believe X11 made plenty of wrong choices, and Wayland made much more informed ones.
But my attempt at staying neutral only fueled the Wayland vs. X11 debate. I stepped away entirely from open-source graphics in 2016, but it still goes on today.
As someone who is working with X11, I strongly disagree. It really needs replacing badly. It doesn't even work; only a few code paths even get tested on for example GLX, which is why complex X11 apps like browsers need blacklists.
> There is a reason why Intel still stuffs the i368 instruction set in their processors
If you want a CPU analogy, Wayland is more like ARMv8, which was mostly a clean break in terms of the AArch64 ISA, but which still supports 32-bit at the process level. 32-bit and 64-bit code cannot be intermixed in the same address space, but you can run 32-bit and 64-bit apps simultaneously on the same OS, analogously to the way you can run X11 and Wayland apps on the same Wayland desktop using Xwayland. This actually worked out beautifully, and I would have preferred that x86 had followed the same trajectory.
> As someone who is working with X11, I strongly disagree. It really needs replacing badly. It doesn't even work; only a few code paths even get tested on for example GLX, which is why complex X11 apps like browsers need blacklists.
Blacklists are needed because OpenGL drivers for specific hardware have bugs and can have missing functionality. Those can affect Wayland too, since the same drivers are used.
Modern Xorg supports direct rendering well, Keith Packard authored a lot of modernized infrastructure such as DRI2/DRI3, and even support for VR (work he's been doing for Valve).
There's also the important VK_display_timing extension implementation he's done with Croateams Alen Ladavac:
Sadly, I don't see much more recent activity from him or Alen about these topics, but it shows that Xorg is still the main platform of inovation for Linux real-time graphics needs.
X11 and its ecosystem of applications isn't abandoned by Wayland, because you can run an X11 server on Wayland. By all means, debate the merits of Wayland itself, but don't lean on inaccuracies and conspiracy theories.
My window manager won't work, my hotkey daemon won't work, my screenshot tool won't work, any X11 program that draws to the root window won't work. Xwayland is obviously not a working solution to fix the ecosystem problem. Who is inaccurate now?
> debate the merits
Wayland has no merits. That is kind of the problem. It is just a reimplementation of stuff that worked.
Also, even the limited X11 support which there currently is under Wayland will probably go the way of the dodo because it involves about the same amount of code as supporting X11 natively. For example, it requires every Wayland compositor to be a X window manager with all the complexity that implies, plus running a full hardware-accelerated X server and the OpenGL driver support for that. I can't imagine the desktop environments and distros who are so keen on dropping legacy features like X and support for 32-bit apps will keep all that code around for long.
Once the integrated X support is gone you can't just replace it with a standalone X server of the sort that's available on OSes like Windows and Mac OS X either - by design, the standard Wayland API doesn't provide the functionality required to run X (or Windows) apps. In particular, it provides no way to set or get the absolute position of top-level windows, which apps on those platforms need to support things like menus. In a few years time, I can easily see modern Linux becoming the only current desktop OS that cannot run X apps.
Xwayland doesn't need a hardware specific DDX driver, because firstly, many of the features can be implemented by forwarding to the Wayland server, and secondly, the drawing operations (which don't exist in Wayland) have been implemented hardware independently, based on OpenGL, in the GLAMOR backend.
If hardware vendors don't have to care about it, then Xwayland should not create a huge maintenance burden and thus I'm pretty confident it will be available until 2030; enterprise distros likely won't be able to remove it earlier anyway.
X doesn't need a hardware specific DDX driver either when running on hardware supported by Wayland+Xwayland. The GLAMOR backend works just as well for a standalone X server, and it can just use the same libraries and APIs as Wayland does for the rest of the stuff. (There's not really any kind of hardware abstraction layer in Wayland itself.) The main reason why people use the hardware specific X drivers still is because they're more mature and robust than the Wayland stuff.
Maintaining Xwayland basically means maintaining all of Xorg except the hardware-specific drivers. And Red Hat doesn't want to maintain that code base anymore. Therefore, Xwayland will soon be abandoned, then deprecated, then removed entirely.
The wayland project exposes what marvelous piece of software X11 really is since Wayland is still not able to compete, otherwise the whole Linux community would use it already.
I respect the intention and efforts of the Wayland community to make things better. However, I wished the community would have focused on making X11 better (easier to maintain with modularity etc.) instead of following the mainstream which is focused on eye candy and other sensual features to attract customers.
The unfortunate developments of KDE3/4, Gnome2/3 and Win7/8 exposed that boring usable functionality is much more important than exciting sensual functionality.
It is, including the author of this post. Unlike other controversial things like systemd or pulseaudio, Wayland was started by major Xorg contributors. I've watched some of their talks and can see why they went down this route.
I've tired Wayland a few times and whenever I do, I give it an honest shot for a few weeks, but tend to run into issues (Sway being pretty buggy or oddities with old X apps or Steam, etc.) I've heard Sway has gotten a lot better and I should really try it out again; probably the next time I setup a laptop.
I do wonder what plans FreeBSD and OpenBSD have for the future. Currently they main their own Xorg trees, but I wonder if we'll ever see Wayland implemented on other platforms.
I used Sway during the 0.14 cycle and ran into some issues e.g. with pointer warping in X-based games, so I decided to wait until 1.x. I just switched to Sway again, with 1.1.2 AFAIR, and mostly everything is working smoothly for me this time around, including Minecraft. :)
Things that don't work for me:
- mirroring (i.e. two displays showing the same windows at once), but AFAIK this is being worked on
- KDE Connect (not Sway's fault, but not having this working really hurts sometimes and might just make me go back to Plasma)
Windows 8 was a major improvement saddled with an untested UI that someone was using to make their political mark. The internals of Windows 8 and Windows 10 are a major, major step forward from 7.
Do you have a horse in this race? I ask because the writer's background is in writing Wayland, a lot of other commenters are writing software related to compositing, and there are even a few developers of X Window Systems.
I only ask because it's very easy to be cynical online and I'm not sure that this comment comes with much to back it up. There's very much an 'old man shouting at cloud' vibe to it, even if unintended, but I ask in good faith in case you have insights we don't know yet. :)
I just witnessed how complete ecosystems are destroyed all the time in the name of incremental updates.
Some prominent example are Python3, Gnome3 or KDE4. Every time such a transition takes place lots of people that are not full time programmers but are using and creating in the FOSS realm mainly in their spare time have to spend a lot of time to adjust to the radical changes. And every time some useful legacy software just gets lost because the maintainers don't bother with porting and just scrap the thing altogether. A permanently moving target makes FOSS that is dependent on many hobbyists just impossible. Just stop changing things all the time. FOSS is clearly reliant on hobbyists and those usually have real jobs and other things to do in their life.
The funny thing about that stuff is it works better with multimonitor setups than every other environment which followed later. By that i mean laptops in docking stations with one or two external displays, not necessarily the same size and resolution, and hotplugging.
But how do i dare to speak of that when the herd moved on and bleats bitrotten!,deprecated!,unsafe!,unsupported!,obsolete!
/me mumbles something about stoned sheeps which obviously had too much of the wrong weeds...
I run a Arch with the newest GPU drivers and the Compton compositor. I still have significant screen tearing in every app except video games with v-sync enabled and mpv with the vulkan backend.
I feel like this is short changing X. I have seen it as a good local solution over 20 years or so.
Did you ever see those 90s commercial Unix workstations? Like from SGI?
How about a Nokia N900? X on a smartphone. It ran pretty well. Today's smartphone hardware would run circles around that thing so I can only assume if somebody ported X to those it would also do fine.
I feel like people see things they don't like or imperfections and bash the whole thing as useless. X had a lot of mileage. I use it frequently today and don't see a lot of problems.
This was a really nice demo! It can even be used to understand concepts used in the HTML5 canvas, such as aliasing. Hell, it could possibly used as en entry to learning how to program ray tracers. At the bare bones, construction of complex bodies in POV-Ray was used by Constructive Solid Geometry hearkening back to the basic union, xor, intersection, etc, operations done in the demo, at least on a conceptual level.
I haven't used linux as my primary OS for many years now. If not X11 or Wayland, what other alternative? Is there some other relevant development going on?
There's Arcan: https://github.com/letoram/arcan . In the words of the developer, it is a "multimedia engine" (a combination of display server, game engine and multimedia framework).
I had been working on Linux professionally for about 6 years at that point, and got burned out on it. I have a lot of thoughts on the success of Linux as a general-use platform but mostly keep them to myself these days.
I still have a deep love for graphics, and have since moved onto working in the games industry, where I am much happier. I also still have a passion for explaining graphics topics.
The basic problem with X11 is it wraps a whole bunch of things into one software package. So in theory you have to know about fonts, OpenGL, graphics drivers & memory management, the window model and networking simultaneously to configure and run the thing.
Wayland is a protocol for a well designed future where all that functionality is broken up into separate pieces of software. Wayland handles the window model and most of the other parts already live in their own software projects. The problem is that Wayland makes some changes to the basic window model on the way through that are a big deal and so far there is no great software written that uses it.
Adoption is very slow because the best case scenario is there a Wayland compositor that has feature parity with an X11 window manage and a bunch of weird problems cropping up because screen recording is a to-be-solved problem.
My position from the armchair is that Wayland really needed a complimentary official protocol for how to request another application's graphical output. Not coordinating that means that anything that supports Wayland is going to be practically inferior to an X11 solution. It'll get fixed sooner or later I expect, it isn't a technically hard problem.
The issue is I need the powerfull features, like I have a script that OCR a region of the screen when I press a button (the coordinates depend of the app that runs) so I do not want a GUI provided by the DE to let me capture the screen.
Anyway the issue with Wayland is that it is not ready, the important screen recording feature is yet not here,I seen also some complaints from gamers and their issues are a won't fix, probably when Wayland will be actually ready we will need something to replace it or a Wayland 2 that learned from first attempt mistakes.
Having recently moved from X to Wayland there are still some things missing sometimes but they're not many. If you want a DE at least GNOME and I think KDE as well are making the switch reasonably seamless. And there are some features that Wayland solves much more thoroughly (e.g., tear free output, fractional scaling). The network transparency part, that is often brought up but rarely used in X is also being delivered:
I suffer of macular degeneration https://en.wikipedia.org/wiki/Macular_degeneration , so I am having issues reading texts , I am using TTS and Zoom features for most applications but the big issue I have is with video games. For open source game engines I can patch them to have the text pushed to my TTS scripts but for closed source game engine OCR or zoom is the only solution but it has a lot of issues.
1 my zoom follows the mouse but this is broken is nome games where the trying to move the zoom area to the edge where the text is would move the camera in game , pausing the game is a solution but some games would put a Pause screen on top of te text I want to read.
The OCR has issues when games use fancy fonts, old games use some pixelated fonts that are impossible to OCR, other games use more artistic fonts that are also hard to OCR, other games use text over semi transp[arent background and it makes hard to separate the text from the background.
For DirectX I am thinking if I could maybe intercept the draw text calls, I am considering making a Stack OVerflow post but I am a bit anxious that I am not sure what the correct title and tags should be and I could get the off topic and the question closed.
Quoth the wiki 'PipeWire is a rather new multimedia framework by GNOME'.
There will be people who don't want to use GNOME frameworks and don't want to use the GNOME window manager.
Wayland is enforcing a policy ('don't access other devices graphics output') where it should be providing capability ('an applications graphics output is private by default but that can be configured'). However it looks like the design of Wayland purposefully omitted that sort of configuration from the protocol. So instead of having a security model that is sufficiently complicated to allow screen recording, the security model is 'No'.
To get a framework independent solution all the compositors would need to implement an identical API for accessing graphics output. That needs to be coordinated in the same fashion as the Wayland spec. As the spec stands, it is just asking for every framework, compositor or application to go implement its own different solution.
The Wayland designers may even have tried to avoid explicitly creating a security model, but in practice it comes with an implicit security model that is inflexible, unconfigurable and difficult to coordinate around.
From what I gather, Pipewire isn't part of Gnome, i.e. it's generic and can be used with any DE.
For example, all compositors are using libinput, which replaces legacy ways of X handling input. It's not mandated by the protocol either. So I don't see why they can't use Pipewire for audio and video needs.
If anyone wants to make a better alternative to standardize on - they are free to propose it.
But libinput is a freedesktop.org project, ie, supported by the same people who are working on Wayland. It was specifically built to support Wayland.
Pipewire looks like some audio-visual framework for GNOME and GTK apps and while it might be fantastic it is both new and not very comprehensibly advertised. And it will probably be very complicated and involve some rather interesting design issues compared to building something in when the Wayland protocol was created.
We're coming out of a world where the standard used to be "run graphics drivers in userspace!" so any weird compromise is workable (including an audio framework deciding on graphics security policy). But ultimately the idea that a GNOME-sponsored effort is going to be what sorts out the security model for desktop apps is weird. I can't find anything to suggest that Pipewire is intended to be the definitive way to do screen recording, it looks to me like that might be a happy side effect. It wouldn't be surprising if it turns into two or three competing ideas.
You're mistaken with pipewire, it's a daemon intending to replace pulseaudio at some point, it's not a gnome component.
Screen recording is also available in wlroot based window managers.
The wayland environment is maturing quite fast now.
So far video applications on Linux have accessed the
hardware directly, meaning that two applications can’t
access e.g. a webcam simultaneously. Solving that
problem, and screen sharing with Wayland, were to my
understanding the main motivations for creating a video
daemon.
So screen sharing on Wayland (that covers capture use case) was one of the main motivations behind it.
If both Gnome and KDE (the most widely used DEs) are behind it, I'd say it's already a major consensus on where things are going.
From my perspective, Wayland "fixes" things that could allow bad things but were also useful. For instance, screenshots/recording took a long time to be implemented, and is still constrained, because of the security risk of arbitrary programs recording the screen. Because that is a risk... but users still want it. As another example, I use xdotool to inject input to programs in X; in Wayland, that would need to be explicitly supported per-compositer.
Well, removing or not implementing such features because of security is always a big fat sign of something sneaky or even worse, outright incompetent going on. As such security risks of features can always be addressed through some sort of authorization mechanism.
I'm running Sway WM right now, a Wayland compositor that implements an i3-compatible window manager, and in fact working on the code (hopefully merging my first contribution soon.)
I'd say the biggest issue is really maturity. I actually find Wayland to be a surprisingly nice piece of software engineering. I don't know if I like the architecture really, but it comes together fairly nicely in the end.
For example, there's unanswered security questions. It would be good if applications couldn't just go and manipulate and spy on other apps like they can with X11. Wayland indeed solves this issue, but it leaves other problems. How do I capture the desktop?
wl_roots offers some extensions that help alleviate the problem, but they don't solve the full problem; for example, you can capture the screen, for which programs and an OBS plugin exists, but there's no upstream Wayland extension that can make WebRTC desktop capture work. I think what we really need is the ability to integrate policy and consent requests, but I'm not sure whether there's progress on that front, or how long it will take. Once the problem is solved, upstreamed to WebRTC, and that upstream version trickles down to Electron apps, Firefox and Chrome stable versions... well, things will be in a better place.
Speaking of web browsers, today Firefox still has some issues to go before it is fully working in Wayland natively. You can do it today, but it is not perfect; there are sometimes places in the code where it assumes it is running on X11, and things simply break. The screenshot tool is just flat out not present when running under Wayland due to the above issue.
Applications are slow on the uptake too. XWayland covers most of them, but it is not perfect. Like anything that captures the screen, of course, is not going to work under XWayland, but also, XWayland apps may behave a little off. In Sway WM, one bug is that when the mouse travels from an X window to a Wayland window, sometimes it behaves as though the mouse is still hovering over the X window at the last position it touched, which leads to tooltips showing up where they shouldn't. There's still a decent amount of GTK2 apps, and even Qt apps aren't all Wayland yet; Krita plans to start working on support I believe next year.
In my case, here's the itch I'm currently scratching: Sway WM also does not have support for drawing tablets. I found this out, by, well, loading up a drawing program and noticing the lack of pressure sensitivity. So, that's what I'm currently working on. Needless to say, this is a non-starter for anyone that relies on a graphics tablet, although if you are just using GNOME 3 it is unlikely to be an issue since I'm sure GNOME 3 supports graphics tablets just fine.
NVIDIA drivers are still a pain in the neck. I gave up and bought an AMD graphics card, and frankly I can't recommend it enough. Basically everything works better on Linux, but certainly Wayland works better. It's worth giving up some raw power. Still, for many people the proprietary NVIDIA drivers are going to need to work, and today support varies from lackluster to poor depending on what compositor you use.
So Firefox supports screenshots when running on the Gnome compositor, but not running on sway? Looks like we're heading into interesting times where applications are getting tied to certain DEs with little to no cross-compatibility.
IDK. Sway is my daily driver. I just checked Firefox, and the screenshot tool works as expected (including clipboard). So unless GP is talking about capturing non-Firefox windows with that tool, I do not have that issue.
This Firefox instance is running on native Wayland. The only issue I have with it is sometimes the coordinates for the context menu are wrong.
But yeah, there might be compat issues, if some protocols are required for stuff. But Wayland itself is the baseline: you'll always be able to draw windows and interact with them.
What's left to extra protocols isn't guaranteed to work: control of the compositor with a client, for instance (color correction, virtual keyboard, screenshots, window placement, screen geometry, window decorations, etc.) is completely up to the compositor (and the protocols it can speak with the client). So is other stuff, like graphics tablet input (with pressure sensitivity, tile, etc), minimizing windows, window miniatures as used for KDE's taskbar as well, I think.
But then, it's not like everything worked flawlessly everywhere on X either :)
And as long as the baseline is solid, we can build on it.
I may have done something wrong. I am not highly knowledgable here. For example, I just learned of Pipewire.
That said, the problem occurred for me when using the NixOS package firefox-wayland. It is possibly better to use upstream firefox with Wayland, but I have not tried; I am just running firefox on Xwayland for now.
I think capturing desktop and screenshots should be addressed by standard Wayland extensions with integration with something like Pipewire as far as I know. I didn't follow the progress of that though.
As for Nvidia - don't waste your time on their proprietary drivers on Linux. They don't to play by the rules (refuse to upstream), so no need to spend time supporting their blobs.
Like I said... I gave up and bought an AMD card. Not everyone is going to do this. It's a step down from my NVIDIA card.
I think there is still hope. Obviously today's reality is still bleak with signed blobs and plenty of question marks, but I think open-gpu-doc is a sign that NVIDIA does actually want to square away this problem, maybe they're just not sure how.
I do believe NVIDIA is mostly responsible for Wayland's poor adoption, and I firmly believe that this also lead to Wayland not getting as much developer attention as it could've gotten. The fact that one company wields this much power is a serious problem. I kind of understand the hostility Linux kernel developers seem to have towards proprietary drivers under Linux now.
Kernel developers are right to push back against blobs. Nvidia is the bad player here.
AMD is doing fine though. I have RX 5700 XT, and besides for some rough edges due to it being very new and drivers still fixing bugs, it's a very good card.
I expect Nvidia usage on Linux to plummet, due to their refusal to provide open drivers. It's already happening, just will take some time for it to drop to the point where they have little influence.
Personally I’m not the biggest fan of wayland (I haven’t found a compositor I like and using OpenGL for everything including VTEs feels like a mistake.) but I will say it’s common knowledge (as far as I’m aware) that nvidia drivers on Linux are pretty bad and you’re not going to have a good experience with them no matter what kind of display manager you use.
In what world are nVidia drivers crap? I'm regularly doing some pretty funky stuff with OpenGL on top of X11 and nVidia and this setup fails me less often and less hard than any equivalent AMD setup has done so far. Unless I'm trying to allocating tons textures or ridiculously sized vertex buffers, I don't see any hickups at all.
In the Linux world. All of their problems (roughly you can call them poor integration with the Linux stack) are consequences of them refusing to upstream their kernel driver.
The fact that Nvidia blob fails you less is not a good indicator for you, unless you test with Mesa as well in each case. Mesa is a lot stricter about OpenGL spec compliance. So if you only test on Nvidia, you likely have bugs that you didn't notice.
Every time I tested with Mesa and had issues, it turned out that Mesa was out of spec and quite specifically lying about that fact. And this was with absolutely essential features like rendering into floating point textures. So no, Mesa is not an indicator for spec compliance, either.
So when was it exactly? I doubt recently. Mesa progressed to the point of full compliance a few years ago. Today, it's trusted a lot more than the blob, due to the later silently swallowing errors frequently and messing around with substituting shaders behind the scenes.
Try testing current Mesa versions. Previous experiences are no longer relevant.
> Sway WM also does not have support for drawing tablets. I found this out, by, well, loading up a drawing program and noticing the lack of pressure sensitivity. So, that's what I'm currently working on.
Hey, thank you for working on this! The other day I plugged a tablet in, and found myself quite dumb with it not working properly. I wanted to have a go at implementing the tablet v2 protocol under wlroots, but it sounds like I'll hack on something else : P
It has been an interesting exercise so far, and frankly a quite pleasant one. Rather than autotools or CMake, many of these projects are using Meson, which is just so much nicer. I also really have learned to appreciate Sway WM's code, which I personally think is quite nice now, after having worked with it (though, admittedly, the ease of segfaulting does make you long for something like Rust.)
wlroots already has first-class drawing tablet support, but the "process input events" parts and the "dispatch input events to clients" parts are separated and it's up to the compositor to connect them in the ways they best see fit.
In the past that was a common reason. Today it's not relevant anymore. And downsides of the blob make buying Nvidia a non starter. You can clearly observe this trend today among Linux users.
For Windows users coming to Linux with Nvidia, this usually simply involves one extra cycle of GPU upgrades. By the next one, it's AMD for them. So in the long run - still not an issue.
Every time I see the X11 protocol, I realize that the UNIX philosophy (everything is a file) is dead.
It’s a pity that instead of improving the speed of the file system, everyone started to ignore it, hence making it’s speed even less important, and on and on...
And now nobody uses the file system for these purposes.
Not disagreeing we've moved on from it a bit in some places, but.. "Everything is a file" never meant "everything is a disk file" - think of std{in,out,err} or network sockets. The implications are more around having a standard set of syscalls for moving data in and out of a program, abstracting away where it actually lives and treating the data itself as something opaque.
> Not disagreeing we've moved on from it a bit in some places, but.. "Everything is a file" never meant "everything is a disk file"
You're right that it never meant 'everything is a disk file', but generally the implication of the unix philosophy was that everything could be accessed from the filesystem.
Have a look at Plan9, which is essentially directly descended from 7th edition UNIX and IIRC was built from what was going to be 8th edition UNIX. It has a HTTP interface from the filesystem -- one can literally do GET requests on documents from the internet just by opening a file on the filesystem.
Original parent commented that UNIX was dead, given that Wayland does not seem to have an accessible interface from the file system, and nor does SystemD, Pulseaudio, or any of the major new things on the desktop system, I am inclined to agree. Of course Xorg is accessible from the filesystem, but it's probably one of the last technologies to have such an interface.
On my machine systemd is configured by creating files in /etc/systemd, /lib/systemd, and potentially /run/systemd (even temporary/generated config turns into files) and then sending a message to /run/dbus/system_bus_socket. How does it work on your system?
There is a Wayland compositor called Wio that is looking for help in implementing a file system interface for window management, similar to the one in Plan 9. If you have experience writing FUSE file systems and want to help out, please contribute! Previous HN discussion here: https://news.ycombinator.com/item?id=19797152
Umm, but Wayland and PulseAudio do have accessible interfaces from the file system... Wayland even requires it, as opposed to X11 which could work over TCP socket as well. No idea about systemd though (I think it uses dbus?)
These are only sockets through, through which you have to multiplex-serialize all the commands.
In the Plan9 model, each and everything is represented by an individual file system entry. In Rio (Plan9's windowing system) every window is represented by a directory in a 9p tree, and you draw to it by opening a framebuffer file inside the window directory and manipulating that.
Furthermore, terminals know about their windows, and can hand it over to whatever is started inside them, which essentially makes it possible to open windows "inside" a shell environment.
It's kind of a unified screen/tmux environment, and you can stack, mix and match how you desire.
Sure, but that's the Plan 9 philosophy. UNIX doesn't even have /proc/$pid/fd/ - there's no way to refer to another process's stdio unless you have the fd passed to you, and stdio is kind of the basis of UNIX. So by UNIX's definition of "everything is a file," we're still there.
As mentioned earlier, Plan9 is what UNIX research edition 8 grew into (as I understand it), it's possibly one of the best examples of the UNIX philosophy.
It's important to note that UNIX the actual implementation is a moving target because of the many versions, just because the designers didn't see something could be achieved in UNIX (resources, time, effort, etc.) or just plain didn't think about it, doesn't mean that it doesn't embody the UNIX philosophy :)
> "Everything is a file" never meant "everything is a disk file" - think of std{in,out,err} or network sockets.
Yeah, and shouldn’t that (theoretically) file systems with very small delay? (For example, RAM based file systems for inter-process communication.)
I thought that it isn’t the slow storage that is blocking use of files, but more of a software problem that has significant overhead to use files as a communication method. (I’m not an expert here; please correct if I’m wrong.)
If you're sending a message to some process for it to consume, like X11 or like standard input to your shell, sure, you could technically interoperate via disk, but
- UNIX itself has no inotify/kqueue/etc. feature, blocking reads are via pipes or sockets, not regular files (plain UNIX tail -f polls the file every fraction of a second)
- you're appending to the end of the file for a message that's going to get read once, and the file model of UNIX doesn't let you truncate a file from the beginning, so it will just grow indefinitely for no reason
- one benefit of the open socket/pipe approach is you get notified if the other side exits/crashes, you can't detect that from a disk file
> I realize that the UNIX philosophy (everything is a file) is dead
Was it ever alive? As far as I know lo0, eth0 and others where never accessible as a file interface and we still rely on them. Is there something to gain to give them a name on the filesystem?
You can't read and write packets via the interface, but there are configuration knobs and statistics you can access.
I think it's quite messy and inconsistent that some network interface attributes are only in sysfs, and others are only accessible via AF_NETLINK (usually via the "ip" tool). But it's there.
The problem with file systems is that it is extremely difficult to build reliable systems on top of them. It’s shared mutable state with barely any error handling possibilities.
> Every time I see the X11 protocol, I realize that the UNIX philosophy (everything is a file) is dead.
Brings back memories of the old days of the late 1990s trying out so many workstations running obscure Linux distros to actual UNIX OSes like IRIX, Solaris and HP-UX.
They all ran some variant of X11 which was part of the UNIX philosophy which I found interesting (X11 forwarding, startx, etc) but as the requirements of the desktop kept evolving over time, X11 just kept on showing its age.
Nowadays, Wayland is showing promise as a successor but I really don't have time to be messing around with these .xinitrc, .Xresources or dotfiles to get HDPI working on my MacBook these days.
The uncomfortable truth today is that, while having to touch an X config file hasn't been a thing in more than 10 years, it will take you some fiddling with your favourite Wayland compositor (of which only three -- two massive DEs and a tiling compositor -- are anywhere near being useful) just to get it to stop crashing on that one application that you need, or to get things like multiple monitors or middle-click copying and pasting between two particularly misbehaved applications to work. Ironically enough, the least pretentious of them, sway, is by far the most solid.
It's basically all the fun we had with X11 back in 2003, just to get an environment that's about as dependable (though admittedly much faster) as X11 back in 1993.
It's one of those cases where the "release early, release often" mantra kindda seems to work against us. I'm sure ten years from now Wayland will be just as solid as X11 was back in 2013, aka "good enough", but there's a different kind of crowd looking at this stuff today.
Unfortunately, the anti-Wayland buzz propagates easily because the community response is inevitably unsympathetic. Mention things like "I tried Wayland for an afternoon, applications kept crashing" and you'll inevitably get a dozen answers, all of them along the lines of nah bro I'm running Gnome Wayland which is the default in Fedora 67 on my ThinkPad X1 Carbon and I'm not getting any crashes. Well, yeah, 90% of the projects that failed to get traction and adoption in the field of graphical systems did so because they couldn't make the transition from "running on its developers' and fans' computers" to "running on everyone's computer". I can run Wayland on my system just fine, too, but I'm the only one in my immediate circle of nerds who can.
Yeah, I also should have said that the response of the other part of the community isn't exactly sympathetic (or useful), either.
Truth is, a lot of the reason why X11 works so well today is that a bunch of us put up with a lot of X11 crap back in the day. E.g. things like xdotool haven't existed forever, either, and it took a lot of itches (and imperfect way of scratching them) before someone wrote that. X11 wasn't exactly hassle-free for a very long time. And you get to fiddle with config files with Wayland, too, but none of these files are as long as a small section in XF86Config.
That's how free software matures -- in the open. If you want to contribute towards hiring a team of testers, you're welcome, because no one does that as a hobby. And all this is in a context were fewer and fewer developers are interested in desktop development, and fewer and fewer companies are interested in investing in desktop development.
> And all this is in a context were fewer and fewer developers are interested in desktop development, and fewer and fewer companies are interested in investing in desktop development.
Which could make X11 the last linux/unix desktop thing. Anything new would have to come with the whole desktop paradigm shift.
How do you think it should have evolved? I personaly think posix way of filesystem needs to be changed into specialized oject filesystems, but I am not sure what object filesystem for a UNIX OS should look like specially since it needs to be generalized into something all the other sub-systems can benefit from.
Personally I also preferred the ‘Everything is an object’ from the lisp machines.
However, I also believe that files are an acceptable object format, since if several applications know how to manipulate a file format, it’s basically duck typing.
I believe file systems could have a more formal method to query the file’s format, and has less latency so that applications could use files as an inter communication format.
> Every time I see the X11 protocol, I realize that the UNIX philosophy (everything is a file) is dead.
As "dead" as Android (which is based on Linux, a Unix clone) and OSX (a BSD Unix) and iOS (child of OSX)? Why does Microsoft vibrantly support Linux in Windows 10 (an abomination to Ex-CEO Steve Ballmer) if Unix is "dead"?
I think the very opposite is true. Some developers realized that the KISS principle (including the Unix philosophy where everything is a stream of bytes) is not a bad idea after all.
When I started this, I was working on GNOME's window manager full-time, and wanted to learn intricately how X11's drawing model worked, so over the course of a few weeks in a hotel room, I recreated large parts of X11's drawing model in a web browser, fixing artifacts as I went along, until I feel I had a really good grasp of it. My initial test scene was a traditional desktop-like approach with a taskbar and xeyes, both of which are still in the codebase today, but untested [0].
I didn't know what I wanted to do with it, until I settled upon using snippets of it to build a long-form article. I learned a lot about the difficulty of writing, of pedagogy, of that blurred line between being technically correct 100% of the time vs. telling a few small lies here and there to keep the flow consistent and help people see the broader picture.
At my day job, I had mostly moved onto Wayland, where some of the bits I picked up here really helped me design better protocols and systems. My goal with the series was to try to be as neutral as possible, and my original design was to have a giant caution sign around "Author Opinion Zones" where I would talk about how certain design features haven't held up well in practice. But quickly, people on Hacker News or Phoronix or Reddit seemed to skim the article, pick up a piece here or there, and go on straight to bashing Wayland, gleefully unaware that I was one of the people making it.
So, the end result was that I basically stopped working on Xplain basically after the second article. The COMPOSITE article was one I made after a colleague was having trouble understanding COMPOSITE, and I figured it was easier to write with my framework than explain in a chatroom, and maybe some others would appreciate it.
I have a deep passion for sharing my knowledge, and Xplain was the format I first really used to do it widely, so I tried to keep it exciting for me by changing it from "Xplain" to "Explanations", and opening up the topics from X11 to just about anything, but at some point I was just unhappy working on it.
The last thing I was working on was a continuation of my Basic 2D rasterization article, where I had a fun code editor you could use to make your own graphics [1] [2], but as fun as the technology was, I couldn't find a satisfying flow to the article, so I stopped it. Parts of it were later recycled for an article on the histories of 2D and 3D graphics. [3]
Around mid-2016, I had stopped working on Linux and open-source graphics entirely, and by 2017 I had exited the open-source industry completely and jumped ship to professional game development. I still have a deep love for graphics and a passion to explain things. I just released a new side project a few days ago for it, even.
Here's some stuff I'm working on these days. It's much cooler than X11/Wayland flamewars, in my opinion.
https://noclip.website/
https://blog.mecheye.net/2018/03/deconstructing-the-water-ef...
https://www.youtube.com/watch?v=8rCRsOLiO7k
--
[0] https://github.com/magcius/xplain/blob/gh-pages/src/clients/...
[1] https://u.teknik.io/XdSbC.webm
[2] Sort of up here at https://magcius.github.io/xplain/article/rast2.html
[3] https://blog.mecheye.net/2019/05/why-is-2d-graphics-is-harde...