I didn't see Xpra mentioned, but Xpra does desktop and window streaming with fast video encoding using H.265 (or H.264 and others options), and it has an experimental UDP transport - though I haven't needed it yet. In the Windows world, gamers use Parsec which matches those two network transport and encoding design decisions (which I am sure are the right direction).
To use Xpra as part of a secondary monitor solution, you might need to define a "dummy" window region for it, and I suspect X server can be configured to allow this but I don't know how yet.
I'm interested in any solutions for Linux secondary monitor devices too, especially more simple ones.
Edit: kbumsik link below uses xrandr to add a new display region and then uses x11vnc with a "clip" argument. I couldn't see such an argument for xrandr.
This would work nicely on Pinephone, because it has very efficient accelerated decoding of H.264 codec. gstreamer using kmssink consumes very little power decoding H.264 video. It would likely run at ~2W with display on, so for about 5 hours on battery (at reasonable screen brightness)
That low latency is an impressive achievement, makes me want to attempt to convert all my old tablets to a real Linux install (no chroot jails etc), but the risk of bricking them is really high. I wish there was some tool to probe the hardware under Android, possibly against a distributed database, to know at least if the device is similar to a more known one (it could be the case among no name cheap Chinese tablets) and if there are chances of success before a blind attempt.
Most SOCs have a mode you can put them into with just the hardware switches that runs firmware in the mask ROM which waits for a bootloader over USB. It's pretty hard to actually brick modern devices.
Yeah, but most non-brand manufacturers don't release the tools, driver packages or the full firmware images at all - meaning you have to rely on questionable third party sites, hackers and leakers to obtain them to recover your device. I don't even want to know how many people ended up with malware from shoddy SEO-spam domains advertising Samsung's ODIN tool.
If Google had forced everyone to use the aboot boot loader and to allow rooting, we wouldn't be in this damn mess of people using all kinds of highly questionable stuff.
I get 100-200ms including network latency to a faraway server that has no hardware for encoding or decoding video via webrtc in a browser with vp8 without even trying. Anyone who can freely choose their encoder and decoder including hardware acceleration should be well within 50ms on a local network.
In fact, the biggest problem for me isn't the lack of HW encoding, it is the lack of HW decoding because it makes 60fps videos on YouTube unplayable.
I think it'd be cool if you could use a cell phone as a mouse. You'd have a mount that holds the back of the phone slightly off the table, and the phone would track its position using the IMU and maybe motion tracking with the camera. The phone display could show a subregion of the monitor and you could click by tapping the phone screen.
Forget mouse, I want to use a phone as a computer. I think the only hardware you can buy for less than a thousand dollars with arm SVE support is a phone, would be nice to just boot into debian seamlessly and run a test suite.
What about standard, probably proprietary, solutions, like Miracast ("Wireless Display")? I wanted to use my old tablet with Windows as secondary monitor, but no luck when running Linux on primary PC. Best option is only mirroring desktop with GNOME Network Displays.
It would be great if there would be out of the box support for wireless displays on Linux.
Im surprised Figma doesn't allw for this... only because their app dev kit with phone mirroring desktop is pretty sweet... I could see how it coud be adapted for this purpose
If you haven't already checked it out, I'd suggest investigating use of `evdi` ("Extensible Virtual Display Interface") for the virtual display aspect of this:
> Wish that I could use my iPad as a second screen.
No wishing necessary. Before apple sherlocked the idea with sidecar there have been multiple apps that let you do that and are (somewhat) cross-platform (Luna display, duet, ...). These apps are still around and much better than the apple one for specialized use cases like art, they cost money though.
Or, if you don't need high speed, just go the simple vnc route, e.g.
There's definitely ways you can do that. Look into programs that use HDMI dummy plugs.
It's a little stub that plugs into an HDMI port trick the OS into rendering a separate screen. Then there are a lot of apps that let you display/interact with that virtual screen, usually through some kind of browser. Acts exactly like any monitor you plug in, regardless of OS.
It seems silly to need hardware for this, but it's a very clean/stable way to do this without mucking with config files or using weird hacks.
I use Deskreen to attach an Android tablet to my PC. All it requires is (I think) a modern browser for the mobile device (and you currently need an hdmi dummy plug on the PC).
I like the UDP approach. I tried doing something similar to multicast video streams to Raspberry Pi 1s quite a while ago (back when it was impossible to run a modern browser on them and we wanted live dashboards across the building), and it can be very efficient.
You can also multi-boot using kexec (using Linux to boot other Linux image) or just by booting into initramfs that has a GUI program for selecting how to continue boot (which root filesystem to start from), etc.
That's super easy already. You can make the phone act as a USB sound card, or you can use PulseAudio's network transparency (see https://puri.sm/posts/speak-to-me/), or you can broadcast an RTP stream, or you can use Bluetooth audio...
No. It's not. Fast video playback and low latency was a requirement, and you can't do that over the wire with X11 or VNC. Pipewire is the solution for this.
This is why SirCmpwn was right, and the anti-Wayland crowd needs to just shut up. Wayland+Pipewire is the future of remote desktop. Together they solve issues X11 by its nature cannot solve.
Maybe there wouldn't be a substantial antiwayland crowd if wayland proponents hadn't started saying that when wayland was an impractical mess years from being ready back in 2015-2017
Offering a superset of functionality in a prominent option without major downsides might have worked too.
At least it would have worked better than offering limitations and telling people to shut up and deal with it.
The Wayland architecture was decided on because it offers performance and security in a way that mirrors how the graphics stack actually works. So says everyone with deep subject matter knowledge about the graphics stack.
The Wayland debate is like the global-warming debate: on one side are the experts, and on the other side are grognards and reactionaries.
> Analogies are false understandings. They confuse patterns for truth, but the evidence for these patterns are often based on further analogies or tenuous connections. The human mind looks for patterns, and isn’t above creating the patterns that it wants to see.
Its most often true that what is different between the analogy and the real is more important than what is the same and that deconstructing the simulacrum and the thing is more complicated and convoluted than simply speaking plain words about boring facts.
For example the global warming debate at it's simplest is two groups arguing for binary and mutually exclusive choices. Either the earth is getting warmer or it's not. Either its caused by our actions or its not and we all live in the same environment so if I'm wrong you are right or vice versa.
Meanwhile Wayland and X are only mutually exclusive in the same session you can in fact log out of one and into the other between one comment and the other if you please. Or you can use Wayland and I can use X to have this very conservation which is in fact very likely what is happening right now. The same way I can use Firefox and you can use Chromium if you prefer.
Both Wayland and X offer an abstraction on how the graphics stack actually works. An abstraction no users should give two shits about because nobody on earth cares about plumbing until the water doesn't come out when you turn the knob. A user cares about the system as a level of abstraction where they click a button and a window appears. What they don't want to do is change a damn thing about how they use their computer or accept even the slightest decrease in functionality in order to personally accommodate a difference in plumbing while opting out is entirely trivial.
To ignore "advances" in plumbing that don't yield advances in practice doesn't make one a reactionary or a grognard it simple pragmatism. When was the last time you saw someone say
"Let me buy this ugly car that doesn't turn so well because the way the engine is laid out really sparks joy!"
It is the conceit of someone who really does care about how the engine is laid out to suppose that people ought to care about that rather than how it is actually experienced by the user.
For example Ubuntu didn't get a version that could actually accelerate xwayland windows under Nvidia until last month and it still suffers from bugs not apparent under X. Furthermore any setup that requires fractional scaling or mixed DPI will result in a xwayland apps looking like a blurry mess so you either need to stay away from such configurations that work entirely fine under X or exclude any apps that aren't wayland native from your computer.
Some of us would prefer to simply using our computers instead of beta testing the future and putting up with limitations for someone else's sense of aesthetics.
Posted from Firefox running on X11 with 3 monitors with 3 different DPIs attached to an Nvidia GPU wherein all outputs are perfectly scaled to be identical and everything is crisp and clear.
I think telling people to shut up is a fast way to shut down valuable discussion. We can have good discussions about the value of particular technologies without being uncivil. Having strong opinions makes it easy to be unkind (especially online). I admit I also struggle with this. I think everyone could dial it down a bit from time to time.
The X protocol is extremely chatty, does not compress images, and uses TCP. It's virtually impossible to make any sort of latency guarantees with that sort of protocol. Pipewire can use several compressed formats and the OP used UDP to stream the media off Pipewire to the remote display.
Yeah, I know running X11 over the wire doesn't hold up, but that's not what's being done here with Wayland either; since we're locally grabbing pixels off the virtual display and then flinging them over the wire via custom UDP protocol, X11 vs pipewire only matters for the initial handover.
The compositor may be able to transfer image data to PipeWire in zero-copy fashion, which may then be consumed by hardware video encoder without any copy on the path. I'm not sure whether you'd be able to do that using Xorg as display server.
Actually it is possible to do low latency while using X server (for those not on Wayland), but it seems much faster to stream a video of the screen area than to use X protocol directly (which I have often used over SSH in the past to control another desktop system) - Xpra seems much faster to me. I've tried x11vnc too. Xpra is cool. Just my subjective experience, but proper benchmark stats would be cool to see.
To use Xpra as part of a secondary monitor solution, you might need to define a "dummy" window region for it, and I suspect X server can be configured to allow this but I don't know how yet.
I'm interested in any solutions for Linux secondary monitor devices too, especially more simple ones.
Edit: kbumsik link below uses xrandr to add a new display region and then uses x11vnc with a "clip" argument. I couldn't see such an argument for xrandr.