Here we are again, locking useful features behind HTTPS :)
Background: I have been working on a software that creates and streams extra virtual displays via the local network (like Duet Display or Apple's sidecar), and the easiest way to get started for everyone is to stream to a web browser.
WebRTC is kinda easy to implement thanks to webrtc-rs, but has unacceptable latency as I do not have control over whether or when exactly a frame is rendered (I need to be able to drop outdated frames). This API has the potential to do exactly what I want, but:
* It is not possible to connect to a local server if the viewer page is served via HTTPS, as the local server won't have HTTPS.
* It is possible to generate CAs and certificates locally and instruct users to install them, but I don't think that anyone out there will like this solution.
* We have WebTransport that might be able to do this, but it is an overly complex technology that uses HTTP/3 (plus no support on anything but Chromium), and tools like localtls[1] that requires internet connection and my own domain (I would really want this tool to be able to run completely offline).
Modern web APIs are kinda useless when you don't have internet, but the ship has sailed.
I was able to achieve a latency of ~30ms with a custom Android client by dropping outdated frames, but I could not find out how to do this with WebRTC, since its buffering and rendering is entirely controlled by the browser with almost no tuning knobs.
WebRTC as a technology is also quite complex, providing less flexibility than a custom TCP or WebSocket stream. For example, to further reduce latency, it is possible to tunnel TCP traffic through USB with the help of Android's debugging interface, but this interface does not allow tunneling of UDP, nor does webrtc-rs allows listening on 127.0.0.1 (it is hardcoded to ignore the loopback interface).
Insertable streams looks interesting, but it seems that Firefox and Safari (especially on iOS, where I can not easily relese a native client) does not support it at all, while WebCodecs is at least experimental.
Does TCP actually works on webrtc-rs and Chrome? ADB does not support UDP at all. If it works, then having the ability to gather loopback interface is certainly a plus for my particular use case.
WebRTC supports TCP candidates, so webrtc-rs can do the same. If that unblocks you happy to implement. Getting people with interested use cases involved in the project helps a lot.
Background: I have been working on a software that creates and streams extra virtual displays via the local network (like Duet Display or Apple's sidecar), and the easiest way to get started for everyone is to stream to a web browser.
WebRTC is kinda easy to implement thanks to webrtc-rs, but has unacceptable latency as I do not have control over whether or when exactly a frame is rendered (I need to be able to drop outdated frames). This API has the potential to do exactly what I want, but:
* It is not possible to connect to a local server if the viewer page is served via HTTPS, as the local server won't have HTTPS.
* It is possible to generate CAs and certificates locally and instruct users to install them, but I don't think that anyone out there will like this solution.
* We have WebTransport that might be able to do this, but it is an overly complex technology that uses HTTP/3 (plus no support on anything but Chromium), and tools like localtls[1] that requires internet connection and my own domain (I would really want this tool to be able to run completely offline).
Modern web APIs are kinda useless when you don't have internet, but the ship has sailed.
[1] https://github.com/Corollarium/localtls