How come this is hard to do with 100 participants? Not a crypto or video encoding expert, but I'd have thought each participant would have a decryption key for their stream, and this key they would transfer to others by encrypting it separately with each other participant's public key and then broadcast that bundle to everyone. Thus everyone else can decrypt only their entry in that bundle using their private key, and gain access to the shared decryption key for that individual stream. When a new person joins, they add their public key to the central list of public keys, and request that everyone update their bundle to include an entry encrypted with their public key. So basically, from the perspective of the new joiner, each stream would be encrypted until that streamer had updated their bundle to allow the newcomer access to their decryption key. But each streamer would still only need to encrypt their stream once, and then share access to the decryption key only to participants. Scaling wise, sending only incurs the cost of encrypting once - though as a receiver, you'd need to decrypt N streams using N different keys - but is that much harder to do than decoding N video streams in the first place?
If you want a HD stream for someone who is talking, and an SD stream for each person in grid view and users on slow connections, you either need access to the unencrypted HD so you can downsample them, or all platforms to support Scalable Video Coding [1] so you can downsample by dropping otherwise-opaque encrypted packets.
Consider resolution. With non encrypted video, you receive one hd stream. With 100 people, you'd need 100x the bandwidth, and supporting things like highlighting speaking people would need to be done client side.
I honestly thought that was the case already (that highlighting is happening client side, and that my client receives each stream separately). So what you're saying is that currently a central server is performing the task of merging those 100 streams into a single stream for users. So it needs to do this separately for each user configuration of how they want to see the streams?
1. Basically, in a peer to peer topology you’d have 100x99 streams going. Everyone sending to everyone.
2. With a star topology and a central SFU you have 99x2 connections only. Much less bandwidth being used. The economics is that everyone pays the central server.
3. Now, I would wager there is a way to do some sort of sparser version of option 1 with beefy peers playing the part of “supernodes”. I think that’s how skype used to work before M$ bought them and made it less P2P
But regarding the encryption, yes you can have an SFU or whatever but you won’t have adaptive resolution of the SFU or supernodes can’t decrypt your stream in transit.
Yes, large videoconference tools like zoom meet etc. all use a central server to do intermediate processing. One on one (or small group) apps are often p2p, and can therefore be e2e encrypted.
Currently it encrypts the stream using TLS, just swap out that encryption key with one only known to the participants. Generate and share a symmetric key for the call (under the hood, of course; users should never have to deal with this unless they actively seek out the key verification feature) and things would work exactly the same from there, the server just needs to forward traffic.
It might currently not work in a browser because native TLS is way faster than doing the crypto in JavaScript, but in principle it should be equally heavy whether you make it end-to-end-encryption or encrypt-to-server.
