From my read, it counters Bryan Cantrill's claim that, "unikernels are undebuggable".
From personal experience I'm also quite certain Bryan Cantrill's claim is spurious in that regard, as I've used both debugging and tracing facilities w/ LING unikernels to assess a number of runtime and clustering issues.
Yup, If you're running unikernels in a VM for "improved performance" you're doing it wrong. Containers are a much better solution. Unfortunately people see containers now as a packaging mechanism rather than an alternative to hardware virtualization. Also doesn't help that everyone runs their applications in The Cloud where you're required to run on a VM. I'd really like to see containers services like Joyent's take off.
>I'd really like to see containers services like Joyent's take off.
Interesting. I'd like to see a return to owning/renting whole machines and running containers on them directly (as several major tech companies do). A lot of the value proposition for the cloud service providers is a multi-host hypervisor that's really easy to get started with (compared to buying vSphere). If it becomes really easy to deploy your own instance of an open-source container scheduler across your own boxes, we can return to cost-competitive commodity boxes instead of overpaying for AWS.
Agreed, private cloud systems that leverage OS level virtualization like Joyent's solution is completely production ready from a software perspective. IMO, the biggest challenge past getting CTO approval is finding production 24/7 support for the complete stack.
For SMB private clouds, its a no-brainer; I just buy a FreeNAS or TrueNAS box (FreeBSD based) from iXSystems and have them customize the box for extra ram and CPU. You get ZFS, DTrace and a wicked NAS, OS level virtualization with jails and even linux binary support if required. I've done this successfully in production and could not be happier.
Note: I don't work for iXSystems but I do love their products and services.
I'd really like to see containers services like Joyent's take off.
Same here, especially since the zones that the Joyent containers are built on are really easy to use and provide entire miniature yet full fledged UNIX servers, not to mention it’s all open source and gratis and the community is really competent.
Apart from disassembling the machine code, assuming one could even attach a debugger to such an application, how would you debug a unikernel application in production?
Just like debugging Java and .NET applications running bare metal on production.
By having a rich runtime (kernel) that exposes the internals to the world, like Mission Control, TraceViewer and the respective debuggers, when the right set of flags/authentication are enabled.
Unikernel are no different from running embedded applications, bare metal.
I know they are no different since they are one and the same. Now imagine your unikernel application is running inside of a vehicle and there is a bug in the head-up display code. Without re-inventing DTrace and kdb from Solaris / illumos from scratch, how would you debug your unikernel application in order to find and even fix the bug? (With kdb, it can be done on the fly.)
Inside of a vehicle, not yet; production example, yes.
Back in the day, there was a kernel bug which prevented Solaris 10 installer from booting on systems with a Pentium III processor. The workaround was to boot the kernel with -k, let it crash and drop into kdb, then patch a certain kernel structure and drop out of the debugger, continuing execution. The same mechanism would work in a car or a Boeing 787, which I understand were actually running Solaris as the control & navigation system.
How are theoretical applications which are designed to do just one specific thing a threat to a fully featured hypervisor with 15000 packages used in production?
This is quite interesting, are there major performance benefits to using something like AtmanOS? My main concern is if something like this will ever be able to run bare metal, and cut out the overhead of a hypervisor, if performance is critical, then a dedicated box is likely.
Then again, this brings us to the same issues that the *BSD and Darwin kernels have, with really lackluster driver availability. While some big corps like Sony might get AMD to build them a performant GPU driver, the tragedy of the commons situation repeatedly occurs with BSD licensed projects, major improvements aren't contributed upstream reliably as there is no business reason.
That is why you use Xen as a base. Xen lets the dom0 (primary domain) handle the device drivers and then provides guests access to them via paravirtualised interfaces. I.e you get Linux compatibility and then you can run any OS that has paravirtualised drivers written for it.
Generally if you are going to use something like this or MirageOS you will choose to pass through certain real hardware to the guest and write drivers for it.
Say for instance pass through an Intel NIC and then have your application embed a TCP stack and DPDK like components so that you can run your application at line rate.
Not trying to be stupid, but I'm not really clear on what the advantage is of this system. Do I understand correctly that the dom0 OS is still providing drivers and hardware abstraction? So both that and some userland exists somewhere in the stack; it's just not duplicated in the virtualized OSes?
Is the main goal simplicity, performance, or something else?
Userland isn't duplicated, it's 2 totally separated userlands. The userlands may be the same distro in case of Xen (for example RHEL 7) but they still lead separate lives; They are different installations. So that's also multiple userlands in which you have to for example deploy patches (RPM's, Deb's, etc.) for your SSH install.
And yes, I see some value in the reduction of maintenance costs, but you're not doing this manually--you have a package manager of some sort and you're automatically tracking some standard image (either for your company or from some upstream maintainer like Canonical). So conceptually I get the simplicity argument, but practically speaking, it's not really more work to maintain two userlands vs one, right?
I guess there's also an argument of resource (disk, memory footprint, etc) overhead of the second userland. It's not clear to me how significant that is, which was part of my question.
Squawk[0] ran pretty much directly on the hardware, Sun had a JVM-on-Xen project, OSv provides a lightweight pseudo-OS (not unlike rump kernels it looks) which support JVMs. Possibly one step down, Oracle has a JRockit VE edition which runs directly on the Oracle Hypervisor, but despite being Xen-based it's unclear whether that can run on raw hardware.
Not really. Now instead of having to break the application, then break the kernel it's running under and then attack the VM host, you only have to attack the app and can then go directly at the VM host.
Unikernels just remove a whole security layer. May as well run the app as a user process on the host and forget the VM.
It depends on what you're securing and what the weakest link in your chain is. The assumption is that your app is doing something valuable (i.e. there is value in compromising the app, not just as a means of compromising the host).
Perhaps I should have phrased that differently, but you've reduced the attack surface for compromising your _app_ via the environment (instead of the environment via your app).
(big disclaimer: this is assuming Xen bugs are much more valuable than app bugs and someone with a Xen exploit won't be focusing on you.)
unikernels do not have to run in ring 0 although a lot of early ones did. You can run them in ring 2, and they probably never need ring 0 access after boot so there does not have to be any way to return.
Wrong. Cracking the app does not give you access to the VM host. These apps are running in a VM. There's no relationship at all between cracking the app and cracking VM host.
I believe what zlynx meant was that if you want to escalate from the VM to the host, you usually do that via the virtualised devices. On a normal system, your path (for a web service on Linux) would be:
App exploit (-> LSM breakout?) -> local to root escalation -> VM to host escalation via device.
Maybe I'm mistaken but it seems to me that the parent's argument was that running your app in a unikernal prevents another app from crapping all over it, ie: root escalation happened in an adjacent app and now your app is at risk.
It does move the hypervisor up a few levels of abstraction, which could be dangerous, but (more to the point) the benefit is isolation from other misbehaving apps.
Somewhat related: for anyone who is interested in unikernel development, there is a new proposed project called Unicore under the Xen Project umbrella.
That project aims to reduce the effort for porting applications to run in unikernels on different platforms (Xen, KVM and baremetal).
Which is to say, DIY symmetric multiprocessing, or BYO kernel code in-application.
I'd have to read a compelling technical explanation before believing this could perform better than a Linux or BSD kernel.
In most cases, the Go code is going to be single CPU, and that ain't the way the world works anymore. There's going to be a bunch of wasted computing power on that VM
Where people seem to really care about perf is networking. What that means in non-crazy land is you cut out the kernel from that part, do networking in userspace, and pin a thread to a CPU to poll for packets. Like it's all MMIO and DMA anyway so ring 0 doesn't have to get involved if it doesn't want to. Then for other stuff you have convenient slow OS facilities. Including threads, processes, and disk I/O! Luxurious.
So it's not just that this has to perform better than a kernel, it has to perform better than not involving the kernel in the critical path, and it has to perform so much better that it's worth wanting to beat yourself in the face with a hammer after trying to diagnose the latest lockup. Unikernels are interesting to me, but like the demoscene or a semi tractor doing wheelies. Not for production usage.
You can bind processing for a consistent subset of requests to an individual cpu core - essentially sharding requests across cpu cores and benefitting from very high l1 and l2 cache utilization. The idea is to treat the system with multiple cores as bunch of single cpu nodes connected via bus instead of network and without the unnecessary overhead of thread and process related context swtiching.
Instead of sharing the code at runtime, i.e. what an OS does. You could easily share code at compile time, i.e. statically link a library.
Because of sharable code, "implementing * in application" should always be at-least as performant as the best generic implementation (i.e. the implementation you find in a general purpose OS). However, when appropriate, customizing the implementation for the application would allow it to become even more performant.
From my read, the benefits do not outweigh the costs. If you want light weight microservices, OS level virtualization is the way to go.