QEMU is absolutely incredible. I think its most undersold feature is the fact that it runs on Windows hosts[0] thanks to MinGW. This in itself isn't unique, but what is unique is that by default it runs entirely without admin privileges. I've been playing with shipping server apps as QEMU VMs and it works surprisingly well. QEMU itself is small enough (~20MB) to ship the entire thing with the app, including DLLs.
With -accel whpx you can even get pretty solid CPU acceleration. WHPX[1] is sort of like KVM for Windows, only takes a couple clicks and a reboot to enable, and ships on both Windows Home and Pro. Other than the heroes who did the actual work, we probably have Android to thank[2] for this situation.
If you're willing to get your hands a little dirty, QEMU is a fairly viable alternative to WSL or even Docker on Windows.
QEMU and FFmpeg are sort of in this category of tooling that are incredibly powerful. The amount of things you can do with it, the technical achievement, is incredible.
And the only thing holding them back is that you need 3 pages of command-line arguments to invoke them properly :-)
(EDIT: I was just reminded that both were started by the same author: Fabrice Ballad)
Yes, we were very excited to get WHP (Windows Hypervisor Platform) support integrated. Before that, you could only use it with the HAXM hypervisor on Windows, which conflicts with the Windows hypervisor (used by Hyper-V, WSL2, and the Windows Subsystem for Android).
It's huge honestly. Do you think there's any chance of Microsoft enabling WHP by default? I think using microVMs for sandboxing apps could be amazing for securing user applications, but currently you still have to instruct users how to turn it on and reboot.
Also, how active is the WHP work in QEMU? Is it mostly done by the QEMU folks or inside MS? Is anyone other than Android actively banging on it?
Sorry for the deluge of questions but I think there's a ton of untapped potential in this tech.
I didn't see anyone mentioning about GPU acceleration for 3D graphics so just in case for those who are not aware, Virtio-GPU, a virtual GPU implemented in Qemu for Linux and classic Windows OS guest machines, which works under Windows host. Just like VMWare.
You can watch some demo videos from the YouTube channel "KJ Liew",
Enabling Hyper-V on Windows currently causes problems with VirtualBox, so that’s one reason. They’re (vbox) working on fixing it, but it hasn’t landed yet.
Enabling Windows Hypervisor Platform doesn't require enabling Hyper-V. Hyper-V doesn't even exist on Windows Home, but WHP does. That said, I have no idea whether WHP interferes with VirtualBox.
Since QEMU devs/maintainers are lurking, i just want to say thank you for a fantastic project.
I work for a company (Datto) that backs up Windows/Linux machines, which can then later be run as VMs for disaster recovery. Our product heavily relies on QEMU, and would be significantly harder to implement and maintain without it (believe me, it was built around VBox in the past). Thank you!
I work for an MSP that uses Datto to back up lots of servers and saved our bacon after the Kaseya hack! (Ironic how they're buying you now.) And I remember when you were using VBox, QEMU is much improved. Having QEMU boot up the backed-up VM to a screenshot that you can just email to us really helps prove to auditors that yes, our backups are tested! Now just replace all the spinning hard disks in your data centers with NVMe so we can get decent performance. :)
That is a fantastic idea. We are a Silver sponsor for Let's Encrypt, so this is not entirely impossible that we'd donate to QEMU as well :-D I'll bring it up to the people who can make these calls.
How do you handle the missing drivers when it comes to running under the VM? Do you modify the backups, or do you opt for emulation with good old standard hardware instead?
Windows is.. Windows. Always pain with drivers there. Linux should be the easiest story, but since RHEL7 (and the derivatives from there), dracut has defaulted to "host-only" mode which leaves the initramfs so skinny and devoid of drivers not related to the host.
We have a process called Hardware Independent Restore (HIR), which injects drivers into Windows before booting it in the virtual environment. If you Google "datto HIR" you'll find some resources (e.g. https://help.datto.com/s/article/KB217259206), but it's obviously a lot of secret sauce, so I can't talk about the details.
While this is true, please don't let it overshadow the tremendous work (possibly even the bulk of it by now) by subsequent maintainers. In the ffmpeg case that involved fighting a well-resourced hostile fork over a period of years, in the Qemu case I believe it means at least 10 years of additional substantial feature development.
I wonder what Bellard's overall hit rate across all those projects looks like in terms of staying around for chores after delivering the initial act of wizardry
I agree with you, but I also think it's pretty impressive that Fabrice has started multiple projects that many other people have devoted their entire careers to expanding/maintaining. Almost seems to challenge the common assertion that execution is more important than a great idea.
These weren't un-executed ideas, though. I think it lines up very well with my general sense that the best way to get to a really good implementation of something is to publish one that's lacking in some way that annoys people. Then you have an idea plus something people can contribute work to improve.
Not intentionally… I am arguing that “Almost seems to challenge the common assertion that execution is more important than a great idea” is not the case. Because these ideas were executed.
One that have the same binary name as yours for compability reasons, but then proceeds to be not so compatible with slight command line options. Thus, made a whole lot of confusion because people didn't expect that depending on what distro they were using they could be using similar but incompatible software with the same executable filename.
First off I want to make it clear that this isn't intended as criticism. I recognize the QEMU devs are busy, do incredible work, and don't owe me anything.
That said, I started a project recently that I'm very excited about. It wouldn't be possible without QEMU. I've had to get deeper into somewhat lower-level features, and it's likely I'll have to implement some new devices and interfaces for QEMU itself eventually.
Honestly I'm struggling to get my foot in the door. My questions on the user mailing list go mostly unanswered. I'm not aware of any QEMU book. Documentation feels sparse and is fragmented across multiple locations and often requires hoping someone has written a blog post about the specific piece you're interested in. I wish there were more descriptions of architecture and design decisions.
I hesitate to use the development mailing list since it appears dominated by patches (a ton of them; QEMU is very active), and my requests don't quite seem to be development-level (yet).
Any suggestions for how to get help learning QEMU as a "power user" hoping to eventually contribute as a dev? I'd happily pay for consulting if I knew the right people to ask.
The -devel list is dominated by patches just because that's how our workflow works; but if you have a technically focused question or one that's about the internals or the source code then you should feel free to ask it there (though of course I can't promise anybody will answer it...) The user list is lower traffic and tends to be more "users helping other users with straightforward queries"; not all the developers read it.
The documentation is indeed not great, and documentation of the internals is worse. I think this is a mix of people often not having the time or priority to write documentation, being on the inside and not seeing the gaps in the docs that are obvious from outside, and sometimes the docs not having a structure that gives a simple place to add the needed new information. Plus as a low-level tool we assume to some extent that most users will be using a higher-level management app like libvirt which has hopefully better UX and docs. I don't have any bright ideas here, so if anybody does I'd be interested in them. Bug reports of the form "functionality X is undocumented" would be useful too I think.
> The -devel list is dominated by patches just because that's how our workflow works
I guess the community likes it this way, but it's my least favorite part of working with upstream QEMU. I think partitioning patches and discussion into separate lists would be a (IMO) small impact but would significantly facilitate communication.
> Any suggestions for how to get help learning QEMU as a "power user" hoping to eventually contribute as a dev? I'd happily pay for consulting if I knew the right people to ask.
I'm not being cheeky: read the source.
I worked with qemu for years, and the only way I could get real answers was to find the libvirt functionality and work backwards from it. Once I got good enough with that, I could navigate the qemu codebase on my own and just read.
Almost all the docs are going into libvirt, and libvirt's (IMHO) terrible abstractions. The reality is that almost all the investment in qemu's user-facing capabilities are done via libvirt, so it makes sense.
> find the libvirt functionality and work backwards from it
This is a great suggestion, thanks.
> Almost all the docs are going into libvirt, and libvirt's (IMHO) terrible abstractions. The reality is that almost all the investment in qemu's user-facing capabilities are done via libvirt, so it makes sense.
Yeah it's unfortunate that not many people seem to use QEMU directly. I can't use libvirt for my project because a) I'm targeting windows and b) libvirt seems to assume that it will run with root privileges, and I'm targeting userspace.
Honestly other than a few things like CPU pinning, I haven't seen much to justify libvirt over plain QEMU. This would be especially true if the QEMU CLI was simplified slightly and better documented.
CPU pinning is something I'd really like to see in plain QEMU as well. Another is some equivalent to the pre- and post- hooks that libvirt supports, though I can get around this with a bash script.
Yah I had some similar constraints, also I just didn't want to bring the entire mess of libvirt into my project's life.
> This would be especially true if the QEMU CLI was simplified slightly and better documented.
I think this is the biggest issue. I can't complain because I haven't helped fix it, but the command line flag explosion, coupled with the qemu -readconfig/-writeconfig sucking is a big reason people just check out mentally and write big XML descriptions of their VMs.
QEMU releases are calendar-based, every four months, so "7.0" just means "first release of 2022", not "we got lots of new and shiny stuff so we bumped the major version" :-) More generally, what goes on in QEMU these days is mostly continued progression of existing work rather than sudden new developments: it's an established project making incremental improvements and following along with updates to hardware and other standards.
It was simpler to just move from "major.minor and we bump the major infrequently and semi-arbitrarily when the minor is big enough to feel unwieldy" to "bump major for first release of the year" rather than completely change the version number format.
Merely for your consideration, I believe it'd be possible to match both goals by setting next year's version to 2023.0.0
I also didn't know about the leading number being a yearly increment and would have thought 7.0.0 was a substantial change over 6.0.0; setting the number to 2023.0.0 makes that distinction a lot more obvious
is not any better, I think that starting from 2018 the number means (roughly) the year:
3=2018 (or also early 2019)
4=2019
5=2020
6=2021
7=2022
but I can see no evidence of a 4 month fixed interval release.
As the person who tagged v7.0.0 yesterday, I do in fact know our release cadence :-) As bonzini says, .0.0 is April, .1.0 August and .2.0 December. The x.y.1 are stable-branch releases which happen slightly more irregularly.
I don't doubt you know that cadence :), but without the additional info you just provided with the x.y.0 and x.y.1 distinction it didn't seem accurate when compared to the actual dates of the releases.
So, to clarify:
x is year (counting from 2016=1) ? (OR first release of a year gets a major version bump)
IF last digit is 0 then y is to be decoded as:
0=April
1=August
2=December
OTHERWISE IF last digit is 1 the y may (or may not) be decoded as above.
Yes, except that if the last digit is not 0 then it only tells you "this is on the stable branch that started at the x.y.0 tag", and says nothing about year or month. Stable branch releases usually happen about 4 months after the .0, but there's no regular cadence to them, nor any rule about whether there is none, one, or more from any particular mainline release. (The entire release process for them is largely separate from the main releases.)
Modern computing is somewhat exciting. We finally got two competitive architectures (x86 and arm) and risc-v slowly building its momentum. This can only result in a better outcome: architectures will compete with each other and improve at faster rate; software will be forced to be cross-platform which is known to improve robustness even if you don't care about other platforms (some bugs are more likely to manifest on other platforms).
We got more than 2 competitive architechtures. There was Alpha, Sparc, MIPS, PA-RISC, ARM, X86 and POWER. Now Alpha and PA-RISC are dead, POWER and Sparc are niche, MIPS 64 is dead.
I like QEMU a lot but VirtualBox is my goto because the cli commands work fairly reliably and simply and I'm ashamed to say the simplicity of the UX is compatible with my laziness. For server workloads KVM/Qemu is great but imho, Vbox or Fusion are the best in my experience.
Does anyone know if there are Qemu wrappers with a nice UI/UX?
My current use cases aside, it's amazing the number of nice use cases Qemu is made use of. GNS3 was my favorite when I used it many years ago (Cisco IOS).
On the Linux side of the house, virt-manager is pretty great, though it leans much harder into KVM, so you don't necessarily get all the exotic architecture support of QEMU built in.
QtEmu for Windows gets you there, but it's very much so an API frontend, so if you're looking for simple from a choices perspective, this ain't it (though you might figure out your sane defaults and go from there).
Since you mentioned Fusion though, I'm guessing you're a Mac user, and UTM might be the ticket for you. The project started as a side-loadable/jailbroken device iOS VM system, but the project also makes a macOS app as well. https://mac.getutm.app/
This works both on Intel and Apple Silicon Macs, though in order to get virtualization acceleration, it's best to run workloads built for your native architecture (ARM on AS, x86(_64) on Intel).
> On the Linux side of the house, virt-manager is pretty great, though it leans much harder into KVM, so you don't necessarily get all the exotic architecture support of QEMU built in.
That's kind of funny because I end up accidentally creating non-KVM machines with virt-manager all the time...
On the Mac UTM is a nice wrapper around QEMU. They have a convenient gallery of preconfigured guest machines you can use and it's easy to export the QEMU cli command for any of the hosts you config in UTM if you need to.
As far as I know, running UTM/QEMU is the only method so far to run an x86 VM on the M1 Mac.
It works reasonably well, but I wish it was easier to do things like device passthrough or adding additional networks. If you get into the Advanced mode, you can pass in QEMU command flags directly which is nice.
What are your use cases? I have used to use UI/UX (virt-manager) in the past but finally just went for direct CLI on Qemu, because UI/UX is often missing some features.
In most of the cases, you only need to build one optimized single command and then you can reuse it everywhere. Over time, you also learn options pretty well.
Just save the command for somewhere.
My only lamentation of QEMU is their rapid development. (I did say “only”).
We had an open source CPU emulator (of many CPU arch) called “Unicorn” that was a heavy test instrumentation points inserted widely throughout and into QEMU 4.0. I spent 2 solid years on these dynamic API bindings of Unicorn into QEMU.
It was an awesome piece of SW that allowed us to recreate the behavior of malware.
Asking the QEMU team to insert some 50-odd test points into the QEMU code was proved to be a maintenance nightmare.
Even if it just an #ifdef of C language, it was too “cluttery”, it was still an awesome automated detection of code generation of just the affected malware portion.
I still believe this approach to be a significant tangential vector of prime investing for a startup.
The upstream now has TCG plugins (https://qemu.readthedocs.io/en/latest/devel/tcg-plugins.html) which allow for a degree of instrumentation. The implementation is architecture agnostic and also tested within the code base. There are still features missing but it does provide a base for dynamic analysis of guest code.
The plugins have access to the instruction stream to make architecture specific decisions. What I meant by architecture independent is that it doesn't involve per-guest annotations in the frontends to handle - any guest using the common translator loop (which is all of them now) can be instrumented by plugins.
However I absolutely agree its not currently as full featured as we would like. The next step when I get time is re-factoring the handling of register values in the core QEMU code so we can expose them to the plugins in a clean API.
// shameless ad, mods please delete if it's not permitted //
> It was an awesome piece of SW that allowed us to recreate the behavior of malware.
If you have experience w/QEMU and you are interested in working on it more, my team is doing work with QEMU. For the most part we have tended to focus on architecture-specific stuff but trying to grow our expertise.
Personally I pronounce it "kwem-yoo" (first syllable starts like
'quest', second syllable like the second-person pronoun 'you'). But I think there are multiple pronunciations floating around...
How’s macOS virtualization performance these days? Had to pony up for a Parallels Desktop license as QEMU was 2-3x slower and just wasn’t up to snuff for daily driving on my dev machine.
From my toying around with it it seems quite good. But I haven’t done any stress-testing/benchmarking. NB: Important to use the HVF accelerator! Otherwise it will surely be slow.
Any idea if this is enabled by default (or how to check/enable it manually) on UTM.app? I have the "Use Hypervisor" option enabled on the VM but it's dreadfully slow even with just a headless linux server.
Strange. I would guess that it’s enabled by default. You might want to submit a new discussion on their GitHub project: https://github.com/utmapp/UTM/discussions The devs are very active there.
QEMU is great for debugging the Linux kernel. But does anyone know if there are any paid alternatives? (QEMU is a bit clunky and the configuration is arbitrary and frequently broken)
With -accel whpx you can even get pretty solid CPU acceleration. WHPX[1] is sort of like KVM for Windows, only takes a couple clicks and a reboot to enable, and ships on both Windows Home and Pro. Other than the heroes who did the actual work, we probably have Android to thank[2] for this situation.
If you're willing to get your hands a little dirty, QEMU is a fairly viable alternative to WSL or even Docker on Windows.
[0]: https://qemu.weilnetz.de/w64/
[1]: https://docs.microsoft.com/en-us/xamarin/android/get-started...
[2]: https://developer.android.com/studio/run/emulator-accelerati...