VR music has quietly evolved from fringe experiment to serious, fun musical tool – and it’s done so entirely through one app, PatchWorld. Released publicly last summer, it’s gotten steadily more capable and reliable, including on Meta’s affordable Quest 2 headset. But most recently, on top of additions like Ableton Live integration, PatchXR has added a true killer app: multiplayer.
Disclosure statement: I have worked in the past as a consultant to PatchXR and collaborated with them on a series of hackathons for artists to work on live experiments on the platform.
It’s difficult to show immersive VR experiences on a flat screen – you really need the headset to grasp how they work. But Apple’s launch of Vision Pro got one overwhelming reaction from online audiences: the technology looked really lonely. You saw people putting ski goggles on and basically becoming antisocial.
That’s odd, because multiplayer is where this tech really shines. Just as spatializing sound and instrument positions assists improvisation in shared physical space, it’s a huge boon when improvising with other musicians online. VR is clearly not for everyone; it requires an additional hardware purchase and some people find the headsets uncomfortable or have difficulty with the 3D effects, especially on low-cost hardware like the Quest 2. But PatchWorld is the one tool I’ve seen that demonstrates why you would really want this as a music tool – and multiplayer is obviously the technology’s real killer app.
And while the demos are cute with this animated character, this project uniquely scales. You can just toy around with it like a musical game, even with no music experience. But you can also go deep and patch custom creations with a library that’s reminiscent of modular environments on desktop – seriously.
Here’s an update on how PatchWorld has evolved since its public release on the Meta Quest Store.
Aside: you don’t need a Facebook login for your Quest any more, and for that, you can thank the German government. I think it’s a solid example of how oversight can ultimately benefit consumers.
PatchWorld works with mixed reality, and integrates with Ableton Live
PatchWorld last year added support for mixed reality – camera passthrough. On the Quest 2, this is via a fairly lo-fi black-and-white camera, but it’s usable even there, and you get higher-quality color not only on the spendy Quest Pro but also the forthcoming Quest 3, due later this year.
Mixed Reality support is more than just supporting the passthrough image. PatchWorld has a number of tools for actually using the physical environment around you – like bouncing balls off your living room table to make music. Here’s a deep-dive explanation of what that entails:
Mixed reality transforms how you jam and play: PatchWorld update
Passthrough also means the ability to overlay VR with other views. There’s a toolset available for integrating Ableton Live with PatchWorld, for instance. Here it is in action:
This is thanks to integrated support for OSC in PatchWorld. That means you can also roll your own integrations, but Ableton Live support is provided by this Max for Live device from PatchXR themselves:
There’s a deep toolset
There are just a ton of tools you can use in the built-in library – a steal for 30 bucks. There are lower-level objects you can use to create your own custom instruments and audio and visual tools, including really a lot of fun with sampling. When you want this to be a full modular environment in VR, it totally can be, for your own custom synths, if you desire. But there are also higher-level, ready-to-play tools – like, recently, some very good-sounding classic 303 and 808 additions:
Multiplayer is extraordinary
Multiplayer is one of those places where VR actually solves a problem. It’s hard to show in videos, and there’s not much documentation from PatchXR’s side. But I did have Gad Baruch Hinkis from PatchXR walk me through a multiplayer session. The spatialization is really essential, because you hear the location of what instrument is being played, and even the spatialized voice of other people you’re playing with, even apart from seeing their visual avatar.
That really provides an immediacy and intimacy that’s lacking in other multiplayer tools. It really feels like you’re spending time in the same room as someone else and playing with them in a shared space. The pandemic revealed just how much isolation can prove a challenge; this seems a crucial tool for distance work, especially for those who can’t be physically in a location.
Jamming is really easy, and it opens up not only playing together but patching together. Combined with avatar recording, that makes multiplayer a powerful teaching tool. It’s what we were missing, frankly, in the early Patchathons – we didn’t have a good way of getting PatchXR’s developers to show users how to work.
Here’s an idea of how multiplayer can work, as the developers first unveiled it:
And how a jam session can work:
You can definitely teach workshops in this environment, and some folks are even getting fairly avant-garde.
VR is a great tool for natively working with 3D
Because it’s open to custom content, PatchWorld has proven a powerful tool for rapid development of 3D audiovisual work – natively in the 3D environment, rather than having to work separately in authoring tools and then test how things work in VR. We saw that come across brilliantly in some of the results of the Patchathon, and this was with a much earlier version of the authoring tools, without multiplayer:
You can also do things like bringing in your own 3D assets – but manipulating them natively in the 3D environment, with spatialization built in, which isn’t possible like this in other tools. And indeed, if Apple wants to make Vision Pro a pro device – an equivalent to the Mac – this is the sort of stuff they’ll need from developers. (This series covers a lot of other tutorials, too, for those interested.)
Like I said, VR I know isn’t for everyone. Musicians are as always going lots of different directions – from patching physical modular to bringing back trackers. But if you want a glimpse of what VR could do, PatchWorld is really the most compelling current example. PatchXR also is building the competencies to work on future platforms. And yet you also don’t have to wait – this is all usable with an inexpensive app on the entry-level Quest 2 headset (which is kind of a marvel of engineering).
For more on Patch:
