They’ve finally nailed hand tracking for music! (ROLI Airwave demo)

[@andrewhuang]

Interested to hear what you think about this tech! Thanks again to ROLI for sponsoring - check out their site for more: shorturl.at/GExWw

[@RegebroRepairs]

I mean, for $300. It's not MiMu, but hey...

[@blackviperofdeath]

That looks dope AF. Reminds me of a theremin. I can hear some cool song ideas in the demo section.

[@modular_hall]

One step closer to the "MUSICAL RENAISSANCE"!!!! It feels like a liberation of motoric expression in electronic sound and performance. From the beginning of electroacoustic creation to modern electronic music, creativity has always been bound by technical constraints. It's a design that represents an idea-an idea that is abstract, perhaps even transcendent, but ROLI seems to take the concept behind their products very seriously. I have already ordered my Airwave!!! I'll take this opportunity to say THANK YOU ANDREW HUANG for your way of life! I don't know you but I can feel that you mean it.

[@nobodyhome3753]

Why don't you know what MPE is by now?

[@modular_hall]

@@nobodyhome3753 That's an interesting point you raise. I think it's no longer about MPE, but about its utilization. At the moment we are dealing with the problem of the "right" design.

[@Rwko.]

the force is strong with this one

[@UmasPapa]

The picking was ripe with this comment

[@driftliketokyo34ftw35]

And groovy!

[@JensHolgerHopp]

Finally, the Theremin approach is coming into MIDI realm. I hope, this will be further developed. These ideas are coming to me immediately: first of all, both hands detected separately (as you already mentioned, Andrew); then the next step: detecting every single finger's position on the keyboard. Using a proper software solution, combining the data from this thing with the note data from the keyboard should be possible. This would add MPE capabilities (or even more) to traditional keyboards. If done properly, this could be another game-changer. Well done, Roli!

[@famitory]

at that point you could add a laser projector and have an instant keyboard on any surface

[@ET2carbon]

Moog had mini thermini

[@jasonk125]

kzbin.info/www/bejne/fmqapWRthsR1mK8 has been around for years. This is a free browser based controller.

[@bewareofthecow]

@@jasonk125 this probably could of been done with a kinect on an xbox 360 but the hardware is the most insignificant part. being able to do something interesting and natural requires good software

[@ItsAbrahamSamuel]

Exactly 😅 Theremin dropped the mic on this long time ago. You're thinking what I am thinking mate! 😌👊🏽

[@translucentlyopaque]

In the realms of any sufficiently advanced technology being indistinguishable from magic, this is very magical

[@petermgruhn]

I'm thinking it's magical only if you don't watch the video.

[@ToyKeeper]

For those times when the Osmose just isn't expressive enough.

[@phelper4554]

Leap motion does the exact same thing for many years 😅

[@cockur]

Just like the leap motion was 20 years ago. This is rehashed nonsense that barely fits on your desk

[@junodark]

@@phelper4554 The hand tracking in this *is* Leap Motion! The project was a partnership with Ultraleap, the bone tracking at 5:00 shows the Ultraleap visualiser

[@SarahCharles92]

This tech gives me some serious Imogen Heap vibes. I feel like what she did a decade or so ago with her glove was the blueprint for tech like this.

[@N-trepid1]

Could not agree more. She used an xbox 360 camera I believe to track her hand gestures? Time to give Roli a call Imogen! ;)

[@AndyNicholson]

It's been around for a while already, go check out the "Mimu Glover" software and demonstration videos (that I think Imogen was involved with) being controlled by a Leap Motion controller. It wouldn't surprise me if the hardware in this is just a Motion Leap 2 that's been licensed out. I'm not hating on Roli, there is some interesting things happening here they deserver credit for, but they're not the first to do this and the people at Mimu should get some credit for it.

[@LoftyAssertions]

Yes!! And also Onyx Ashanti for his incredible homemade cyberpunk-like music making suits. I wonder if we would have gone these directions technologically without art pointing and creating the way, did artists see the potential futures so accurately or have scientists been inspired and followed their visionary imagery? I have been wondering why these two paths are so eternally related. I definitely think the latter is more likely from the scientists I know personally and parasocially.

[@AnalogFlava]

Totally, even the style of this track kinda took me back to frou frou!

[@trebmaster]

DJs are going to have so much fun in the next 20 years with this stuff.

[@tawmifm]

The Roli brand is gonna need some serious improvements for that to happen 😔

[@Medniex]

no more problems with hot knobs

[@AnalogFlava]

Itll be fun for 1 year not 20

[@Emtrax.]

It will be fun in 20 years for me when I can get it used

[@FranFuzz]

🤤 this into orchestral sounds will be amazing! Imagine the control over a violin vibrato, expression… omg

[@henrikpetersson3463]

Exactly my thought!

[@ThomasLoyd]

Next level Midi Theremin. LIKE!

[@Jonas_Mikkelsen]

Burst Generator is a beautiful piece of music.

[@4CloudySky]

So now it's better not to scratch your nose during the performance 😅

[@Xominamir]

Best comment

[@lucastrapeze]

Who knows? It might get a happy accident 😅

[@Xominamir]

@@lucastrapeze 😅😅😅

[@55pl3]

I won't be surprised if a style will be made after a couple of occurrences like these lol

[@alexanderjamesl4868]

Stand up job going through the tech and showing just how expressive this can be! I was really excited to see this finally come out as I was one of the folks working on the hand tracking side. Really looking forward to seeing what people can make with it :D

[@moamed2006]

for neurofunk, future garage the bass control sounds sick

[@RedMeansRecording]

Great track dude 😊

[@zavarmusic]

If you want similar but without being limited to ROLI devices just get a Leap Motion controller (Mk1 is fine, I got an used one for $10) and use it with GECO or MidiPaw.

[@ChristopheTornieri]

That’s exactly what I did when I saw the product. I found my forgotten leap motion mk1, downloaded the drivers and bought GECO midi for 20 bucks on the App Store and voilà. Same experience , with left and right hand tracking and more gestures.

[@johnbroadway7709]

Leap Motion is very responsive - smaller and adaptable. - The Neova Ring (got a bargain at £25) is excellent both studio and live (Understand Roland have acquired Neova Technology). The $250 price tag will put most people off.

[@smwells5]

I found the Leap Motion controller to be finicky and unpredictable when used in anything but ideal lighting. Hopefully the Airwave handles this better.

[@ChristopheTornieri]

@@smwells5 I don’t think it will be better because it’s probably based on the same technology.

[@zavarmusic]

@@smwells5 You're right and don't get me wrong, I'd love to see development with similar tech but this one feels like repackaged stuff from 10 years ago. Seems like the Airwave is trying to solve the same problem the Leap Motion has with the led strip but yeah, hopefully there's more to it.

[@Frostwriter111]

The gestures remind me a lot of conducting a choir or orchestra, which may be why it feels so natural

[@misterTavao]

I think someone performing with this would be so cool to watch live

[@124myth]

Yeah now we just move our hands (on some others things than a Theremin) and music starts playing automatically. This is literally magic

[@GerenM63]

I got to play with one of these at the AES show in New York today. It’s very cool, fun, and very intuitive.

[@ValsVirtuals]

it's just a leap motion controller in a stand, believe it or not this hand tracking is 10 years old tech, though this is a new application for it which is nice to see, seems like it has a lot of potential

[@giolabs]

That is not new either, geco MIDI for leap motion, Google it and you'll realize that also was implemented already, it was some hidden app that didn't got any additional update but already had 6 or seven different types of movement detection, but what this might bring is the marketing visibility and ease of use that a big brand can do.

[@smwells5]

I worked extensively on trying to make a Leap Motion theremin, but found that the Leap Motion controller tracking was rather finicky and unpredictable with anything but ideal lighting.

[@whyZaya]

The Leap Motion was such a cool idea with such a cool demo that was so terrible in practice. It really makes me not excited about this product.

[@ValsVirtuals]

@@whyZaya software has come a long way since, and Leap motion 2 refined the hardware, now they're pretty much state of the art in term of quality i heard

[@glennzone]

As Andrew mentions, it’s actually more like two Leap Motions error correcting each other, and in calibrated fixed positions to provide further stability. As one who bought the original Leap Motion at the time it came out, I found they had the right idea and direction, but the execution just wasn’t quite what it could have hoped to have been, especially in the knuckle joint movements. Having said all that, it seems Roli’s implementation is a good one. :-)

[@Jazz-nm1kf]

This is like bringing the espressivness of a theramin to the precision of a keyboard. magical. thanks andrew!

[@reset_rt]

Love it ❤ can't wait to see an update where both hands can independently control parameters ❤

[@MakerFarmNL]

This adds next level MPE to any traditional keyboard! The only disadvantage to the current design IMO, is that it stands between you and your monitor screen all the time, thus blocking the view significantly. So a version that could be mounted on top of your monitor would be solving that issue. Nevertheless a great achievement from Roli!

[@VincentKnaus]

Mine will be arriving in April. I cannot wait!

[@smthnew861]

they couldn't find a better person to advertise this! awesome!!!!

[@WindlessZephyr]

oh that is friggin' cool. maybe someday the tech can expand even further into full body tracking and we can combine things like dance and sound in entirely new ways :o

[@55pl3]

Wow. Using this to change rhythmic properties like arp behavior time signatures and changing how it sounds gritty or smooth long or short with a gesture... WOW!

[@xJayAikox]

that looks dope af fr

[@NeonVines]

🙌🏻💜💜💜

[@Alon21Ashkenazi]

Awesome review! This looks like the coolest thing in existence

[@fumarolas]

Fascinating. They should really make it clear if this can be used with any controller and if gestures can be mapped to any software synth. Watched all their vids and am still not completely sure

[@ShallieDragon]

I would be curious to see this used for VR applications. Like, if it can transmit the tracking data to SteamVR (for example), it could be a way to get accurate hand tracking without needing something like a Quest 3. While you're limited to using it while sitting, it could give you a way to accurately track your hands while playing piano. This could be really neat for people who perform music in VR.

[@DavidJFulde]

Maybe, but I think it's just a leap motion hardware wise so.... that's cheaper and has lots of support/tutorials to get it working in VR :)

[@Metapharsical]

Yep, I have a Leap Motion I've mounted on the face of my VR headset looking at my hands. Butt it's a novelty I've only used it for one title, Virt-a-Mate 😅

[@richardgperez]

Cant wait for full orchestra presets, ultra maestro mode. Great preview Andrew!

[@Vintagestep]

That's amazing, hand tracking has been a challenge for a lot of products attempting motion tracking. I wonder if you can take not only the MIDI CC output but also the tracking data to feed other software.... That's possibly unallowed in the ToS of their product but so many possibilities, not just influencing music but you could use it to operate plugins in other software, or have better hand tracking when using virtual avatars.

[@Metapharsical]

Hand tracking has existed for some time.. it's a product called Leap Motion. It looks like Roli basically repackaged Leap Motion into a desk stand and implemented the MIDI plug-ins. They did a nice job with it, but it's nothing new

[@Vintagestep]

@@Metapharsical Yeah, maybe I lived on a cave for some years lol

[@Jmjazz4711]

Wow! This definitely gives me Theremin vibes but takes it to a whole new level! I can already envision and hear my compositions utilizing this great technology! Fantastic video, as always!

[@WeAreROLI]

Hey, thank you for the comment! ROLI Airwave isn't just a modern take on the Theremin (though, let's be honest, the Theremin is a timeless classic). With ROLI Vision tracking every unique hand gesture and nuanced movement, you get control over more than volume and pitch. ROLI Airwave will include 100 bespoke Airwave sounds, finely tuned to 5 Dimensions of Air for each gesture you make!

[@petermgruhn]

And yet... it can't even be a Theremin. See the comment from ROLI, "Airwave isn't... [a] Theremin." They KNOW they have quash that before they get sued.

[@troytinsley148]

Hey Andrew, this is wild, and I really dig Roli and their innovations. I know it's been a while since I've been paying attention. I know a while back, things weren't looking good your way. I remember you have been losing your hearing over time. I know how it is since I'm almost blind, but not hearing must be incredibly challenging since you're a musician. All this is to say I'm sending some love your way!!!

[@nomoresaul]

I saw this exact technology in a video many years ago, but this is the first I’ve seen it applied to anything more than just an experiment. Somehow, I hadn’t imagined it for music, but now I don’t know why I hadn’t.

[@markquavertune2003]

WOW ! I already have equator .I should start to make presets so I can use them when I get a Roli and this very useful device .Exciting musical times ! Thank you Andrew for this good video .Makes me think about doing modular hardware too .

[@professor_corrin9350]

That bone tracker would actually be super useful for animation of musicians. Wish it was publicly available.

[@Cyrax89721]

Simultaneously outputting the 3D joint vertices to something like Touch Designer to modulate a live video screen would be dope.

[@AndyNicholson]

Go check out Leap Motion Controller 2, I'm fairly certain this is just one of them packaged differently.

[@speedmetalftw]

​@Cyrax89721 run it into source film maker

[@KaiMattingly]

you are the mad scientist of music.

[@L0ps1d3]

This is literally the future of music, wow.

[@martiboucat]

Finally the classic piano hand gestures will do something

[@marclupiane4843]

Just amazing!

[@LibbyTehHuman]

It's like a theremin for FX. It's so cool.

[@matheusempixels]

Can't describe it, so perfect!

[@WeAreROLI]

Thank you ❤

[@mopishlynx2323]

See, this is cool and all, but I feel like the Seaboard already was as expressive as I want. Roli has a history of buying into some early tech, selling some units, then abandoning the ecosystem. This happened for Blocks, which was supposed to grow over time, but only really changed with Lumi, which was much more geared towards beginners than working musicians. I guess that comes with being bought out by Apple.

[@SeanHyland]

When were they bought out by Apple?

[@rayNmanMusic808]

@@SeanHyland 🤔

[@mopishlynx2323]

@@SeanHyland they went bankrupt years ago and Apple got them out of it. That's why Blocks never went anywhere.

[@dannyC3827]

Roli customer support isn't the greatest either. My Roli Seaboard 49 decided to brick itself, and I'm going on week twelve tomorrow just to hear back on the part they think I need to fix it. I want to believe in Roli as I LOVED playing the Seaboard and Lumi's I have, but I have very little faith in support when something goes wrong. I wanted to get a Seaboard 2 but I don't think that's going to happen based on my experience with customer service. Apple has also been caught bricking their own tech, just so you HAVE to buy their new stuff. That business model is disgusting, especially for instruments. I hope that is not happening with Roli.

[@mopishlynx2323]

@@dannyC3827 that's what I hate; their instruments are actually really good! If they'd spend the money to give a quality experience and actually make good on their promises, they'd do super well. Think about how much money Arturia made on the Microfreak bc its an amazing synth that they continued to support and update for years, even after the Minifreak released. I went to a Guitar Center the other day, and to *this day* they have 3 Microfreaks in stock, the only synths they had. If you treat musicians with respect, you're gonna get hella money from it.

[@AugustineKizis]

Wow!! I love it!

[@BlufyreAudio]

At some point, I'd really like a breakdown of Burst Generator. I've wanted to make music like that for a long time, stuff with tons of layering, but always have trouble mixing everything so that the sounds don't clash with each other and make everything super muddy.

[@kitebabe05]

Music through gestures , this is the beginning of future kind of music making

[@mopishlynx2323]

Wow I wonder if they'll take Blocks seriously again one day

[@rylyn_burne]

I know. I love my block. Wish they kept them going

[@wega4105]

UDO Super 6 desktop synth are used w/ seabord. Really nice.

[@steveelbows3797]

Unlikely, they even removed some of the 'DNA connectors' from later products like the updated Seaboard Block (now known as Seaboard M) and the Lumi (now known as Piano M). I think they removed the connectors from the top edge of the device, and only retained the ones that are necessary to expand the octave range of the Seaboard M and Piano M by joining multiple ones together side by side.

[@earthman6700]

​@@rylyn_burne I am sure they do sell them. Called the Seaboard M now.

[@VDJ4500]

...looks funny, sounds okay 😂, a nice review

[@cyb]

That technology seems like inadvertently perfect for vr

[@jcalerre]

This is essentially how hand tracking currently works in headsets like the Quest and Apple Vision

[@glnnk_art]

It’s probably based off of VR tracking tech

[@swampflux]

You are... actually correct! The original product was called Leap Motion. It was not intended to be a VR device but something that laid in front of your computer keyboard to do... whatever you programmed it to do. Not too many people bought them except developers trying to dream up a use case. Later, the first Occulus Rift Dev Kit headset came out, and it wasn't too long after before someone finally realized they could glue a bracket to the front of the headset to hold the Leap and have hand tracking in VR. (they had to of course build their own software to make this a reality though). Later hand tracking implementations licensed the Leap's patented tech for years. I think it's hilarious that this is just.. a leap motion back on the desk and we're surprised like it's new and not just standing on some popsicle sticks. The magic here is in the synth UI and the already set up bank of presets that take advantage of the tech to make it feel like a finished product. That's what the on-desk Leap was always missing: something that felt like a reason for the tech to exist. (Like releasing a wiimote without a wii or a zelda game to go with it.)

[@OlivioSarikas]

VR can do way more than this already. you can have a complete modular sythn setup simulated in VR and control it with your hands like a real one. have a look at SynthVR

[@widgity]

@@swampflux Yeah, my first experimentation in VR was a DK2 with a leap motion. I wonder if you could use this software with a standard leap motion, or if they have locked it to their version of the hardware...

[@teally-bop]

THIS IS SO COOL!!!!!!!!!!!!!!!!!!!!!!!!!!

[@rishabhrajan]

So it's basically like a Leap Motion 2 controller but with a giant structure in front and can only be used with Roli controllers?

[@samumohacsi]

this is like a Rain World soundtrack machine. really cool.

[@midnitejesus]

Absolutely underrated gem of a game. Wish more people knew how great it was!

[@angelgcastro1]

I am loving this thing...preordered!

[@chrisolivtreez8017]

Piano in it self just went flying! 🎹 🕊️

[@jaffasplaffa1]

Pretty impressive tool!

[@alejandroacevedo9409]

Sounds amazing

[@all_that_gazz]

I remember as a teenager, watching Jean Michelle Jarre play the Laser Harp, and was like WOW! Just had one of those moments again 🙂

[@DesertCow1000]

I've dreamt about using technology like this before. What a time to be alive

[@OatsJenkins]

finally, theremin 2

[@ElricMelni]

Nice ... Thanks for showing ... Think about to order it ..

[@severusnare]

Thanks for the video, I'm thinking about the possibility of extending this tech to other devices, amazing.

[@tl2402]

Damn, I have to get back to this video later and do some anatomy study sketching on your hands; the lighting makes them really interesting in that sense. There's so much material in this video, lol.

[@familycraftdad]

I used to use the Leap Motion from years ago to do similar things. But to have a dedicated interface like that with all the presets.... wow that would be so much fun!

[@familycraftdad]

Wait! That backend program you showed looked nearly identical to Leap!! I wonder if that's coincidence or...

[@TSGEnt]

Kinda a theremin keyboard. What I appreciate about you is you say, "let s try this out" and you actually make a a brilliant piece of music. ( yeah I know it's not real time, at least the full tune, but you take the time to make some music.) Thank You!

[@giraffeapproved]

This is fucking insane. The song was FIRE too. Good stuff Andy!

[@OfficialStevenCravis]

Many people are mentioning Theremin, and Roli could really easily add and position this as Effects or Pitch mode in Equator. Pitch could really function like a theremin.

[@smwells5]

Yes, from a hardware perspective, it has everything needed to be programed as a theremin. If enabled, you wouldn't even need a keyboard to use it as a theremin. Set the left hand to control volume, and right hand to control pitch, with whatever range and sound you choose. For $299, it would be a relatively inexpensive theremin.

[@bubume8277]

Magical! 🧙‍♂🧙‍♂🧙‍♂ You literally are casting spells with it!

[@oceanus123]

Very nice tune !

[@Steolepandaxiongmao]

this is really cool, like a modern Leap Motion that feels reliable!

[@AndyNicholson]

My v1 controller and Glover software has been reliable for years, but I believe this might be a v2 embedded controller, judging by how it's working and the "developer" graphics.

[@AndyChamberlainMusic]

now this is cool very cool

[@craigritchie8470]

This is moving slowly to the air keyboard. We won’t even need a physical keyboard. In fact it won’t be long until our entire body IS the instrument. Imagine a dancer making the music as they dance!!

[@benas_st]

I think that would already be possible! Just perhaps not to the regular consumer. But body tracking for different CGI work has been available for a while, I'm sure that input could be turned into musical expression! Though I do have to slightly disagree on the keyboard not being needed. It would be very difficult to accurately play notes if there isn't any haptic feedback and you weren't really sure what keys you're pressing. And if you do need to look at a screen to know that, then I feel it kind of destroys the purpose, at least for me it would :))

[@aaronellson9849]

Look at mimu gloves. And glover software.

[@VarionJimmy]

I made a working prototype of that around 2006-2007. (I used gloves with accelerometers and OpenGL with two web cameras.) Unfortunately my webpage is down at the moment (https problems) but it’s there somewhere.

[@Metapharsical]

Some guy did this (full body midi control) about ten years ago using an Xbox Kinect camera. He even released it as a PC app, called Kinectar !

[@HeroNoodle]

Honestly this feels like it was made for orchestra conductors wanting to produce, that and also anyone who morphs their hands in line with dubstep basses lol

[@alexandervonheereman6168]

This is really cool. I imagine version 2.0 and how much more you can do with that one plus the refinements in the software. Could be kind of a game changer fpr certain people. I want one. :)

[@drsamurai009]

Almost like working as an orchestra conductor performing the timbre. Very cool!

[@apiksDen]

I do love the music you do

[@karsinokuuni3208]

biblically accurate theremin

[@williamshaneblyth]

Crazy expansive like all roli stuff

[@gphiproductions]

Nice demonstration Andrew! Reminds me of a modern version of The Theremin instrument. 👍🏽

[@OlivioSarikas]

So this is basically a xbox kinect combined with midi. you can also do this with a Quest 3, but actually using all your fingers and both hands. have a look at SynthVR

[@tenma4219]

That's so cool! I wonder if one could achieve similar result with a leap motion 🤔

[@WarrenPostma]

1:14 this is awesome. what a great track

[@andrewhuang]

Thank you! Full thing is on streaming services, Andrew Huang - Burst Generator

[@Centomila]

This replaced my desire to buy a theremin. EDIT: Damn. Works only with a ROLI keyboard.

[@nullbyt]

Imagine the same concept but with hand tracking stuff used by vtubers remapped to control midi

[@Centomila]

@@nullbyt I'd rather not.

[@eghho1]

Search for midipaw and leap motion 😊

[@strixcz]

Yeah, also Mac only - that's a hard pass for me 👎

[@ANTheWhizkid]

It seems to work like you want it to and appears less experimental like similar devices of the past. Looks legit.

[@Korgi_Polymer]

1:12 kinda sounds like the main theme from the Amiga game Weird Dreams, really dig the sound!

[@jwrightgardening]

I first saw the commercials that focused on beginner learning and thought, fun but a lot of money for something I don't really need... Now seeing ALL THE THINGS it can do 🤤🤤🤤

[@EmlynInTheMix]

Can’t wait to try this!!

[@Pauluz_The_Web_Gnome]

Hahahahha Stinky Karate Face @1:16 😂😂😂

[@MitchLJay]

Omg I need one!!! ❤

[@NondoPondo]

I saw a guy in a band who was using his theremin to add expression to his guitar rig. This reminded me of that.

[@naping]

I need this.

[@richsentertainmentproductions]

I could only imagine a full body tracker as someone dances to create the music they are dancing to.

[@chrisolivtreez8017]

Lets call Mezerg !!!!❤ theremine + air piano combo

[@LoftyAssertions]

I can easily see mechanical interfaces such as mice and keyboards being archaic vintage gear in a few decades, it's definitely a clunky remnant of the past, as well as 2D media and displays instead of AR and holographic displays. 2D photos giving a huge sense of utopian nostalgia etc. I simultaneously see the godlike magic of our current state - as well as it as a primitive past of a future that has completed more technological pathfinding.

[@Jamman610]

I love it. Imagine wearing a hooded cape, skeleton gloves that glow in the dark. The theatrics are endless. My Juno G has the D Beam that’s similar. I would love to get this and a ROLI Sea board. I played one of the 61 keys at Guitoy Centre years ago. It felt so natural to play. I’m a guitarist and playing a regular piano type instrument feels limited. I’m using MIDI Guitar 2 & looking forward to the MIDI Guitar 3 to come out in iOS soon with a Hexaphonic pick capability. This system would be cool to incorporate.

[@unternatuerlich]

I see a lot of potential in this device.

[@Ghozer]

It's like a modern version of or an evolution of the "Alesis AirFX" - that was awesome at the time :D

[@thummumcrysanth]

Burst Generator is awesome!

[@michaelschnell5633]

Seems like a must have :)