They’ve finally nailed hand tracking for music! (ROLI Airwave demo)

[@andrewhuang]

Interested to hear what you think about this tech! Thanks again to ROLI for sponsoring - check out their site for more: shorturl.at/GExWw

[@RegebroRepairs]

I mean, for $300. It's not MiMu, but hey...

[@blackviperofdeath]

That looks dope AF. Reminds me of a theremin. I can hear some cool song ideas in the demo section.

[@modular_hall]

One step closer to the "MUSICAL RENAISSANCE"!!!! It feels like a liberation of motoric expression in electronic sound and performance. From the beginning of electroacoustic creation to modern electronic music, creativity has always been bound by technical constraints. It's a design that represents an idea-an idea that is abstract, perhaps even transcendent, but ROLI seems to take the concept behind their products very seriously. I have already ordered my Airwave!!! I'll take this opportunity to say THANK YOU ANDREW HUANG for your way of life! I don't know you but I can feel that you mean it.

[@nobodyhome3753]

Why don't you know what MPE is by now?

[@modular_hall]

@@nobodyhome3753 That's an interesting point you raise. I think it's no longer about MPE, but about its utilization. At the moment we are dealing with the problem of the "right" design.

[@Rwko.]

the force is strong with this one

[@UmasPapa]

The picking was ripe with this comment

[@driftliketokyo34ftw35]

And groovy!

[@translucentlyopaque]

In the realms of any sufficiently advanced technology being indistinguishable from magic, this is very magical

[@petermgruhn]

I'm thinking it's magical only if you don't watch the video.

[@ToyKeeper]

For those times when the Osmose just isn't expressive enough.

[@phelper4554]

Leap motion does the exact same thing for many years 😅

[@cockur]

Just like the leap motion was 20 years ago. This is rehashed nonsense that barely fits on your desk

[@junodark]

@@phelper4554 The hand tracking in this *is* Leap Motion! The project was a partnership with Ultraleap, the bone tracking at 5:00 shows the Ultraleap visualiser

[@JensHolgerHopp]

Finally, the Theremin approach is coming into MIDI realm. I hope, this will be further developed. These ideas are coming to me immediately: first of all, both hands detected separately (as you already mentioned, Andrew); then the next step: detecting every single finger's position on the keyboard. Using a proper software solution, combining the data from this thing with the note data from the keyboard should be possible. This would add MPE capabilities (or even more) to traditional keyboards. If done properly, this could be another game-changer. Well done, Roli!

[@famitory]

at that point you could add a laser projector and have an instant keyboard on any surface

[@ET2carbon]

Moog had mini thermini

[@SarahCharles92]

This tech gives me some serious Imogen Heap vibes. I feel like what she did a decade or so ago with her glove was the blueprint for tech like this.

[@N-trepid1]

Could not agree more. She used an xbox 360 camera I believe to track her hand gestures? Time to give Roli a call Imogen! ;)

[@AndyNicholson]

It's been around for a while already, go check out the "Mimu Glover" software and demonstration videos (that I think Imogen was involved with) being controlled by a Leap Motion controller. It wouldn't surprise me if the hardware in this is just a Motion Leap 2 that's been licensed out. I'm not hating on Roli, there is some interesting things happening here they deserver credit for, but they're not the first to do this and the people at Mimu should get some credit for it.

[@LoftyAssertions]

Yes!! And also Onyx Ashanti for his incredible homemade cyberpunk-like music making suits. I wonder if we would have gone these directions technologically without art pointing and creating the way, did artists see the potential futures so accurately or have scientists been inspired and followed their visionary imagery? I have been wondering why these two paths are so eternally related. I definitely think the latter is more likely from the scientists I know personally and parasocially.

[@RedMeansRecording]

Great track dude 😊

[@ThomasLoyd]

Next level Midi Theremin. LIKE!

[@4CloudySky]

So now it's better not to scratch your nose during the performance 😅

[@Xominamir]

Best comment

[@lucastrapeze]

Who knows? It might get a happy accident 😅

[@Xominamir]

@@lucastrapeze 😅😅😅

[@trebmaster]

DJs are going to have so much fun in the next 20 years with this stuff.

[@tawmifm]

The Roli brand is gonna need some serious improvements for that to happen 😔

[@Medniex]

no more problems with hot knobs

[@snailfox7]

If you want similar but without being limited to ROLI devices just get a Leap Motion controller (Mk1 is fine, I got an used one for $10) and use it with GECO or MidiPaw.

[@ChristopheTornieri]

That’s exactly what I did when I saw the product. I found my forgotten leap motion mk1, downloaded the drivers and bought GECO midi for 20 bucks on the App Store and voilà. Same experience , with left and right hand tracking and more gestures.

[@johnbroadway7709]

Leap Motion is very responsive - smaller and adaptable. - The Neova Ring (got a bargain at £25) is excellent both studio and live (Understand Roland have acquired Neova Technology). The $250 price tag will put most people off.

[@smwells5]

I found the Leap Motion controller to be finicky and unpredictable when used in anything but ideal lighting. Hopefully the Airwave handles this better.

[@ChristopheTornieri]

@@smwells5 I don’t think it will be better because it’s probably based on the same technology.

[@snailfox7]

@@smwells5 You're right and don't get me wrong, I'd love to see development with similar tech but this one feels like repackaged stuff from 10 years ago. Seems like the Airwave is trying to solve the same problem the Leap Motion has with the led strip but yeah, hopefully there's more to it.

[@cyb]

That technology seems like inadvertently perfect for vr

[@jcalerre]

This is essentially how hand tracking currently works in headsets like the Quest and Apple Vision

[@glnnk_art]

It’s probably based off of VR tracking tech

[@swampflux]

You are... actually correct! The original product was called Leap Motion. It was not intended to be a VR device but something that laid in front of your computer keyboard to do... whatever you programmed it to do. Not too many people bought them except developers trying to dream up a use case. Later, the first Occulus Rift Dev Kit headset came out, and it wasn't too long after before someone finally realized they could glue a bracket to the front of the headset to hold the Leap and have hand tracking in VR. (they had to of course build their own software to make this a reality though). Later hand tracking implementations licensed the Leap's patented tech for years. I think it's hilarious that this is just.. a leap motion back on the desk and we're surprised like it's new and not just standing on some popsicle sticks. The magic here is in the synth UI and the already set up bank of presets that take advantage of the tech to make it feel like a finished product. That's what the on-desk Leap was always missing: something that felt like a reason for the tech to exist. (Like releasing a wiimote without a wii or a zelda game to go with it.)

[@GerenM63]

I got to play with one of these at the AES show in New York today. It’s very cool, fun, and very intuitive.

[@Geo-wc7jc]

I couldn't think of anyone more fitting to demonstrate a product like this. True techno-wizardry.

[@funkingitup1805]

Loopop. Daniel Fisher.

[@misterTavao]

I think someone performing with this would be so cool to watch live

[@mopishlynx2323]

See, this is cool and all, but I feel like the Seaboard already was as expressive as I want. Roli has a history of buying into some early tech, selling some units, then abandoning the ecosystem. This happened for Blocks, which was supposed to grow over time, but only really changed with Lumi, which was much more geared towards beginners than working musicians. I guess that comes with being bought out by Apple.

[@SeanHyland]

When were they bought out by Apple?

[@rayNmanMusic808]

@@SeanHyland 🤔

[@mopishlynx2323]

@@SeanHyland they went bankrupt years ago and Apple got them out of it. That's why Blocks never went anywhere.

[@dannyC3827]

Roli customer support isn't the greatest either. My Roli Seaboard 49 decided to brick itself, and I'm going on week twelve tomorrow just to hear back on the part they think I need to fix it. I want to believe in Roli as I LOVED playing the Seaboard and Lumi's I have, but I have very little faith in support when something goes wrong. I wanted to get a Seaboard 2 but I don't think that's going to happen based on my experience with customer service. Apple has also been caught bricking their own tech, just so you HAVE to buy their new stuff. That business model is disgusting, especially for instruments. I hope that is not happening with Roli.

[@mopishlynx2323]

@@dannyC3827 that's what I hate; their instruments are actually really good! If they'd spend the money to give a quality experience and actually make good on their promises, they'd do super well. Think about how much money Arturia made on the Microfreak bc its an amazing synth that they continued to support and update for years, even after the Minifreak released. I went to a Guitar Center the other day, and to *this day* they have 3 Microfreaks in stock, the only synths they had. If you treat musicians with respect, you're gonna get hella money from it.

[@fumarolas]

Fascinating. They should really make it clear if this can be used with any controller and if gestures can be mapped to any software synth. Watched all their vids and am still not completely sure

[@craigritchie8470]

This is moving slowly to the air keyboard. We won’t even need a physical keyboard. In fact it won’t be long until our entire body IS the instrument. Imagine a dancer making the music as they dance!!

[@benas_st]

I think that would already be possible! Just perhaps not to the regular consumer. But body tracking for different CGI work has been available for a while, I'm sure that input could be turned into musical expression! Though I do have to slightly disagree on the keyboard not being needed. It would be very difficult to accurately play notes if there isn't any haptic feedback and you weren't really sure what keys you're pressing. And if you do need to look at a screen to know that, then I feel it kind of destroys the purpose, at least for me it would :))

[@aaronellson9849]

Look at mimu gloves. And glover software.

[@VarionJimmy]

I made a working prototype of that around 2006-2007. (I used gloves with accelerometers and OpenGL with two web cameras.) Unfortunately my webpage is down at the moment (https problems) but it’s there somewhere.

[@Metapharsical]

Some guy did this (full body midi control) about ten years ago using an Xbox Kinect camera. He even released it as a PC app, called Kinectar !

[@ValsVirtuals]

it's just a leap motion controller in a stand, believe it or not this hand tracking is 10 years old tech, though this is a new application for it which is nice to see, seems like it has a lot of potential

[@giolabs]

That is not new either, geco MIDI for leap motion, Google it and you'll realize that also was implemented already, it was some hidden app that didn't got any additional update but already had 6 or seven different types of movement detection, but what this might bring is the marketing visibility and ease of use that a big brand can do.

[@smwells5]

I worked extensively on trying to make a Leap Motion theremin, but found that the Leap Motion controller tracking was rather finicky and unpredictable with anything but ideal lighting.

[@KaiMattingly]

you are the mad scientist of music.

[@moamed2006]

for neurofunk, future garage the bass control sounds sick

[@MakerFarmNL]

This adds next level MPE to any traditional keyboard! The only disadvantage to the current design IMO, is that it stands between you and your monitor screen all the time, thus blocking the view significantly. So a version that could be mounted on top of your monitor would be solving that issue. Nevertheless a great achievement from Roli!

[@FranFuzz]

🤤 this into orchestral sounds will be amazing! Imagine the control over a violin vibrato, expression… omg

[@Jonas_Mikkelsen]

Burst Generator is a beautiful piece of music.

[@Jazz-nm1kf]

This is like bringing the espressivness of a theramin to the precision of a keyboard. magical. thanks andrew!

[@professor_corrin9350]

That bone tracker would actually be super useful for animation of musicians. Wish it was publicly available.

[@Cyrax89721]

Simultaneously outputting the 3D joint vertices to something like Touch Designer to modulate a live video screen would be dope.

[@AndyNicholson]

Go check out Leap Motion Controller 2, I'm fairly certain this is just one of them packaged differently.

[@speedmetalftw]

​@Cyrax89721 run it into source film maker

[@mopishlynx2323]

Wow I wonder if they'll take Blocks seriously again one day

[@rylyn_burne]

I know. I love my block. Wish they kept them going

[@wega4105]

UDO Super 6 desktop synth are used w/ seabord. Really nice.

[@steveelbows3797]

Unlikely, they even removed some of the 'DNA connectors' from later products like the updated Seaboard Block (now known as Seaboard M) and the Lumi (now known as Piano M). I think they removed the connectors from the top edge of the device, and only retained the ones that are necessary to expand the octave range of the Seaboard M and Piano M by joining multiple ones together side by side.

[@WindlessZephyr]

oh that is friggin' cool. maybe someday the tech can expand even further into full body tracking and we can combine things like dance and sound in entirely new ways :o

[@halsmypal]

Imagine what Lang Lang can do with this thing.

[@reset_rt]

Love it ❤ can't wait to see an update where both hands can independently control parameters ❤

[@samumohacsi]

this is like a Rain World soundtrack machine. really cool.

[@midnitejesus]

Absolutely underrated gem of a game. Wish more people knew how great it was!

[@Vintagestep]

That's amazing, hand tracking has been a challenge for a lot of products attempting motion tracking. I wonder if you can take not only the MIDI CC output but also the tracking data to feed other software.... That's possibly unallowed in the ToS of their product but so many possibilities, not just influencing music but you could use it to operate plugins in other software, or have better hand tracking when using virtual avatars.

[@Metapharsical]

Hand tracking has existed for some time.. it's a product called Leap Motion. It looks like Roli basically repackaged Leap Motion into a desk stand and implemented the MIDI plug-ins. They did a nice job with it, but it's nothing new

[@Vintagestep]

@@Metapharsical Yeah, maybe I lived on a cave for some years lol

[@Pauluz_The_Web_Gnome]

Hahahahha Stinky Karate Face @1:16 😂😂😂

[@LoftyAssertions]

I can easily see mechanical interfaces such as mice and keyboards being archaic vintage gear in a few decades, it's definitely a clunky remnant of the past, as well as 2D media and displays instead of AR and holographic displays. 2D photos giving a huge sense of utopian nostalgia etc. I simultaneously see the godlike magic of our current state - as well as it as a primitive past of a future that has completed more technological pathfinding.

[@matheusempixels]

Can't describe it, so perfect!

[@Centomila]

This replaced my desire to buy a theremin. EDIT: Damn. Works only with a ROLI keyboard.

[@nullbyt]

Imagine the same concept but with hand tracking stuff used by vtubers remapped to control midi

[@Centomila]

@@nullbyt I'd rather not.

[@eghho1]

Search for midipaw and leap motion 😊

[@Steolepandaxiongmao]

this is really cool, like a modern Leap Motion that feels reliable!

[@AndyNicholson]

My v1 controller and Glover software has been reliable for years, but I believe this might be a v2 embedded controller, judging by how it's working and the "developer" graphics.

[@all_that_gazz]

I remember as a teenager, watching Jean Michelle Jarre play the Laser Harp, and was like WOW! Just had one of those moments again 🙂

[@MrFlottgote]

Just 100 years after synthesizer technology came up with this. I can't beleive we're making such progress!

[@WeAreROLI]

Here's to the next 100 years! 🚀

[@Frostwriter111]

The gestures remind me a lot of conducting a choir or orchestra, which may be why it feels so natural

[@AugustineKizis]

Wow!! I love it!

[@VincentKnaus]

Mine will be arriving in April. I cannot wait!

[@OatsJenkins]

finally, theremin 2

[@alecrisser12]

That long hair suits you, especially considering you're the mad scientist of the modular synth.

[@ShallieDragon]

I would be curious to see this used for VR applications. Like, if it can transmit the tracking data to SteamVR (for example), it could be a way to get accurate hand tracking without needing something like a Quest 3. While you're limited to using it while sitting, it could give you a way to accurately track your hands while playing piano. This could be really neat for people who perform music in VR.

[@DavidJFulde]

Maybe, but I think it's just a leap motion hardware wise so.... that's cheaper and has lots of support/tutorials to get it working in VR :)

[@Metapharsical]

Yep, I have a Leap Motion I've mounted on the face of my VR headset looking at my hands. Butt it's a novelty I've only used it for one title, Virt-a-Mate 😅

[@NeonVines]

🙌🏻💜💜💜

[@ElricMelni]

Nice ... Thanks for showing ... Think about to order it ..

[@OfficialStevenCravis]

Many people are mentioning Theremin, and Roli could really easily add and position this as Effects or Pitch mode in Equator. Pitch could really function like a theremin.

[@smwells5]

Yes, from a hardware perspective, it has everything needed to be programed as a theremin. If enabled, you wouldn't even need a keyboard to use it as a theremin. Set the left hand to control volume, and right hand to control pitch, with whatever range and sound you choose. For $299, it would be a relatively inexpensive theremin.

[@Lewious]

It's like a theremin for FX. It's so cool.

[@BrunodeSouzaLino]

Whenever anything Roli comes up,I always remember when LMNC did a video disassembling the smaller seaboard along with other people and started giving the Roli rep shit because the unit used Pentalobe screws.

[@Madmohawkfilms]

Oh look, Roli reinvented the Roland D Beam.

[@getrichquicc]

This is how a wizard would make music

[@ArrangerKeyboardGuy]

This is insane. We live in such a wild time for technology!

[@WeAreROLI]

Thank you so much!

[@Fpeyro]

I missed these lil videos where you're just reviewing things and having fun

[@HeroNoodle]

Honestly this feels like it was made for orchestra conductors wanting to produce, that and also anyone who morphs their hands in line with dubstep basses lol

[@PlugInGuruVideo]

At $299 ( Euro / that = $328 USD) they priced this at a place mortals can afford. I can see everybody benefiting from this. Live shows will take on new experiences if the keyboard player controls the synths & the laser lights! #excitedforthisone

[@607]

I can barely believe that this video is only 8 minutes long! Cool device. I do wonder why some features that seem like they could be quite central to me, are not yet developed at launch.

[@alejandroacevedo9409]

Sounds amazing

[@gphiproductions]

Nice demonstration Andrew! Reminds me of a modern version of The Theremin instrument. 👍🏽

[@Maddin-Kambrium]

I NEED TWO OF THEM! ... for obvious reasons :-D

[@WeAreROLI]

Awesome!

[@troytinsley148]

Hey Andrew, this is wild, and I really dig Roli and their innovations. I know it's been a while since I've been paying attention. I know a while back, things weren't looking good your way. I remember you have been losing your hearing over time. I know how it is since I'm almost blind, but not hearing must be incredibly challenging since you're a musician. All this is to say I'm sending some love your way!!!

[@Night-Wolf]

I can only imagine the music pieces people like Jean-Michel Jarre will make with this included in their instrumental arsenal.

[@angelgcastro1]

I am loving this thing...preordered!

[@f3rny_66]

finally almost 10 year old japanese music arcades technology is coming to pro gear

[@AndyChamberlainMusic]

now this is cool very cool

[@Jmjazz4711]

Wow! This definitely gives me Theremin vibes but takes it to a whole new level! I can already envision and hear my compositions utilizing this great technology! Fantastic video, as always!

[@WeAreROLI]

Hey, thank you for the comment! ROLI Airwave isn't just a modern take on the Theremin (though, let's be honest, the Theremin is a timeless classic). With ROLI Vision tracking every unique hand gesture and nuanced movement, you get control over more than volume and pitch. ROLI Airwave will include 100 bespoke Airwave sounds, finely tuned to 5 Dimensions of Air for each gesture you make!

[@petermgruhn]

And yet... it can't even be a Theremin. See the comment from ROLI, "Airwave isn't... [a] Theremin." They KNOW they have quash that before they get sued.

[@bubume8277]

Magical! 🧙‍♂🧙‍♂🧙‍♂ You literally are casting spells with it!

[@jaffasplaffa1]

Pretty impressive tool!

[@VitorGuerreiroVideos]

Super cool use case for Leap Motion, good to know their tech is still out there even if in different products

[@Mikalyn]

Love your work - ever need vocals hit me up :) - also super cool to see you integrating the Roli into the workflow - amazing

[@calebcole7480]

I've always wanted something like this

[@MetalMan1245]

This seems like it would be ungodly expensive and not work very well, but I love the idea of being able to do all that fluid automation, super cool.

[@michaelschnell5633]

Seems like a must have :)

[@bodhilanders2072]

Damn it Andrew, now i have to get it this looks like what the guy who invented the piano pedals would fantasize about before needing to pull himself back to reality

[@Cynshet]

Nice add-on to the Roli keyboards but it would be more interesting, when the Airwave can detect the fingers and you don't need a midi keyboard anymore.

[@rishabhrajan]

So it's basically like a Leap Motion 2 controller but with a giant structure in front and can only be used with Roli controllers?

[@markquavertune2003]

WOW ! I already have equator .I should start to make presets so I can use them when I get a Roli and this very useful device .Exciting musical times ! Thank you Andrew for this good video .Makes me think about doing modular hardware too .

[@rodolfospartos9633]

Wow it's super impressive. As a conductor wiht passion for synths, that's kind of a way to combine both. 😁 Wondering how it could improve even more.

[@THEHAR0LD]

For some reason this makes my think there'll be a prog rock V tuber that uses this technology to accurately track their hands as the 3D model shreds a keyboard, lol.

[@WarrenPostma]

1:14 this is awesome. what a great track

[@andrewhuang]

Thank you! Full thing is on streaming services, Andrew Huang - Burst Generator

[@Arnoudtunes]

Musical notation is going to be a new challenge!

[@EmlynInTheMix]

Can’t wait to try this!!

[@unternatuerlich]

I see a lot of potential in this device.

[@RP14_Music]

i was on the phone with my friend 2 years ago and i was telling him i was going to use the theramin from moog with the midi out to control cc parameters spefically for strings and brass sections. looks like roli has a better way of pullling that off now.

[@nomoresaul]

I saw this exact technology in a video many years ago, but this is the first I’ve seen it applied to anything more than just an experiment. Somehow, I hadn’t imagined it for music, but now I don’t know why I hadn’t.

[@alexross9630]

The tech looks neat, but I have a hard time seeing this as a tool for a mass market (which Roli is always chasing, perhaps to their detriment). Ultimately it's an iteration on the vision of the seaboard, which sought (and arguably succeeded) to put dynamic expression in your hands. The difference here is that where Seaboard felt like a playable instrument, this is a glorified mod wheel. If they can deliver it sustainably, I'm happy for them, but Roli has a history of flying into the sun... And this tech doesn't look cheap to develop. Hope they survive this.

[@SliRSirGames]

Awesome

[@That__Guy]

I really love the idea of exploring soundscapes with this thing, but I could see how it would be an issue for me in terms of performing or recording music with it, because you can't really use your hands for anything while playing with it or you'll accidentally modulate stuff. At least that's what it looks like. But damn, I was already intrigued by ROLI's controllers and I will probably give it a try.

[@TSGEnt]

Kinda a theremin keyboard. What I appreciate about you is you say, "let s try this out" and you actually make a a brilliant piece of music. ( yeah I know it's not real time, at least the full tune, but you take the time to make some music.) Thank You!

[@apiksDen]

I do love the music you do

[@RachelDavis705]

This is actually pretty cool rather than gimmicky

[@YoanThony]

I was expecting an exorbitant price (2500 or more), but it's only 299?? Wow!

[@ToyKeeper]

If they can get it running completely standalone, just generating midi without a computer, it could be this generation's theremin.

[@_o_]

The "under the hood" skeleton display reminds me of magic leap directly, I wonder if they used patents or partnered with them. I did a VR experience back in 2017 using oculus goggles and hand tracking for a virtual synth oscillator, it was a lot of fun! Great to see how the tech has evolved (and wow it's already been 7 years?)

[@TheJumboBurrito]

imogen heap is calling

[@nullbyt]

Got me thinking you could just use an Xbox connect and some remapping magic to do something similar with regular midi devices 😅

[@BinauralBae]

BONE ROLI IS READY

[@jfredd]

the ultimate neurobass performing tool

[@karsinokuuni3208]

biblically accurate theremin

[@darkhawkwingsgt]

That bit at the end with the dubstep beat. Can you turn that into a full song? 😄 also this thing is awesome

[@oceanus123]

Very nice tune !