Audio In, MIDI Out - Avrosh Kumar & Chris Latina - ADCx SF

Join Us For ADC23 - London - 13-15 November 2023
More Info: audio.dev/
@audiodevcon
Audio In, MIDI Out - Avrosh Kumar & Chris Latina - ADCx SF
Audio plugins enable musicians to apply effects to audio tracks, synthesize sounds through virtual instruments, and even transform MIDI events using MIDI effects. Mixing and mastering engineers also make use of analyzers and mastering plugins that can extract instantaneous and statistical information about the audio track to help guide the mixing process. As plugin developers, we can tinker with audio and MIDI data in several ways to augment musicians’ creative workflows.
In this talk, we will explore a special case of such plugin implementations and generate MIDI output using audio as input. This can be achieved by performing real-time content analysis on an audio track, and produce control signals as MIDI events that represent high-level features extracted from audio. These MIDI events can then be used to trigger or modulate external parameters and engines.
Audio-to-MIDI capability is already natively supported by some DAWs. They convert the melodic contents of audio to its MIDI representation and is even used by some plugins for real-time voice-to-pitch conversion. MIDI is not only a format to represent melodies and harmonies but, is excellent for producing annotated events and control signals using program and control change (CC) messages. We will leverage this in our Audio-to-MIDI plugins to potentailly create controlled chaos (think Buchla).
Slides: data.audio.dev...
_
Avrosh Kumar
Speech Research Engineer @ Pindrop
Plugin Developer, Music Tech Enthusiast
Chris Latina
As a sonic explorer melding hybrid hardware and software mediums, Christopher Latina works at the crossroads of engineering, design and music composition to create immersive environments.
Edited by Digital Medium Ltd - online.digital-medium.co.uk
_
Organized and produced by JUCE: juce.com/
_
Special thanks to the ADC Team:
Sophie Carus
Derek Heimlich
Andrew Kirk
Bobby Lombardi
Tom Poole
Ralph Richbourg
Jim Roper
Jonathan Roper
#audiodevcon #audiodev #midi

[@Starovdenis]

Very interesting project! Would like to see other examples of such plugin use, but with tonal instruments, not percussion.

[@tritonedelta3464]

Sorry, I've used all three tools mentioned @9:57 and they are far from useful tools for actual musicians. The concepts are of interest, but none of these deliver on what on paper sounds great. These are at most, novelty items for curious kids/teens. Take the Dubler for example. Even after extensive tweaking and testing, the tracking produces a very high error rate for rhythmic events. Not only is it difficult to make even a simple stream of 16th notes to a) pick up every event and b) play it close enough to the intended timing, it also has a good 50% error rate for timbral classification. It takes me less time to simply notate the intended beat in a notation program or midi piano roll than it does to edit the midi it spits out.