# VR DAW controller



## Guy Rowland (Mar 27, 2014)

Waiting for a client to arrive... thoughts meandering...

I was trying to imagine a new way of controlling a DAW and - perhaps more importantly - plugins. Or in fact anything on a computer. I guess it could use a Kinect if it were sensitive enough - I don't know much about them. You'd have an area in front of the sensor in which you'd operate, so it wouldn't get set off by every tiny move. Then you'd put your hand in the zone (if you will), and on screen you'd get a virtual hand - sort of translucent edges that means you can still read everything clearly, but it's easy to see where your virtual hand is. You know, like a Klingon cloaking device effect or something cool like that. As you move your whole hand down, it would zoom the display in a little to hone in on where you want - up back out. Then it would be sensitive enough to recognise a pinch - just moving your index finger and thumb down and then in to grab a knob and turn... or a fader or a switch - hopefully with a visual confirmation of a highlight round the knob/fader/switch. It might conceivably be clever enough to work out several fingers doing several things at once by moving the fingers down in relation to the hand.

It would all be in the execution I guess. If it was even remotely a PITA not doing what you'd want it would never catch on. But if it were effortless and intuitive - the illusion of intelligence to interpret your moves in the way we operate analogue kit without the physical objects... I can see it being pretty useful. More appealing than touch screens somehow.

Am I just bored / mad? Is the tech there now to do this? Near future? Never?


----------



## paulmatthew (Mar 27, 2014)

There are several items out there that allow different ways of controlling daw or fx nowadays. I have no experience with them but they look interesting. There's also a handheld device called Orbit from Numark .This is a video for a product called Hot Hand . Have a look:


----------



## Daryl (Mar 27, 2014)

Something useful for more than very broad strokes would take a huge amount of skill to control, not unlike learning to play a instrument, so I doubt there would be many people willing to learn the skill.

D


----------



## tfishbein82 (Mar 27, 2014)

Guy Rowland @ Thu Mar 27 said:


> Waiting for a client to arrive... thoughts meandering...
> 
> I was trying to imagine a new way of controlling a DAW and - perhaps more importantly - plugins. Or in fact anything on a computer. I guess it could use a Kinect if it were sensitive enough - I don't know much about them. You'd have an area in front of the sensor in which you'd operate, so it wouldn't get set off by every tiny move. Then you'd put your hand in the zone (if you will), and on screen you'd get a virtual hand - sort of translucent edges that means you can still read everything clearly, but it's easy to see where your virtual hand is. You know, like a Klingon cloaking device effect or something cool like that. As you move your whole hand down, it would zoom the display in a little to hone in on where you want - up back out. Then it would be sensitive enough to recognise a pinch - just moving your index finger and thumb down and then in to grab a knob and turn... or a fader or a switch - hopefully with a visual confirmation of a highlight round the knob/fader/switch. It might conceivably be clever enough to work out several fingers doing several things at once by moving the fingers down in relation to the hand.
> 
> ...


Leap Motion was designed with the potential to do this. It can detect all 10 fingers independently. I use it to control performance of a VI in 3-dimensions with the GECO app, which doesn't leverage individual fingers, just hands. It's quite good at this. The Leap Motion has the potential for more, but I don't think the technology is quite good enough yet. Plus someone needs to write app(s) to do what you want.

Leap Motion
GECO


----------



## Guy Rowland (Mar 27, 2014)

Paul - yeah I'm sorta aware of some of these, it's not really what I'm talking about. Those are creative new ways to control sound itself, all a bit Jean Michel Jarre (whom I love, btw... well the early funny stuff anyway... no, that's Woody Allen... getting confused now....).

Maybe Leap Motion in time can evolve into something more practical. I'm really not intererested in vague waving about, it would have to be absolutely as good as being able to intelligently read your precise motions to make fine control quickly and efforttlessly, that would require pretty much no learning curve. I think an eye sensor with it would be useful too, so it would follow you as you switch between monitors.

The visual zoom and graphic feedback would be very important - perhaps a subtle magnifying glass effect would work well around your virtual hand, and something concrete to indicate you've locked onto something. If it isn't faster to use than a mouse, its redundant. Mice are actually really good, they zip around super fast and are more practical than most controllers for most tasks. They fall down in certain areas - on knobs specifically - the whole left / right / up down thing doesn't work well for rotary things. Even touch screens aren't too great there - big ungainly swirls rather than the deft real pinch motion. If a sensor could reliably detect your pinch motion, and multiple fingers useful motion like controlling multiple faders at once - that would be cool.

EDIT - I think I'm just losing it while staring a pro tools mixer waiting for clients. I'm now also imagining 3D rendering based on your head movements, so as you look at a mixer (traditionally top down), as your head moves subtly it appears to have depth (like the new IOS7 screens on steroids). Might help with the tactile nature of it. Dammit, then we'd need grown up graphics cards.


----------



## Pegginton (Mar 27, 2014)

Reminds me of this from last year.

http://youtu.be/TjF8JN5aVfc

Still limits you to one hand playing though.


----------



## Hannes_F (Mar 27, 2014)

Daryl @ Thu Mar 27 said:


> Something useful for more than very broad strokes would take a huge amount of skill to control, not unlike learning to play a instrument, so I doubt there would be many people willing to learn the skill.
> 
> D



Not only that, it would be actually be more difficult than an instrument in a way. Because on a physical instrument you have at least some sort of scale or orientation, opposed to these freehand stroke things. I tried 3D gloves, the WII and a 3D mouse - all moot for our purposes.

A graphical tablet is something else.


----------



## Guy Rowland (Mar 27, 2014)

Hannes_F @ Thu Mar 27 said:


> Not only that, it would be actually be more difficult than an instrument in a way. Because on a physical instrument you have at least some sort of scale or orientation, opposed to these freehand stroke things. I tried 3D gloves, the WII and a 3D mouse - all moot for our purposes.
> 
> A graphical tablet is something else.



Ha - the graphical tablet is the most horrific, unintuitive controller I've ever used :D 

I think you and Daryl are missing the point. The entire premise of this idea is that it is completely, seamlessly intuitive, with no user learning curve. If it isn't - it won't work, end of story. If devices until now have been sorely wanting - and I strongly suspect that is exactly the case - the only question worth asking is "is this deficiency inherent in this concept, or is it merely that hardware and software are not sufficiently developed?"

It's a little like the iPhone and the iPod weren't created in a vacuum. Stuff like it existed before, it just didn't work nearly as well. That's how I see all this VR stuff - it's all gimmicky pointless crap until it reaches the point where it's sufficiently advanced to actually do the job it was originally designed to do. The question is - will it ever get there, or is there some inherent flaw in the concept?

Fortunately for me, I know someone whose job it is to design UIs. I'll drop him a line


----------



## Guy Rowland (Apr 12, 2014)

Guy Rowland @ Fri Mar 28 said:


> Fortunately for me, I know someone whose job it is to design UIs. I'll drop him a line



Forgot to reply to this - I did talk to my friend. He thought most likely "not in our lifetime", but he did qualify that with "keep a check on the tech every few years, you never know - big breakthroughs do get made".

Not anticipating anything like my dream being in the Black Friday sales...


----------



## d.healey (Apr 12, 2014)

Wait for the Oculus Rift, combine it with the leap motion and write a GUI, shouldn't be too much trouble for someone with the right know how.


----------

