Working on multiple monitors in VR

[mothershout]

Crossposting from the long thread on voice synthesis, but it’s more of a workflow post.

I thought others might be interested to see a little of what it's like to work on multiple monitors in a VR headset.

These two shots show three monitors on a Macbook running Logic Pro. Because the field of view is wide, I had to take two screenshots; the first screenshot shows the Mixer window on the left monitor and part of Logic's Tracks window on the middle monitor. The second screenshot shows the Tracks window again and a voice-synthesis plugin open on the right monitor.

I find this a nice way of working for some parts of my workflow; when working on getting a vocal line right, it's great to have a huge monitor to see the UI in detail. When I'm mixing, I find it very helpful to have the Mixer and Tracks both visible at once. Often I have a standalone piano VI running on one monitor so I can experiment with harmonies.

This is all happening in a Quest 3 VR headset, using the Immersed app to let me connect to my Macbook and have three independent virtual monitors. I use Immersed also because you can see I created a large "passthrough" space so that I can see my piano and Apple keyboards and my aftertouch/drum pads.

For audio, I use my regular Beyerdynamic headphones over the headset, because I don't want any audio lag and I do want maximum sound quality.

Click to read more...

 

[MartinH.]

I recommend you align the virtual distance to your virtual screens to the physical focal distance of your headset. If you don't, you risk messing with your ability to focus your eyes on the correct distance on a neurological level, because you would be decoupling vergence from focal depth in an unnatural way.

 

[mothershout]

I recommend you align the virtual distance to your virtual screens to the physical focal distance of your headset. If you don't, you risk messing with your ability to focus your eyes on the correct distance on a neurological level, because you would be decoupling vergence from focal depth in an unnatural way.

Hm. What do you mean by physical focal distance of your headset?

 

[osterdamus]

This is inspiring, thanks for sharing

Do you feel any fatigue from the headset during your sessions? How much CPU do the three virtual screens require from your system? Any issue working with the headset in combination with your cans?

 

[MartinH.]

Hm. What do you mean by physical focal distance of your headset?

I mean the distance your eyes focus on to see the pixel-grid of the display in focus. I can't find definitive numbers for quest 3. For the quest 2 people say it's about 1.3 meters. Looking for it I found a reddit thread calling this issue "vergence accommodation conflict" and linking this talk at this timestamp:



It will not solve the problem, but it might help with understanding the problem and why placing the screens at the right virtual distance is your only solution on a device without varifocal lenses.

 

[mothershout]

It will not solve the problem, but it might help with understanding the problem and why placing the screens at the right virtual distance is your only solution on a device without varifocal lenses.

Thanks for the information, though I’m not sure what the problem is that it won’t solve. I’ve set the correct inter-pupil distance carefully, and I did a number of tests with the virtual monitors at various distances, and I feel no eye-strain, nor do I notice any after-effects (I used to work with multiple real monitors, and had some training in setting up an ergonomic workstation, including how to notice effects on vision).

 

[mothershout]

Do you feel any fatigue from the headset during your sessions? How much CPU do the three virtual screens require from your system? Any issue working with the headset in combination with your cans?

I don’t notice any extra fatigue; in fact, it’s better than working on the Macbook alone, because I put the virtual monitors higher up in my field of view, compared with the laptop screen, and that definitely improves my posture and I no longer get the mild shoulder and neck stiffness I used to.

That said, I would not want to do this on the Quest 2, which has lower resolution displays (bearing in mind the useful comments from @MartinH.) The Quest 3 is the first headset I’ve found that has a clear enough virtual display that I don’t have a problem reading text on a virtual screen. Also, I can’t afford an Apple Vision Pro to compare!

I don’t notice any significant CPU load on the Macbook. Activity Monitor shows a small load, but my main CPU meter doesn’t move. This is on an M1 Macbook, which is definitely a powerful beast. One of my tests was to play a large Logic project with many tracks using dynamic EQ and modelling plugins, and there were no CPU issues doing that in VR.

Also no issues using cans and the headset - the Quest 3 strap and “earpieces” are pretty thin, and I found my cans fit nicely over them.

 

[MartinH.]

Thanks for the information, though I’m not sure what the problem is that it won’t solve. I’ve set the correct inter-pupil distance carefully, and I did a number of tests with the virtual monitors at various distances, and I feel no eye-strain, nor do I notice any after-effects (I used to work with multiple real monitors, and had some training in setting up an ergonomic workstation, including how to notice effects on vision).

In reality your eyes' vergence and focus are always linked. You look at something very close to you (e.g. your hand 0.3 m from your eyes), the eyes rotate inwards slightly, and the lenses focuses on the close distance.
In VR vergence is dynamic but focus stays static. So when you look at something very close in VR, your eyes still rotate inwards, but your lenses either stay focused on the ~1.3 m focal distance of the display, or your vision turns blurry if the lenses actually adjust to the short viewing distance that the vergence tells them they should be focussing on.

If you use VR a lot at virtual viewing distances where vergence and focus are not aligned in a natural way, I think this can potentially untrain your brain from keeping vergence and focus in sync and the results are problems with focusing your eyes on the correct distance outside of VR. I have felt glimpses of that when I used VR semi-regularly and in the past I have looked for more info on that, but found very little. Possible explanations for that lack of reports are a) people don't use VR enough to do damage, b) it's not a problem for most people regardless of how much they use VR, c) people are having these problems but their eyesight was bad enough to begin with, that they don't notice or they don't make the connection. I remember reading about one case of a VR game developer that was quite severe and an eye doctor that was very concerned about the VR tech in general. Personally I think if you take care to have your virtual screens at the right distance, it's safe-ish, but if you sit virtual 30cm away from your virtual screens for 8 hours a day I expect you'll over time be able to give us a more detailed description of how these focus issues feel when not using VR. That's simply what I'm trying to prevent.
Aside from that it should just feel way better for your eyes to have this aligned correctly.

 

[mothershout]

Personally I think if you take care to have your virtual screens at the right distance, it's safe-ish, but if you sit virtual 30cm away from your virtual screens for 8 hours a day I expect you'll over time be able to give us a more detailed description of how these focus issues feel when not using VR. That's simply what I'm trying to prevent.
Aside from that it should just feel way better for your eyes to have this aligned correctly.

Ah, now I see what you mean, and thanks for the clarification; today I learned .

Yes, I have been working with displays that are about 1.3-1.5m away (judging by eye because of course there's no scale) and had found that to be the sweet spot for me, for clarity of vision and eye fatigue. It's good to better understand why this works.