Field Notes: Streaming VR to Wireless Headsets Using NVIDIA CloudXR
AWS Architecture BlogField Notes: Streaming VR to Wireless Headsets Using NVIDIA CloudXRby William Cannady | on16 FEB 2021| inAmazon EC2,AR & VR,Architecture,Technical How-to|Permalink|ShareIt’s exciting to see many consumer-grade virtual reality (VR) hardware options, but setting up hardware can be cumbersome, expensive and complicated. Wired headsets require high-powered graphics workstations, and a solution to prevent you from tripping over the wires. Many room-scale headsets require two external peripherals (or ‘light towers’) to be installed so the headset can position itself in a room. These setups can take days to tune, and need resetting if the light towers are moved.With the release of theOculus Quest, users of virtual reality were delighted with a wireless, room-scale headset with dual hand tracking. They could enjoy VR without worrying about light towers or a high-powered graphics workstation. However, because the Quest was battery powered, it used an inherently low-powered central processing unit (CPU) and graphics processing unit (GPU). As a result, VR content had to be simplified to run on the Quest. This prevented customers from using the Quest for the most demanding graphics experiences, such as Computer Aided Design (CAD) review, or playing games such as Half-Life: Alyx.Customers were faced with a difficult choice: expensive, complicated setups, or reduced-fidelity experiences.In this blog post, we show you how to stream a full-fidelity VR experience from a computer on AWS to a wireless headset such as the Quest.Overview of architectureArchitecture diagram updated March 12, 2021NVIDIA CloudXRtakes NVIDIA’s experience in GPU encoding and decoding, and streams pixels to a remotely connected VR headset. By doing this, the rendering and compute requirements of visually intensive applications take place on a remote server instead of a local headset. This makes mobile headsets work with any application, regardless of their visual complexity and density.To provide global scalability,NVIDIA announced the CloudXR platform will be available on G4 and P3 EC2 instances. It provides the following benefits:At a global scale, customers can stream remote AR/VR experiences from Regions that are close them.It enables centrally managed and deployed software experiences on Amazon Elastic Compute Cloud (Amazon EC2) instances. Previously these required physical transportation and implementation of devices and server hardware.Lastly, IT administrators can now centrally manage content that may be sensitive or require frequent changes.WalkthroughUsing CloudXR on AWS requires EC2 instances with NVIDIA GPUs (that is, theP3orG4instance types) running within your virtual private cloud (VPC). The instance must be network accessible to a remote CloudXR client running on a VR headset. Connections are 1:1, meaning that each CloudXR client is connected to a dedicated EC2 instance. If needs require multiple CloudXR clients, you can deploy multiple EC2 instances.Note that the process outlined is accurate as of January 2021. CloudXR, andX Reality(XR) overall, is rapidly changing. Consult thelatest information about CloudXR from NVIDIA. Using CloudXR within your AWS account requires you setup P3 or G4 EC2 instances, as you would within an Amazon VPC. You must also add a security group that allows the ports required for CloudXR communication. These specific ports can be found in theCloudXR documentation, available from NVIDIA.We have created a CloudFormation template that deploys an EC2 instance with CloudXR configured for reference, linked in the prerequisites. Because it makes reference to a private AMI, it must be shared with your account in order to deploy successfully. If you’re interested in using this template, contact your AWS Account team.PrerequisitesThe following steps describe how to configure the EC2 instance manually. CloudXR streaming requires using a connection other than Windows RDP to connect to the remote EC2 instance. We use NICE DCV, which is provided at no cost to EC2 instances for remote connectivity.For this walkthrough, you should have the following prerequisites:AnAWS accountTheNVIDIA CloudXR SDKfrom NVIDIAPermissions to create G4 or P3 EC2 instancesA VR headsetDeploy CloudXR Server onto Amazon EC2It’s important to note the steps outlined are for configuring a G4 instance. If you’d prefer to use a P3 instance, manually deploy your P3 instance and install NICE DCV as described in thedocumentation.Log into your AWS Account and navigate to theAWS Marketplaceto install an EC2 instance with NICE DCV configured.Create a new security group during deployment that matches the CloudXR port settings and apply it to your instance. Consult theCloudXR documentationfor the latest port settings.Wait 5 minutes for everything to initialize properly. Make note of the issues public IP address (or attach an Elastic IP address to the instance).Navigate tohttps://<IP-OF-INSTANCE>:8443to connect to NICE DCV web-browser client. b.Use the credentials created during EC2 initialization to Log inNICE DCV login screen on Web Browser Client5. Once logged into your EC2 instance, install SteamVR and CloudXR onto the remote EC2 instance. SteamVR is used as an OpenVR/XR proxy between your VR application and CloudXR. CloudXR is used to stream the SteamVR experience to a remote CloudXR Client.Install SteamVRInstall CloudXR6. Verify installation of the CloudXR plugin into SteamVR by navigating to the Manage Add-ons page within the Advanced Settings option. Make sure it lists CloudXRRemoteHMD as an addon and is set toON.Verification of CloudXR Installation7. Add an allow entry to the Windows Firewall Entry forVRServer.exe. This allows SteamVR to use the CloudXR to stream properly. By default this file is located at %ProgramFiles% (x86)\Steam\steamapps\common\SteamVR\bin\win64\vrserver.exe\8. Install a CloudXR client onto your VR headset. If using an android-powered headset (that is, the Oculus Quest), you can use the sample APK within the CloudXR SDK9. SelectFinish.Connect to your CloudXR Server and start streaming1. Launch SteamVR on your remote EC2 instance by logging into your Steam account or configuring a no-login link following the Installation/use of SteamVR in an environment without internet accessinstructions.2. When loaded, it will report a headset cannot be detected. This is OK.SteamVR will display Headset not Detected—this is OK.3. Within your Client headset, load the CloudXR Client application you recently installed.4. Once connected, the headset will start display the SteamVR “void”. You also should see a view of your headset if SteamVR mirroring is enabled. The status box on the SteamVR server application will show a headset and 2 controllers attached as well.SteamVR “Void” and Headset Connected icons5. Congratulations. You’re now connected to an AWS EC2 instance using NVIDIA CloudXR! Any VR application you now run on the EC2 server that uses OpenVR will be streamed to your VR headset!Cleaning upEC2 instances are billed only when they’re being used. You’ll want to make sure to stop your instance or shut it down when you are finished with your session. Terminating your instance is not necessary. You will also want to review your specific network requirements to connect to AWS and discuss this with relevant internal stake-owners or teams. Even though the EC2 instance has an AWS security group applied to it, production networks are complex and unique.ConclusionIn this blog post, we showed how to stream a full-fidelity VR experience from a computer on AWS to a wireless headset. Having the ability to remotely connect to GPU-powered servers to run graphic workloads is not necessarily new, but connecting to a remote server with a VR headset and having full interactivity certainly is. With this architecture, you realize the benefits of CloudXR combined with the agility and scalability available on AWS. It becomes less challenging to manage content played on VR headsets because content doesn’t reside on the VR headset—it lives on the EC2 server.Deploying to any AWS region where GPU instances are available allows you to offer CloudXR to your users at global scale.  As networks get faster and closer through services likeAWS OutpostsandAWS Wavelength, remote VR work will become possible for more customers. We’re excited to see what new workloads come next as this way of working grows.Field Notes provides hands-on technical guidance from AWS Solutions Architects, consultants, and technical account managers, based on their experiences in the field solving real-world business problems for customers.Leo ChanLeo Chan is the Worldwide Tech Lead for Spatial Computing at AWS. He loves working at the intersection of Art and Technology and lives with his family in a rain forest just off the coast of Vancouver, Canada.TAGS:Field NotesWilliam CannadyWilliam Cannady is a Sr. Specialist Solutions Architect focusing on AR/VR. He enjoys discovering new technologies and working on bleeding-edge solutions. When he is not tinkering with something, you can probably find him wandering on trails or camping in the woods.
It’s exciting to see many consumer-grade virtual reality (VR) hardware options, but setting up hardware can be cumbersome, expensive and complicated. Wired headsets require high-powered graphics workstations, and a solution to prevent you from tripping over the wires. Many room-scale headsets require two external peripherals (or ‘light towers’) to be installed so the headset can […]
William Cannady
Amazon EC2 AR & VR Architecture Technical How-to