A large portion of today's network traffic consists of streamed video of large variety, such as films, television shows, live-streamed games and recently omnidirectional videos. A common way of delivering video is by using Dynamic Adaptive Streaming over HTTP (DASH), or recently with encrypted HTTPS. Encrypted video streams disable the use of Quality of Service (QoS) systems that rely on knowledge of application-dependent data, such as video resolution and bit-rate. This could make it difficult for a party providing bandwidth to efficiently allocate resources and estimate customer satisfaction. An application-independent way of measuring video stream quality could be of interest for such a party. In this paper, we investigate encrypted streaming of omni-directional video via YouTube to a smartphone in a Google Cardboard VR-headset. We monitored such sessions, delivered via both WiFi and mobile networks, at different times of day, implying different levels of congestion, and characterised the network traffic by using the Coefficient of Throughput Variation (CoTV) as statistic. We observe that this statistic shows to be able to indicate whether a stream is stable or unstable, in terms of potential video playback freezes, when the DASH delivery strategy is used.

Virtual Reality (VR) is becoming increasingly popular, and wireless cable replacements unleash the user of VR Head Mounted Displays (HMD) from the rendering desktop computer. However, the price to pay for additional freedom of movement is a higher sensitivity of the wireless solution to temporal disturbances of both video frame and input traffic delivery, as compared to its wired counterpart. This paper reports on the development of a test-bed to be used for studying temporal delivery issues of both video frames and input traffic in a wireless VR environment, here using TPCAST with a HTC Vive headset. We provide a solution for monitoring and recording of traces of (1) video frame freezes as observed on the wireless VR headset, and (2) input traffic from the headset and hand controls to the rendering computer. So far, the test-bed illustrates the resilience of the underlying WirelesslID technology and TCP connections that carry the input traffic, and will be used in future studies of Quality of Experience (QoE) in wireless desktop VR.

Remote rendering for VR is a technology that enables high-quality VR on low-powered devices. This is realized by offloading heavy computation and rendering to high-powered servers that stream VR as video to the clients. This article focuses on one specific issue in remote rendering when imperfect frame timing between client and server may cause recurring frame drops. We propose a system design that executes synchronously and eliminates the aforementioned problem. The design is presented, and an implementation is tested using various networks and hardware. The design cannot drop frames due to synchronization issues but may on the other hand stall if temporal disturbances occur, e.g., due to network delay spikes or loss. However, experiments confirm that such events can remain rare given an appropriate environment. For example, remote rendering on an intranet at 90 fps with a server located approximately 50 km away yielded just 0.002% stalled frames while rendering with extra latency corresponding to the duration of exactly one frame (11.1 ms at 90 fps). In a LAN without extra latency setting, i.e., with latency equal to locally rendered VR, 0.009% stalls were observed while using a wired Ethernet connection and 0.058% stalls when using 5 GHz wireless IEEE 802.11 ac. © 2021 Viktor Kelkkanen et al.

This paper shows the impact of bitrate settings on objective quality measures when streaming non-panoramic remote-rendered Virtual Reality (VR) images. Non-panoramic here refers to the images that are rendered and sent across the network, they only cover the viewport of each eye, respectively.

To determine the required bitrate of remote rendering for VR, we use a server that renders a 3D-scene, encodes the resulting images using the NVENC H.264 codec and transmits them to the client across a network. The client decodes the images and displays them in the VR headset. Objective full-reference quality measures are taken by comparing the image before encoding on the server to the same image after it has been decoded on the client. By altering the average bitrate setting of the encoder, we obtain objective quality scores as functions of bitrates. Furthermore, we study the impact of headset rotation speeds, since this will also have a large effect on image quality.

We determine an upper and lower bitrate limit based on headset rotation speeds. The lower limit is based on a speed close to the average human peak head-movement speed, 360°s. The upper limit is based on maximal peaks of 1080°s. Depending on the expected rotation speeds of the specific application, we determine that a total of 20--38Mbps should be used at resolution 2160×1200@90,fps, and 22--42Mbps at 2560×1440@60,fps. The recommendations are given with the assumption that the image is split in two and streamed in parallel, since this is how the tested prototype operates.

Rendering VR-content generally requires large image resolutions. This is both due to the display being positioned close to the eyes of the user and to the super-sampling typically used in VR. Due to the requirements of low latency and large resolutions in VR, remote rendering can be difficult to support at sufficient speeds in this medium.In this paper, we propose a method that can reduce the required resolution of non-panoramic VR images from a codec perspective. Because VR images are viewed close-up from within a headset with specific lenses, there are regions of the images that will remain unseen by the user. This unseen area is referred to as the Hidden-Area Mesh (HAM) and makes up 19% of the screen on the HTC Vive VR headset as one example. By remapping the image in a specific manner, we can cut out the HAM, reduce the resolution by the size of the mesh and thus reduce the amount of data that needs to be processed by encoder and decoder. Results from a prototype remote renderer show that by using the proposed Hidden-Area Mesh Remapping (HAMR), an implementation-dependent speed-up of 10-13% in encoding, 17-18% in decoding and 7-11% in total can be achieved while the negative impact on objective image quality in terms of SSIM and VMAF remains small. © 2021 IEEE.

Remotely rendered Virtual Reality (VR) typically has to rely upon less capable and less reliable communication channels as compared to locally rendered VR. Such communication channels pose the risk of impairing the VR experience with temporary disturbances in form of frame stalls that would not be present in local VR. However, it is not obvious to which extent the durations of temporary stalls affect the Quality of Experience (QoE) in interactive 6-degrees-of-freedom (DOF) VR, and neither to which extent present reprojection/warping techniques mitigate any adverse effects of such stalls on user perception.

In this paper, subjective experiments were conducted with N =29 users to quantify the relationship between QoE and single, isolated stall durations in interactive 6-DOF remote VR when stalls either freeze the frame in place, or when frames are reprojected by the VR driver. Results indicate that given a 90 fps headset and a simple task that requires moderate head-movement; (1) short, isolated stalls of up to 4 frames go unnoticed on average, no matter whether reprojection is used or not, (2) the number of perceived stalls when using reprojection is the same or lower than the reference (no stall) in the range 1-4 frame stalls, (3) reprojection entails significantly higher user ratings for stalls ranging from 8 frames and beyond, (4) the improvement of using reprojection over freezes in terms of making more stalling events imperceptible is highest in the range 8-16 frames. © 2022 IEEE.

All virtual reality (VR) systems have some inherent hand-controller latency even when operated locally. In remotely rendered VR, additional latency may be added due to the remote transmission of data, commonly conducted through shared low-capacity channels. Increased latency will negatively affect the performance of the human VR operator, but the level of detriment depends on the given task. This work quantifies the relations between aiming accuracy and hand-controller latency, virtual target speed, and the predictability of the target motion. The tested context involves a target that changes direction multiple times while moving in straight lines. The main conclusions are, given the tested context, first, that the predictability of target motion becomes significantly more important as latency and target speed increase. A significant difference in accuracy is generally observed at latencies beyond approximately 130 ms and at target speeds beyond approximately 3.5 degrees/s. Second, latency starts to significantly impact accuracy at roughly 90 ms and approximately 3.5 degrees/s if the target motion cannot be predicted. If it can, the numbers are approximately 130 ms and 12.7 degrees/s. Finally, reaction times are on average 190-200 ms when the target motion changes to a new and unpredictable direction.