Evaluating the Influence of the HMD, Usability, and Fatigue in 360VR Video Quality Assessments

Description

This site contains supplementary material associated to the evaluation of the influence of the HMD, usability, and fatigue in 360VR video quality assessments proposed in [*]. In particular, it contains:

• SRCs details + objective metrics results (PSNR, WS-PSNR, CPP-PSNR, VMAF [1], SSIM, MSSSIM).
• Head tracking data and video quality rates obtained from 48 participants during free-viewing experiments with two HMDs: Samsung GearVR and Lenovo Mirage Solo.
• Presence questionnaire answers, specifically TPI (Lombard et al.) & PQ (Witmer & Singer), obtained from 48 participants.
• Statistical analysis notebook.

The work [*] presents an experiment where video quality and sense of presence are jointly assessed in two of the most popular HMDs.

The hypotheses evaluated in the experiment are:

• The HMD and the order in which conditions are evaluated have influence on: sense of presence (H1), quality (H2), and perceived usability (H3).
• sPQ and TPI provide similar measurements (scores) for spatial presence (H4).

For any question about the article [*], please contact Marta Orduna at This email address is being protected from spambots. You need JavaScript enabled to view it..

Citation

[*] M. Orduna, P. Pérez, C. Díaz, N. García, “Evaluating the Influence of the HMD, Usability, and Fatigue in 360VR Video Quality Assessments'”, IEEE Conf. on Virtual Reality and 3D User Interfaces, IEEE VR 2020, Atlanta (GA), USA, pp. pp. 682-683, 22-26 Mar. 2020.

Test material

We used six representative sources with the characteristics presented in Table 1. The original resolution and framerate were maintained throughout the experiment to preserve the source characteristics. The SRCs were selected with varied features in terms of color, texture, camera motion, and type of content in the scenes, from as many immersive VR video sources in equirectangular format. Here, we present a brief description of each source and the Spatial Information and Temporal Information (SI, TI) indicators as expressed in Recommendation ITU-T P.910, proving the variety of the selected sequences. The clips were of a duration of 30 s without abrupt scene changes and stitching problems to no disturb the evaluation. All sequences were selected with audio to help the user immersion.

Additionally, we present the objective metrics computed on the sequences used in the experiment. Specifically, Peak Signal-to-Noise Ratio (PSNR), Weighted to Spherically PSNR (WS-PSNR), Craster Parabolic Projection PSNR (CPP-PSNR), Video Multimethod Assessment Fusion (VMAF), Structural Similarity Index (SSIM), and Multi-Scale SSIM (MS-SSIM).

Here, you can find the original sources of the videos:

Statistical analysis

RESULTS.csv file first presents the TPI responses and then those of the sPQ. Specifically, items 3, 10, 12, 14, 26, 27, and 30 of the original questionnaire.

The analysis can be downloaded from here.

References

[*] M. Orduna, P. Pérez, C. Díaz, N. Garcí­a, "Evaluating the Influence of the HMD, Usability, and Fatigue in 360VR Video Quality Assessments", IEEE Conf. on Virtual Reality and 3D User Interfaces, IEEE VR 2020, Atlanta (GA), USA, pp. xxx-xxx, 22-26 Mar. 2020.

[1] M. Orduna, C. Díaz, L. Muñoz, P. Pérez, I. Benito, N. García " Video Multimethod Assessment Fusion (VMAF) on 360VR contents", IEEE Trans. Consumer Electronics, vol. 66, no. 1, pp. 22-31, Feb. 2020.