AR_SituationalAwareness

 

Research  

 

GTI Data   

 

Open databases created and software developed by the GTI and supplemental material to papers.  

 

Databases  


LaSoDa (2021): 60 annotated images from matches in five stadiums with different characteristics and light conditions.

EVENT-CLASS (2021): High-quality 360-degree videos in the context of tele-education.

Parking Lot Occupancy Database (2020)

Nighttime Vehicle Detection database (NVD) (2019)

Hand gesture dataset (2019): Multi-modal Leap Motion dataset for Hand Gesture Recognition.

PIROPO Database (2016): People in Indoor ROoms with Perspective and Omnidirectional cameras.

ViCoCoS-3D (2016): VideoConference Common Scenes in 3D.

LASIESTA database (2016): More than 20 sequences to test moving object detection and tracking algorithms.

Hand gesture database (2015): Hand-gesture database composed by high-resolution color images acquired with the Senz3D sensor.

HRRFaceD database (2014):Face database composed by high resolution images acquired with Microsoft Kinect 2 (second generation).

Lab database (2012): Set of 6 sequences to test moving object detection strategies.

Vehicle image database (2012)More than 7000 images of vehicles and roads.          
 

 

Software  


SETForSeQ (2020): Subjective Evaluation Tool for Foreground Segmentation Quality.

SMV Player for Oculus Rift (2016)


Bag-D3P (2016): 
Face recognition using depth information. 

TSLAB (2015): 
Tool for Semiautomatic LABeling.   
 

   

Supplementary material  


Evaluating the Influence of the HMD, Usability, and Fatigue in 360VR Video Quality Assessments (2020)

Automatic soccer field of play registration (2020)  

Augmented reality tool for the situational awareness improvement of UAV operators (2017)

Detection of static moving objects using multiple nonparametric background-foreground models on a Finite State Machine (2015)

Real-time nonparametric background subtraction with tracking-based foreground update (2015)  

Camera localization using trajectories and maps (2014)

 

                                                                                                                                                                                                                             
 

                                                                   
 
                                                                                                                                                             
 

      

 

 

Augmented Reality Tool for the Situational Awareness Improvement of UAV Operators

Description


This site contains some supplementary material associated to the Augmented Reality tool proposed in [*]:

  • Downloadable software pack.
  • Results.

The work [*] proposes an Augmented Reality (AR) tool for UAV operators. Common Ground Control Stations (GCSs) provide information in separate screens: one presents the video stream while the other displays information about the mission plan and information coming from several sensors. The proposed AR tool avoids the burden of fusing information displayed in those two screens.

The AR system has two functionalities for Medium-Altitude Long-Endurance (MALE) UAVs:

  1. Route orientation: It allows the operator to identify the upcoming waypoints and the path that the UAV is going to follow.
  2. Target identification: It allows a fast target localization, even in the presence of occlusions.

For any question about the article [*] or about the described test data, please contact Susana Ruano at This email address is being protected from spambots. You need JavaScript enabled to view it..

Citation


[*] S. Ruano, C. Cuevas, G. Gallego, and N. Garcí­a, "Augmented reality tool for situational awareness improvement of UAV operators", Sensors, vol. 17, no. 1, article ID 297, 16 pages, 2017 (doi: 10.3390/s17020297).

Software


The software can be downloaded here.

The binary has been compiled for 32-bit Windows 7 and it depends on the following external libraries to be installed:

  • Microsoft Visual Studio 2010
  • OpenSceneGraph 3.4
  • Eigen
  • ffmpeg

Five arguments must be indicated in the execution:

  • [arg0] binary file of the ARTool
  • [arg1] MPEG-2 TS in a file or UDP stream
  • [arg2] xml with mission CRD
  • [arg3] xml with target information
  • [arg4] obj with terrain information

Results


The proposed AR tool has been tested in a GCS demonstrator in AIRBUS facilities, during a mission that takes place in the south of Spain.

The objective of the mission was the identification of several targets that were reported to the operator. The targets chosen for the test were buildings, and the operator had to check if the targets were actually present in the indicated locations or not. This assignment was framed in a reconnaissance procedure. The UAV followed a route that was predefined according to several restrictions (e.g., non-flying zones) during mission planning. The UAV is flown with an automated control system and the operator is responsible for the supervision the flight, the alerts and the payload. The operator can control the camera sensor manually with a joystick.

Several tests were carried out with different operators and two representatives moments of the mission are illustrated below.

Augmented targets:

This video shows a moment of the mission in which the operator is moving the joystict to try to visualize four targets, which correspond to four buildings. The targets are highlighted, allowing the operator to easily identify them. Moreover, the presence of occlusions is taken into account so that the operator can reduce the time to find themand prevent the use of the camera zoomwhen it is not necessary.

Augmented route:

This video illustrates a moment of the mission in which the camera points forward and does not change its position. Therefore, it allows to see the augmented flight route. This example shows that the augmented route benefit the world exploration, since the operator can infer the remaining time to visit a waypoint and which will be the following movements of the UAV.