LASIESTA (Labeled and Annotated Sequences for Integral Evaluation of SegmenTation Algorithms)
Description
LASIESTA is composed by many real indoor and outdoor sequences organized in diferent categories, each of one covering a specific challenge in moving object detection strategies.
In contrast to other databases, it is fully annotated at both pixel-level and object-level (all the sequences have been labeled through the Tool for Semiautomatic Labeling TSLAB). Therefore, it is suitable not only for strategies exclusively focused on the detection of moving objects but also for those that integrate tracking algorithms in their detection approaches.
Additionally, it contains sequences recorded with static and moving cameras and it also provides information about the moving objects remaining temporally static.
Each sequence contains the following information (where "Id" is the sequence identifier):
- Original video: A folder named "Id" that contains a 24bpp BMP file for each frame in the original vídeo.
- Labeled images: A folder named "Id_GT" that contains a 24png BMP file with the ground-truth (labels) corresponding to each frame in the original video. Taking into account that the sequences of this database contain a maximum of three moving objects, the values of these images have been set as follows:
- Black pixels (0,0,0): Background.
- Red pixels (255,0,0): Moving object with label 1 along the sequence.
- Green pixels (0,255,0): Moving object with label 2 along the sequence.
- Yellow pixels (255,255,0): Moving object with label 3 along the sequence.
- White pixels (255,255,255): Moving objects remaining static.
- Gray pixels (128,128,128): Uncertainty pixels.
- Static moving objects: A .xml file with information regarding to the set of images in which a moving object remains static. This file appears only in the sequences with static moving objects.
The uncertainty pixels are those around the contour of the moving objects for which it is not possible to determine if they belong to a moving object or to the background.
For questions about this database, please contact Carlos Cuevas at ccr@gti.ssr.upm.es.
License: CC BY-SA 4.0
This dataset is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0). By accessing or using this dataset, you agree to abide by the terms of this license.
Under this license, you are free to:
- Share: Copy and redistribute the material in any medium or format.
- Adapt: Remix, transform, and build upon the material for any purpose, even commercially.
However, if this dataset is used as part of research, publications, or any other form of academic work, the following conditions must be met:
- Attribution: Proper citation and acknowledgment must be given to the original dataset creators. This includes citing the original paper or publication associated with this dataset. Please refer to the citation information provided below.
Citation Information:
C. Cuevas, E. M. Yáñez, and N. García, “Labeled dataset for integral evaluation of moving object detection algorithms: LASIESTA", Computer Vision and Image Understanding, vol. 152, pp. 103-117, 2016. (doi:10.1016/j.cviu.2016.08.005).
Your cooperation in providing proper attribution is appreciated and contributes to the integrity of academic research and the acknowledgment of contributors.
For any inquiries regarding the use or attribution of this dataset, please contact ccr@gti.ssr.upm.es.
Indoor sequences (I):
Simple sequences (SI): Sequences not containing camouflage, occlusions, illumination changes, modified background, camera motion, or bootstraping.
Id: I_SI_01 Description: Three people cross a room, walking perpendicularly to the the optical axis of the camera. Num. of frames: 300 Num. of moving objects: 3 Main characteristics: Ssmooth shadows. |
 |
Id: I_SI_02 Description: One person crosses a corridor going towards the camera. Num. of frames: 300 Num. of moving objects: 1 Main characteristics: Smooth shadows. |
 |
Camouflage (CA): Sequences with moving objects remaining temporally static on background regions with similar color.
Id: I_CA_01 Description: A person crosses a room and remains static for a few seconds in front of a door with similar color to his clothing. Num. of frames: 350 Num. of moving objects: 1 Main characteristics: Temporally static moving object, smooth shadows, camouflage. |
 |
Id: I_CA_02 Description: A person appears and stands in front of a wall with similar color to his T-shirt. A plant is constantly moving in the background. Num. of frames: 525 Num. of moving objects: 1 Main characteristics: Temporally static moving object, camouflage, dynamic background. |
 |
Occlusions (OC): Sequences containing totally or partially ocludded moving objects.
Id: I_OC_01 Description: A person crosses a room and passes behind a large column. Num. of frames: 250 Num. of moving objects: 1 Main characteristics: Moderate shadows, total occlusion. |
 |
|
Id: I_OC_02 Description: One person goes down some stairs and walks behind a railing. Num. of frames: 250 Num. of moving objects: 1 Main characteristics: Smooth shadows, partial occlusion. |
 |
Ilumination changes (IL): Sequences with global ilumination changes.
Id: I_IL_01 Description: A person who is going through a room turns on the lights. Num. of frames: 300 Num. of moving objects: 1 Main characteristics: Smooth shadows, illumination change. |
 |
Id: I_IL_02 Description: One person walks towards a window and opens the blinds. Then the person walks out of the scene. Num. of frames: 525 Num. of moving objects: 1 Main characteristics: Hard shadows, illumination change, permanent changes in the background. |
 |
Modified background (MB): Sequences showing situations in wich background elements are subtracted or where some objects are abandoned.
Id: I_MB_01 Description: One person enters a room with a bag on his shoulder, leaves the bag on the ground, and goes out of the scene. Num. of frames: 450 Num. of moving objects: 2 Main characteristics: Smooth shadows, abandoned object. |
 |
Id: I_MB_02 Description: One person enters a room, picks up a bag that is on the ground, and goes out. Num. of frames: 350 Num. of moving objects: 1 Main characteristics: Moderate shadows, permanent changes in the background. |
 |
Bootstrap (BS): Sequences containing moving objects from the first frame.
Id: I_BS_01 Description: Two people shake their hands in a corridor and walk in opposite directions, entering two different rooms. Num. of frames: 275 Num. of moving objects: 2 Main characteristics: Bootstrap, moderate shadows. |
 |
Id: I_BS_02 Description: A person stands shortly in front of a sign before walking out of the scene. Num. of frames: 275 Num. of moving objects: 1 Main characteristics: Bootstrap, moderate shadows. |
 |
Moving camera (MC): Sequences recorded with non-completely static cameras (handy cameras or pan/tilt motion).
Id: I_MC_01 Description: One person climbs some steps in a hall with columns. The camera sweeps the scene. Num. of frames: 300 Num. of moving objects: 1 Main characteristics: Camera motion, smooth shadows. |
 |
Id: I_MC_02 Description: A person crosses a room. The camera is not completely static (soft jitter). Num. of frames: 250 Num. of moving objects: 1 Main characteristics: Camera motion, moderate shadows. |
 |
Simulated motion (SM): Set of sequences simulating different types and intensities of camera motion.
Id: I_SM_01 Motion type: Pan Motion intensity: Low Num. of frames: 300 Num. of moving objects: 3 |
 |
Id: I_SM_02 Motion type: Pan Motion intensity: Medium Num. of frames: 300 Num. of moving objects: 3 |
 |
Id: I_SM_03 Motion type: Pan Motion intensity: High Num. of frames: 300 Num. of moving objects: 3 |
 |
Id: I_SM_04 Motion type: Jitter / rotation Motion intensity: Low / low Num. of frames: 300 Num. of moving objects: 3 |
 |
Id: I_SM_05 Motion type: Jitter / rotation Motion intensity: Low / medium Num. of frames: 300 Num. of moving objects: 3 |
 |
Id: I_SM_06 Motion type: Jitter / rotation Motion intensity: Low / high Num. of frames: 300 Num. of moving objects: 3 |
 |
Id: I_SM_07 Motion type: Jitter / rotation Motion intensity: Medium / low Num. of frames: 300 Num. of moving objects: 3 |
 |
Id: I_SM_08 Motion type: Jitter / rotation Motion intensity: Medium / medium Num. of frames: 300 Num. of moving objects: 3 |
 |
Id: I_SM_09 Motion type: Jitter / rotation Motion intensity: Medium / high Num. of frames: 300 Num. of moving objects: 3 |
 |
Id: I_SM_10 Motion type: Jitter / rotation Motion intensity: High / low Num. of frames: 300 Num. of moving objects: 3 |
 |
Id: I_SM_11 Motion type: Jitter / rotation Motion intensity: High / medium Num. of frames: 300 Num. of moving objects: 3 |
 |
Id: I_SM_12 Motion type: Jitter / rotation Motion intensity: High / high Num. of frames: 300 Num. of moving objects: 3 |
 |
Outdoor sequences (O):
Cloudy conditions (CL): Sequences that have cloudy conditions.
Id: O_CL_01 Description: A car crosses a parking and passes behind a lamppost. The wind moves the vegetation and clouds are reflected in parked cars' windows. Num. of frames: 225 Num. of moving objects: 1 Main characteristics: Dynamic background, smooth shadows, partial occlusion. |
 |
Id: O_CL_02 Description: Two people go up a street, one immediately after another. The wind moves the vegetation Num. of frames: 425 Num. of moving objects: 2 Main characteristics: Dynamic background, smooth shadows. |
 |
Rainy conditions (RA): Sequences presenting dynamic background due to falling rain.
Id: O_RA_01 Description: In rainy conditions, a person walks in a parking. After a while, a parked car starts moving and leaves the scene. Num. of frames: 1400 Num. of moving objects: 2 Main characteristics: Dynamic background, moderate shadows, rain, partial occlusion, permanent changes in the background. |
 |
|
Id: O_RA_02 |
 |
Snowy conditions (SN): Sequences presenting dynamic background due to falling snow.
Id: O_SN_01 Description: It's snowing. A car crosses a parking and disappears under a gate. Num. of frames: 500 Num. of moving objects: 1 Main characteristics: Dynamic background, snow, smooth shadows. |
 |
|
Id: O_SN_02 |
 |
Sunny conditions (SU): Sequences recorded in sunny scenarios and showing hard sahdows of moving objects.
Id: O_SU_01 Description: A person crosses a street in a sunny area and a car moves in the background of the scene. The silhouette of the person is reflected in a puddle. Num. of frames: 250 Num. of moving objects: 2 Main characteristics: Dynamic background, camouflage, hard shadows. |
 |
Id: O_SU_02 Description: A person crosses a street in a sunny area. Later, another person crosses in opposite direction in a shaded area. Num. of frames: 400 Num. of moving objects: 2 Main characteristics: Dynamic background, hard shadows. |
 |
Moving camera (MC): Sequences recorded with non-completely static cameras (handy cameras or pan/tilt motion).
Id: O_MC_01 Description: A person crosses a parking and the camera sweeps the scene. Num. of frames: 425 Num. of moving objects: 1 Main characteristics: Smooth shadows, camera motion. |
 |
Id: O_MC_02 Description: A person crosses a parking. The camera is not completely static (soft jitter). Num. of frames: 175 Num. of moving objects: 1 Main characteristics: Dynamic background, smooth shadows, camera motion. |
 |
Simulated motion (SM): Set of sequences simulating different types and intensities of camera motion.
Id: O_SM_01 Motion type: Pan ant tilt Motion intensity: Low Num. of frames: 425 Num. of moving objects: 2 |
 |
Id: O_SM_02 Motion type: Pan ant tilt Motion intensity: Medium Num. of frames: 425 Num. of moving objects: 2 |
 |
Id: O_SM_03 Motion type: Pan ant tilt Motion intensity: High Num. of frames: 425 Num. of moving objects: 2 |
 |
Id: O_SM_04 Motion type: Jitter / rotation Motion intensity: Low / low Num. of frames: 425 Num. of moving objects: 2 |
 |
Id: O_SM_05 Motion type: Jitter / rotation Motion intensity: Low / medium Num. of frames: 425 Num. of moving objects: 2 |
 |
Id: O_SM_06 Motion type: Jitter / rotation Motion intensity: Low / high Num. of frames: 425 Num. of moving objects: 2 |
 |
Id: O_SM_07 Motion type: Jitter / rotation Motion intensity: Medium / low Num. of frames: 425 Num. of moving objects: 2 |
 |
Id: O_SM_08 Motion type: Jitter / rotation Motion intensity: Medium / medium Num. of frames: 425 Num. of moving objects: 2 |
 |
Id: O_SM_09 Motion type: Jitter / rotation Motion intensity: Medium / high Num. of frames: 425 Num. of moving objects: 2 |
 |
Id: O_SM_10 Motion type: Jitter / rotation Motion intensity: High / low Num. of frames: 425 Num. of moving objects: 2 |
 |
Id: O_SM_11 Motion type: Jitter / rotation Motion intensity: High / medium Num. of frames: 425 Num. of moving objects: 2 |
 |
Id: O_SM_12 Motion type: Jitter / rotation Motion intensity: High / high Num. of frames: 425 Num. of moving objects: 2 |
 |
Results
The following table shows the F-score (harmonic mean of the recall and the precision) values obtained with some outstanding moving object detection strategies on the sequences of LASIESTA database. Algorithm ranks are given by the numbers in brackets. The last column contains the average for all the tests.
The evaluation of the sequences has been carried out without training period. Additionally, all the algorithms have been run with a single set of parameters throughout the sequences. The pixels labeled as part of static objects have been interpreted as uncertainty zones (i.e. the results obtained for these pixels have not been taken into account to compute the F-score values).
References
C. R. Wren, A. Azarbayejani, T. Darrell, and A. P. Pentland, “Pfinder: Real-time tracking of the human body,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 780–785, 1997.
C. Stauffer and W. E. L. Grimson, “Learning patterns of activity using real-time tracking,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 747–757, 2000.
Z. Zivkovic and F. van der Heijden, “Efficient adaptive density estimation per image pixel for the task of background subtraction,” Pattern Recognition Letters, vol. 27, no. 7, pp. 773–780, 2006.
L. Maddalena and A. Petrosino, “A self-organizing approach to background subtraction for visual surveillance applications,” IEEE Transactions on Image Processing, vol. 17, no. 7, pp. 1168–1177, 2008.
L. Maddalena and A. Petrosino, “The sobs algorithm: what are the limits?,” IEEE International Conference on Computer Vision and Pattern Recognition, pp. 21–26, 2012.
C. Cuevas and N. García, “Improved background modeling for realtime spatio-temporal non-parametric moving object detection strategies,” Image and Vision Computing, vol. 31, no. 9, pp. 616–630, 2013.
T. S. Haines and T. Xiang, “Background subtraction with dirichletprocess mixture models,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 4, pp. 670–683, 2014.
D. Berjón, C. Cuevas, F. Morán, and N. García, "Real-time nonparametric background subtraction with tracking-based foreground update", Pattern Recognition, vol. 74, pp. 156-170, 2018 (doi: 10.1016/j.patcog.2017.09.009).
