LASIESTA

 

Research  

 

LASIESTA (Labeled and Annotated Sequences for Integral Evaluation of SegmenTation Algorithms)

  

Description

LASIESTA is composed by many real indoor and outdoor sequences organized in diferent categories, each of one covering a specific challenge in moving object detection strategies.

In contrast to other databases, it is fully annotated at both pixel-level and object-level (all the sequences have been labeled through the Tool for Semiautomatic Labeling TSLAB). Therefore, it is suitable not only for strategies exclusively focused on the detection of moving objects but also for those that integrate tracking algorithms in their detection approaches.

Additionally, it contains sequences recorded with static and moving cameras and it also provides information about the moving objects remaining temporally static.

 

 

 

Each sequence contains the following information (where "Id" is the sequence identifier):

- Original video: A folder named "Id" that contains a 24bpp BMP file for each frame in the original vídeo.

- Labeled images: A folder named "Id_GT" that contains a 24png BMP file with the ground-truth (labels) corresponding to each frame in the original video. Taking into account that the sequences of this database contain a maximum of three moving objects, the values of these images have been set as follows:

- Black pixels (0,0,0): Background.

- Red pixels (255,0,0): Moving object with label 1 along the sequence.

- Green pixels (0,255,0): Moving object with label 2 along the sequence.

- Yellow pixels (255,255,0): Moving object with label 3 along the sequence.

- White pixels (255,255,255): Moving objects remaining static.

- Gray pixels (128,128,128): Uncertainty pixels.

- Static moving objects: A .xml file with information regarding to the set of images in which a moving object remains static. This file appears only in the sequences with static moving objects.

The uncertainty pixels are those around the contour of the moving objects for which it is not possible to determine if they belong to a moving object or to the background.

For questions about this database, please contact Carlos Cuevas at This email address is being protected from spambots. You need JavaScript enabled to view it..

 

License: CC BY-SA 4.0

This dataset is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0). By accessing or using this dataset, you agree to abide by the terms of this license.

Under this license, you are free to:

- Share: Copy and redistribute the material in any medium or format.
- Adapt: Remix, transform, and build upon the material for any purpose, even commercially.

However, if this dataset is used as part of research, publications, or any other form of academic work, the following conditions must be met:

- Attribution: Proper citation and acknowledgment must be given to the original dataset creators. This includes citing the original paper or publication associated with this dataset. Please refer to the citation information provided below.

Citation Information:

C. Cuevas, E. M. Yáñez, and N. García, “Labeled dataset for integral evaluation of moving object detection algorithms: LASIESTA", Computer Vision and Image Understanding, vol. 152, pp. 103-117, 2016. (doi:10.1016/j.cviu.2016.08.005).

Your cooperation in providing proper attribution is appreciated and contributes to the integrity of academic research and the acknowledgment of contributors.

For any inquiries regarding the use or attribution of this dataset, please contact This email address is being protected from spambots. You need JavaScript enabled to view it..

 

Indoor sequences (I):

Simple sequences (SI): Sequences not containing camouflage, occlusions, illumination changes, modified background, camera motion, or bootstraping.

Id: I_SI_01

Description: Three people cross a room, walking perpendicularly to the the optical axis of the camera.

Num. of frames: 300

Num. of moving objects: 3

Main characteristics: Ssmooth shadows.

Download

 

 

Id: I_SI_02

Description: One person crosses a corridor going towards the camera.

Num. of frames: 300

Num. of moving objects: 1

Main characteristics: Smooth shadows.

Download

 

Camouflage (CA): Sequences with moving objects remaining temporally static on background regions with similar color.

Id: I_CA_01

Description: A person crosses a room and remains static for a few seconds in front of a door with similar color to his clothing.

Num. of frames: 350

Num. of moving objects: 1

Main characteristics: Temporally static moving object, smooth shadows, camouflage.

Download

 

Id: I_CA_02

Description: A person appears and stands in front of a wall with similar color to his T-shirt. A plant is constantly moving in the background.

Num. of frames: 525

Num. of moving objects: 1

Main characteristics: Temporally static moving object, camouflage, dynamic background.

Download

 

 

Occlusions (OC):  Sequences containing totally or partially ocludded moving objects.

Id: I_OC_01

Description: A person crosses a room and passes behind a large column.

Num. of frames: 250

Num. of moving objects: 1

Main characteristics: Moderate shadows, total occlusion.

Download

 

 

Id: I_OC_02

Description: One person goes down some stairs and walks behind a railing.

Num. of frames: 250

Num. of moving objects: 1

Main characteristics: Smooth shadows, partial occlusion.

Download

 

 

Ilumination changes (IL): Sequences with global ilumination changes.

Id: I_IL_01

Description: A person who is going through a room turns on the lights.

Num. of frames: 300

Num. of moving objects: 1

Main characteristics: Smooth shadows, illumination change.

Download

 

Id: I_IL_02

Description: One person walks towards a window and opens the blinds. Then the person walks out of the scene.

Num. of frames: 525

Num. of moving objects: 1

Main characteristics: Hard shadows, illumination change, permanent changes in the background.

Download

 

 

Modified background (MB): Sequences showing situations in wich background elements are subtracted or where some objects are abandoned.

Id: I_MB_01

Description: One person enters a room with a bag on his shoulder, leaves the bag on the ground, and goes out of the scene.

Num. of frames: 450

Num. of moving objects: 2

Main characteristics: Smooth shadows, abandoned object.

Download

 

Id: I_MB_02

Description: One person enters a room, picks up a bag that is on the ground, and goes out.

Num. of frames: 350

Num. of moving objects: 1

Main characteristics: Moderate shadows, permanent changes in the background.

Download

 

 

Bootstrap (BS): Sequences containing moving objects from the first frame.

Id: I_BS_01

Description: Two people shake their hands in a corridor and walk in opposite directions, entering two different rooms.

Num. of frames: 275

Num. of moving objects: 2

Main characteristics: Bootstrap, moderate shadows.

Download

 

Id: I_BS_02

Description: A person stands shortly in front of a sign before walking out of the scene.

Num. of frames: 275

Num. of moving objects: 1

Main characteristics: Bootstrap, moderate shadows.

Download

 

 

Moving camera (MC): Sequences recorded with non-completely static cameras (handy cameras or pan/tilt motion).

 

Id: I_MC_01

Description: One person climbs some steps in a hall with columns. The camera sweeps the scene.

Num. of frames: 300

Num. of moving objects: 1

Main characteristics: Camera motion, smooth shadows.

Download

 

Id: I_MC_02

Description: A person crosses a room. The camera is not completely static (soft jitter).

Num. of frames: 250

Num. of moving objects: 1

Main characteristics: Camera motion, moderate shadows.

Download

 

 

Simulated motion (SM): Set of sequences simulating different types and intensities of camera motion.

Id: I_SM_01

Motion type: Pan

Motion intensity: Low

Num. of frames: 300

Num. of moving objects: 3

Download

 

Id: I_SM_02

Motion type: Pan

Motion intensity: Medium

Num. of frames: 300

Num. of moving objects: 3

Download

 

Id: I_SM_03

Motion type: Pan

Motion intensity: High

Num. of frames: 300

Num. of moving objects: 3

Download

 

Id: I_SM_04

Motion type: Jitter / rotation

Motion intensity: Low / low

Num. of frames: 300

Num. of moving objects: 3

Download

 

Id: I_SM_05

Motion type: Jitter / rotation

Motion intensity: Low / medium

Num. of frames: 300

Num. of moving objects: 3

Download

 

Id: I_SM_06

Motion type: Jitter / rotation

Motion intensity: Low / high

Num. of frames: 300

Num. of moving objects: 3

Download

 

Id: I_SM_07

Motion type: Jitter / rotation

Motion intensity: Medium / low

Num. of frames: 300

Num. of moving objects: 3

Download

 

Id: I_SM_08

Motion type: Jitter / rotation

Motion intensity: Medium / medium

Num. of frames: 300

Num. of moving objects: 3

Download

 

Id: I_SM_09

Motion type: Jitter / rotation

Motion intensity: Medium / high

Num. of frames: 300

Num. of moving objects: 3

Download

 

Id: I_SM_10

Motion type: Jitter / rotation

Motion intensity: High / low

Num. of frames: 300

Num. of moving objects: 3

Download

 

Id: I_SM_11

Motion type: Jitter / rotation

Motion intensity: High / medium

Num. of frames: 300

Num. of moving objects: 3

Download

 

Id: I_SM_12

Motion type: Jitter / rotation

Motion intensity: High / high

Num. of frames: 300

Num. of moving objects: 3

Download

 

 

Outdoor sequences (O):

 

Cloudy conditions (CL): Sequences that have cloudy conditions.

 

 

Id: O_CL_01

Description: A car crosses a parking and passes behind a lamppost. The wind moves the vegetation and clouds are reflected in parked cars' windows.

Num. of frames: 225

Num. of moving objects: 1

Main characteristics: Dynamic background, smooth shadows, partial occlusion.

Download

 

Id: O_CL_02

Description: Two people go up a street, one immediately after another. The wind moves the vegetation

Num. of frames: 425

Num. of moving objects: 2

Main characteristics: Dynamic background, smooth shadows.

Download

 

Rainy conditions (RA): Sequences presenting dynamic background due to falling rain.

 

Id: O_RA_01

Description: In rainy conditions, a person walks in a parking. After a while, a parked car starts moving and leaves the scene.

Num. of frames: 1400

Num. of moving objects: 2

Main characteristics: Dynamic background, moderate shadows, rain, partial occlusion, permanent changes in the background.

Download

 

Id: O_RA_02
Description: In rainy conditions, two people (one after another) go walking in front of a garden.
Num. of frames: 375
Num. of moving objects: 2
Main characteristics: Dynamic background, moderate shadows, rain.

Download

 

Snowy conditions (SN): Sequences presenting dynamic background due to falling snow.

 

Id: O_SN_01

Description: It's snowing. A car crosses a parking and disappears under a gate.

Num. of frames: 500

Num. of moving objects: 1

Main characteristics: Dynamic background, snow, smooth shadows.

Download

 

Id: O_SN_02
Description: It is snowing. A car traveling very slowly appears in the bottom right corner of the scene.
Num. of frames: 850
Num. of moving objects: 1
Main characteristics: Dynamic background, snow, smooth shadows, partial occlusion.

Download

 

Sunny conditions (SU): Sequences recorded in sunny scenarios and showing hard sahdows of moving objects. 

 

Id: O_SU_01

Description: A person crosses a street in a sunny area and a car moves in the background of the scene. The silhouette of the person is reflected in a puddle.

Num. of frames: 250

Num. of moving objects: 2

Main characteristics: Dynamic background, camouflage, hard shadows.

Download

 

Id: O_SU_02

Description: A person crosses a street in a sunny area. Later, another person crosses in opposite direction in a shaded area.

Num. of frames: 400

Num. of moving objects: 2

Main characteristics: Dynamic background, hard shadows.

Download

 

Moving camera (MC): Sequences recorded with non-completely static cameras (handy cameras or pan/tilt motion).

 

Id: O_MC_01

Description: A person crosses a parking and the camera sweeps the scene.

Num. of frames: 425

Num. of moving objects: 1

Main characteristics: Smooth shadows, camera motion.

Download

 

Id: O_MC_02

Description: A person crosses a parking. The camera is not completely static (soft jitter).

Num. of frames: 175

Num. of moving objects: 1

Main characteristics: Dynamic background, smooth shadows, camera motion.

Download

 

Simulated motion (SM): Set of sequences simulating different types and intensities of camera motion.

 

Id: O_SM_01

Motion type: Pan ant tilt

Motion intensity: Low

Num. of frames: 425

Num. of moving objects: 2

Download

 

Id: O_SM_02

Motion type: Pan ant tilt

Motion intensity: Medium

Num. of frames: 425

Num. of moving objects: 2

Download

 

Id: O_SM_03

Motion type: Pan ant tilt

Motion intensity: High

Num. of frames: 425

Num. of moving objects: 2

Download

 

Id: O_SM_04

Motion type: Jitter / rotation

Motion intensity: Low / low

Num. of frames: 425

Num. of moving objects: 2

Download

 

Id: O_SM_05

Motion type: Jitter / rotation

Motion intensity: Low / medium

Num. of frames: 425

Num. of moving objects: 2

Download

 

Id: O_SM_06

Motion type: Jitter / rotation

Motion intensity: Low / high

Num. of frames: 425

Num. of moving objects: 2

Download

 

Id: O_SM_07

Motion type: Jitter / rotation

Motion intensity: Medium / low

Num. of frames: 425

Num. of moving objects: 2

Download

 

Id: O_SM_08

Motion type: Jitter / rotation

Motion intensity: Medium / medium

Num. of frames: 425

Num. of moving objects: 2

Download

 

Id: O_SM_09

Motion type: Jitter / rotation

Motion intensity: Medium / high

Num. of frames: 425

Num. of moving objects: 2

Download

 

Id: O_SM_10

Motion type: Jitter / rotation

Motion intensity: High / low

Num. of frames: 425

Num. of moving objects: 2

Download

 

Id: O_SM_11

Motion type: Jitter / rotation

Motion intensity: High / medium

Num. of frames: 425

Num. of moving objects: 2

Download

 

Id: O_SM_12

Motion type: Jitter / rotation

Motion intensity: High / high

Num. of frames: 425

Num. of moving objects: 2

Download

 

 

 

Results

The following table shows the F-score (harmonic mean of the recall and the precision) values obtained with some outstanding moving object detection strategies on the sequences of LASIESTA database. Algorithm ranks are given by the numbers in brackets. The last column contains the average for all the tests.

The evaluation of the sequences has been carried out without training period. Additionally, all the algorithms have been run with a single set of parameters throughout the sequences. The pixels labeled as part of static objects have been interpreted as uncertainty zones (i.e. the results obtained for these pixels have not been taken into account to compute the F-score values).

 

References

 

C. R. Wren, A. Azarbayejani, T. Darrell, and A. P. Pentland, “Pfinder: Real-time tracking of the human body,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 780–785, 1997.

C. Stauffer and W. E. L. Grimson, “Learning patterns of activity using real-time tracking,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 747–757, 2000.

Z. Zivkovic and F. van der Heijden, “Efficient adaptive density estimation per image pixel for the task of background subtraction,” Pattern Recognition Letters, vol. 27, no. 7, pp. 773–780, 2006.

L. Maddalena and A. Petrosino, “A self-organizing approach to background subtraction for visual surveillance applications,” IEEE Transactions on Image Processing, vol. 17, no. 7, pp. 1168–1177, 2008.

L. Maddalena and A. Petrosino, “The sobs algorithm: what are the limits?,” IEEE International Conference on Computer Vision and Pattern Recognition, pp. 21–26, 2012.

C. Cuevas and N. García, “Improved background modeling for realtime spatio-temporal non-parametric moving object detection strategies,” Image and Vision Computing, vol. 31, no. 9, pp. 616–630, 2013.

T. S. Haines and T. Xiang, “Background subtraction with dirichletprocess mixture models,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 4, pp. 670–683, 2014.

D. Berjón, C. Cuevas, F. Morán, and N. García, "Real-time nonparametric background subtraction with tracking-based foreground update", Pattern Recognition, vol. 74, pp. 156-170, 2018 (doi: 10.1016/j.patcog.2017.09.009).