FALEB

FALEB

A Multimodal Dataset for image processing using body worn cameras

Eurecom

Description

Body worn cameras (BWCs) have become more and more popular over the last decade. They are becoming one of the essential tools for law enforcement officers to carry with them for surveillance purposes.

Generally, videos captured by BWCs are used a posteriori through visual inspection in case of major problems between police officers and citizens.

FALEB is a dataset created by use of a body worn camera and an additional sensor for processing of videos and images captured by a BWC and help the law officers in proper evaluation of the footage collected.

Below figure shows the camera and watch used for dataset collection.

camerawatch

FALEB

FALEB is a multimodal dataset that contains data for the activities of face recognition, action recognition, license plate recognition, and egocentric views of different users, all done with a body worn camera.

  1. The first part contains 485 videos from 97 subjects for each environment for facial recognition: indoor, outdoor, and dark.

  2. The second part of the dataset contains 99 annotated videos of actions specific to body worn cameras, along with the metadata such as GPS position, and heart rate of the user.

  3. 18 videos are present from different parking lots for research in the area of license plate recognition for identifying the angle and time that the officer should focus on recognizing a license plate from a busy road or a parking space.

  4. We also provide videos with egocentric vision of 23 subjects to study and investigate whether it is possible to identify the user of the camera by observing patterns in the way he/she walks or runs.

Body Worn Camera

Body-worn cameras are small, portable recording devices typically worn by individuals, such as law enforcement officers, security personnel, or other professionals, to capture audio and video footage of their interactions and activities. These cameras are designed to be attached to the user’s clothing or equipment, providing a first-person perspective of events as they unfold.

Recording setup

The recording took place over different sessions spread across a week in a controlled environment. The dataset is recorded using a camera and a watch. We use Cammpro I826 GPS Body Camera and Garmin vivoactive 5 to record events. All the recordings were done with a video resolution of 2304x1296 pixels at 30 fps for all the activities. The camera was fixed on the middle of the chest of the user (for every activity) and the distance between the user and the subject is kept between 5-6 feet.

  • Face recognition: The indoor environment was well-lit with a uniform background and lighting conditions, and the dark environment was in the same place with the lights switched off. For the outdoor environment, it was recorded in natural sunlight conditions with varying intensities of light. The recorded videos for indoor and outdoor environments lie in the visible spectrum. The recording in the dark environment was done using the infrared feature of the camera, which produced near infrared (NIR) spectrum videos.
  • Action recognition: The action recognition activity was shot in an outdoor setting. Garmin vivoactive 5 was used as an additional sensor for this activity so that we also have the GPS data and heart rate information of the user.
  • License plate recognition: The recording was done in 18 different parking lots of UPNM for license plate detection and recognition.
  • Egocentric vision: Different subjects are provided with the camera, where they have to fix the camera in the middle of the chest and take a walk so as to capture the patterns of their movements as seen through the first-person perspective of the subject.

ACQUISITION PROCESS

  • Face recognition: We recorded 5 videos per subject, each showing them talking for 10-15 seconds. These videos specified the emotions of neutral, happy, angry, sad, and ended with a neutral expression. We record the subject from the head to the torso. These recordings captured the facial expressions/emotions, speech, hand gestures, and head movements of each subject.

  • Figure below shows samples from randomly selected subjects in different environments.

    FR envs
  • Action Recognition: It was divided into 2 scenarios. In the first scenario, we include the actions of walking, talking, showing hands, sitting, going forward and backward, standing, pushing, and running away. The second scenario has the same actions as the first one with some additional actions (an arrest is made instead of the subject running away). So, the additional actions comprise of hands behind the head, turning around, sitting inside the car, and opening and closing the car doors.

  • Figure below shows some examples of actions from randomly selected subjects.

    S1 actions
    S2 actions
  • License plate recognition: The user takes a normal walk in the parking lot and records the plates from different angles.

  • Egocentric vision: The subjects recorded two 8-minute sequences in which they walked from point A to B inside the campus and then followed the same path back (B to A) with a slow jog.

  • Figure below shows some examples of recording in a parking lot, and the egocentric vision of some random subjects when walking / jogging across the 2 endpoints.

    LPR/EV

Download

A download link for the dataset compressed and a password for decrypting the compressed FALEB ZIP files will be provided after receiving the duly signed license agreement. Please fill in the license agreement and send a scanned copy by e-mail at faleb@eurecom.fr

Reference

TODO

Contact

support

If you have any question or request regarding the FALEB Dataset, please contact Prof. Jean-Luc DUGELAY via jld@eurecom.fr