EVEREST study: Evaluation of the performance of mountain guided transport safety video systems (EVEREST)
The EVEREST project was launched in 2016 to contribute to the development of systems using automatic video image analysis to assist operations during boarding (detection of retention bars that have not been lowered or wrong positioning of passengers on chairlifts) and disembarking (detection of passengers on a chairlift beyond the disembarkation point) phases of chairlifts.
For boarding, the aim is to assist the supervisor by detecting any abnormal situation that could lead to a fall from height as early as possible. These kinds of falls are often associated with improper boarding and can lead to serious injuries.
The project was initiated by STRMTG and carried out in partnership with Université Gustave Eiffel (ex-IFSTTAR). The steering committee also included ropeway manufacturers and operators.
The EVEREST project had several objectives:
• raise awareness about the field by providing annotated images to developers of real-time image analysis algorithms;
• develop a tool for evaluating intelligent video surveillance systems (IVSS) for monitoring the boarding and disembarkation of chairlifts;
• organise a national workshop dedicated to these systems for this issue.
To this end, videos of boarding and disembarkation at several installations were acquired. The various images were then annotated whenever they contained an abnormal situation such as “retention bar not lowered during boarding”, “wrong positioning of a passenger during boarding” or “presence of a passenger after the normal disembarking point”.
Some of the annotated images were then provided to the scientific community as part of a challenge, with the aim of facilitating the development of image analysis algorithms dedicated to these issues. The challenge brought together two participants: Bluecime and SIA Partners.
After the challenge, a collaborative workshop was held at STRMTG from 9 to 12 May 2022, to evaluate the participants’ algorithms on images from unpublished video databases.
Evaluation procedures had been defined by the EVEREST project steering committee based on quantification of the sensitivity and accuracy of each of the algorithms developed, the number of true positives (correct detections of an annotated event), false positives (false alarms) and false negatives (missed detections).
To round off the workshop, a half-day feedback session for professionals (members of the EVEREST steering committee, other manufacturers and operators, challenge participants) was organised at STRMTG on the morning of Thursday 12 May 2022, attended by around twenty participants.
This feedback session provided an opportunity to present the work of the challenge participants, the results of the algorithms in real-life situations, and the prospects and challenges for analysis techniques via presentations and discussions.
In summary, the evaluation showed that while there was room for improvement with the solutions developed by the participants, given the time devoted to their development and the tuning/optimisation of the algorithms, the results were encouraging.
It should also be noted that the video databases available, both for algorithm development and evaluation, remain limited in terms of duration and representativeness (photos taken on a total of 5 cameras, over a total of 5 days).
For the future, STRMTG hopes that the work undertaken will be continued by the two challenge participants, and even extended to other anomalies to be detected, potentially through partnerships with operators and/or manufacturers, so that operating assistance tools can eventually be implemented in the field.
In addition, the various “test” video databases remain available to potential future developers, and the project website is still active at this stage to provide an overview (requests should be sent to STRMTG, via the “Contact us” section of the website).