• 6.2020-1873

    Final published version, 4.96 MB, PDF document


A method to automatically identify pilot actions from cockpit camera footage was invented.Although they have long been considered for the enhancement of flight safety, cockpit image recorders have not yet been standard equipment in aircraft cockpits; however, the rules onFlight Data Recorder were changed to include a cockpit image recorder as one of the safety devices, and it is recommended to be installed in small aircraft as a substitute for a FlightData Recorder. With cockpit images becoming available, it would surely be useful for accident analysis as well as for daily flight analysis. Especially for the latter purpose, pilot behavior should be automatically analyzed and classified into specific actions, or procedures. The authors conducted a study to assess the feasibility of automatic detection of pilot actions in the cockpit by a machine learning process. The results show that even with a small amount of training data, the resulting algorithm could identify some typical actions, such as manipulation of the switches on the glare shield, with 80% accuracy. Even when one button is next to the other switch, ’pushing the switch’ and ’Pushing the other button’ were distinguished throughMachine-Learning. Furthermore, it is found that the accuracy is improved up to 90% in the case of using the training data focused on the pilot’s body part rather than on the whole body.In this paper, we report those results.
Original languageEnglish
Title of host publicationAIAA Scitech 2020 Forum
Subtitle of host publication6-10 January 2020, Orlando, FL
PublisherAmerican Institute of Aeronautics and Astronautics Inc. (AIAA)
Number of pages9
ISBN (Electronic)978-1-62410-595-1
Publication statusPublished - 2020
EventAIAA Scitech 2020 Forum - Orlando, United States
Duration: 6 Jan 202010 Jan 2020


ConferenceAIAA Scitech 2020 Forum
CountryUnited States

ID: 68466511