The answer to the first question is an easier one, thanks to the work the (Line Operations Safety Audit) LOSA Collaborative did in establishing a methodology to analyse the efficacy of the change and in training the LOSA observers to accurately assess crew performance. We had three primary outcomes we wanted to measure during LOSA observations: did the crew properly anticipate the relevant threats present; was the briefing interactive between both pilots? and was the briefing appropriately scaled given the level of complexity, risk, familiarity, proficiency, etc…present at the time?
A graph showing the number of briefing errors compared to the LOSA archive average over a period of time also tells an interesting story.
Desired outcome 1
Anticipation. According to the LOSA Collaborative, properly anticipating a threat is a two-step process: (1) the crew must first identify the threat as a relevant risk to flight safety and then (2) manage the risk the threat presents by applying an effective countermeasure to maintain an adequate margin of safety. Out of all flights observed, 86% of flight crews adequately anticipated each relevant threat that the LOSA observer indicated was present during the time of the briefing.
Desired outcome 2
Interactive. To meet the standard of ‘interactive’, both pilots had to contribute at some point during the briefing process. Of all briefings observed, 82% of them involved both pilots during the discussion.
Desired outcome 3
Scalability. The typical airline briefing for many decades was a one-size-fits-all, one-sided, rote recitation of a list of required items. What we were seeking was to promote a more scaled approach in which the level of detail of the briefing matched the level of complexity and risk that was present at the time of the briefing. While no empirical measure is available for this desired outcome, the LOSA observers did record a wide variety of detail across the data set. There were observations in which there were numerous threats discussed and a lengthy briefing to a small percentage of flights in which there were no relevant threats discussed and an appropriate scaled-down briefing.
We also asked the LOSA observers to make a subjective assessment of the overall briefing they observed using the following scale below:
Roughly 75% of the briefings were in the Good to Outstanding category with the remainder leaving room for improvement.
A graph showing the number of briefing errors compared to the LOSA archive average over a period of time also tells an interesting story.
The increasing trend in briefing errors was effectively reversed, as the airline identified that the old briefing procedure had lost relevance with the crews and a safer, more effective briefing was established.
Dr Klinect, PhD, CEO, LOSA Collaborative had this to say: “In 2014, we presented our report and Alaska responded with FH Revision 12 [the new briefing]. We have never seen such an effective, industry-leading innovation with as many positive outcomes as were revealed in LOSA 2019.”
While the data from Alaska Airlines is very encouraging, given the range of operators that are moving forward with implementing this new briefing style, we thought it would be useful to get some feedback about their experiences so far. It is our hope that, by sharing the experiences of so many different operators, the lessons they have learned from their successes and setbacks during the process may be useful to others. To this end, we sent a survey out to all the airlines that had contacted us requesting information and asked them to answer some questions that would allow us to see how they were progressing. Their responses make up the basis of this article. While we have not been able to include all of the information sent back in the surveys, we have tried to highlight the most useful data.
Outstanding – Followed the threat-forward format, was interactive with both the PF and PM both contributing, and appropriately scaled.
Good – Followed the threat-forward format, was interactive with both the PF and PM both contributing, but the observer felt there were some details missing.
Poor/Marginal – Inverse of Outstanding/Good ratings. Additionally, the crew did not identify some critical/relevant threats that observers believed needed to be discussed given the conditions and environment.
Flickr/Kent Wien
Part 1
Demographics
All respondents worked for commercial operators which were either exclusively short-haul or mixed short and long-haul. The operations were distributed across North America, Europe and Asia and the respondents were mostly in senior management or training positions.
training
Since the publication of the original article, both of us have been contacted by airlines around the world looking for more information about how they could most successfully implement these new techniques in their own operation. So far, 30 different operators have been in touch from a variety of operation types and from many different countries. Among the numerous inquiries we have received, there is one question that many have asked…is it working? By that, they mean, are the pilots performing to the desired standard and is it positively affecting the overall safety at the airline?