PRESS RELEASE: Airside Labs Launches Pre-Flight AI Benchmark on GitHub

Aviation AI Testing Framework Now Available for Industry Contributions

LONDON, UK - Airside Labs has released the initial version of its aviation-specific AI benchmark tool, Pre-Flight, on GitHub today. The open-source project aims to create a comprehensive testing framework for validating AI models in aviation applications, with a call for contributions from aviation professionals worldwide.

Following its announcement at the Royal Aeronautical Society event in November 2024, Pre-Flight has been developed to address the unique challenges of implementing AI in aviation contexts. The benchmark contains hundreds of aviation-specific questions designed to test whether AI systems truly understand aviation concepts or are merely generating plausible-sounding but potentially unsafe responses.

"Today marks an important milestone for aviation AI safety as we open Pre-Flight for community contributions," said Alexis Brooker, founder of Airside Labs. "By making this framework accessible to the entire industry, we're fostering a collaborative approach to ensuring AI systems can be properly evaluated before deployment in aviation environments."

The Pre-Flight repository includes testing frameworks for both commercial and open-source AI models, allowing organisations to assess their systems against aviation-specific scenarios covering regulations, operational procedures, safety protocols, and technical knowledge. The tests are designed to expose common weaknesses in AI reasoning when applied to complex aviation problems.

Airside Labs is actively seeking contributions from aviation professionals including pilots, air traffic controllers, flight dispatchers, aviation engineers, and safety specialists who can help expand the benchmark with their domain expertise.

"The strength of Pre-Flight will ultimately come from the diversity of aviation knowledge reflected in its test scenarios," Brooker noted. "We're inviting aviation professionals to contribute questions, scenarios, and edge cases that reflect the complexity of real-world aviation operations."

Contributors can submit new test cases, improve existing ones, or help develop advanced evaluation metrics through the GitHub repository. Airside Labs will maintain a private test set in addition to the public repository to ensure the benchmark remains an effective evaluation tool even as AI models improve.

Organisations interested in using Pre-Flight to evaluate their aviation AI systems or professionals looking to contribute can access the data set at Hugging Face, access a deployment helper repository at github or learn more at www.airsidelabs.com.