FOR IMMEDIATE RELEASE
February 1, 2025
Aviation AI Testing Framework Now Available for Industry Contributions
LONDON, UK - Airside Labs has released the initial version of its aviation-specific AI benchmark tool, Pre-Flight, on GitHub today. The open-source project aims to create a comprehensive testing framework for validating AI models in aviation applications, with a call for contributions from aviation professionals worldwide.
Following its announcement at the Royal Aeronautical Society event in November 2024, Pre-Flight has been developed to address the unique challenges of implementing AI in aviation contexts. The benchmark contains hundreds of aviation-specific questions designed to test whether AI systems truly understand aviation concepts or are merely generating plausible-sounding but potentially unsafe responses.
"Today marks an important milestone for aviation AI safety as we open Pre-Flight for community contributions," said Alexis Brooker, founder of Airside Labs. "By making this framework accessible to the entire industry, we're fostering a collaborative approach to ensuring AI systems can be properly evaluated before deployment in aviation environments."
The Pre-Flight repository includes testing frameworks for both commercial and open-source AI models, allowing organisations to assess their systems against aviation-specific scenarios covering regulations, operational procedures, safety protocols, and technical knowledge. The tests are designed to expose common weaknesses in AI reasoning when applied to complex aviation problems.
Airside Labs is actively seeking contributions from aviation professionals including pilots, air traffic controllers, flight dispatchers, aviation engineers, and safety specialists who can help expand the benchmark with their domain expertise.
"The strength of Pre-Flight will ultimately come from the diversity of aviation knowledge reflected in its test scenarios," Brooker noted. "We're inviting aviation professionals to contribute questions, scenarios, and edge cases that reflect the complexity of real-world aviation operations."
Contributors can submit new test cases, improve existing ones, or help develop advanced evaluation metrics through the GitHub repository. Airside Labs will maintain a private test set in addition to the public repository to ensure the benchmark remains an effective evaluation tool even as AI models improve.
Organisations interested in using Pre-Flight to evaluate their aviation AI systems or professionals looking to contribute can access the data set at Hugging Face, access a deployment helper repository at github or learn more at www.airsidelabs.com.
About Airside Labs
Airside Labs is a specialised software, data, and AI experimentation startup focused on the aviation sector. The company conducts innovative research and development projects, validates AI performance, and assists clients in achieving product-market fit more efficiently. Through a range of professional services including data analysis, software development, and AI model creation and testing, Airside Labs helps aviation-related businesses accelerate their innovation processes and optimise operations through data-driven insights and cutting-edge technology solutions.
Contact:
Alexis Brooker
Founder, Airside Labs
