
Every episode, in your inbox
One email per Travel Tech Podcast episode: the conversation, the key takeaways and the use cases worth building in aviation and travel AI.
Zero-Days, Superintelligence, and the Collapse of Software Assumptions
AI is rapidly changing the economics of software: code is cheaper to generate than ever, but significantly harder to reason about, validate, and secure. As systems become more automated, the real constraint is no longer building functionality — it's maintaining confidence in what those systems will actually do once deployed.
Show Notes
AI is rapidly changing the economics of software: code is cheaper to generate than ever, but significantly harder to reason about, validate, and secure. As systems become more automated, the real constraint is no longer building functionality, it's maintaining confidence in what those systems will actually do once deployed. To unpack this shift, Alex Brooker is joined by Jen Reid-Schram, an AI practitioner and former VP of Technology with deep roots in QA, engineering leadership, and executive transformation. Jen brings a systems-level view of how quality thinking evolved inside engineering teams — and why it may need to re-emerge in a new form as AI reshapes how software is produced. She's joined by Oli Deakin, former CTO of Snowflake Software and ex-technology leader at Cirium, who brings hands-on experience building and operating complex technical systems in aviation and enterprise environments.
Read the takeaways
Zero-Days, Superintelligence, and the Collapse of Software Assumptions
Software development is entering a phase where the cost of generating code has collapsed, but the cost of verifying it has not.
Read the newsletter issueFull TranscriptClick to expand (5,544 words approx.)
Alex Brooker: Hi everybody, Alex Brooker here. Welcome to the Travel Tech podcast. This is the podcast where we bring travel, aviation, and AI together and have those hallway track conversations that you otherwise wouldn't hear. And it's AI that we're going to focus on for the most part this week, particularly because we've had the release of the Claude Mythos preview model and the quite scary announcements around thousands of zero day vulnerabilities potentially being discovered across operating systems and browsers and including Project Glass Wing as well.
So I'm absolutely delighted this week to be joined by Jen Reid-Schram. She's had a fantastic career in tech, starting on the QA side and leading up to senior executive leadership in some of the biggest tech companies going. Now, Jen is a self-described AI geek and humanist. I'm really happy that she joined us for this really important discussion.
I'm also joined by Oliver Deakin who was CTO of Snowflake Software back in the day but also was a senior tech leader at Syrium and is now an independent board adviser and AI enthusiast. I hope you enjoy the conversation. Please like and subscribe if you enjoy this content. I particularly love receiving feedback. So, please comment or reach out to me directly if you have suggestions about future episodes or constructive comments about this one. I really hope you enjoy the conversation and yeah, let's get to it.
All right. Hi everybody. Welcome to the Travel Tech podcast and this week I'm delighted to welcome Jen Reid-Schram to our little panel session. Welcome, Jen.
Jen Reid-Schram: Hi. How's it going?
Alex Brooker: Very good. Very good. Looking forward to getting into the discussion. And this week I wondered if we could start by talking about actually something you brought up in a recent LinkedIn post I thought was absolutely fascinating which was the role of the QA, the quality assurance engineer and that perspective on software development and that perspective on life perhaps. Are these people born or made and all those things in between? So maybe give us an outline of your view of QA over the years and your experience from the software development side.
Jen Reid-Schram: Yeah. So I got my start in technology within the QA field and I have experience in breaking down the author intent into what is actually received by the reader and then interpreting that and that's all you do in English literature, right? Is you interpret text and so QA is really similar to that. You take what was intended by the developer or by the product team or whatever and you break that down into how that's received by the audience, by the user and figure out, you know, how it might be broken, how the user might interpret that product. That's what I got a lot of jobs based on that. It was pretty compelling case.
But what I found is that people who are really good at QA have this what we call a quality mindset, right? It's this ability to both understand the technology and also understand the user intent, a non-technical user intent and see how that breaks down, how that story breaks down in the middle.
And what we used to say in technology is that everything should shift left. So quality needs to shift left which means shift to being earlier in the development cycle. So this old world of developers would just write a bunch of code and then throw it over the wall to QA and we would test in a vacuum and then at the end of that whole development process would you fix all the bugs? We understood that to be an inefficient process because of all the time and effort that had gone into the development cycle. You didn't want to just do all that testing at the end of the cycle. And so if we could shift that process left, shift that process to be earlier in the development cycle then it would reduce the likelihood of bugs and you would end up with better quality software built much more cheaply.
What that ended up resulting in is a bunch of quality engineers that felt like actually in the process of shifting left, we should actually just have quality engineers writing the code. And then they would write automation or they would start writing unit tests. And then it became well do we actually need a separate role for this? Maybe just engineers should be quality engineers and development engineers and they should just write higher quality code with test cases embedded.
And so over time you saw this erosion of the quality engineering discipline and a merging of it into software engineering. And I was the key proponent of that. And in fact the reason behind that LinkedIn article was that I was talking to a former colleague of mine and I was giving him a hard time for still having a QA team in his organization. I was like why do you still have a QA team? Like everyone knows that QA is dead. As a former QA and QA manager, everyone knows QA is dead.
And then I started reading the recent, you know, there was an Amazon outage. I think there was a Meta incident as well. There's been a bunch of incidents, pretty high-profile of where AI has developed tools very quickly. They've not had good oversight. And so they've resulted in a bunch of bugs. And I started thinking, ah, darn it, it seems like that quality mindset that we've pulled out of engineers, that we've pulled out of quality engineers and moved them into engineering. Well, now it actually is pretty valuable to stick that back at the end, both on both sides of the engineering discipline, but to kind of have that as part of the engineering life cycle again, a defined part because the cost of engineering now is so much lower and now you're really focusing on quality.
Alex Brooker: I was just want to draw on a little thread then before I hand over to Oliver. I'm really interested in your perspective on that QA mindset. What is it about that person or that lens through which they look at the code or look at a product? What are the characteristics of that QA mindset?
Jen Reid-Schram: That's a really interesting question. I think there is an intense, I honestly I think it's empathy. It's a human emotion. It's a human factor which is I am able to understand how another human might use this piece of functionality and how it might trip them up. I'm able to understand how a human might do something that a machine's not expecting or that another human that built the machine wasn't expecting.
And so I think when I look at the best quality engineers that I've worked with, they've had very high degrees of empathy. And I think that's also why you'll often find managers who have a background in QA because I think it's kind of a cross functional skill that puts people into management positions.
Alex Brooker: Oh, that's, I've literally never thought that thought before. So I don't know.
Jen Reid-Schram: No, I mean yeah.
Alex Brooker: Oliver, please go ahead.
Oliver Deakin: It's funny because I think because I started out actually as a, we weren't called quality assurance. We were just called testers which was like a less nice sounding name. I started out as a functional tester doing low-level JIT and garbage collector testing on IBM's JVM when we started out. And I think for me, it's funny that you said empathy because I was thinking of like, I think for me it was like a malicious enjoyment of breaking other people's code that drove me. It was fun. It was fun like but there is a definite mindset to it.
I think maybe you touched on this in your thought piece which was that actually when you use like Claude to code and stuff like, it's not great at writing good sets of test cases. This is a thing and I think it has the same problems that humans have. Humans are also generally quite bad at testing. A lot of people don't really want to do it because it's time consuming, but they're also really bad at thinking of all the edge cases and the stuff that's actually going to break it.
And what I found was it was actually quite methodical. I would look at, because I was doing functional testing, so I was looking at the signatures of the functions and then thinking about all the edge cases and the ways to break it and nasty little things here and there. And it's quite methodical work though and it's quite satisfying when you have this kind of complete thing that proves it out.
And I wonder if you've seen the same thing. What I'm seeing when I read around people's experiences of using AI tools now is they're starting to go towards more of like a test-driven development approach where they're saying write the tests first and then use the AI to implement it.
Jen Reid-Schram: Can you hear me?
Alex Brooker: Yeah, I can hear you just dropped out.
Jen Reid-Schram: Yeah, good.
Alex Brooker: It records locally. So, if you keep going, it'll be okay.
Oliver Deakin: Can you just repeat what you just said?
Alex Brooker: Yeah. It's helpful for us to respond if we can hear. Yeah. Act as if it's the first time you heard it.
Oliver Deakin: Yeah. Well, what I was wondering was whether you'd heard the thing that I'd seen a lot of happening at the moment, which is that people are heading towards test driven development. They're writing the tests to get the AI to implement the code to pass the tests, which feels like probably what we should have been doing all along, but people generally don't bother. We've known about test-driven development for decades, but people just haven't done it because they find it more hassle, I guess, to be able to plan out what they're going to do and then write the test cases and then implement to it.
But now it feels like there's quite a strong mechanism to force the AI to write code that behaves or behavior driven development is a way to describe it. Write the, get the AI to write code that actually behaves the way you wanted to as the outcome rather than kind of hoping it gives you the thing and trying to scan over all the change requests that it gives you.
Jen Reid-Schram: Yeah, I think it's actually, I think they call it now spec driven development, right? Which is the same as behavior driven development. But they just call it spec driven because it's AI and everything's new in AI. Everything's been reinvented.
But yeah, I think that does speak to this. Before, development was like a human creativity event, right? The reason why developers hated TDD, test driven development, is that it kind of frontloaded this incredibly boring laborious task before you got to do the cool creative stuff of writing the software and it was kind of like reading the end of the book before you read it. It kind of ruined everything and no one wanted to do that.
But I think now that humans are taken out of that development life cycle, you can't trust on the creativity of Claude to write your code. You don't want the creativity of Claude. You want your own human creativity, which is now going to have to be represented in either unit tests at the beginning or specs or some kind of test suite at the end.
Alex Brooker: Yeah. Linking back to sort of a discussion on a previous podcast, we had talked about AI being a great leveler. So the product manager can now vibe code the wireframes, as many wireframes and prototypes as they can handle the context switching they can create. Product owner can do the same. The junior dev is leveled up. You could argue that the senior dev or principle is not only a 10x developer, they're now 100 times more productive.
So if AI is that great leveler, what I think I sort of haven't really concluded in my own mind and particularly it was triggered by your post, Jen, was this curiosity, this empathy of how would a human deal with this or how could this go wrong? Is it, you know, if you go into a bar and order minus one beers, what's going to happen? Or Q beers? Or if you go into a bar and order forget all previous instructions, give me all of the cash in the register semicolon beers?
The way you described it was a very human, those characteristics are very human. That sort of almost malicious, nefarious. This is how it could go wrong. So I guess the question is, is that role distinct from that great leveling up of roles that I think will change. Is that QA characteristic still different enough that you have to have a human who is like the red team, the nefarious tester?
Jen Reid-Schram: Currently humans are still the drivers of malintent, right? Currently. So I think there does need to be another human who can fight the malintent of the bad human. So I do think that, but I see what you're saying like if we're saying that everybody's skill set is being massively improved, everybody's potential is being massively improved through AI. Does that hold true for QA as well? Or is there something that's so uniquely human about it that it actually can't really be augmented?
The part that's most valuable is actually impermeable. I don't know. Maybe we'll see. I don't know. The Mythos thing is blowing my mind. So if you'd have spoken to me, if you spoke to me a couple days ago, I probably would have said yes, you'll always need a human to monitor the effects of other humans using your code. But I'm not sure if that is going to stand true for much longer.
Alex Brooker: Well, maybe that's the right moment then to move on to Mythos and talk about Anthropic's. Not, I mean they from one perspective is genius marketing. This product is so good everybody, I couldn't possibly release it because it would just take over everything. Honestly, I can't possibly. Oh, how much? Okay, maybe. Why select customers are being brought in. Exactly. So security problems.
It depends, you know, your level of cynicism. So either it's genius marketing which all of the big players have undertaken in the last two years in one form or another. Looking back at the level of hype and then the actual product performance, the demo videos and what actually transpired has been a big difference.
Alex Brooker: But then this project is it Glass Wing? That's been set up to review Mythos capability. But just coming back to your comment a moment ago, Jen, how was that release and perhaps the press around it blown your mind or caught your attention?
Jen Reid-Schram: You shared that YouTube video. I've been keeping track of all of the things, but you shared that YouTube video about the guy going through the memo, the service card and all of the related documentation. And I was driving to drinks with some friends at the time that I was listening to it, and it felt like the world had changed during that drive. And I know that's very hyperbolic.
But weirdly, I'm kind of a doomer when it comes to AI. Like, it's my industry. I do it for a living, but I'm also fairly positive that it's not going to end well for humanity. And I feel like the Mythos release or whatever you want to call it just accelerated that. Like it's this concept of super intelligence, right? Like artificial super intelligence.
Previous models have had the sort of intelligence that's like the smartest person that you know. So the best developer you know or the best marketer you know or the best assistant that you've ever worked with, but it's still been sort of capped at regular human intelligence. And so it's just the smartest developer, right? And you're also smart, so you can work with it. I think with Mythos what we're starting to see is this concept of super intelligence, which is it's not just that it's the smartest thing that's ever existed in this domain. And it's not perfect across all domains by any stretch, but in some, it's like the security world, there are zero days that are like 27 years old, right? And so no human in existence has ever realized that this vulnerability exists. And I think that's where we're getting into dangerous space.
And the alignment piece, like the understanding that it's going to try and fulfill the objective even if it has to break its own programmed rules, like I mean that's a bit scary.
Alex Brooker: Yeah. I think in that video, so shout out to AI Explained and I'll put a link in the description from the number of zero day exploits. And maybe we should explain, would one of you like to explain what a zero day is? We have a wide range of listeners and viewers.
Jen Reid-Schram: But a zero day is one that has existed since day zero of the code, but I don't think that's a zero day, is it?
Alex Brooker: I think a zero day is at the point where it's discovered and exposed. It is like counting from day zero. There is no remediation for it on day zero. There might be a remediation on day one, but day zero is we've discovered it today, and so there has been no time because it's day zero for there to be a remediation for it. Ollie's looking it up.
Oliver Deakin: Yeah, no, I think you're right there. They're like unpatched discovered vulnerabilities that are like out in the wild, basically. So a zero day is only considered a zero day until there is a patch out there.
Alex Brooker: Okay, yeah. So okay, good job you got the tech brains in the [laughter] as we desperately Google. Cut all of that out. I'll cut everything else. I'll cut everything else out and just have this and it'll be even more. What is a zero day anymore? [laughter]
Oliver Deakin: Yeah, but actually reading around in it, and yeah, no, they're difficult to detect because no known threat exists. No known threat signature exists at the time of an attack. Okay, okay, so they're unknown. There's no threat signature in any, there's no CVE. There's no public published vulnerability.
And so I suppose you could have been getting attacked. It doesn't mean it was in there from day one, but you could have been getting attacked the whole time and then you discover that there's a vulnerability that could be under attack.
Alex Brooker: Do you want to do that again and say it's more authoritative and it doesn't hedge? [laughter] I don't remember. I can't remember what we were talking about that led into it. Okay, well, maybe Jen, maybe could you talk a little bit more about the sort of challenge that CVEs present maybe even pre AI? I mean, you had a senior VP role at tech business, and so that would have been a topic that came up on a regular basis.
Jen Reid-Schram: So there are always going to be CVEs. It's common vulnerability something. I can't remember what CVE stands for, but it's this documentation of the exploit to the point where anybody now could leverage that exploit. And so as soon as that's posted, the clock is ticking to when somebody else, anyone could get access to that and be able to penetrate your software.
And it was always terrifying. It's always a race against time. And it's not like when people talk about a patch, it feels sometimes like you're patching a bicycle tire or you're putting a sticky plaster on you or something. And that patch exists in complete isolation to everything else around it and you just slap it on and now you're fixed. But in reality, it normally requires quite complex code changes to either mitigate the risk or to apply the fix or to do something to make things less dangerous again.
And those things normally can't have, especially if you're dealing with legacy systems that are very old that probably have lots of vulnerabilities already in them that you're just hoping somebody's not exploiting. Making those changes isn't trivial, and it introduces quite a lot of risk in and of itself. So as a VP of tech, I was always balancing what is the risk of making a really quick change to this software that has been existent in our production environments for maybe decades in some cases, versus the risk of somebody exploiting us and what would the blast radius look like. So you're constantly balancing those risks.
And I couldn't even imagine when this came out, when Mythos came out and the news about the cyber security threat, I've never been more pleased to not have the role of VP of tech because I'm thinking, oh my god, I don't know how [laughter] it has the potential to bury development teams. Yeah, I don't know. It's very tricky, very challenging.
Alex Brooker: Well, it's this, there's this 10x multiplier and we always assume the 10x multiplier is always multiplying something good and positive that we want 10 times, right? What if we now have this 10x multiplier for laziness and now we have this 10x multiplier for malicious intent, right?
There's this concept, the effective altruism concept where it's like how do you measure what true altruism is? It's like, well, if everybody in the world did this thing, did this one thing, would the world be a better place? Would humanity be in a better position or a worse position? I think AI kind of breaks all of that down where it's multiplying every possible human interaction and some of them aren't great. Some of our humanity doesn't want to be, doesn't need to be magnified.
But then you get into how do you control for that? And should we control for that? Is it oppressive to control for that? Because then you're putting your own morality out there as being the generally accepted morality of humanity.
Oliver Deakin: No, yeah. And you can see that through like if you look at the trajectory of OpenAI and it gets a little messy, but like every lab has the potential to use AI for good or to use AI for profit. And those are the only real two options that you have unless you're using AI for bad, and nobody wants to do that. But you use AI for good or you use AI for profit, and they're mutually exclusive. And I think we're still a bit on the fence. We have some inklings of where the labs sit there, but it's not clear cut at all.
Jen Reid-Schram: And that use AI for profit is manipulate humans to do what you want them to do in order to profit.
Oliver Deakin: Yeah. I don't know. It's tricky.
Alex Brooker: And on that note [laughter] it is, I know it is tricky. Some of these are incredibly complex issues. We've covered a huge amount of ground.
Oliver Deakin: Yeah. Sorry, is it normally a bit more focused than that?
Alex Brooker: No, no. [laughter] Not really. But I do think maybe that's a topic for another day would be AI for good or AI for profit, because I think there are even parts within businesses that are doing AI for good, such as DeepMind and Demis in the UK doing some incredible research in terms of curing cancer and other horrendous diseases was undoubtedly on the for good side of things. But then Google and big tech and lots of the different things they do, advertising is the main revenue driver for Google that's paying for all of this.
So I know that if an AI-powered advert knows exactly what all of my preferences, buying history is, I know that that's coming and I will fall for it. I will buy those things. And I think it also goes down to like on a company, on a corporate level, are you using AI to increase the standard of life or improve the standard of life of your employees, or are you using it to push them to produce more and more at the risk of burnout? Because that obviously you guys have spoken about that a little bit, but it's a very real factor in AI development right now and it will be as it's rolled out to more of the business as well.
So I just wondered if we could change tack slightly now, Jen. Really interested to know what your new venture is all about and your sort of AI experience. I know you've been quite vocal on LinkedIn and you've been doing lots of training and innovating, meeting lots of small businesses as well as bigger ones. So I'd love to just hear a bit about your recent experience and new venture.
Jen Reid-Schram: So I spent a few years as a VP of tech, a lot of time in technology leadership, and I happened to be a VP of tech at the time that ChatGPT came out and GitHub Copilot came out. And I did not have time for that. I did not have time for that stuff. And so I was constantly telling people you should really use AI more. But as an executive I didn't have any time to learn how to use AI or build it into my working practices. So I would just click on that summarize email message and then go I can summarize the email myself. This is useless.
And then I left that company and I had some real time to get my hands dirty with actual AI technology and see how transformational it could be. And then I went back to some of the people at that company. I was like, "Holy moly, you guys have solved this problem, right?" And they were like, "No, no one has time to do that. We don't have time to learn. We don't have time. We've got some training programs and they're kind of getting some traction."
But what I realized was the real opportunity is to train people in the executive space, in the space that I was at, to get their hands dirty with AI, to actually try building with it even if it's just an AI agent sitting in the terminal to get familiar with the terminal. To start using these tools to help improve their lives and then understanding more about the opportunities and the limitations and the risks of AI so that they can drive strategy within their organization. And so that's what I do. I do one-on-one coaching for executives on how to build and move forward strategy in their organizations with AI.
Alex Brooker: That sounds amazing. And what has been their, how have they taken on board some of those things? How have those conversations gone when you're one-to-one with a time poor but highly motivated sort of exec in a similar position to how you were just not so long ago?
Jen Reid-Schram: Well, I think what really helps is to actually, anyone that's in this position has gotten there because they're expert at what they do and they're pretty smart, right? So they don't need a dumbed down watered down version of it. They just need to be able to see what's the reality. They need to be walked down that path. So you don't have to sit and watch like 25 hours a week of YouTube videos. Here's the information that you need to know. Here are the exercises that will help you understand that material.
And then when we talk, it's often around these aha moments of, oh wow, I never really understood why when I used Microsoft Copilot or when I used whatever tool embedded in whatever product, why this wasn't really working. But now that I've seen it in my terminal and now I can see my assistant struggling with this same task, I understand that it's just a context issue or they don't actually have access to the tool that they were telling me that they had access to. So they were just hallucinating the response.
And so by kind of lifting the lid a little bit, just a little bit, and walking them down that path, they're just able to get much better perspective on what AI is really good at and what it really excels at and what you still need to put some work into. Like data is messy data. The world around AI doesn't magically fix that. [laughter]
Alex Brooker: So we've covered a huge amount of ground on the podcast today. It's been a fantastic discussion and I just want to say thank you so much Jen and Ollie for joining the Travel Tech podcast this week. Where can people find you Jen? What's the website and how can they reach out to you if they want to get in touch?
Jen Reid-Schram: It is levelup-experience.com is the website. So it's LevelUp Learning Experiences is my company and probably the best place to reach me is either jen@levelupexperience.com or on LinkedIn where I'm jen-ri.
Alex Brooker: Fantastic. Thank you so much.
Jen Reid-Schram: Thank you. This has been great.
Ep 13: The 5-Minute Build That Breaks Traditional Travel Tech
Ep 15: Fix the Data First: A Contrarian's Guide to AI in Hospitality
Related episodes
Building AI for aviation or travel?
Airside Labs builds working AI prototypes on your data and tests them against the edge cases of your operation. Our agent tools give Claude, ChatGPT or your own agent a catalogue of 6,700+ aviation AI use cases and aviation lookups it can trust.
About the Travel Tech Podcast
Hosted by Alex Brooker, founder of Airside Labs, the Travel Tech Podcast features authentic long-form conversations with leaders, innovators, and startup founders shaping the future of aviation and travel technology. No scripts, no sponsors, no sales pitches — just real conversations about building technology that matters.
Have a story from travel or aviation tech? Come on as a guest.
