Trust hijacking, AI phishing and calendar invite attacks

In this episode

• From targeted spear phishing to AI carpet bombing
• Trust hijacking and the growing social perimeter
• Calendar invite worms that spread through contacts
• Why training and compliance alone are not enough
• AI versus AI: catching attacks sent from trusted infrastructure
• Humans on the loop, not in the loop

Speakers

Cy Khormaee, CEO and co-founder, AegisAI
Ryan Luo, co-founder and CTO, AegisAI
Dineshwar Sahni, almost 20 years in financial services security

Transcript

Machine transcript from Riverside, lightly corrected. Unclear words are marked [unclear].

0:00 Ryan Luo: Yeah. It's good to meet everyone and thanks to Dineshwar for stepping on as well. Yeah, Cy and I have had a long history of fighting phishing malware on the Google side, took some of what we learned and applied it out here and we're excited to talk about what we've learned over the last year or so.

0:14 Cy Khormaee: Yeah. What I love about Ryan, by the way, he is a true purist. He's literally only had one job role his entire career, which is detection engineering against phishing. Like that is all he's ever done. Nine years at Google, and so Aegis is the one, the only difference is where he's doing it, but he's very focused on that. So, you know, just this is kind of what we're doing today, is you know, all three of us have spent at least a decade at the front lines combating this threat. And so as we dig into some of these new AI attacks and

0:44 Cy Khormaee: how they affect our world today, we wanted to share the perspective from the three of us on that nature. And something I love about Dineshwar, by the way, he's not just a security guy, he's also a big history guy. And so one of the things we love to talk about is how we can look back to our history to inform what is happening today. Because look, it doesn't repeat, but it rhymes, I think is the common saying. And if we think back to World War I, you know, what did bombing look like?

1:13 Cy Khormaee: Bombing was, you know, very bespoke, a very hand-done craft, as you can see here, literally. In this case, I think they were literally trying to aim the bomb into a trench, you know, back in that era. It's a single aircraft, single pilot. It's slow, it's expensive, it's very, very targeted. And that's kind of what a lot of the statecraft looked like, you know, 10, 20 years ago. They target very specific individuals like John Podesta, and we'll dig into that in a second. But it's an n of one activity.

1:42 Cy Khormaee: What we're really seeing the rapid evolution towards now is what we saw in World War II, where they discovered, you know, aiming is hard, but planes and bombs got a lot cheaper. So rather than aiming, what I'm gonna do is fly a whole bunch of TNT over a city and drop it and see what happens. And that really is for us kind of the framing and the setup for the talk today where

2:06 Cy Khormaee: attacks used to be squads, 30, 40 people at a state actor who are trained, you have analysts, you have engineers, you have data folks, who are all focused on one human and compromising one human. Now with AI, that's collapsed into however many agents you think a state actor can afford, which is pretty much all of them. And so now they're starting to do much more of these carpet bombing tactics. And

2:31 Cy Khormaee: one of the examples we wanted to talk through, because you know, we lived through a lot of it in the aftermath at Google, was the early artisanal spear phish, in this case John Podesta, the Clinton campaign chair. So really, you know, a state actor sent a very targeted, you know, fake Google reset request to this person. It was designed to slip through all of the existing kind of phishing controls. And just through that one single attack, they're gonna exfiltrate

2:59 Cy Khormaee: not just fifty thousand emails, but fifty thousand emails from a political party, which of course has massive implications for the democratic process in the US. And again, it's like a fairly bespoke attack. So they're you know creating the spear phish, they have a specialized link, there's a you know IT connection there, and then of course the classic like custom harvesting pages.

3:27 Cy Khormaee: And then all of that is, you know, probably used. I know it says here it was, you know, weaponized through WikiLeaks. Almost certainly the way state actors work is they'll collect the data, they'll use it for opposition research, and then later they'll release it to hide their tracks. It's kind of a good way to kind of obfuscate who it was you know doing the attack. And at Aegis, we're seeing something very similar. You know, we're seeing that these core attacks, the same behavior is happening, that fundamental idea of send the lure, send the URL,

3:57 Cy Khormaee: custom phishing page, that stuff hasn't changed. What has changed is that we're seeing it generated, generated for an individual, like it was for John, but in this case, we're seeing it rise to about twenty, twenty five percent of total volume that we're seeing, which is just a seventy one percent increase over the last quarter. And I'll play a short video here. You know, even our friends at CrowdStrike are noticing this trend.

5:29 Cy Khormaee: And that's Mike Sentonas, the president of CrowdStrike, you know, someone who's certainly been in this industry for quite a while and just seen this evolution. You know, by the way, not even their core business, but he just sees it as someone who's downstream. Work also with Alex Ionescu on that side, who built a lot of the core Falcon technology. And he likes to say, you know, when we as email providers screw up, it just lands with a bigger mess on the EDR side. So it's something we all need to get better at.

5:55 Cy Khormaee: The last thing I would say is, you know, aligned with Mike's commentary, you know, we see the same thing in our systems where just this rate of phishing is exponentially increasing, the rate of these attacks exponentially increasing. They're bypassing filters half the time, and then users are clicking on them 60% of the time because they're so much more compelling. And really what we're seeing this all kind of come together at, the same attacks, but at volume with better bypass rates, is what we really like to call, you know, trust hijacking, where

6:23 Cy Khormaee: rather than that targeted single bomb attack, we're really seeing state adversaries spread it out, kind of get whoever they can get through these automated attacks at mass volume. And that starts to enable them to chip in through the social perimeter. So if you think about, you know, think about your phone, who are the top ten contacts on your phone and how good is their security? Which one of your top ten contacts, people you love, people you trust, are maybe not the best at securing themselves

6:51 Cy Khormaee: and could easily be compromised? And then how easy is it to make that leap from that person you love and you trust to you. And so one of the latest threats that we saw, actually let me have Ryan, do you want to talk through kind of some of the latest research we've seen around Midnight Blizzard, which is a great example of exactly this.

7:10 Ryan Luo: Yeah, and what we're finding is oftentimes MFA is not the end all solution to stopping phishing. And there are many problems that have existed for a long time, like man in the middle attacks, relays, stealing device codes, and this is part of what we were seeing with, you know, attacks like Midnight Blizzard,

7:34 Ryan Luo: where organizations, governments, were targeted by AI agents to automate some of that phishing infrastructure, but more importantly malware distribution. And the device code phishing was, these are oftentimes what we find is like a lot of these services, like you know,

7:55 Ryan Luo: like a lot of our different login services, authentication services, will often have special ways to kind of unblock business. And these special ways often leave backdoors that attackers can take advantage of, right? And you know, we've seen this on the email side with Direct Send as well. So a lot of these, it's kind of this whack a mole situation where you know it's really, really hard as a security practitioner to cover all the gaps. And what's really important is to have a strong defense in depth strategy to make sure we don't leave anything up to the human. And Cy will talk a lot more

8:25 Ryan Luo: about that too. Speaking of, go ahead.

8:26 Dineshwar Sahni: And I just wanted to add here, 'cause you brought a very good point, recently Anthropic released a report where, since we were talking about nation state actors, they've built a rewriting malware till it gets undetected. There's a Russian state actor working on it. And Iran's targeting US naval pipelines.

8:56 Dineshwar Sahni: And somebody else, a militant group, is building a weapon system using AI. And all of that is being weaponized using AI, and most importantly, the spread is like a virus, you know, the epidemiology of the whole thing, if you look at the detection and response, is gonna be very similar. We need to react,

9:24 Dineshwar Sahni: build a COVID vaccine very quickly. Detection is important. Hence, what do you say to that, Ryan?

9:34 Ryan Luo: Definitely. The detection is the first step. Like just, you know, the fewer opportunities you have to be exposed, the less exposure will happen. And then there's the, you know, when you talk about the vaccination side, there's also the herd immunity side as well, right? The harder we make it for attacks to spread, the more difficult it becomes to be an attacker when we attack that economic chain. Things just get better for everyone overall.

10:00 Cy Khormaee: Sorry, that's why I had to move to this slide. I mean, you guys were already talking about it. So continue the conversation because you're absolutely right. Like Ryan, you want to talk through the calendar worm, and Dineshwar, like I love the analogy of the vaccine. We absolutely need it. 'Cause this attack is the perfect example of carpet bombing. You're right. It's not even carpet bombing. It's like, I don't know what the word is, but like the biological attack where we just put a virus in the city and you see what happens. That's exactly what we're seeing adversaries do. Ryan, do you wanna quickly talk through the attack and then we can discuss this one a little bit too?

10:29 Cy Khormaee: 'Cause it's right on the money.

10:29 Ryan Luo: Yeah, yeah. Well I'll give just a quick overview and then we should hop into the discussion. But you know, on the carpet bombing side, this is where we've seen this like really pop up over the last couple of weeks across many of our customers and also just across, you know, news sites around the world too, that these calendar invite attacks are hitting not just one tenant, not just one particular government entity, but everyone.

10:55 Ryan Luo: Almost no matter how large the tenant, they're getting these strange calendar invitations. Some of them are kind of like malicious from the get, some of them are kind of empty and they get updated later. There's a lot of cleverness that kind of is being deployed here just around like how can you convey a malicious payload to a target. That question has always been, you know, core to how attackers operate. But what's different is like now they're so much more creative and they can do this at a scale that hasn't quite

11:14 Cy Khormaee: Okay. Yeah, and just to really summarize it, because like Ryan always gives a very technical explanation, but just in simple terms, they send you an email. That email compromises your Gmail account or your email account. And then it sends that same email to all of your contacts. And so actually one of the ways I even saw this amongst like my own social network, I watched these worms progress, of course, at Aegis, but as well as just my personal life, because I'm a human who uses email, is you know, folks or maybe investors got compromised somehow.

11:56 Cy Khormaee: And then they're sending this to all their portfolio companies. And then they're sending it to customers and they're sending it to vendors. And like we can literally watch this spread, you know, as Dineshwar said, it's very virus-like in nature. We're just seeing this toxicity spread across all the weak links, not just at work. You know, again, adversaries are not, you know, well behaved and constrained just to like business hours, only use people's business addresses. They're going across

12:24 Cy Khormaee: personal addresses, family, mobile, kinda every channel, every link in that social chain is what they're gonna use to exploit. So I don't know, Dineshwar, do you wanna speak a little bit more to this, because I think this is right on the money for where you were going, but it was just too good of an example to pass up.

12:38 Dineshwar Sahni: It is, you know, thank you. So if you look at it, there was a time one of the biggest attack vectors was the thumb drive. We trained people out of it. Nobody's using a thumb drive anymore that they can stick in the corporate laptop, and then we got rid of the port. But on the regulatory side we have to go through a certain phishing campaign every year, report every year. That's a very

13:10 Dineshwar Sahni: interesting process and everybody has to go through it and you spend about two hours a year going through the training. And we are talking about machine scale. Can you train people out of getting in a situation where they are vulnerable to these attacks, and if not, what's the best approach to take care of it? MFA is not working, training is not working.

13:38 Dineshwar Sahni: What's your approach to solve this problem?

13:43 Cy Khormaee: Yeah, it's a really tough one. I mean, Dineshwar brought up a really good point where training, you know, honestly, we haven't really believed training has worked for a long time. I mean, even at Google, I think we're not allowed to kinda orange team social engineering because like, yeah, we know it works. Go do something more useful. And our view on what more useful is, is better detection. So you mentioned MFA. Look, MFA is very, very helpful, but there's many other controls, especially using AI, to detect and stop these threats in multiple layers,

14:13 Cy Khormaee: at the front door with email, then in your NDR, then in your EDR, and making sure everyone's talking to each other, including SASE, right? So like we talked to Sanjay Beri at Netskope about too, where it's really important that like all of the legs of the game are connected. And we saw this at Google as well, where look, we owned the entire kill chain, all the way from your ads to Gmail to your Chrome interactions to your account in Google, we call it Gaia.

14:40 Cy Khormaee: So we own that entire kill chain, we saw there's so much value. Because look, by the way, no one is ever perfect. Stuff absolutely got through. But you can see how every layer was affected, what parts failed, what controls failed, what in the stack failed, and then obviously fix it and make it harder. What I think maybe is a big thing for us as an industry and vendors to do is like honestly like play better together where we're able to have these deeper integrations.

15:05 Cy Khormaee: So the data can kind of flow across that faster. Because, you know, I do believe that kind of this detection really has to be the answer. Because by the way, the broader theme is exactly what you both said and touched on, which is that they're going from this surgical approach to saturation. I'm sure everyone has heard this way too much with Mythos and like Glasswing, that like all vulnerabilities will be exploited instantly. Which look, it's absolutely happening, so you know, it's not wrong.

15:35 Cy Khormaee: I think the same is true for social vulnerabilities, where now again they're going from this very surgical, targeted, let's go after John Podesta approach to, as Dineshwar said, like this viral approach where we're just gonna spread this everywhere and kind of see where it lands. And so the vulnerability is going to be extreme. And we like to think about it as also, you know, I don't know if folks know the term dynamite fishing, but you can go fishing with a rod and a lure, or you can throw a stick of dynamite in the pond and see what floats to the surface.

16:02 Cy Khormaee: Again, I think that's very much the behavior of the state actor or the criminal organization. Because again, they're not well behaved. They're not thinking about how do I do this surgically with minimal damage. They're just thinking about, hey, in this campaign for this month, how can I get what I can get? And, you know, with all the AI labs, I think there's a lot of discussion around moving slowly and responsibly, which is great. But I'll tell you the adversaries are not.

16:29 Cy Khormaee: Right. So they're just going full steam ahead. And for better or for worse, and I was talking about the trend of technology, what we do know is the trend of technology will almost never be stopped. And so whether it's not stopped here or not stopped in the adversaries, you've got to understand that the frontier will be pushed forward and the results of that are going to happen. So we covered, you know, worms, we've covered credential attacks, we covered retargeting as well. And again, we saw a lot of this at Google, retargeting around

16:59 Cy Khormaee: credential stuffing. I mean, Ryan, you were right at the forefront of that, leading reCAPTCHA at the time. Do you want to maybe talk through what happened in twenty seventeen with like mass loss of credentials, then credential stuffing, and then reCAPTCHA as a control? Because I think that very much mirrors what Dineshwar is talking about as what controls might work in this world where humans can't be relied on anymore. By the way, ourselves included.

17:24 Ryan Luo: Yeah, there was this big, I think in seventeen, you're talking about the big like password dumps that were coming out, and in those moments, well, reCAPTCHA was already installed in many websites to stop automation. And automation, you know, used to just be like, hey, like stop this crawler from like, you know, crawling my site or just like generally keep my services online, stop, you know, fake

17:50 Ryan Luo: ads from kind of being triggered, those kinds of things. But then when paired with credentials, it became like a way to acquire and start from a credential stuffing approach and just try to steal account access at scale, at a machine scale, which is a lot of what we're talking about now. And so this transition actually helped Cy and I build some of the business around reCAPTCHA, bringing it to the top Fortune three hundred web pages and making sure that, you know,

18:20 Ryan Luo: we could stop automation from happening, but also the most sensitive, the most dangerous automation from occurring, which eventually prevented a lot of account takeovers, which was the real reason reCAPTCHA existed in the first place. The real reason being keeping websites and keeping companies and organizations safe.

18:37 Cy Khormaee: Yeah, it's a big difference. I mean, I'm sure you all, most are security folks who have loved ones, and I'm sure you asked them to please rotate their passwords. And if your loved ones are anything like my loved ones, they didn't really listen to you. And so I think that's just the secret of security, is you know, humans are humans. They are going to behave how they are. I'm not going to change human nature. You know, Lord knows we try, even including myself. But we really have to as practitioners build controls around the animal that we know.

19:07 Cy Khormaee: And so, speaking of attacks, I know Dineshwar, this is one of your favorites. It's a little sweet treat here. But do you wanna talk us through kind of maybe what you're seeing in like, you know, the Snickers campaign and maybe this new era of exploits we could be seeing?

19:14 Dineshwar Sahni: Yeah, I'm hungry. It's fascinating. And as a person who's interested in both the cyber aspect of it and the marketing aspect, it is fascinating because everybody is talking about Snickers now in terms of what a good way to advertise. It's the new way to advertise, but I think it is a good way to show what injection means and how you're gonna be participating in prompt injection across

19:51 Dineshwar Sahni: everybody that I know of who is a Snickers fan, they've been using it. And the only thing that you do is ask a prompt, you wait for the question, and then you give this prompt and then it provides really interesting output. And I won't recommend using it, but it's worth a try on your own. But it is a good example where, it's a social engineering attack.

20:20 Dineshwar Sahni: Everybody is going to use this. Who finds this fascinating? It's kind of similar to your script kiddie doing something. And it's gonna go on machine scale and it's gonna go in a community scale. So how do we build herd immunity around this kind of attack? Because if you look at, you know, again, giving your example of history, the transition: first there was targeted bombing.

20:50 Dineshwar Sahni: You go up in the air and you throw whatever hits it, and then you target a specific formation of the enemy. And then came carpet bombing. And then was followed by the nuclear bomb. And then everybody, good people got together and said, no, this is something we need to work towards. Let's talk about the nonproliferation treaty.

21:19 Dineshwar Sahni: Everybody signs on it and everybody acts on it, but it did not stop certain nation states, certain bad actors from doing that. Same thing is happening with AI. Yes, we are talking about pacing. What we need to do is kind of think about how do we, you know, control, or how do we harden our defenses? This is where I would be interested to ask you.

21:45 Dineshwar Sahni: How would you in an organization, how would you ramp up your detection engineering towards making sure these types of attack are not becoming socially engineered and cumulative attacks, maybe along with the proverbial machine speed?

22:02 Cy Khormaee: Yeah, it's a great question. Actually, I even have slides for that too. So we'll go a little bit deeper in the talk here. 'Cause I think, by the way, if I go back here, there's actually two things that are embedded here that are fascinating about this case. That's why we love it. First of all, I love Snickers. I think we all love Snickers. So like that's reason one. Now I'm hungry. I want to go get Snickers. But in the other one, there's two. Like one is how do you think about, you know, safely deploying AI in your own infrastructure? So we talk about herd immunity. This is kind of the first thing we're all worrying about,

22:30 Cy Khormaee: especially those of us who are kind of deploying it via third-party, you know, frontier models. Like you're really relying on a SaaS service for your entire infrastructure now. That's a big scary leap. And then second, to the specific point, is how do we deal with this at the adversarial level when the adversaries have kind of moved into this full strategic bombing era? And I think we all agree, like they are fully there. They're running this at scale. Now we haven't seen the impact yet, but it's coming. Just like with Ryan's case.

23:01 Cy Khormaee: There was about two years of massive credential breaches. And the state adversaries used those, they did it for whatever their purposes were. And then they just dumped all the credentials on the internet. So all of a sudden there's like three to four billion credentials on the internet. And again, people don't mutate their credentials that much. And so if I get you on account A, I'll get you on B, C, D, and E with simple variations. You know, by the way, I'm sure security people know this, changing the one number in your password doesn't fool anybody. So it makes it very easy to exploit.

23:32 Cy Khormaee: You know, to a point, you cannot train our way out of it. Click through rates here at fifty plus percent. I mean, at this point, that control of training is at best a coin flip. We also find for us the false positive rates for users are like in the eighty percent. So users are just very much struggling to either identify positively phishing or understand if there's a risk in their email. So it's just a thing I think is a very, very unreliable signal.

23:59 Cy Khormaee: You're almost better off just flipping a coin rather than asking your users in a lot of cases. And it's because all the things we used to train don't work anymore, right? That phishing was kind of poor grammar, didn't make sense, it wasn't personalized. Well, now it's perfect. It's all well crafted, it's all designed, and it's all coming across trusted infrastructure. You can't even do those things like where you look at the URL, you say, hey, did this come from a vendor I trust? You know, was it

24:30 Cy Khormaee: Google.com, not like goggles.com, you know, is this really coming from the right place? But look, compromises happen. Like Google is compromised, Zoom is compromised, Salesforce is compromised. Like, if I look across the vendors that we observe in our infrastructure, we probably see at least, I don't know, don't quote me on this, but like around five percent that are compromised, maybe at any given point in time. Like we just expect that. And that's just, by the way, not the hyperscalers. That's if you know your business works with

24:57 Cy Khormaee: smaller vendors. We work with a lot of these manufacturers that have thousands of contractors, mom and pop shops doing machining and all sorts of really important stuff, especially in the world of defense, but they're not IT security experts. They're 10 people. There's no one, you know, like a Dineshwar on their staff. They just can't ever afford anything like that. And so they're kind of left to their own devices. So imagine, you know, your average user who's fairly non-technical is actually responsible for a security program that can impact

25:25 Cy Khormaee: a defense contractor down the road. Like that is a real situation we see frequently and one that's really scary. And I think when the attack happens, it's going to be through that route versus others. So one attack I wanted to walk folks through, just to give some color on this, especially for you know the audience that's maybe more consumers than pure enterprise, is, you know, Robinhood was compromised with sending malicious emails out from their own servers. And so if you look at

25:54 Cy Khormaee: even the analytics, like SPF, DMARC, DKIM, all of that passes perfectly. It's from the right domain. Everything is right. And [unclear] asked, kind of, how could we detect this? How could one even figure this out? Even with AI. And you can see a little analysis that we'll run with our agents in the background. And I think the first secret is with AI, it's all about data. It's like, how can I use more data than the other guy? And where you get a really big asymmetry

26:21 Cy Khormaee: is you have an AI attack that's very complex. There's always flaws, but it's very complex and very well crafted. And then you have a human on the other side trying to figure out if that's good or not. That asymmetry of information will ensure the adversary always wins. But if it's AI versus AI, the subtle mistakes the first AI makes, the second AI can catch. And in this case, one of the first suspicious signals is that this doesn't use the same kind of CMS fingerprints we'd expect from something from Robinhood.

26:50 Cy Khormaee: So it's clearly crafted by a different source. And again, no analyst is going to spot this, but the AI can. And then it can investigate and do live sandboxing, it can open its own browser, it can run through with the email contents and kind of go through the path they're usually going through and see the entire chain of the attack. And again, most security crawlers of the prior era, they can't even click on links really. And so you see a lot of, if everyone has seen like a really easy CAPTCHA or a thing that just like,

27:18 Cy Khormaee: click once and win a prize in order to get to the next stage. The reason they do that is just to evade security crawlers. That's the whole reason they do that, because that's all it takes is just one click. This is an example of where there's a highly complex attack. It's well crafted, it's personalized, and it's using trusted infrastructure, but it can still be spotted with the right applications of AI. And again, it's not AI that's kind of like the old world of machine learning where it's the

27:47 Cy Khormaee: same patterns over and over again, you're pattern recognizing. It's actually investigative and it's actually behaving exactly like phishing analysts do today, just at infinite scale with nearly complete data, because again, our agents can see everything. And they mutate as quickly as the AI attackers. And I think that is really the critical piece for the world to evolve into,

28:13 Cy Khormaee: is getting AI defenses and detections that evolve at the same speed as attackers. So we don't have to rely on machine learning or training or rules or things that we've seen before. Because inherently AI is creative. You know the adversaries will be sending you zero days every time. And so the new defenses have to be almost zero day first. Like that's what they're built and tuned for, because that's the norm. It's not the irregularity anymore. So let me take a pause for any discussion on this topic.

28:41 Cy Khormaee: I don't know, Dineshwar, Ryan, any thoughts, kinda given that's kinda where we're at today, which is an interesting place.

28:48 Ryan Luo: Yeah, and so you bring up a great point on how subtle some of these attacks are. And I think, you know, it seems like Robinhood in this particular case, you know, one could make the argument, well, like how often is a company like Robinhood going to be compromised? But the reality is that it's not necessarily a compromise as much as they have taken advantage of a delivery system that has existed for a long time. And like a lot of the reputation of a lot of these organizations, whether it's Robinhood or whether it's any other legitimate organization, they've established

29:18 Ryan Luo: these channels of information delivery. That's what they have to do. That's what the world operates on. So the question is, can you trick one of these countless, infinite different ways that trusted information can flow across the world? Can you just trick that infrastructure to pass your message along instead? And I think when we think about it that way, the reality is that these attacks are happening all of the time and faster than ever, because if it's not Robinhood, it's gonna be someone else, right? So not to pick on one company.

29:44 Cy Khormaee: Totally, not at all to pick on Robinhood, but they're just public about it and I think a lot of us are users, so it's a relevant one. But yeah, to double down on Ryan's point, it's like everyone has been compromised. We just see this rotating stream of similar attacks across all major vendors, across hundreds of minor vendors. Minor vendors just like a thousand times worse. You just never hear about it. I don't know, Dineshwar, what do you th

30:03 Ryan Luo: That's right. The question is, do you send emails? And if the answer is yes, then you can be compromised to send someone else's email, right?

30:09 Cy Khormaee: That's a great point, too. As we're all thinking about this as security folks, too, and I'll push it a little bit more in this too, because Ryan made this point where it wasn't necessarily a compromise, people took advantage of a delivery mechanism. Really, what they're doing is they're taking advantage of like sending infrastructure. This is like a side marketing system. It's probably not considered part of your like crown jewels you're really defending. It's maybe the vestigial tail of the security program because it doesn't affect like the core data that you're really trying to protect. But

30:39 Cy Khormaee: maybe, Dineshwar, your point on herd immunity, the impact to the rest of the herd is really, really damaging. So it almost makes, in this case, the victim a carrier versus truly getting sick.

30:52 Dineshwar Sahni: Another thing I would say, you know, with AI, all of the, you know, phishing, vishing, smishing, all of the star-dot-ishing has become machine scale, more complicated, very difficult to detect. And I think we do need to kind of think out of the box and figure it out.

31:12 Dineshwar Sahni: Who's susceptible to this? You cannot train them out a hundred percent. You can, but the impact is not there. How do we enhance detection, response, and improve the evolution from where we were and where we wanna go? All of the, you know, standards and all that, we will be passing. But if you look at a few of the compromised situations, most of them were compliant with one regulation or the other. If you look at the MGM attack, if you look at Capital One, if you look at British Airways, they were all a hundred percent compliant. But in the end, the weakest link was humans. With AI you're gonna get better deepfakes you cannot detect. There was a time, DPRK workers who would say, you know, do this, do that, this angle, that angle. But with AI, the deception of nation states at machine scale is kind of repeated ad infinitum. We need to kind of work together towards understanding how we are going to solve this and understanding how we're going to detect it in each individual. Of course they will be carpet bombed and dynamite in the pond.

32:35 Dineshwar Sahni: But we do need to kind of see what's the exposure and kind of build the layers of protection. Everybody talks about zero trust, defense in depth. We need to apply that here too. Most of the time when we talk about security architecture, we think about infrastructure. But when we talk about human resources, the human infrastructure, we need to think about defense in depth in terms of social engineering and how we're gonna protect that.

33:04 Ryan Luo: And maybe this is a good transition back to kind of Cy's next topic, which is really around like these traditional systems. Just as Dineshwar was saying, because the underlying systems are not intelligent enough, we've kind of cheated. We've cheated on the side of trust. When you cheat on the side of trust, then we allow these messages to be delivered. And so the question that Cy's gonna answer is like, well, if these messages are being delivered, what do we really need? What intelligence do we really need to stop this? Right?

33:32 Cy Khormaee: Yeah, by the way, we're chipping away. I don't think we have the solution yet. By the way, one thing I did want to touch on too that Dineshwar, you mentioned, that I think is very, very present, is the divergence between security and compliance. You know, look, compliance is a hard job and it always has to lag a little bit behind security, because everyone needs to agree on a compliance standard. It's almost like the baseline. Like if you're not doing this, you're just like kind of not doing your job.

33:59 Cy Khormaee: And then security is all the other things on top of it to make yourself actually secure. With AI, I think it's created a massive divergence. And it's funny, you mentioned MGM. I have two funny stories about that. One is I was actually at the MGM when they were attacked. And so I was there when the credit card systems went down. I'm sure there is a piece of paper with my like receipt and handwritten credit card number, because they were doing everything with paper, still sitting in a vault in the MGM somewhere.

34:24 Cy Khormaee: And then two, I was at MRC, which is the Merchant Risk Council, on the fraud side in Vegas just a year ago. And one of the biggest themes was, these are all compliant organizations. But because they're compliant organizations, the process for them to adopt defensive technology is slower than the adversarial evolution. And so compliance in that case wasn't even just lagging behind the security organizations. It was preventing them from being safe. The adversaries had gone from, you know, a single bomb to carpet bombing to nuclear weapons, and these guys were still kind of

34:54 Cy Khormaee: thinking about if it was okay to use an airplane or not. Right? I mean, that's really where they were stuck. And I have a lot of empathy on both sides, because I think it's a really difficult problem to ask any financial institution to kind of break away from the existing controls they have that have kept them safe for 100 years. But I do think we're entering an era where that very much is the case. And I think the other kind of reaction we think about is,

35:22 Cy Khormaee: and I have a strong perspective here, you know, look, everyone's saying slow down with AI models, and that's maybe true for the frontier models, but I think the message for everyone else who's not building a frontier model is speed up and build more and build faster and go harder, because it is an arms race. We're falling behind in an arms race. I think the only way to catch up is to really move faster, because the adversaries are not, you know, playing by the same rules.

35:48 Cy Khormaee: But I don't know, Dineshwar, you've been in this space for a long time too. Like what do you think? You struggle with both sides, you have a lot of visibility across the challenges, across multiple stakeholders. Like how do you think about, you know, if you're someone in the audience running a large, you know, security organization, how do they think about this balance?

36:08 Dineshwar Sahni: I think the biggest thing that we all need to do is break down the silos. You know, your compliance needs to kind of be in lockstep with the R and D team, with your insider threat. You need to be working together with your SOC. You need to be working together with all of the, you know, parts of the cyber kill chain. They are working together, but somewhere we do need to have a singular vision and, most importantly, the adaptability. You know, a bad actor can use your frontier models to build malware

36:44 Dineshwar Sahni: and kind of rebuild itself till it becomes undetected. We gotta reverse engineer that mindset to kind of continue to evolve till we ensure that nothing goes undetected.

37:01 Cy Khormaee: Yeah, I love it. That's a great one. And I totally agree with the organization point. I mean, Dineshwar, you've got a phenomenal perspective on this too, where it just really is breaking down the silos. And a lot of what we see is like starting to flatten the organization so things can move quicker. I mean, one example, even at Aegis, I think we have almost fifty people now, but it's just two layers of management really. There is no like third layer. We're gonna resist that for a long time, because we accept a little bit more chaos and a lot more self direction from our team in order to say, look,

37:30 Cy Khormaee: you're on the ground, you gotta make the decision with the data you have. Like that's just what you gotta do. We can't rely on hierarchy and decision making several levels up to react to the threat in real time, because, you know, it's literally evolving. You know, for us, we see brand new patterns of behavior almost on a monthly basis, certainly on a quarterly basis. Back at Safe Browsing at Google, building protections into Gmail and Chrome, we would see attacks rotate annually.

37:59 Cy Khormaee: And you had like a whole quarter or two to react to it. Here, the new attack wave has come and you gotta react and figure it all out. It's gonna come, you know, day one, and then we gotta react on top of that. So I mean, one of the biggest themes too that we see, that we're talking about, is, you know, really humans, you gotta get humans, you know, out of the loop here. I was chatting with an analyst a while back and he had a phenomenal perspective, which is it's gotta be, you know, humans on the loop, not humans in the loop.

38:29 Cy Khormaee: Meaning humans should sit on top, observe the behavior of kind of what's happening inside and tune and trim and organize, but you can't have them be in the action. One of my favorite cases was, I was at RSA last year and I was having a conversation with a customer and he literally like saw a text, got up and ran out of the room. And he did that because one of the people in his organization had been compromised and there was an exfiltration in progress. And the posture of his organization

38:57 Cy Khormaee: was then contingent upon his foot speed and his ability to get Wi-Fi at RSA, which of course is one of the most difficult things in security, getting Wi-Fi at RSA. And we really want to move out of a world where we're dependent on great people doing heroic things and instead move to a world where it really is automated by default. [unclear] You've got AI that's refining this malware. You're not going to have a reverse engineering team do the same thing ad infinitum. You must have

39:25 Cy Khormaee: a machine on the other side. I think that even gets harder as we talk about the perimeter and the social perimeter expanding, where again it's no longer our organization. It's, you know, I think about my family, it's my wife, it's my daughter's daycare. I always bring that up as an example. Look, I love them, they're phenomenal, but like, look, they send me a lot of email. I don't know how good they are at security. Like if you want to phish me, not to recommend it, but that's probably one of the approaches I'm gonna watch the closest.

39:54 Cy Khormaee: And then pick 50 other vendors and friends I interact with and really enjoy, but like who knows what their security posture is. And so I can already hear our SOC team starting to wrap more controls on my accounts as I say this. But it's really like a thing to think about. The social perimeter is almost everyone. If you think about one layer out, who you touch, it's now a hundred people. And if you're at an organization of 10,000 people, you don't need to worry about 10,000 people or 20,000 people. You need to really worry about

40:24 Cy Khormaee: a million people. And that is a new level of challenge in this world of trust hijacking. And something where I just think we'll have to be very aware, not just of identities, but also relationships. And this spans across data leaks, insider threats, behaviors, as well as like business to business relationships, which again is where we're seeing the most pain right now: small vendors, big vendors getting compromised and used as vectors of attack. And that trust that Ryan kind of brought up

40:53 Cy Khormaee: really becomes the vector of compromise. And so as we think about this, you know, our push just as practitioners, and we should discuss this more, is to move AI from the frontier really into the pipeline. I think about, and I teach this class, I teach the AI course at UCLA and I've taught it for three years. The first year it was like a curiosity. It's like, there's this crazy guy. And look, I've been in the AI space for 20 years. So I've been the guy sitting out there with a sign saying, you know,

41:22 Cy Khormaee: the end is coming, for about 20 years now. So I've believed in this for a long time. I've long been a zealot of this. And I was a little bit of that at UCLA when we were starting. Year two, all the students are doing their papers with ChatGPT. Now it's done badly, it's very obvious, but you can see they're all using it. Year three, a third of my class had OpenClaw deployed natively, like day one, without me saying anything. And so that evolution with students, I think, is the same evolution we need to see in the industry. And it's funny, actually, I was at another talk a while back.

41:51 Cy Khormaee: You know, hey, as a professor, like what do you think students should know about industry AI? Actually, you know, it's the other way around. When I go to industries and companies and we give threat briefings and we discuss AI transformations, I actually have to do a lot of convincing. A lot of people are kind of not really going into AI to the degree they need to. And there's a long discussion and change management process to get there.

42:17 Cy Khormaee: I have none of that with students. Like again, I walked in, I take zero credit for that. I walk in and day one, I remember this because I asked, hey, who's using AI? Everyone. Who's using harnesses like OpenClaw or even more exotic things? A third of the class. Like these are truly AI natives. And so maybe the push here is to look at, like look at what maybe your teenage kids are doing. And that's probably the right answer for where it should be going as an AI native company and AI native world. And

42:45 Cy Khormaee: in this arms race, it's the same thing. It's like if you think about, you know, from the war analogy, in World War I, it was the machine gun. If you had a cavalry charge against a machine gun nest, it's going to go poorly for you. In World War Two, it was the carrier and the airplane versus the battleship. If you have twenty, you know, airmen and aircraft versus a battleship with big guns,

43:10 Cy Khormaee: they're just really bad at combating that new technology and they're gonna lose. In this third era, as Dineshwar mentioned, nuclear power is a big one. In this fourth era, potentially it's cyber, it's data, it's kind of what's flowing around these wires, and that's what we're gonna see next. So I don't know, let me open up to the group a little bit more to discuss, you know, how do we enable this? How do we as practitioners help people get towards that vision? 'Cause I think what I hear all the time, Dineshwar, I've heard it from your talk too, is like no one disagrees this is where we should be going.

43:40 Cy Khormaee: But there's so much pain in getting there. And if we do things, I think, the traditional way, we're not going to get there faster. The adversaries will beat us and we will lose. So how do we kind of, I don't know, get over the pain or take the pain all at once so that we can get to this place, or at least be playing the same game as the adversaries, because we must.

43:59 Dineshwar Sahni: Professor, I gotta ask you a question for this question. So the New York Times just published an article about the cognitive surrender of the students, and as an enterprise, it's my personal opinion, there is a certain cognitive degrading. Of course, my efficiency is going up, but certain curiosity is kind of going down. So even with my email, my social protection, social engineering, phishing, vishing, smishing, and star-dot-ishing, I'm gonna be reliant on the systems. Right? So how do I ensure, you know, the systems are

44:45 Dineshwar Sahni: checking and evolving and working together as a larger team, different silos, working together in a single pane, and improving our program along with improving the relationship amongst the different domains within cyber.

45:05 Cy Khormaee: Yeah, I mean that's a really good question and a hard one. Let me say two things. You mentioned there's this, yeah, like the lack of thinking. And I think there's two ways to use AI. One is you can use it to replace your brain, right? So this is kind of the car method. You sit in the car and you do nothing. The other method is like the bicycle, where it's a thought accelerator. If we think about

45:30 Cy Khormaee: dealing with colleagues via email versus dealing with colleagues via mail, it's a classic example. Mail, and I remember my mom was a salesperson way back when her CRM was mailing stuff back to the home office. You know, it was way back when she worked for Merck, right? That was a very slow data process, slow learning. Email is faster. AI can be even faster. So if you use AI as a tool to amplify your mind, I think you'll have a dramatic spike in ability. And so I tell students, and we see this with the student population,

46:01 Cy Khormaee: there's a huge divergence between students who are like, hey, I have this homework. I'm going to turn in the same paper I would before, but just use AI, and I'm going to use zero brain cells to do it. They are going to be the losers. They are already behind, and they will be largely irrelevant in the future world. Then there's other students who are like, wow, I can now stand toe-to-toe with someone like Cy. Because look, Cy has spent 20, and I tell this all the time, I've spent 20 years in this space.

46:30 Cy Khormaee: I wrote a book, I teach the topic, I spent a lot of time and energy here. How much more do I know than a student in my class about deploying OpenClaw on a personal device? Probably zero, if not negative. They probably know more than me. So in this moment of change, they have this very unique opportunity. We all have this very unique opportunity to leverage this new knowledge, and the playing field is totally flat. And I'm saying this as someone who's invested his entire life in getting to this moment. My view is my advantage getting to this moment is zero.

46:59 Cy Khormaee: And so we must accelerate. Now, to your specific question, how do we apply that to the practitioner and the infrastructure? And a lot of my lessons have actually been taken from Ryan, [unclear], who'll take off his detection engineer hat and put on his CTO hat at Aegis: how do we think like engineers? How do we think in systems? How do we think in terms of logging and debugging and data integration? So I don't know, Ryan, as someone, by the way, building AI

47:28 Cy Khormaee: with AI at the front lines, if you kind of think about, you know, what advice you'd give folks with more traditional kind of IT infrastructure and digital transformation, what would you tell them?

47:40 Ryan Luo: Yeah. That's a great question, Cy, something we think about every day. How do we keep the human on the loop, not in the loop? You know, I think we've talked about this analogy a lot around how attackers are able to just inject AI wherever they want and defenders are nervous about it. And a lot of it comes down to the fact that, you know, safety is important to the defender, but not important to the attacker. If the attacker breaks something with their AI, you know, look, they're meant to break stuff anyways, right? So it's okay. And I think that's probably the first place that I would point to for

48:09 Ryan Luo: practitioners: make sure that you have, you already have automations. The question is like, which automations are you not confident about? Right. And so like really looking at building this harness, this system around kind of the decisions your AI may make autonomously without human guidance. What kind of impact can be created from that if a mistake happens? And then also how can you kind of audit the correctness of your system over time, right? How do you look for

48:39 Ryan Luo: anomalies? And this is probably something you already have. Like you likely already have some kind of system that audits manual human decisions. So like extending that to the machine is actually probably the first step that I would take. And then starting to find ways to kind of experiment at scale and roll out slowly is generally what I've always found to be, you know, staged rollouts are always the path forward, even at a systems level, making sure you understand what system correctness is and how you kinda roll that out slowly and thoughtfully over time.

49:09 Ryan Luo: Dineshwar, does this kind of align with any of the things that you've tried to build over your career? I know like adding and integrating AI safely is probably like the main question that you and other practitioners have.

49:20 Dineshwar Sahni: I think we are at the moment, since we were talking history, we are building the seat belts. The car's speed is not gonna slow down. It will accelerate. You're gonna break the sound barrier. And the people behind you are going to be doing the same thing. What do you do to keep yourself secure? I mean I don't wanna be wearing a helmet while driving, but

49:31 Cy Khormaee: Yeah.

49:44 Dineshwar Sahni: I wanna have safe harnesses, I wanna be, you know, educating, I wanna, you know, have a program where, as I mentioned earlier, you know, every part of the cyber domain, every domain of the cyber kill chain, as well as protection, as well as architecture, they are talking to each other and building a better reporting mechanism, better fine tuning mechanism.

50:13 Dineshwar Sahni: The protection should go beyond infrastructure and applications. The eighth layer in the OSI, the human layer, is I think the weakest layer, we need to focus on that. So this is where, you know, we need to put together, you know, a program where we are not just looking at compliance because we need to be compliant.

50:37 Dineshwar Sahni: Compliance is not negotiable, as I mentioned, because we need to set certain standards which we all agree upon. But we need to kind of go beyond that compliance and make sure, you know, we are looking at where the adversaries, you know, not only a script kiddie but also a nation state actor, where they're heading, you know, share the information within the industry, and in panels and forums like this, we should be able to communicate, converse and build a better way of rolling out and safeguarding AI over the years.

51:16 Dineshwar Sahni: Because eventually you're gonna have another campaign which will ask everybody to put in the prompt to get a wonky response out of it. You know, as a Star Trek fan I would say this is the final frontier. We all need to boldly go where no one has gone before, and it is happening right now.

52:08 Ryan Luo: Thanks, I think.