Snyk's Manoj Nair: Skills and MCP Are the New npm

0:00 Someone said Minthos basically was a margin call on your technical debt. And last week I was with a you know like 100 plus C sales in New York and people are now starting because of this like you know real notion that there is an autonomous attacker out there, lots of attacks have happened, speed has increased, security is not kept up.

0:19 We've seen 10x increase year over year on the backlog. And for every issue that is being fixed, eight new issues are being created.

0:30 The AI Native Dev is a podcast with developers and engineering leads at the cutting edge of AI identic coding. Join your host, Guy Fajani, and meet Simon Maple every week as we chat with the most exciting voices in AI and tackle the biggest questions facing developers today. This is the AI Native Dev.

0:52 Yesterday, you were writing code. Today, you're prompting agents. Tomorrow, you'll be running a software factory. Agentic coding is moving fast enough to give anyone whiplash. This June, AI DevCon London sold out. Hundreds of developers, engineering leads, and VPs packed in to hear speakers from Anthropic, OpenAI, Netflix, and GitHub cut through the hype and get to what actually works.

1:17 This November, we're bringing it back to New York. Bigger, bolder, and full of people actually building the agentic future. Three days of real talks, real tools, real answers, whether you're directing the agents, building the loops themselves, leading the teams, or setting the strategy. As a podcast listener, you can get 15% off your ticket using promo code pod50.

1:41 We have so much planned, and we can't wait to have you join us in New York.

1:50 Hello everyone, welcome back to the AI Native Dev. I'm Guy Pajarni and we're here in the uh SNC AI Security Summit, or the SNC-backed AI Security Summit, with Manoj Nayer, who is uh heading uh sort of product, innovation, marketing. I forget now your sort of title, but like you know, kind of key product leader in SNCC.

2:06 Uh Manoj, thanks for joining the podcast.

2:08 Thanks for having me, guy.

2:10 Um really fun event here today, you know, various uh various companies of various stages. But I I figured what we should talk about today, mostly also relevant to the kind of ANA to dev audience, is to talk about the enterprise and kind of the state of AI in uh in the enterprise and agent adoption. Because I feel you know, oftentimes in this podcast and in general, maybe like at the hype of the era, uh, you know, we uh we get drawn and we dig into just sort of like this incredible futures on it, and enterprises have certain burdens, a lot of urgency to move, but certain sort of burdens that prevent them from doing it.

2:48 And I think Sneak is in a really, really good um uh kind of a point of view on it, working with some very modern. So I guess just to kick it off, like how um like what is it that you'd say is the current state of you know enterprise adoptions as you're sort of seeing it? And then we'll dig a little bit into like why are like the ones that are behind what's holding them back, and the ones that are ahead, you know, how are they getting there?

3:15 Yeah, I it's a really good question. First of all, thanks for having me here, and thanks for being a sponsor of AI Security Summit. So uh it's become a really good you know collection of people talking about security in AI. A year ago, no one was really cared. Now that seems to be the biggest topic. So uh it's it's been a really good turnout and good to be here.

3:36 I find uh just as a bit of a side note, it's like when I founded Sneak uh many, many years ago, I debated whether to come back to security because I was saying on how like you go to a security event and you come back and you kind of want to cry in a corner, you know, like curl up in a corner and cry.

3:50 Uh as uh it's just like the world is doomed, you know, there's so much problem. And you know, you're going to like a DevOps conference and it's all like singing Kumbaya and the web is amazing, and we can kind of build these great things on it. Um, and and so I I think like in the world of AI, it's just this sort of odd combination of both.

4:08 It's like you have these conversations about these incredible things you do in building and in security, like all these things that AI unlocks, and there's definitely a lot of doom, you know. There's definitely a lot of like the world is collapsing on us, agentic attackers, you know, what can we do on it?

4:22 So it's just sort of an interesting thing.

4:24 I think I I'll start there. You know, I think we had I hope we had the healthy dose of optimism too from all the speakers.

4:29 Uh a lot of both, yeah.

4:31 But uh I liked how an enterprise customer recently classified it, and I'll give you a little bit going back to the first question. AI is definitely an attacker now. AI is also a new threat surface. And then, like, we had like Satya Nadella talking about it like recently, AI is an insider and an agent.

4:50 Uh the fourth thing, actually, is the interesting part in all this. Like AI is also a defender. So we can unpack that. Yeah. But like if you take those first three things, I think from an enterprise perspective, you think about using AI versus using agents. And are you just using or are you building?

5:08 Right? So we classify it, like, think about this like a two by two, like you know, use AI, building with AI, use agents, building with agents. And so categories of companies that I would say are almost everyone 100% is using AI, yeah, Chat GPT, you know, Gemini. Yeah, let's assume that's now pretty much the standard.

5:28 So weaved into like a lot of their existing kind of enterprise tools, it's just sort of it's there.

5:32 And then you got you know, building with AI. So you had the copilot era, and it was, you know, it was so interesting. Like maybe sitting 18 months ago, we thought like it was game over on you know AI coding is copilot. Um, but there's still a lot, a healthy percentage of the enterprise customers because they were early to copilot, they were Microsoft and GitHub customers, so that natural gravity was there, but that was easier because of the connection I feel for a lot of enterprises.

6:02 So 50-60% of the enterprise probably is kind of you know in that some kind of AI, you know, assistant mode. Right.

6:10 Uh still autocompleting and sort of sidebar chat.

6:13 Sidebar, like yeah, it's a humans very much orchestrating. And so then enters the agentic, you know, building with agents. And that's where I think like it's usually the cursor, you know, claw unlock that happened. Um that's not still a factory, but it's you know, getting to a point where okay, the agents are doing work.

6:33 I can shut down my laptop and and and go away. And yeah, um there, you know, there's a risk aspect in all this. That's where the attack surface, you know, AI as an attack surface really exploded over the last six months with skills and MCP servers and all that. And then the last one is like building agents.

6:51 So you're like actually, and I am modifying that last category a little bit. I think maybe part of our conversation is you are shipping production agents, whether for internal use or external use. So you're not just using agents, you're yeah, but I also think those people are increasingly the ones that are, you could probably say that's also where the software factory is coming to life too the most, you know, as they're building the last category, right?

7:15 And so, you know, I would say like there's a lot of people shipping agents now. We are just releasing the state of uh agencai AI in the enterprise, volume three. I never thought we would do the one of these reports three times in a year.

7:27 Yeah.

7:28 It's so random.

7:29 It's just so quickly, yeah. You have to be for it to be relevant. Yeah.

7:33 First time we did it was January. Surprise, surprise, open AI was number one in terms of models being used for building. By the second report, Entropic really closed the gap. And this third one, we are really focusing on the agents, not just models being used. Uh, like what are the harnesses, what are the agents, and and we're starting to see like this is interesting term of reach of the risk, right?

7:57 So you could use, you know, we see lots of different models. Open models have become much more prevalent than than even six months ago. Uh, but there's a lot, you know, a lot more agents in there too.

8:08 In the path. So let me uh echo back a little bit and just sort of like drill into a few things before we go further. So the first thing you're saying is there's actually still a fair bit of that sort of co-pilot style, sort of self-uh kind of uh development uh in organizations. We actually see this a lot at Tesla as well.

8:21 Uh and I think it's it's almost like the the in action versus the action, or it's a little bit of the uh feel-good appeal of like, yeah, yeah, of course we're sort of building with AI. Um and it's interesting because I feel like cursor, like with the cursor as cloud agents, there was sort of a lot of this conversation.

8:36 Like in some of the kind of more frontier development sort of teams, not frontier models, uh, the you know, the cursor cloud agents are convenient, but they weren't necessarily like seen as sort of something that's novel. But I think what they cracked pretty well is actually this uh like fit and convenience into indeed these sort of enterprises just sort of smooth their journey right into that app.

8:57 Um that's the twos to the threes that I think about. Like a lot of customers transitioning, you know, from being just using AI as a co-pilot to using, you know, doing more agent deck development. That uh transition was easier because of with that sort of cursor cloud agents.

9:15 So as they as they as they move into that, so maybe like to drill into that and then we'll sort of go a bit into these others that are building agents and into the attack surface. I guess what has been what what do the companies need to overcome, right? Or like what is it that helps them uh uh make that leap, whether it's from a security or risk or compliance or maybe even cultural reason to be able to go from I'm just co-piloting to uh I am now kind of agentic, uh developing agentically.

9:45 Honestly, the number one um propellant of that I feel is leadership and culture, more than anything else. We have stories of, you know, like some of the largest financial institutes in the world where the CEO and leadership team are actually using agents on a weekend and they come and talk about it. And so it becomes a the pace of change of these enterprises are so slow typically.

10:09 Yeah, but when it is a top-down push saying, no, you are going to be more productive doing agent development, that makes timelines that like think about like 50,000 uh clawed code agents getting rolled out in an enterprise. That would be a year-long project. It was uh, you know, we were notified in in uh May that it's going to be by August.

10:32 That sounds like an aggressive timeline. And then uh end of May, I got a call saying no, it's being pulled in to July 10th. So top-down pressure really works.

10:42 Yeah.

10:42 Um and then there's pressure, this is obviously security concerns, right? And so when you have this conflict between, then what happens is it goes to the CTO versus the CISO. So the CTO is the proponent for the change, and we are seeing a lot more of the power of shifting, shifting the culture from goes from CEO to CTO.

11:02 Yeah. CISOs, unlike in the cloud era, where they were slow to get to it, they don't they don't want to be left behind. And we can unpack that too separately. But they don't want to be left behind.

11:14 Some of them are even driving it because of the agenda attacker, which will be. Exactly.

11:17 So so then it becomes like, what are the biggest uh you know, known security risks? And it really breaks down to in coding at least, what what do agents do? They produce, there's an output, yeah, and in the process of producing it, the output could be insecure. You know, the code might be bad, they might use bad dependencies.

11:36 Yeah, uh no different than devs, but just faster.

11:39 Yeah.

11:40 Um, then there's you know the environment. Again, there's a new supply chain. Yeah. So you started sneak, and you know, uh it was like the dependency supply chain risk was a big concern. The new ones, you know, like skills and MCP servers are the new NPM of these agents. So those are, you know, there's toxic flows, there's toxic skills, like you know, three lines of English can be malware.

12:05 What is the intent of the skill? Uh and then finally the behavioral risk, right? So like the Pocket OS moment was like a big moment for a lot of CISOs, even though it was a safety thing, you would think. And uh this is you know, Pocket OS is uh this company that's building uh rental car, you know, management uh portal and uh software.

12:26 And I think the headline story of this was the agent had was overly permissioned and deleted the production database and uh you know Rails API backend or something. And the copy. Yep. And it was uh it was one where the the the famous line I remember there was you know, when when introspective, the agent said, oops, that was bad, I should not have done it.

12:48 And so this oops, it was bad, I shouldn't have done it, became like a big headline. Like, okay, what are all the ways that what is really a safety concern became a CISO concern very quickly, right? Safety, security. But it really comes down to like those three things like the output, the behavior, the supply chain.

13:04 Yeah.

13:04 And I think the best CISOs and organizations involved are like looking for a cutting-edge solution, whether they build or buy, uh, or some combination to overcome that and and move the ball forward. Right.

13:15 So I feel um, again, kind of sort of repeating a little bit, right? So we have these enterprises, they're they're sort of moving to agentic. They have to overcome kind of these concerns, is like I guess kind of paraphrasing a little bit, like would they generate secure code? Yeah. So like this notion of secure the output, there's definitely like a whole period of like, you know, these like these agents are producing insecure code.

13:33 We had all these stats on it. It wasn't even wrong at the time, except you know, I think models have gotten better, but it's true that also you generate volumes of it, you have to secure the output. And we'll get to agentic attackers in a sec, you know, instead of that uh that driver of urgency. Uh then you have the uh sort of supply chain element.

13:48 It's interesting because like skills and MCPs are there, but there's also this whole world of like uh of tricking the agents to actually just download malware, right? They're sort of falling for the same type of mistakes. So I don't know if that's sort of the first bucket or the second.

14:00 Yeah, kind of blurs the line out.

14:02 Um I forget what type of squatting they kind of called that uh slop squatting. Slop squatting, yeah, of sort of creating, uh you know, just like making up uh dependencies and then going off to download them. Uh and and then you have this notion of of behavior, so more sort of securing the the environment, which starts veering a little bit into like enterprise security land on it.

14:21 So we'll we'll uh come back. So so you have to overcome these three concerns, and and I guess also echoing like a lot of that is the top-down pressure to just happen. One one thing that is um uh that I sort of see when I talk to like you know the big banks or or others, uh, is that you know, when again for most people listening, they think about running an agent, they assume a bunch of permissions for that agent.

14:42 So like it is running agentically, surely it has access to the internet, surely it has you know like some ability to do things. Um and I see like a lot of organizations uh that are like the banks, they're they're they're embracing it, they'll tell you everybody's using flood code or whatever it is, um, but they are running those agents uh in a very confined kind of uh uh setup, right?

15:03 Like they can use intelligence, but they don't really have a lot as much information access. Um I guess is that what you're seeing as well? And do you have a view? Like, is that um like I guess how how do you feel uh about the sort of the the sort of security functionality trade-offers? Like, are these agents almost as effective even though they are kind of locked down?

15:25 Is it losing 80% of the value?

15:27 Uh I think it's an extension of the AI as a copilot era. So these are agents, they're not copilot, they're not doing autocomplete. Yeah, they are taking, let's say, you know, task and going and being able to achieve it. So they can do that. I think it'll prevent them from being real factories, right?

15:45 And so now you see all kinds of like the the enterpr the most common enterprise solution for like an agentic coding is you know, they'll use something like skills scanning and MCP scanning from us. They'll have a secure sandbox, you know, Docker or something else. They will look at either creating permissions and separate identities for the agents, or the more sophisticated ones are exploring a lot of you know, more dynamic.

16:12 More dynamic authorization, you know, ephemeral um authorization on the fly. Uh I think that's an unsolved problem, but a lot of a lot of POCs going on there. So the minimum, just everyone is a teammate, they have their own uh identity, and they're locked out of the internet. In fact, like you know, how do I let my agent securely browse the internet but not the open internet has become another thing.

16:34 And so I think that's that next phase that's curating the internet. Starting to curate the internet, and there's companies that are are agentic um, you know, search tailored for agents that are now taking off and saying, hey, we could do that part where you don't expose anything that you don't want your agent to have, but it has most of it, you know.

16:55 So uh I think what is missing there a little bit is is uh security is always like thinking about all the layers and the risks, but it goes back to what are you trying to enable that organization to do and that agent to do. Uh and I think it's just time to unpack it. So let's let's allow agent take, but let's close as many doors as possible using a little bit of yesterday's model versus tomorrow's model.

17:21 You know, so so that's where I feel like that's the step between the three and the four that it's a pretty big unlock required um before they can go off and do that. And that's what they don't get if they like run in such a kind of more over.

17:35 Yeah, in the uh yeah, and I think it's interesting because like I feel on one hand it is a real limitation. On the other hand, this is uh this is an era in which like the best thing you can do is just just get going. Yeah. And so if the way to get going is to run agents without internet access, fine, do that.

17:51 There's probably enough value in that anyway for you to continue. And then alongside that, you have to think about how do I start enabling so so I guess there's a compliance aspect too here.

18:02 Like a lot of these organizations feel like there's a compliance reason for a human approver at the end. Yeah. So I talked to some companies who have already gone and changed their job descriptions or or or their contractual agreement saying, hey, it doesn't matter if you're using AI or not, you own the output and you're responsible and all those you know controls in place.

18:24 Uh and I think that's fascinating too, because like that is a barrier, and I've started challenging like what is the actual law that says this? Like what is the compliance, what is the regulation, or is it just the is this just how we interpret things?

18:38 Um Yeah, I think it's uh it's a little bit like uh it's like insurance, it's like self-driving cars are running along if a self-driving uh driving car uh while self-driving, you know, makes uh makes uh you know like causes an accident, who's to blame? Uh and I feel you know, in in the in the context of a self-driving car, typically the same company owns the entire stack, you know, you are kind of running along and you're building it.

19:04 But in the case of you know it's a uh it's a production or like a product change that has happened, uh it's it's it's sort of a bit of an evolution, right? If you don't have a name to attribute to it, there's an accountability problem. So you need the organization, I think, to more broadly uh decide the changes of a of a of a certain type or to a certain application or such, the company is willing to take kind of responsibility, so you know they're sort of stomaching the risk uh in uh in favor of the uh the change.

19:37 And then you need to, and then you need to kind of convince the uh convince the auditors that that's the case.

19:43 And that this is so fascinating. We had a customer summit yesterday. We did a poll of who is responsible in your organization. Is it the dev? Is it the business? Is it the organization? And it was fascinating. The more AI native organizations were like further along, they said it is a shared organizational risk.

20:02 Yeah.

20:02 We decided we will back this risk for the benefits. And I think that is probably one of the hardest things for that next leap for people to like go off. And there's a lot of agents being built and shipped too, so there's that risk assessment happening on both fronts in software development and in the software that is powering their business.

20:26 Right. It's different sets of risks. Here there's there was a human involved. Yeah. What can I do?

20:31 You had someone to fire if a thing was problematic, you could uh there were heads to sort of roll, right? Right.

20:37 In this case, it was already like your shipping apps, the business was responsible, quote unquote, the business. You had product security, you had appsec, you had all these functions. So I think they this is why I feel like we see a lot more agent apps in production than true agentic government, software development.

20:55 And it's, you know, like if you think about it, like it's like, well, how's that possible? This should be a sequential journey. No, it's like people are leaping forward there because the organizational notion of risk was already established, and we have to move from that individual risk to organizational risk and software development.

21:13 Yeah.

21:13 I'd also say that the the agentic, like as you talk about building agents to be sort of an evolution of using agents, I think the sort of the caveat to that will be that building agents is is a specific team. So in in those 50,000 developers you have, maybe a hundred, maybe a thousand, you know, are building agents.

21:33 Uh but the changing the 49,000 uh is really sort of like a big endeavor. And so it's actually easier to get a team to be forerunners than it is to sort of get an entire organization to sort of level it up. I guess we've sort of seen this with DevSecOps as well. You'd always have the sort of the forerunning teams that sort of set the North Star for the organization and that deal with all of this compliance.

22:00 And getting the auditor, so you can't get everybody in. And then you have the rest that you need to like gradually move them up the ramp, you know, as they uh as they build out. Um yeah, and this risk management. I mean, I was just hearing my talk at uh at uh at ASS, you know, talking about how there's there's this interesting inversion of control or sort of inversion of responsibilities uh that is happening in which uh uh security needs to build more automation and fix more issues on it.

22:25 And uh uh engineers, if they want to manage that risk, have to build security controls. And you know, for the for the security, you don't I guess you do blame security engineers and such, but you don't expect sort of security people to inspect everything to say if it's an attack. You know, you expect them to build automated controls that inspect to see whether something was an anomaly or not.

22:47 So to an extent, we probably need to get to that spot.

22:51 Yeah, I think that uh you know it's fascinating you're bringing this uh notion up. I think it's a good segue into a little bit of that autonomous attacker.

23:01 Yeah.

23:02 I just for the first time in my software engineering career, or or even like you know, in security or outside security, start hearing the notion of trade-off between operational risk and security risk. It's always been operational risk. Uh and last week I was with a you know, like 100 plus CISOs in New York, and people are now starting because of this like you know, real notion that there is an autonomous attacker out there, lots of attacks have happened.

23:30 They can automate, you know, finding these chained exploits and vulnerabilities using lower, you know, things that basically have been our you know backlog, right? Like someone said, Mythos basically was a margin call on your technical debt. And so now how do you how do you erase that security technical debt?

23:47 Right. And the only way to do it is to make that trade-off, saying, I am okay accepting this because I will automate fixing this issue. And maybe that is that culture change that changes that last, you know, that the stragglers, because you have to do that in mass, you can't do that in pockets, you can't just have security engineers doing that.

24:07 Yeah. And I actually think that could be an unlock, right? Like a silver lining here is as organizations, as more organizations start asking their their security folks and their businesses, like, hey, we need to make this trade because it's more important that we fix all that. That starts changing the culture, and sometimes a crisis is what you need to change the culture to tip forward.

24:28 So the I so I love that, and I've actually really felt this in the conversations here with sort of different CISOs. Uh, this sort of appreciation. And you know, to be clear, this is like some of these are banks or like very large, like Fortune 100 uh organizations, so sort of uh very sort of security-minded organizations, but also very uh sort of ops and uptime minded organizations, like they don't really tolerate sort of and and you really see you know kind of explicit statements.

24:56 It's like I think we are near, you know, some are already at, a few are already at, but many are saying that they are near uh the point where saying I would I would just accept an outage. Now I want to minimize the risk of an outage, which I gotta say from the Tesla lenses, that's good, you know, that's a part of what we do, good code review, and sort of you build those out, and and so like we'll we'll kind of help you maintain that quality.

25:18 But uh, but it's it is interesting to say okay, you just go all the way the patch. And uh this was very much like my conversation with Caleb uh Sima as well, very much around it. But he's again, he's in the in the cutting edge, so it's I almost expect to sort of hear that from him. Uh and I feel what's really interesting is to hear CISOs of of very large companies saying this is the conversation.

25:41 And someone yesterday told me that I feel like you know, for the first time, I feel like I can go into the CEO's office and say we need to shut down all development as we just invest in building uh in and kind of covering the sort of technical debt and building this pass. And it would be a a viable conversation, uh, which is pretty strong sort of statements.

26:02 Yeah, I think I think it is uh to your point also, this is why I feel like this could be that uh you know urgent crisis trigger for a big change that needs to happen. And the second part, well, I think generally, I feel like software engineers, we get trained on innovation and building, not on safety and security.

26:22 Right. And so as a result, what are we good at? We're good at fixing our application if it breaks. Because we understand the business logic, we understand the app, we understand the operational part, there's platform engineering teams, you know, who have for you know, like DevOps, to SRE, to platform engineering.

26:38 Uh, you have a lot more of this, you know, automation and scaffolding there on the operational side. Um, I think like if I were to say two things, one is security teams how to like push that, you know, and especially product security to get that level of scaffolding for that security, you know, auto PR and merges, and we are trying to help there, but the second part is assume that we will be good at fixing the operational issues faster than security.

27:08 What else could you do to get ready if you feel like you're going to take more operational risk? Right. And maybe that, you know, is part of that impetus to go to a true software factory in some sense.

27:18 Hey everyone, hope you're enjoying the episode so far. Our team is working really hard behind the scenes to bring you the best guests so we can have the most informative conversations about agentic development. Whether that's talking about the latest tools, the most efficient workflows, or defining best practices.

27:35 But for whatever reason, many of you have yet to subscribe to the channel. If you're enjoying the podcast and want us to continue to bring you the very best content, please do us a favor and hit that subscribe button. It really does make a difference and lets us continue to improve the quality of our guests and build an even better product for you.

27:54 Alright, back to the episode.

27:56 So let's uh let's talk a little bit about the use of uh security in in agentic uh kind of development or the sort of world of software defense factories and sort of building those. Um maybe I'll kind of tee this up a little bit, right? When we when we talk about uh um security in the in the agent era and the sort of the software factory era, um we we end up talking about this is sort of true in Tesla and true in sneak, so this is actually like a good overlap area uh for it, which is we talk about this sort of this division of securing the product and securing the factory.

28:28 And so in the securing the product side, very much about automation of uh the um uh of securing the product, it's a lot of what we've just discussed on it, just bringing in the sort of vulnerability patching and sort of other security controls into it. And the securing the factory is around you know a bunch of these sort of agent protection and these problems you mentioned before.

28:48 Um so maybe within that context, uh like how do you feel? I know it's sneak sort of moved into the sort of agentic app set of products on it from the scan. You want to say a little bit about how do you think about the securing the product side of it from from Snake and the sort of the new lens, you know, and how has that maybe changed your views of how did we see it before?

29:11 Uh I would say, you know, traditional app sec, which Sneak, you know, uh thanks to you.

29:19 It's a little bit funny to sort of ask you about Snake instead of doing it, but you're actually a lot more in the detail of it. I'm just that I'm just like a boring board member at this point.

29:26 If you if you go back to why you did it, it was to s change the traditional app sec uh era. The you know, I remember our first conversation, the auditor, the you know, the gatekeeper trying to do essentially became a C by A. And if you think about what they were trying to optimize for was that scarcity of human bandwidth.

29:44 I can have only that much dev bandwidth to fix issues. So I'm gonna like, and and then you do it late, so you just like you know, piling up these issues, so you can fix fewer issues. We feel, I feel over the 11 last 11 years are being able to like shift that in into the you know dev loops and it it's we're here, we're obviously successful, almost 5,000 customers starting with the biggest in every category.

30:09 Yeah, there has been progress, but I don't think we moved the toil. We moved it you know to a point where if you really did a good job, you can enforce the ID and the PR, and uh but it I think that breaks at the pace of our true Agentic software factory. Uh and so one of the notions there is could you what are you doing?

30:32 You're providing security context automation and learning to the dev. What if you provided that to the agent? Can you shift it into the agentic loop itself? And guy, you brought this example of of uh packages, right? Like there's there's bad, you know, unhealthy packages that are more susceptible to risk.

30:48 Yeah, there's ones with vulnerabilities, and now there's you know, with uh zero days, there's a lot of malicious packages. Yeah. And so if you could block that right at the point where the agent is making a choice or provide the agent, like, you know, this this one is a QR code scanning package that you know has not been updated for 10 years, and this has been updated yesterday.

31:06 And as long as the one updated yesterday doesn't have malware, yeah, that's probably a much better choice. So if I can provide that context in the loop right before you get the output, the best part, right? And so what else could we do there is what we started thinking, and like and it's a prevention part, right?

31:22 And prevention, I feel, can be a lot more effective than even in shift left with the devs. So shifting into the dev context is one thing that is changed with oh, there's agents producing code. Yeah, actually, good news. They're better at following instructions, yeah. Mostly you gotta keep an eye on whether they're doing it or they're getting overloaded, too much context, too much cognitive, you know.

31:44 Yeah, but it's more about the effective, there's no reluctance because there's no there's no agenda, you know. The agent is just right.

31:50 It's just a technology problem you have to solve on how to do it effectively. Speed is another thing, like you do have to be very fast. You can't just put a model in there that's gonna take a long time crunching. So it's a great way of taking some of our deterministic tools and technologies and Intel and customer, you know, like global context.

32:06 Uh so that's something that we're seeing really good pro like this is about 1,500 plus enterprise customers who adopted this notion of a sneak studio, which is what where we are feeding our context to the agent. Right, yeah.

32:19 And I love that sort of phrasing of it of we've been providing context and tooling to developers, now we provide the context and tooling to the agent as a as a new as a new coder, right? Like it's sort of a new entity that is now producing the code.

32:31 Exactly. So that that's step one. Now the second step is interesting because everyone has a backlog. I have a backlog. I don't know about you. You probably like uh you know erased your backlog.

32:41 So well at Tesla we're like fortunate in the sort of the remediation, uh the it's all software factory on it, so the backlog did actually get addressed. But it's uh it's the uh the the goodwill of like or the good fortune of a of a new company that is agented from the city.

32:55 So if you have any history, you have a backlog, right? And you got there one way or the other. So right now with the autonomous attacker, that became the biggest problem. Like, you know, prevention is good, but what do I do with my backlog? And and and there's like really stark numbers there because speed has increased, security is not kept up.

33:16 Um, we've seen uh 10x increase year over year on on the backlog. And for every issue that is being fixed, eight new issues are being created. So that's not acceptable. And and yes, prevention can help if you really do that at that point, but I I have a huge growing backlog. So this is probably the most promising use of like an agent to do that remediation.

33:42 Uh, we've been slowly building towards that. We released skills for Cursor Cloud Codecs, and there's organizations like LabelBox and uh Relay Networks, you know, case studies, right? We got to backlog zero just using the skills. Why? The skills were more security context for your coding agent to use to safely get rid of your uh security debt.

34:04 Then we released uh a lot of customers said, well, we have places where we don't have uh you know an actual coding agent. Um it's uh you know, I just need to go do a bad job. Can we get a remediation agent? So we did that, and it was a little bit of bring your own model. We learned a lot there. We were um we were a lot more efficient than just using, you know, we have a benchmark called um Wall and Bump.

34:28 You know, my team's gotten really creative in these benchmark namings. So VULN Bump.com, you can go see the results. But it's it's you know really good to see that okay, we get a 90% merge success rate using us plus cloud versus a 60% just with cloud. Yeah, and it's 61% more token efficient.

34:45 So that's effectively the value of a harness, right? Because you're you're sort of building something that is uh attuned for it. It's not just context and guidance, it is an actual kind of uh built for purpose.

34:55 Built for purpose. And where it's uh taking advantage of the global context is like things like breakability. Like we see across you know millions of repos and upgrades of dependencies which particular upgrade path is going to, for this application, be the safest one to do. So you got a uh network effect data starting to come to play.

35:16 Um we now just today released a cloud remediation agent, so you can kind of do a you know a broader campaign across all of your vulnerabilities, and and we're incorporating a model and a harness and all that in there. Uh the step that is before that, you do want to use frontier intelligence for detection too, which you know you have a deterministic engine, it's finding things.

35:40 Frontier intelligence is finding other things that only you can find only with context. So our first application is we kind of use that for triage so you can reduce the number of real issues versus This is the so uh just sort of I guess give a view of it.

35:54 This is kind of an understanding. If previously this was how you found a vulnerability, here's information about this vulnerability act. And then these skills and all that are really about engaging better with an agent, where the agent's kind of role is maybe to understand a little bit the application and analyze it and all that, and here with this information, and these a little bit of guidance of sequences like verify the fix before you do that, the remediation agent kicks in.

36:15 This is the sort of last part is a lot more about kind of the broader application context of it, like understanding fine, you told me about whatever, like non-lack of input sanitization in the spot on it. But what does that mean for my actual application? Does it mean there's some exploitable path?

36:30 Yeah, and the the the use the thread model, use the full, like you know what business logic is understood. So you can be a lot more effective at saying like these are the real things, like is this is not prioritization. I feel like this is like truly deleting stuff that doesn't matter, right? And then focus on the things that matter, and then you know, like essentially are able to do a batch mode remediation of things.

36:51 So we're finding great results with that internally ourselves. You know, we've put it at like a hundred plus repos that we have subset of our full thing and 93% merge success rate. Learnt a lot, but what about tests? People that have good tests have a good job of you know fully merging. So you then like start going to the point where you're like, okay, I need to like you know, almost like do a uh a you know, PR agent, I think we have talked about this, or a critic of that PR that was opened by the machine, like to say, how good was that?

37:23 You know, is this going to be successful? And I think the final step there is like automatically, like if I can build enough confidence with the human in the loop that this is working well, then there's more likely by policy they're gonna say, okay, this class of issues you can automatically.

37:38 This autonomy. So I think thanks for that. I feel like the sort of last bit is really where it starts touching the factory on it. And and I feel like the reason AppSec existed is that it or like exists today, is that it it carries these sort of two competences in the world, right? Which is like in the organization.

37:52 One is the the sort of the knowledge of uh what is secure and insecure, you know, what like what is correct behavior from like what an attacker might exploit. And the second is like how does an application behave uh and what would break it? And I think agents are just raw intelligence, and so to an extent you can feed in information from both of those, and the opportunity of blending the two together comes in.

38:14 But if you want to get autonomous merging of it, uh that's the part in which you need to get the engineering to manage sort of risk, and maybe that's like a software factory line uh versus if you want to understand which of these risks even you know merits the tokens, you know, and uh which you know really should like maybe a conveying of what's the other side in that risk equation, you know, to the factory.

38:38 Uh that's a lot of this sort of like agentic app sec and sort of uh uh fitting that in.

38:42 Yeah, I think I think I think that's a good way to think about like the separation, and it goes back to what we were talking about the trade-off of operational risk and security risk. That's a perfect place where this comes to life, where the factory with the engineers who are like driving the intent and orchestrating it, if we have this kind of you know agents for doing the app sec, know how to incorporate that into the whole factory process so that it's going to be a lot more effective.

39:11 We're bringing that global context, we're bringing the automation, we're bringing the security understanding, uh, but bringing it in a way that can be leveraged by other higher order agents in in that factory, like almost like the security fabric for that factory.

39:26 Yeah. So I feel we're kind of running out of time a bit here. So we talked a lot about um just sort of in general, like kind of enterprise adoption, you know, and sort of being hindered a bit by that sort of risk and sort of controlling it uh in the future. Uh we talked a bit about some of the current pains around embracing the factory, malicious skills, malicious uh uh MCP servers, uh behavioral kind of uh containment, and then how kind of the agentic appsec kind of evolution is able to secure the product side, right?

39:57 Going from starting from delivering that type of context and information to the agent versus the human to dedicated harnesses around remediation. And uh the first one was more in prevention, you know, the second is more on remediation, to uh much more of like full loops that try to collaborate with hopefully an agentic functionality on the other side uh that is able to remediate things.

40:18 I'm curious, uh maybe as a as a closing question, uh as we as you as you look at these enterprises that are today running agents in like a more contained fashion, they don't kind of provide you know internet access or or others. Uh and and as the factory, like I I think today the products are such an easy target uh for agents that they're sort of pursuing mostly those.

40:42 But as agents target factories more and sort of target that agent manipulation more. How uh like what do you think if you kind of I don't know even what timeline to sort of offer really in sort of factories, but I guess what do you envision as the future of sort of enterprise agentic automation? Like what type of control do they need to have to be able to trust these agents from a security lens, not you know, I think we understand the breakage risk, but from uh uh operating those agents securely, uh what type of control do they need to, I don't know if the right word is let go, right?

41:19 Or like trust enough uh that would run them with broad responsibility.

41:24 Yeah, I I think the biggest, you know, like you talked about like the three aspects, the output, the supply chain, and the behavior. Um there's obviously the I think about it like the infrastructure layer and the application layer, typical security way of thinking. I think in the infrastructure layer, like you know, sandboxing and all those controls will become more mainstream.

41:46 Um you kind of need to like curtail your blast radius, but it doesn't do anything at the application layer. Right, yeah. It's just protecting your local network and machine in that sense. So it's great, you know, do it. Um I think the two biggest things I feel that are kind of fledgling technology capabilities, but not yet fully solved in a robust enterprise forward way, is uh the identity stack for sure.

42:12 Like, you know, what do you really do with these agents and the identity? And eventually I think that part is interesting because it connects back into the you know the the origin source and you know going back to like who created this and yeah, like how the the intent, like your the agent is coming and asking me to do a thing.

42:29 Yeah like what is it trying to do? Like what is it trying to uh that it's asking me for permission to the system, you know, for right.

42:37 And then like solving that on the fly and understanding the intent, you know, uh and then deciding how much permission am I going to give that at the moment. Um so I think that will get solved, but that's one that I think is uh not that needs to get solved for more trust. And then the second one is the behavioral part, like let's even take the internet disconnected thing.

42:59 Part of it is like what do what does this agent pull from the internet? And the internet is a scary place with a lot of, but you know, you talked about slop squatting, right? Like people, these agents are hallucinating a package name. There's a little bit of interesting predictability in the hallucinations, so the packages are malware is being you know put in those uh drop points.

43:20 Those are the kinds of risks. So could you on the fly really figure out based on the memory and the history, what in this particular action, what could the potential behavior be? And not block it because that's old school security. And it's like, what do you do to steer the agent back into the like you know, okay, that yeah, that's not something you should use.

43:43 Steer it back on the rails. And I think those are the two big things I feel like truly unlocks it for the enterprise. Uh everything else is, you know, like it's like the future is here, but it's not evenly dispersed moment. Yeah. A lot of things, it's awareness is missing. Yeah, there's really good supply chain scanning capabilities from us, a few others, good sandboxing, good way to test the output.

44:08 Uh I think that's an awareness gap. This is really a not yet fully solved properly gap.

44:13 To get ahead. Yeah, I I like that. And uh like I really love like you call it, you talked about infrastructure and application layer, and I increasingly think about these as like security and safety. Yeah. Sort of uh imperfect uh split, which is containment, security, as in like I told you to be in the sandbox, stay, you know, like don't kind of go over there.

44:30 And then safety is interesting because it comes back to maybe like the future of AppSec, which is um like if if you think about a skill and it's behaving in a certain way, understanding whether that uh that behavior is uh correct or not depends on understanding what is the skill and what does it do. And so a lot of that intent analysis, a lot of maybe like the skill definition, those are around the sort of the the band, like they need to combine an understanding of what the business understanding of what is it that it's trying to do and the sort of security understanding, and kind of comes back to well, this is kind of what we do in AppSec.

45:04 So and I think they may be the last you just touched on it, it might be the hardest of everything we've probably not discussed, but maybe it's uh it's the next time. Businesses have never been really good. Like I built a data classification taxonomy product like 20 years ago, yeah. And no one knew how to put the rules in for that because, well, the security guys or the admins don't know the business, and the business doesn't understand.

45:28 There's no security. Yeah, I think agents can actually help bridge that. Yeah, but that's a critical bridge. Like, how do I unlock something that is in the organizational brain of the business? Yeah, because that is required to then solve this last part.

45:41 Yeah, yeah. And defining what you what you want and what good looks like. Sort of the thing that we as humans avoid, like a plague. Uh and you know, I guess from the Tesla lens perspective, we often think about this as like these are skills, like you know, that is where you codify it. Uh, that's not to say how do you kind of extract that.

45:56 Yes, how do you extract that? Yeah. Cool.

45:58 Well, Maloj, this was a great conversation. Thanks for taking the time over here.

46:02 Awesome to be on uh you know your podcast.

46:05 And uh thanks everybody for tuning in, and I hope you join us for the next one.

46:09 The AI Native Dev is brought to you by Tesla, the package manager for skills and context. Your hosts are Guy Pajani, and me, Simon Maple. Our producer is Tom Dowler. The AI Native Dev is not just a podcast, it's a community. And we host monthly meetups at the Tesla offices in central London. Visit Tesla.io forward slash community to learn more at our hope to see there.