
Kapitelklicken springt direkt hin
Shownotes
I sit down with Vas from Varick Agents to map out exactly how to break into AI forward deployed engineering — and how to grow into a sharper FDE — in thirty days. We start from a single premise: every company can now buy the same frontier intelligence, so the real advantage moves to deployment. Vas traces the role back to Palantir, explains the judgment that decides where AI belongs, and lays out the audit → evals → deployment loop that turns raw models into measurable business value. He then hands over a full 30-day plan to build, harden, measure, and defend a production-grade agent, so you can do the job before you hold the title. The whole conversation stays tactical and grounded, with clear examples I can apply today.
The FDE Blueprint: startup-ideas-pod.link/fde-starter
Timestamps
– Intro
– What is an FDE
– How Palantir Popularized FDEs
– Deciding Where Intelligence Belongs
– What FDEs Earn
– Two Kinds of Judgment: Communication and Engineering
– How the Work Really Gets Done
– Audit, Evaluation, Deployment
– Which LLM to Choose
– Audit: Finding the Workflow Worth Rebuilding
– Evals: Turn non-determinism into evidence
– Deployment: Build on Existing Systems
– The 30-Day Plan Begins
– Final Thoughts
Key Points
• Intelligence is now commoditized, so the real edge lives in deployment — the job of the AI forward deployed engineer.
• Vas traces the FDE role to Palantir, where engineers embed on-site, learn workflows, and customize the ontology per client.
• The strongest FDEs blend deep technical skill with consulting-grade communication — the rare "art plus science" combination worth up to a million dollars a year.
• The FDE loop runs audit → evals → deployment, and each improved workflow makes the next one clearer.
• Vas condenses a year of learning into a 30-day plan: build an agent, harden it, make it measurable, then defend it like an FDE
The #1 tool to find startup ideas/trends - ideabrowser.com
LCA helps Fortune 500s and fast-growing startups build their future - from Warner Music to Fortnite to Dropbox. We turn 'what if' into reality with AI, apps, and next-gen products latecheckout.agency
The Vibe Marketer - Resources for people into vibe marketing/marketing with AI: thevibemarketer.com
FIND ME ON SOCIAL
X/Twitter: twitter.com/gregisenberg
Instagram: instagram.com/gregisenberg
LinkedIn: linkedin.com/in/gisenberg
FIND VAS ON SOCIAL
Varick Agents: varickagents.com/#hero-section
X/Twitter: x.com/vasuman
AI Forward Deployed Engineers: learn.varickagents.com/fde-in-30-days
Transkriptvom Podcast bereitgestellt · Zeitmarken springen hin
I know it's crazy, but there are people making a million dollars a year as Ford Deploy engineers. But what exactly is an FDE? I know you've probably seen it, but I feel like a lot of people aren't clear as to what it is and how they could become one. Well, in this episode, I brought on my friend Voss.
And Voss is a leading expert when it comes to FDEs with his company, Veric Agents. And in this episode, he gives you his entire playbook, how you could become an FDE in 30 days. Now, this episode is for people who want to become an FDE, but also for people who are just interested in what it means, how they can actually use FDEs in their business to make more money, to be more productive.
And I just think this is the clearest episode, the clearest piece of content on the internet, the clearest masterclass for how to understand clearly what an FDE is, how you could become one, and why it matters. Enjoy the episode, and I'll see you at the end. Voss is here and Voss, by the end of this episode, what are people going to learn?
They're going to learn exactly how to break into forward deployed engineering or become a better FTE in 30 days. The full roadmap for AI forward deployed engineering. I don't feel like this has been shared anywhere. I feel like the term forward deploy engineer is just on my X feet everywhere. So what I'm hoping for, Voss, is for you to clearly explain what this means and all the concepts to it and just break it down for me in a clear, easy to understand way so I could learn from it but so others can learn from it too.
Absolutely, and there's a lot of different definitions. Everyone has their own and I'm going to give you what I think is the clearest explanation. Okay, let's do it. intelligence is becoming commoditized. So the same foundational capability is becoming available to anybody who can pay for it. And that's most companies today.
So you'll see here is a graphic where every company has access to the same foundational model. So if everyone can access it, intelligence can no longer be the mode. I think there was this huge theory that people will be priced out of intelligence and that could be the case in the future. But the reality is today, everyone has access to the same tools.
If you go and talk to 50 different enterprise clients, they're all using the same stack. They're all using Cloud Code, Codex. They're using Cursor for model agnosticism. They're using GitHub Copilot. It's all the same thing. So the reality is everyone has the same capability in terms of what intelligence tap they have access to.
So where does the advantage go? It goes into deployment. So the edge is no longer who has the intelligence. It's where, how, and why they use it. And that is the role of an AI forward deployed engineer. It's allowing companies to harness and take advantage of AI intelligence or software previously to make sure that they apply it the best to their specific company context.
Every single company is different in terms of how their business is structured, what processes they have, how things run, etc. And the advantage will start to become who has that best bridge, that connection between their own processes and the intelligence stack that they have access to. And Voss, this was a term that, correct me if I'm wrong, was popularized by the Palantir team, right?
That's right. Can you tell me about how Palantir works with FDEs? I feel like for a lot of people Palantir is this black box. Can you go a little more into that? Yeah, it's funny. I lived in New York for a few years and I had a ton of buddies who were Palantir 4.0 engineers so I have a little bit of insight into this.
Without sharing what I think of proprietary, Palantir has an ontology. They have a software stack that they work off of and this is full of connectors to software, but also data lakes, which allow enterprises to pipe their data into a unified interface. And what Palantir FDEs then do is they'll actually be deployed on site.
So this is either like enterprise customers or the military, the government, and learn their workflows and then spin up kind of workflows, dashboards, agents that will solve the problem for these companies. So Palantir really popularized the idea of essentially what was consulting, but for the software space, they coined the term forward-deployment engineer and it really started to allow their business to take off.
They had a centralized platform, but its beauty was not in how tech-forward it was, it was in how customizable it was. And then these forward-deployment engineers would go on site, customize it for a client, and it would really solve their pain points better than a generalized service. Cool, and I guess the thesis is that if it works for Palantir, it can work for everyone else, right?
That is sort of the thesis. Yeah, and the Palantir, I think, really solved it when it came to the data age, where you wanted to unify your data sources and then visualize it in more unique ways. I think the AI age is going to demand that a hundred times more, where every single company is going to need customized agents.
And that's actually what we're here to solve as well with our OS. But everyone is sort of coming to the same realization that forward-deployed engineers are a massive reason why AI is going to be powerful for businesses. Cool. Let's keep going. Sweet. So someone has to decide where intelligence belongs, not just where it's applied.
And that person is a forward-deployed engineer as well. So there's kind of three stages to a forward-deployed engineer's involvement at a company. The first is understanding the business reality. So it's how the work actually happens today. And I think this is the part that gets glossed over by people who are very deeply in tech.
In Silicon Valley, we have a mindset where like, okay, well, the beauties in the software, like it doesn't really matter what the business processes are, but I can tell you firsthand, every business is so different. Even in terms of like the same process across multiple businesses, let's go like accounts payable, for example, or sales, for example, at two different companies.
The way that they do that is so different. One has a 10-step process. One has a 30-step process. One is using Salesforce, Gong, and ChiliPiper. The other one is using HubSpot and Apollo and Clay. There's the software differences, but there's also the process differences. And there's the things that matter most to the business being very different.
When things go wrong, like exception handling, the way that that's being done at a company needs to be documented as well. So for deployed engineers, go on site. They'll either interview people or just observe them work or get access to their systems, like their ERPs, their CRMs, to figure this stuff out.
And this is where the bulk of the time goes, in my opinion. It cannot be understated how important this is, both in terms of understanding the business, but also to bring that business along the journey. And this is where communication and analytical ability is incredibly important for a forward-to-point engineer.
the way I like to think of an FDE is the best combination of someone very deeply technical who can understand it but also someone with fantastic communication ability and the ability to get the information out of people that they need to and this is all encompassing for business reality and you say the FDE goes on site when you say on site is that literally walks in the office and starts speaking to people and stuff like that or do you mean like you know, it can, it can certainly be done remotely.
Yeah. And, you know, there's, there's certain times where that's required because the company itself is remote or like the people of the function are not all in one office, but I will say a large majority of the time it is on site. I know that Palantir does this very heavily. We do this as well. And it's not just because, you know, you can't get the information remotely, but it's, it's, it's actually more so because the relationship that you build with the person, like, you know, you're on site, you're part of the team, you'll uncover far more information that way, right?
Like if somebody will, if you schedule a one hour meeting, for example, somebody will walk you through what they think is the job. But if you're on site with them for the full eight, 10 hours, whatever it is, you're actually experiencing the job. Like when something goes wrong, that's not really documented in an SOP or like a, you know, Word doc or Google Docs somewhere, you're going to see that play out.
You know, even consultants at McKinsey, they do this, they'll go on site to a mine, and they'll sit with the miners and they'll watch them do their work, because it's so much more powerful to establish that relationship and get the information that way. Cool. Yeah, the second step is FDE judgment, which is, where does intelligence belong and where does it not?
I think when the AI wave first started, we saw a lot of, you know, let's just slap AI everywhere. Let's give everything to the model and let's let it figure it out. and this is what led the token maxing and hallucinations and you have the MIT stat that 95% of generative AI pilots fail. Again, now the industry is sort of shifting gears and they're realizing that okay, we actually need to be very selective about where we apply this intelligence.
And we also need to be selective about how we design the new stack for this intelligence to play out. So again, for example, you have a 10-step workflow. It might be that that workflow should not be changed by AI. Maybe it's too risky or maybe it's not high enough ROI. Maybe it's already pretty automated.
Why do we need to bring AI into it? It also could be that of those 10 steps, only three of them actually need judgment. So categorization of this lead in a CRM tool, that might be a little bit more non-deterministic. So we're going to bring in LLM there for judgment. But the rest can be solved with if-then-else statements.
It can be solved with API calls. And this sort of judgment is actually far more complex than I'm making it out to be even, but it belongs with the FDE. So the FDE both has, again, the business reality, which is the consulting style, like communication style approach, but also the technical judgment so that they can determine based on their technical background, okay, I think that this design is going to be risky, we're going to have 80% accuracy, it's not worth it, versus this other workflow will have a much higher ROI, we'll be able to build it much faster, it's lower risk, etc.
And that sort of back and forth judgment is where an FDE really shines. It's bridging the gap between the business and the technology. If someone listening to this wanted to become a Ford deployed engineer in New York City that's doing this sort of stuff, how much money could they make? lot of money.
You have no idea how expensive it's gotten, both from a we're hiring perspective, but also in terms of what the market's demanding. This is the hottest role in technology right now. I mean, you could make anywhere from 150,000 base with considerable equity to, I've seen the roles go up as high as a million dollars a year, and I'm not joking.
These are extremely well compensated roles if you are the best combination of consulting and technology. Cool. Deployed AI system, finally the last step is actually going out and building the software itself. This is where it varies wildly company by company. For example, at Palantir even, they have some FTEs that you're not actually writing code, you're mostly spinning up workflows, like text, you're chatting with the software of the Palantir ontology to create some dashboards and to the extent that you're writing code, it's SQL.
But there's other companies where you are fully writing production code, either on-site with a client or you'll go back and do this, but it varies wildly. So there are some FTE roles where you're going to be writing production code. You need to have a background in software engineering and really be confident in your ability there.
There's other FTE roles where it's far more technically light and you can chat to build on top of an existing platform. So this is the part that varies wildly. But either way, you need to have a very good understanding of the software because when the client has an issue with, hey, this doesn't work right, or we have an issue in production, it's basically your ass on the line.
And you have to know who to call, what to do, and how to fix it. In summary, FDEs are in demand because they control how intelligence enters the business, how it's used. And that is where all the value is today in the AI age. Everyone is coming to this consensus, and that's why FDEs are extremely valuable.
Well, it's also in demand because it's new as well. There wasn't super intelligence on tap five years ago. So not only is the idea of super intelligence on tap just absolutely absurd, many trillion dollars of money is going to be changing hands over the next few years, but the idea that now you need a person to actually, people are realizing, hey, you actually need judgment and hey, you actually need to be a systems thinker and hey, actually token maxing isn't the best strategy.
There was a moment in time where token maxing, people basically were agreeing that token maxing was the strategy. It was just like, hey, these models are so good, let's just let them do their thing. That was the thinking. Yeah, that was a funny period in time. I would argue we're still not fully out of that.
But yeah, you're totally right. I mean, this is a brave new world for everyone involved. And I mean, I have horror stories of people, of like C-suite executives I've talked to who have blown through their entire $10 million clogged budget in like three months. It was supposed to last them a year, but because they gave it to everybody, it's token maxing and everyone's spinning up whatever they need.
And the sad reality is it didn't really move the needle for the business either. And it's because that business didn't really invest heavily into forward-point engineering. and so I think you're totally right. Cool. Let's keep going. Sweet. So we've already alluded to this pretty heavily prior but I want to really make sure I hammer down this point which is that there's two sort of streams of kinds of judgment that are required and it's very rare in a single person so I think the unfortunate reality is and this is what's going to happen, it's already starting to happen is as we go from the token maxing let's go all in on token maxing, go, go, go, to now the same thing on FTE, let's go, go, go,
let's hire more FTEs. I want to be very clear about what the role really demands. I think there's a lot of FTEs who are, you know, unfortunately, neither the best communicators and neither the best software engineers. I would strongly urge them to strengthen both of those skills. It's both the understanding of workflows, cost, incentives, risk, adoption, business value, you know, the politics of the internals of a company.
These are all things that you have to manage. And consultants here are incredibly strong, right? You'll talk to McKinsey engagement managers, BCG, Bain engagement managers, they'll be very good at this side. The other side is what they might need some more support in. And same thing, software engineers will be very good at the right side with models, systems, APIs, data, code, reliability, evals, guardrails, you know, more AI-centered terms, harnesses, post-training, fine-tuning.
These are things that are more on the detectable side of the aisle. And software is usually very good at this, but they need to also then kind of drift towards the business side by understanding the left side of the aisle. And FDE is the best combination of both of these. It's not an average combination.
It's not the worst combination of both where you're not the best communicator, but you also can't code. It is truly the best of both. And that is the million dollar hire, where the FDE can turn business understanding into working software end to end. They can do both sides perfectly. put another way if you understand art and you understand science and you can speak both you have what it takes to become the million dollar FD the hard part is usually the people that are good at science are sort of good at science and the people that are good at art are kind of good at art but there is some overlap in the Venn diagram absolutely And that's why it's such a rare role.
But I also firmly believe, and that's kind of the whole point of this presentation, is that you can become this. It's not out of the realm of possibility to become much better at both of these things. You need it cleanly laid out. You need a roadmap. And that's what I hope that I can provide by the end of this call.
Cool. All right. All right, Voss. Let's go. So first, for example, it's understand how the work is really done. because the documented process is very rarely the real process, right? So an email might arrive. Now, this is extremely simple, but the reality is it sounds like a clean trigger, but it's far more complicated than that.
It arrives from 40-plus senders. No two of them are formatted alike. The data is different. Some of it's in a PDF. Some of it's in a screenshot. Some of it's in an Excel spreadsheet, or it's buried in a forwarded thread. It's far more complex than it makes out to be. So if you didn't look into this, if you weren't an FTE and you just asked the person for, hey, what's the first step of the workflow?
They'll tell you when email arrives. And all of a sudden you're building for a system that doesn't map to reality versus the reality, which is that it's so complicated. And half of them are exceptions. It's the same as last time. It ignores the second attachment. Sarah already signed off on this one.
There's no consistent subject line, so you can't route without actually going into it. and usually the reality of how to play this is in one person's head. So one person will know like, okay, yeah, well, when I see this email from this person, I'll send it to this vendor or to this part of our procurement team, but that's not written down.
And if you don't sit with that person, kind of coax this all out of them, they're not going to remember to even tell you. You would think about like at your job today, I ask people doing this, how easy is it for you to really write down every single exception that might happen in your job? I was a software engineer at Meta, and if people asked me for my job, I'd say, well, yeah, I code all day.
I'll get a task, and I'll work on it. But that's not the reality, right? The reality is I have meetings. You know, this happens, something breaks in prod. I have to go fix it. That's what we're getting at here. The second step is, oh, it's copied into a spreadsheet. Same thing here. You get the idea.
One's a real one. Two are stale. Data validation. It's rekeyed by hand. Columns drift. Same thing with checking into an internal system. I won't get into all of this. You get the idea. Every step is extremely complicated. so that's what understanding the work really means it takes time and it takes effort to sit with a person responsible and sometimes multiple people responsible usually usually it's multiple people very often yeah I mean if this company has like 5,000 10,000 people working in it chances are you have a lot of people working on the same thing then you decide how the workshop right when intelligence is built in so again where does deterministic software live in where does the agent act where does the human approve
where does the record get updated? I think the best combination of, sorry, the best solution of AI for most companies is a very good combination of deterministic software. Probably the majority of it is that, but then obviously the judgment that API calls to LLMs can provide. And then finally, human in the loop.
So this is just a fancy little digest, but it's really the agent that can then be deployed into existing systems. It's doing the first half, which is intake validation, agent drafting. then you have a human in the loop for approval. This is something that I strongly recommend my FDEs to push for in an agent implementation.
Once you get approved, it'll then go through the latter half of the steps. And that's what you're building. So the job of an FDE, when you're building has three parts, it's obviously auditing, then creating evaluation suites to make sure the system behaves correctly. This is extremely important in the AI age.
And then finally deployment, which is both handholding the client to make sure that they're adopting it, it's working well for them, but then also the software side, making sure that nothing breaks, You're monitoring all the metrics that matter. You're monitoring KPIs, SLAs, and everything needs to be top-notch for somebody to really trust you as an FTE.
And every stage is a prerequisite for the next. So for an eval, so prove the system behaves correctly. So in a scenario where the outcome is non-deterministic, meaning it's tough to say what success looks like, how do you create an eval? Or can you create an eval for more creative tasks or tasks that are hard to understand if it's successful or not?
Yeah, obviously for more non-deterministic tasks, it's much harder. It's much easier to say, okay, was this email categorized correctly because we have 10,000 previous emails to go off of and it'll be the basis for our eval set. But even for tasks where it is non-deterministic, like for example, creating a presentation.
there's a million different ways to do it and the beauty is in the eye of the beholder where one thing looks good to me might look bad to you here it's very helpful to have as much previous data as possible obviously if you have 5,000 previous presentations to go off of it makes it a lot easier to create this golden data set of what we think matters you can say always put the logo in the top left always have larger font of the styling etc.
etc. But this is obviously where you'll never get to a perfect result with just evals. You need human-in-the-loop feedback to make sure that going forward, you at least have a feedback mechanism that improves your harness, if not like post-train or fine-tunes the model that you're working under. So on one hand, like get as much data as you can and kind of determine like what looks good, what looks bad, identify like what matters to you, but also then always bake in human-in-the-loop feedback because even with a good data set, even with good evals, you'll need to have them constantly improved.
And that's where that feedback mechanism comes into play. And I've noticed in this entire presentation, this entire podcast, we haven't really spoken about which LLM to use. Are you basically agnostic in terms of working with Anthropic or OpenAI or Google? or like if you're an FDE, basically, how do you think about which LLM to work with?
Yeah, great question, actually. Something I probably should have touched on. We as a company are extremely model agnostic. So we think our value lies in our ability to be switching from one model to the next and making sure that your accuracy only improves, your cost only goes down and you're not marrying to one intelligence provider, which we think will be an asset going forward.
You don't want to monopolize your intelligence, your inference layer. That being said, if I was an FTE today, or if I was trying to become the best FTE today, I would stick to one model and one agent building platform. OpenAI has one, Claude has one, Agent SDK, etc. Every single model provider has one.
Get very, very good at one of them, because that will be the foundation that you then, let's try out Claude tomorrow if I'm already good at OpenAI's agent building platform. Okay, I feel more confident about that. Let's go to Kimmy 3 or GLM 5.2. Let's see what the open source models can do. Let's build a proprietary harness.
That's how I would go about it. But I wouldn't really worry about being model agnostic when you're starting out as an FD because that's again not where your value lies. Your value lies in how good are you at understanding both sides of the aisle because that can then apply to any model. Totally. And we're getting to a point where the models are very similar in a lot of ways.
And a lot of the big players, like Google will have their frontier model, but they'll also have an open source model, for example. And so now you're getting to this place where it's like, okay, you can play with their open source model, you can play with their frontier model. And so I expect that the arrow of progress around LLMs is they're going to have a bunch of different products for you to play with.
So yeah, I agree. Pick an ecosystem, bet on an ecosystem that you believe in for whatever reason. Be the best at that ecosystem. And then as you become the best, then it's like, okay, if you want and you're working with a client, for example, and for whatever reason, another model makes more sense, then great.
you can go and recommend that absolutely I would say that's exactly right and then just to really hammer the last point in your ability to determine what model is best for a task relies on your understanding of various different models you're benchmarking them along the way but to your point, don't put the cart ahead of the horse really get good at one before you then try to venture out and make that understanding I would totally agree I mean, that's like, yeah, you don't want to like hammer a specific, you know, model when you don't understand what the system is, the set of tasks are.
It's like the equivalent of, you know, you're a waiter and you just hand someone a glass of Pinot Noir and they're like, I didn't ask for that. you know you you know a good restaurant has a sommelier and and and sommelier's job is to understand what is your palate um do you like dry wines do you like wines from you know uh france you know southern france or northern france or you know yeah i'm not the biggest wine guy so i you know the analogy might break down but that idea i think of just like understanding what people want first and then deploy makes a lot of sense.
Honestly, I thought that was pretty good. Some of the area of agents is an FTE. You really go in and figure out what they want and then you give it to them. You might give everybody a Pinot Noir and it might work for some of them, but it's not going to work for most and that's why, again, most AI pilots fail.
100%. Cool. All right, let's keep going. Sweet. This is, again, more of the same find the workflow worth rebuilding. I'm going to link this. I'll have Greg link this document in the channel. So if you guys want to dive in deeper in here, but the idea is again the same. Collect the context, trace the FDE findings, figure out the bottlenecks, the repetitive work, the judgment points, all that stuff, and then produce the operating map.
And this back and forth is why having the understanding of the business and the tech is super important. Cool. By the way, if you go back to the audit, if you want to be an FDE, we just had an episode with Corey Ganim he came on the podcast and he basically he talked about selling audits as a way to learn about someone's business and then deploy AI afterwards like you can sell the audit right?
Yeah. And then the implementation you can charge like a monthly fee or you can charge a one time fee. How should people think about that? Yeah. So actually, we're in this exact business of, you know, implementing AI across the largest companies on the planet. And we require every single engagement to start with an audit, which, you know, obviously costs money to the business.
This is extremely valuable. I think, again, like there's a lot of misunderstanding, like, you know, you can just throw AI on the company. The audit is worth so much money to a business. I mean, we've had companies that tell us like, the audit was worth 10 times what they paid for it. So it's better than McKinsey because it's so telling.
AI is so new, like you said, no one really understands how to go about an audit. But if you're able to say like, look, here is in your department, here are all the different workflows and we've mapped them out very cleanly, right? We have the full steps, the back and forth, the exception handling. We're going to map that out for you.
And we're also going to tell you what we think is worth automating versus what isn't. give them that priority map, give them that matrix, that ROI matrix, and then go ahead and show them how you would build it and show them the ROI, show them the use case. That is worth so much money to a company. I think this is something that even most consulting firms are not able to figure out.
And this is where you have an edge if you are really up to date on AI and you actually know, live, and breathe it yourself. The audit is worth a ton of money to a business. Totally, it's also a chance for you to build trust with them and show them how you work and under-promise and over-deliver and it gets their creative juices flowing around like, okay, I didn't realize that because you're producing an operating map so you didn't realize, oh hey, I never thought about that use case or I didn't think that this would produce this expected business value, maybe it's worth investing in.
Absolutely. It's funny, when we first started the company, it was last year, this was before FTEs were really a big thing. We used to call the audit the medicine that neither one of us wants to take. A lot of companies are like, do I have to or not, I can't, you just token max and start building. But it really is so valuable, and they realize that as the audit goes on, so I totally agree.
Yeah, we also have an agency called LCA. And LCA is well known for building, working with the biggest companies on the planet and then taking their products from a product perspective and bringing them into the AI age. You work with a Dropbox, and what does an AI-first version of Dropbox or Slack and AI-first version of Slack look like?
and we started doing audits as like, hey, let's audit your product first. What we noticed was the word audit was a tough pill for people to swallow. And we just rebranded audit as a sprint. So it was a design sprint. And we brought in the concept of an audit with it. So we noticed that that worked better.
Just a little tip for folks. Super helpful. For some reason, people have an allergic reaction to the word audit. Well, I mean, they think of a tax audit. Yeah, fair enough. The AI audit doesn't have the same ring to it, for sure. Yeah, cool. Let's keep going. Again, deterministic software versus an agent versus a human in control, I'm not going to beat a dead horse.
You've got to prioritize the high volume workflows where the improvement is large enough to matter. that's your job as an FBE is to figure that out firsthand. Evals, you turn non-determinism into evidence. You've got to make sure that you have the right data, the required steps, it matches the expert, and it's safe to act on.
And you've got to make this kind of matrix. And wherever you feel like it's not safe to act on, you know, you route it to a human, you create an evaluation report, right? So you have 50 runs and 41 of them passed. Of the nine that didn't, let's investigate why. five of them had missing data, four of them had the wrong record pulled and then you use that to improve the system.
Greg, you touched on this earlier, how do you set up evals? This is sort of the framework that I would use. This is cool, by the way. Yeah, it helps to just have a decision tree, a matrix when you're thinking about things. Again, because everything is so new, you could go a million different ways but I'm sure there's other ways to do this, but this is ours and this is just how we think about things at a high level.
it depends on the case like it spaces for sure so how do you make it deploy and work inside the business this is again phase three the first one is the audit the second was evals third is deployment one we really preach about like integrating with what already exists um i think a lot of ai folks are forcing migrations to like new software and the reality is and this is a tip that i give to all my fds you have an edge if you're able to build on top of their systems so you know one of our clients for example said that they spent you know a couple of years a couple million dollars moving to NetSuite, which is an ERP software.
And if your AI solution is like, hey, we have to make you move off of NetSuite, they're going to tell you to get lost. But instead, if you're saying, which is what we do, build on top of NetSuite, make it much better and integrate that NetSuite with your Salesforce, with your SAP, with your Concur, Expensify, Gong, you know, every other piece of software workday, that is a much more powerful system.
And that is we're all the value-wise. Then if you can test it in a controlled environment and really scale up from deployment to shadow mode to increasing autonomy to then being deployed in production, that is going to be your edge as well, where you're not forcing a massive shift. You're kind of walking them through that journey.
And to our point earlier of why you meet them in person, it's because it's a lot easier to kind of guide them along that journey if you've met them face-to-face versus if you're just a guy behind a computer screen saying, hey, now we're going to flip a switch and AI is going to run your business. That's a much different, much more polarizing approach.
I mean, it makes sense, right? Like you did the audit and then if you're going to pitch to them, hey, you've been working with this software stack for the last 20 years, all of a sudden go switch to this thing and it's going to cost you a bunch of money and there's just like so many unknowns. That is a tough pitch to sell.
You know what I mean? if you're pitching anything you want to pitch something that feels like you're fishing with dynamite so it's like how can you how can you fish with dynamite you just say like hey you have this system and this stack it's worked for you I'm going to make it better and it's going to help you all be more efficient it's going to help you reach customers faster.
It's going to drive value for customers. It could increase revenue. When you start saying things like that, it's like, okay, no-brainer, no-brainer, no-brainer. Also, you have to keep in mind that you're pitching to people at a company. And people at a company, I'll say the thing that people don't say, which is, they don't want to get fired.
They want to get promoted, actually. So your job is to help them get promoted. How do you help them get promoted? is probably not by moving from one ERP to another ERP. That may be marginally better. You help them get promoted by driving value cost-effectively. If you can drive value cost-effectively, everyone's high-fiving.
Because when performance reviews comes around, the employee, the executive can point to, I worked on this project. Yes, I worked with an outside agency like LCA or Veric Agents or individual FDE freelancer. But as long as you help them do that, that's what's going to help them get promoted. Yeah, totally.
And just to, again, double down on that, they view you as a risk, right? They can just sit by and let things stay the same and status quo, they'll be fine. But if they instead bring in an FDE who's going to change stuff up and maybe it fails, like they're worried. If this fails, it's a terrible look on me.
Forget like migrating to another ERP. Even just you being involved at all is a risk to them. So you have to de-risk this as much as possible for them if you really want to sell yourself into a company. And what I strongly recommend is like, do the audit for free. Like get your foot in the door, prove value there, come up with a plan and then only get paid when you really prove measurable value.
That is what I strongly, because that de-risks the whole thing. Your first few customers, if you're really starting this out, will teach you so much. They are genuinely worth more to you than you are to them. But after you have one, two, three of those, then you can start charging for this because you're going to be leagues and miles ahead of everyone else in the space.
I promise you, it's still so early. I know this from our company as well. There is so much demand for people who really know how to do this. and quite frankly there aren't that many people who know how to do this. Get started, get your feet wet and really prove that you know what you're doing and that's de-risking the whole thing for them.
Totally. And it's just going to give you the confidence too, which is important. Yeah, and you'll know what matters to them when you're selling. You can touch on different aspects that speak better to this person in the function. It's all really valuable. If I had to put one page cemented, burned in everyone's brain, It's this one, which is you go from audit to evals to deployment.
There's some steps in the way you build, you observe, and you improve, and the loop runs again. Because once you improve one system, the next one becomes extremely clear. There's always interconnected bottlenecks where one workflow is impacted by something upstream, and it frees up something downstream.
And this is why AI is so pervasive in an organization. It's because once you have it in one place, you're going to need it everywhere else. So that you're not just 10x-ing one workflow, 10x-ing another, you're 100x-ing the entire business as a whole. And that's your job as an FTE, is to go from audit to eval to deployment over and over again.
And the next stage that I'm going to show is the 30-day plan of how you can get there from zero to one. If I was starting from scratch, how would I go about it? And you did this, by the way. You started from scratch, right? I didn't know this, but you were an engineer at Meta, right? And then you sort of learned how to do this.
So you're speaking from experience. Yeah, absolutely. I was an engineer at Meta for a few years working on a couple different products. But I was never a consultant. I never really understood what mattered to businesses as deeply as I do now. And we got started again just by doing it. and we had this thesis that like AI needs to be applied and it allows us to get ahead of the curve but you never learn by you know reading and you only learn by doing it so the goal from this is to do like if I could condense what I did over a year and really had the biggest learnings the biggest wins in just 30 days this is what I would do so the first step is build an agent that can complete a real loop right?
Build an agent that's actually useful as a workflow. So like ask Chad, what is one real enterprise workflow in a function of the back office, right? It could be finance, it could be HR, it could be procurement logistics, it could even be front end, it could be sales, anything. Get the workflow in as granular of a detail as possible and build an agent for it.
Even today, it's very hard to build agents, right? We think that it's a solved science. It's not. There's a thousand different ways to do this. Everyone has different definitions of an agent. My definition is this. If I give you a task, can you solve it in as much detail and as high enough accuracy as possible?
It's different than me prompting Claude to go do it. It's far more in the background, and it has much more of a repetitive motion where I'm not reliant on somebody prompting perfectly to make it happen, I can prompt like an idiot, and it will still happen. That's my kind of requirement for you when you're solving for this.
There's a bunch of different aspects to this, but if you have seven days to work on this, you'll be able to pick it up. Agent looping, then tool usage, then guardrails, then context and memory, then the audit trail, which is incredibly important. I want to hammer down here a little bit. If you can't show the client what the agent is doing, they will never trust you.
There's a big fear in AI agents today that, okay, it's going to go off and do something. It's horrible. There's a lot of fear mongering as well. I won't say from who, but everyone knows who I'm talking about. You need to show that the agent traces are logged. Everything, and this is a software engineering problem.
So if you can do that, you're a step ahead. And again, there's a full day for each of these things. To clarify, if you're working 12 hours a day, I don't expect you to be able to pick all this stuff up perfectly on each every single day. But this is what I mean by a 30-day plan, you can space it out as you need.
It's not like you have to get it done in 30 days. Then a real workflow, then a checkpoint. The checkpoint the last day is you have a working agent with tools, guardrails, deliberate memory, and a full audit trail for one task. You might not even understand the task the best, but this is just to get you well-versed in building agents.
Fair. The second week is turning that demo into a system that can recover. Again, very heavily on the engineering side. So define JSON schema, not freeform text. You're validating schema. You have failure modes. And again, I want to call it failure modes. Exception handling, and we also see in 13th day as failure handling, is also extremely, extremely important.
This is where going in deep into a client matters because if you understand, hey, when something goes wrong, how does it go wrong? And let me build the agent around that. That is extremely important. it's far more effective than you're building an agent that solves for just the happy path. It's called the happy path.
If you're building for the unhappy paths, the 1500 different ways it can go wrong, your agent is worth a million times more. The way I say it is this, there's only one way that something can go right, but there's a thousand different ways something can go wrong. So if you're only building for the way it goes right, you're worth nothing.
If you're solving for all the exceptions, that's where you are worth something as an agent. That's week two. then week three is where you start to make it measurable and economically viable, right? So you'll have the retry logic. Yes, this is more engineering. You'll have the golden data set for evals.
You'll make sure that it improves over time. But you'll also start understanding, okay, this is what we talked about earlier, Greg, looking at cheaper models for subtasks, looking at less soda, less frontier models. Can we get this job done with a Gemini Flash? Or can we get the job done with a Muse Spark?
Or probably not a Lama 4, but there's other models that can be a good fit there. This is where you start to do more of this test, which is we have an agent now. Now let's try to optimize it. Let's try to measure, okay, how much is it really moving the needle? If I deploy this in production, how much time am I saving?
How much risk am I mitigating? How much revenue uplift do I have? There's only three buckets of measurement that matter for a business. It's those three. revenue uplift, risk mitigation, and cost savings. So you need to measure your agent across all three of those buckets. And at this checkpoint, you have an evaluated agent with known failure modes, measured costs, and a golden data set.
And the final week is defend the system like an FTE, which is all of the business around it. It's the pain points. It's why AI belongs. It's the architecture behind it. It's the iterations. When you first built the agent, it got this wrong, but then it improved over time. Accuracy went from 70% to 95%.
And the economics around it. So how much time did you save? The error reduced again. Risk, revenue, cost. We talked about this. And you rehearsed this as an engineer. So what was the architecture? What were the decisions that you made? And then you also rehearsed this as a VP. What was the problem that you saw?
What was the outcome? What was the evidence? What was the risk? And this week is where you're going to know, was the system that you built worth the salt? Was it worth the investment? how much could you charge for this when you build it for a customer? That's what this week is for. And I strongly recommend that during this week, you pitch your agent to businesses because they will tell you like, you know, did I get, you'll pitch them, did I get this right?
Did I get the economics right? Am I thinking about this the right way? And they'll tell you point blank, no. Or I want it built in this way. And you'll start to see like, okay, now for my next FD engagement, starting out with an audit when you're actually embedded with a customer, you'll learn much more about that.
Obviously, this 30 days is you're not embedded with a customer because you can't, because you have to become an FTE first. That's when you can finally start to pitch yourself and be involved in a company. So if I had to zoom out, this would be the 30 days. It's doing the job before you have the title.
So on day 30, you understand forward-deployment engineering, but you also have evidence that you can do it. And if you pitch this to a company, they'll be much more likely to give you a shot. and that's my goal Foss, this is perfect this is exactly what I would recommend too what would be so cool is if you actually taught people how to do this and spent 30 days with people to actually do this maybe we do it together just an idea if people are interested I'll just include a link in the pinned comment on YouTube I'm just curious if people people are interested in like because it might feel overwhelming for people to do this on their own I mean I still think you could do it on your own by the way
but I wonder and by the way I'm not promising anything I'm just curious are people into into some sort of program for this because they don't teach you this at school they don't, they should I'm sure we'll have university courses on FDE soon but yeah, people are just I'd love to it just takes so long I remember, I was in computer science school in 2008 no, 2009 at university and I remember the app store had just come out it was so clear in 2009 that mobile apps was the next wave just like it's so clear right now that AI agents and AI is the wave.
And I just remember the textbooks at the time and I went to a top university. Textbooks at the time and the course material was like building old school software. And I remember going to a teacher, a professor, a well-known guy and saying, why can't you teach us how to build an Objective-C and to build for the app store.
And he was just like, yeah, it's just not in the textbook. Just not in the textbook. And that's when I was like, I'm going to drop out of this. I'm dropping out because I don't want to learn yesterday's stuff. I want to learn tomorrow's stuff. Now, there's always the argument to be made that you need foundational work.
and so I learned a lot in university around foundational stuff around math and physics and stuff like that and I actually think that that stuff was really helpful just in learning how to think but the actual tactical stuff did not really learn I want to say this time is different just because it's so powerful AI is so, like you said, it's the wave that I'm hopeful that universities are going to pick up sooner than later, but for some reason I feel like you're right.
I don't think it's going to happen anytime soon. Well, there you have it, folks. What FDEs are, how to become one, a 30-day plan. Voss, anything else you want to share? I think, like you said, you might not find this in university, but you're absolutely going to find it on YouTube. Like Greg is teaching everything that you need to know and Twitter as well.
Those two sources are going to be where everything is released. I mean, even Mark Zuckerberg had to come back to Twitter to announce, you know, the latest model for meta. That's where everything's happening. Study the game there. And you've got a great coach right in front of you with Greg. So hopefully that, you know, people are really taking advantage of this time where there's a significant alpha from going out, learning, doing it yourself, being scrappy with it versus waiting for a university to come by and teach you this because that's not going to happen anytime soon.
And it's free, right? You can listen to this and apply. It's free. So it's just like, why not, right? Voss, thank you for being generous with your, as we see on the channel, the sauce and the tactics and just like breaking this down so clearly. I've been following you for a couple years now almost and you're a must follow I'll include links on where you can follow Voss from Varick Agents in the show notes, in the description please comment what you thought of this episode because I enjoyed myself with Voss I'd like to have him back on the podcast again hopefully he's down to come back on but please let us know I read every single comment you want to like and subscribe for more of this in your feed
Voss any last words for the people? Greg you're a legend thank you for having me on to the people I believe in you I really believe this is a fundamental shift in how work is done and you are if you're listening to Greg and you're watching this you're already a step ahead I'll be reading every single comment too if any questions you have for me let me know but I would say go out and get it go out and get the job done make the most of your ability to understand AI and it's still so early.
So get ahead of it while you can. Greg, thank you for having me on, man. You're a legend. Amen. All right, catch you next time. Cheers.