#S2.19 Agentic Engineering at Scale: Inside Delivery Hero's 18x Software Factory - Rodrigue Schäfer

Rodrigue Schäfer: as we got one thing through in December, we we said, okay, let's take a hundred in Q1, and then we ended up with 1800. And these are merge PRs, right? So these are actual things that that go into production most of the time.

Sebastian: Welcome to another episode of our new season of Beyond Vibe Coding partnering with Impala Search.

André: Go to TAC, an executive search agency in Germany.

Sebastian: In this podcast, we explore the transformational change in software engineering and knowledge work in general. I'm Sebastian Heidemeyer, CTO at Ecosia.

André: And I'm Andre CTPO at Trusted Shops. Great to have you back. Today, we are tackling agentic engineering at scale.

Sebastian: Exactly. Our guest today is the SVP of Tech Foundations at Delivery Hero and he's giving interesting insights into the transformation towards agentic engineering at the company. So listen in, it's going to be really, really interesting. welcome Rodrigue. we're super happy to have you. You recently gave a very interesting interview for the German newspaper Capital. It's not a newspaper, but the German print edition Capital. And yeah, you described a lot how the age delivery hero is approaching agentic engineering. And we thought it would be a great topic, obviously, for our little podcast. So we're super excited to have you. Please, before we jump into the main topic, introduce yourselves very briefly to our listeners.

Rodrigue Schäfer: Yeah, thanks Sebastian. I'm Rodrigue. I'm currently SVP Tech Foundations at Delivery Hero. I've been in the tech scene since, let's say since 1998, I've been involved in more serious projects in the online space. And I've been in leading positions, so CTO or senior engineering manager positions since around 2001. So that's like quarter century of doing

André: impressive.

Rodrigue Schäfer: that kind of job. mostly in the Berlin startup scene and scale up scene with Zolanda and Delivery Hero now. yeah so that's a a quick rundown of what what I've been doing right now at Delivery Hero. I'm focusing a lot on providing infrastructure and tooling for all the engineers at Delivery Hero and all the the data engineers as well and the machine learning engineers and analysts to allow them to build what what we call our everyday app.

Sebastian: Awesome. hence you're exactly the right person for our podcast. so we're s again super excited to have you. But before we jump into how delivery hero is approaching this, we have the small segment of the status quo where we ask our guests how they utilize agentic in their workflows. Do you have yeah, examples for us how you utilize agents in your day-to-day work?

Rodrigue Schäfer: Well, I mean at the job I use it mostly to prepare myself, right? So this is about kind of prepping for the day, getting a summary of what's going on, getting some background context. before I I go into a meeting, for example. It's also about kind of aggregating a lot of information so that I can make decisions quicker. I think I'm I'm probably not the most creative manager in terms of automating all of these tasks, but I I try to to approach it by where where is the return investment biggest for me, right? And then just start with that.

Sebastian: Yeah, that makes makes a lot of sense. And I think we're all doing this. And it's it's really I would say a pattern that we see pretty much with every guest nowadays that meeting prep, day daily briefings, this is pretty much state of the art right now. And specifically also data aggregation, right? Getting context for meetings for topics.

Rodrigue Schäfer: Exactly. Or also actually kind of storing context. For example, what I'm also doing is I'm automatically generating weekly reviews of what I've been doing that week, right? And I store that in a folder on on G Drive and then at the end of the year I have a pretty good overview of what has been going on that year. And this again then also helps me. provide better summaries, for example, also to my manager or group CTO, to to yeah to give him essentially a rundown of what I've been up to since we last spoke. So this is also good. The same can be done with my direct reports as well, kind of due to the the way that we have transcripts of our meetings, at least the ones that are taking place online, I have a pretty good yeah knowledge base about what's what's happening there and I can draw from that as well. So it's not just aggregating information, it's also kind of yeah, creating written context about things that are going on that can then be used later to to inform myself again before a meeting or before I need to to do something with that.

Sebastian: Yeah, agreed.

André: Out of curiosity, is that a very individual setup or do you also share in your leadership team, don't know, prompts and any other kind of automations?

Rodrigue Schäfer: I would say it's mostly personal. some people share their gems and their prompts for cloud and routines and so on, but I it's we we don't have a really good, let's say, discovery around this, right?

André: Yeah, because I could imagine at your scale, it would actually start to make sense. Like there are some leaders in the organization, so with repetitive tasks. So yeah, I was just curious to better understand how you approach that.

Rodrigue Schäfer: Yeah. It's absolutely absolutely a good good observation. It's also the same with skills, right? so right now we we do have a place where people can kind of publish their skill, but there is no curation. There's nobody checking in on these skills and figuring out, you know, is that even something we wanna do? is that something that can be improved? maybe we have 20 times a similar skill. so that that is a bit of a governance problem which we have because all of this is pretty new right and large companies are typically a bit slow in adapting to that and in in in inventing kind of or creating the necessary processes to to have that sort of governance on top of it.

Sebastian: Yeah, maybe final

André: Makes sense.

Sebastian: question on on this one, specifically the sharing approach. If you have a place already, is this a tool like within the anthropic ecosystem or is it just a GitHub yeah, a repository? GitHub, yeah.

Rodrigue Schäfer: It's GitHub. It's it's GitHub based and we we have a website that basically pulls that information and that's it. But yeah, even let's say even ownership of this is not 100% clear, right? So it's really a little bit of free-for-all at this point. and something that we're working on to make that a little bit more controlled, right? The same thing is happening with when you talk about like how apps are actually deployed. Right and made available for others. So of course you have things like artifacts and cloud, right? But that might not meet your purpose all of the time. Then you have our own developer platform where people, engineers can't deploy stuff in a let's say, professional way. but you also have a lot of users that can't use that developer platform but still want to set up some website or so something. And that's something where we're also right now in the process of establishing an environment where they can safely deploy whatever they're building and provide that to others.

Sebastian: Sounds really interesting. Yeah. thanks a lot. I would say with that we can probably conclude the first segment of our podcast and move over to the main section. And yeah, as hinted already, you recently, a couple of weeks ago, I think, gave an interview to Capital where you I think described the agentic engineering approach at Delivery Hero, right? And yeah. That's what we'd like to also hear from you pretty much today. yeah, please let our listeners know how you transformed yeah, the agentic engine or the the engineering organization of Delivery Hero.

Rodrigue Schäfer: Hmm. Yeah, so maybe just to structure that a little bit, I think there are several dimensions that we could talk about. One is how to enable engineers with genetic tooling to be more productive and faster, right? second is the whole autonomous software engineering part. So the the software factory or even Dart Factory is, I think, a label that many people use for that. and CERT is the leveraging agents within consumer or business-facing products, right? Which is also a big deal of what we're doing. the consumer-facing, business-facing topics are not handled by me, but just to give you a quick rundown of the two most prominent projects is on the one hand we're investing a lot in customer service agents, which is something that is already running and mature, where we're trying to just increase essentially the adoption of that. The other is a sales agent that that's really not about like contacting consumers, but it's about helping restaurants with selling their food on Different platforms that we have, very successful one. This is we we have a lot of vendors, right? A lot of restaurants, and providing someone, a real sales agent that's available for them to help them with their sales would be not, it's just not scalable, right? In the past, what we did was more of offering this kind of support for very high profile vendors. while the Azure S didn't really get that kind level of support. Now with agents we can

André: Hmm.

Rodrigue Schäfer: actually provide that support to everyone. And that's that's a for them a pretty amazing thing. It it goes so far to, for example, identify Or make proposals in terms of the menu that they have, in terms of the pricing, in terms of campaigns that they should do, and so on. So it's it's a neat tool. customer services, I think, nothing spectacular at this point. Most companies know that, use that to some extent. There are certain interesting, let's say, aspects to it. One is that they seem to be very successful. So the interactions that we see between customers and our agents tend to have a higher success rate, but people are less happy about it. That's that's a

André: Wow. Okay.

Rodrigue Schäfer: a very strange combination. Yeah. But

André: Yeah, combination, right? Yeah.

Rodrigue Schäfer: yeah, we believe this is probably due to the freshness of that technology, right? Because in the end

Sebastian: Just

Rodrigue Schäfer: yeah.

Sebastian: yeah, sorry, just for me to understand this. So the cases can be solved probably quicker or with l fewer steps, but NPS is lower. Is this how to interpret your underst your the yeah.

Rodrigue Schäfer: Yeah, kind of. We're we're not measuring that with NPS, but similar metric and yeah there is there is a a visible difference between the agentic interactions and the human interactions in terms of how people rate that interaction, right? despite having better outcomes. That is is interesting. so we're we're Looking into that and trying to figure out what's what's going on here. in terms of agentic tooling for engineers, what we're doing is so in the beginning, we were experimenting wildly. So Delivery Hero is a very decentralized group, right? So we have several big subsidiaries, these subsidiaries have their own tech teams, tech platforms, and so on, their own CTOs. And of course, they wanted to try things out on their own. So in the beginning we used that as an advantage by running pilots with several different big players like Cursor or Wins. We had Windsurf before they got yeah bought by by Google and I think Devin bought the rest. I don't I'm not sure.

André: Hmm.

Rodrigue Schäfer: no no it was was it don't don't ask me. we also had Claude of course we had Gemini codicist which we used in in Berlin since mid-24 roughly and yeah people tried things out we tried to to get some qualitative feedback on that and how how they use it and looked at the adoption and discovered some patterns that were kind of common across all of these subsidiaries. Like for example there was always this pattern of a relatively small power user group. Typically around 10 to 20% of the people, while the rest didn't really buzzer, right? very this was early in the phase where the models were also not yet that strong. and at the beginning of this year, we decided to move towards a central stack, at least for the central teams here in Berlin, which is Already a lot of people, right? We're talking about right one and a half thousand engineers here, plus

André: Okay, now...

Rodrigue Schäfer: product. So we wanted to do that mostly because we saw that if everybody's trying out things, we we don't have any level of governance on that. So this is really about things like of course compliance and security play a big role here, but also things like being able to measure. what they're doing with it, right? Because if you have like a dozen different providers that send you a dozen different metric sets, then it becomes really hard to figure out okay how much how much did they use it, what how how much code was actually generated with it, and so on. So this became a little bit of an issue, same with cost control, right? And in general, with figuring out what kind of ROI do we have with that. So we wanted to to centralize that, we want to have some sort of technological ownership on that topic. And then we went with one vendor to to roll that out to at least all the central teams, right? And opt in with some of the subter subsidiaries who also chose. to go that route. And that's what we've been doing since a couple of months. pushing for adoption, pushing for upskilling of people to use these tools more and at the same time of course also trying to to control the costs which obviously can explode pretty quickly when you

André: Absolutely.

Rodrigue Schäfer: you don't have an eye on that right

André: At your scale, imagine this is a huge topic. Are you allowed to disclose which AI provider you are using? Is it something very special or is it like straightforward as you would expect, maybe OpenAI and Tropic?

Rodrigue Schäfer: it's it's relatively straightforward. So in central we're currently focusing on Claude. in the group we have other players as well, such as Gemini, that's our South American company, right? and some of the other subsidiars are also still having some contracts with other vendors. But the central one is cloud and that's what we're pushing mostly because it also combines the engineering support with a non-tech product with cowork right that that people can use

André: Yep. Yeah.

Rodrigue Schäfer: And we wanted to give them some tooling and actually that a lot of people in non tech are very excited about it and try to use it's still like to be absolutely honest, it's still from cost perspective, it's just a fraction, right? Non-tech. But

André: Absolutely. So we are running the same strategy. So I know, but probably in different scales. we'll be also maybe as a follow-up episode to better understand how you do cost control at that scale, because your subscription probably does not fit into a limited model anymore. So you pay by the token,

Rodrigue Schäfer: That's true.

André: same as on our side. So that also creates some new challenges, so to say.

Rodrigue Schäfer: Yeah, yeah, that's true. So we're not on that one hundred fifty people team plan. that's it I mean it can make sense if you are below this one hundred and fifty. I would also use that, right? But in our scale, just the the idea of juggling user accounts across several of these team plans just would be a bit over the top. But yeah, we We don't have that. We manage that through the cloud API, right? We set limits. We we built actually tooling on top of cloud because the cloud functionality was not enough for our needs yet, right? They're working on making that better. But yeah, we we can per organizational unit, we can define limits, we can define limits on individual basis, and we can track. overall spending. So we have that somewhat under control. engineers can tell us what they use it for, right? So if somebody is like requesting another $200, they need to provide some sort of yeah, context about what they're working on to understand is that worthwhile, is that maybe not worthwhile.

Sebastian: Impressive. Yeah. So so this was basically the starting point. Lots of experimentation in the different orcs, right? And then starting to yeah bring this into one structure for governance purposes, right? And this is basically what you then rolled out in Berlin with one provider and other providers in in other companies, right? yeah.

Rodrigue Schäfer: Exactly. So that that setup also allows us to keep track on other vendors and see how yeah how their products develop. And I think that's that's a good thing because this is a very, very fast moving industry. and you want to be on top of it, right? the the third thing that that I wanted to talk about was the autonomous software delivery, right? And here that that's something that we came up with around mid last year. this was when models were still not that strong. So I remember myself playing around with Firebase Studio at the time, right? And it was nice to see that it could create something out of a prompt, and that something kind of worked, right? You could build dynamic web applications with that, with even with interesting business logic behind it and so on. But when you worked for longer on the same code base, it really deteriorated quickly, right? So at that point was clear this is not yet ready for actual production cases. But it was also clear that if you look at how fast models got better, also in other areas, let's say in image generation or video generation. I think personally I I felt I I was very amazed about the jumps that video generation did. I was also following a lot on the image generation since a couple of years already, mid-journey mostly, and what what happened there, the improvements. So it was clear coding will improve soon and really with step changes, right? Not with like small improvements here and there, but I was expecting that this will become Good enough for larger production ready software projects. And that's why we wanted to build a harness around it already, so that when the models become strong enough that we have something that we can run with it. And so we we put on a couple of our so three three really strong engineers, principal-level, staff-level engineers to build that prototype, and they worked on it. Until the end of last year where we had this MVP running, we had the first PR that was done fully autonomously. It's integrated with our environment so it it runs our CI tests, right? and takes any any feedback from CI back, it's integrated with with GitHub and takes any command back with Jyra and or you can interact with it there. So it's there it's basically embedded in what we use as our development tooling already so at that point The the strong models hit, right? This was, I think, around November, December last year. and that was really good timing because when we launched this internally, first with test users in January, we already hit our initial Q1 target during that kind of under-the-rader launch. And then we we upped it, we opened it up for everybody to use, and in in Q one we had to adjust our target several times, right? In the end we we hit like eighteen times the original target in Q one because it really skyrocketed in terms of usage. with a str

Sebastian: So w so it was usage targets or PR targets or what? P R targets, okay.

Rodrigue Schäfer: PR targets. So we were maybe we were also very conservative, but as we got one thing through in December, we we said, okay, let's take a hundred in Q1, and then we ended up with 1800. And these are merge PRs, right? So these are actual things that that go into production most of the time. we were also very impressed by the merge rate in the beginning. So this was quite good. with only I actually don't know what we ended up with as a merge rate in January, but in the beginning we were looking at already at over sixty percent. as a starting point that's pretty strong, right? And nowadays the merge rate is somewhere around I think in September was eighty three percent of all the PRs that are created are are being merged and Mm most of them actually also without any additional change or command by an engineer. So most of them are one-shots, which is also something that we're now trying to improve on. Because if our goal is next year to have an end-to-end software factory working, then of course the one-shot rate is extremely important for this to work. Yeah, since then we have kind of We we've increased the team size, we've worked heavily on that, we've added features to gain more adoption, to improve the quality. and yeah, that's that's what we're still doing. And now I think we're kind of at the same level as some of the other bigger tech players who've built similar things like Spotify or Stripe, right? Uber as well has has done something in that direction has pretty strong numbers that they that they publicize. But yeah, so we're we're in in a good spot and I'm I'm a believer in that we actually can improve it even further in the future.

Sebastian: Very impressive. and how's the interaction with the harness? Is this really Jira tickets and then at some point the PR merge, the manual PR merge? Is this the are these the the interactions?

Rodrigue Schäfer: kind of. We have more interaction points, but the gyro ticket was the original the the the OG of interaction points basically. you wrote a ticket and then you just assigned it to the hero gen user, right? you still had to put in, and that's still the case, unfortunately, the repository that you wanted to work on. something that we're trying to fix with giving it More context in the future. But that was it. And then it went off and created the PR and ran the tests and fixed the tests if something went wrong. And when it saw its final, we also had, by the way, a console of review agents that are looking at the PR before it goes to the human approver. When everything is done, the the Herogen essentially adds to the gyra ticket: hey, I'm done, this is what I've done. changes the status and then the engineer can take a look and can comment on the ticket. If if they comment on the ticket, the Herogen agent picks that up, makes the necessary changes. and in the end we still have the human approval, right? That is something that is right now queuing up a little bit because you can imagine that with all these additional pull requests that need to be reviewed by someone, people Become a bit overwhelmed. So you see that the review time goes up, right? overall. And what we're now planning to address that is to yeah, start with classification of PRs, to see is that dangerous, is that not dangerous, and if it's not dangerous, we can just auto approve, right? So that's the direction we're taking. I think that's by the way, also works for non agentic PRs as well. so that we can what I mean by that is that if a software engineer creates a PR right now they still need approval from someone, right? But if it's a non-critical PR on a non-critical service, we could also auto-approve that. And if things go wrong, tough luck. But the point is to actually enable that auto approval, we will need to also turn up our DevOps practices to 11, right? So we we Need to have safety nets in place, like cannery rollouts with automated rollback, that even if things go wrong through an auto-approval, limit the blast radius of what happens and

André: Absolutely. Yeah.

Rodrigue Schäfer: can also feed back into the agent itself to say, hey, this is what we saw when rolling it out, fix that, right? And then so actually, and this is something that becomes more and more clear is that A lot of the practices on how we get software to production in a safe and fast way have been there for quite a while, right? So we had these DevOps practices for over a decade. And any company you look into has some sort of processes established that pay into that to some extent, but there are only very, very, very few companies who went, yeah, the whole What you say, the whole 10 miles or whatever? They the the only very few companies really did everything and have continuous deployments, have Canary rollouts, have automated rollbacks, and so on. You don't see that a lot. But with agents, you really want to have that in place. because at the speed of the code production that that you can see. You cannot afford to to have either to rely fully on on software engineers kind of verifying everything, or to rely on software engineers dealing with incidents in production, right? At that scale. It's just not not not a viable approach.

Sebastian: For sure. Yeah, thanks. that that actually is really impressive what you describe here. because I feel like yeah, there's it's not many companies that are that far already on the automation, specifically the the route to yeah, fully automate the implementation and then now approach the the reviews. one question there, so is this one size fits all, meaning that it's one harness that you can pretty much like give to any repo and the like it can handle any type of service because the the services can be totally different, right? It could could be something user facing, it could be something that pretty much just handles data or it could be a transaction system. Can it handle everything or is this are there specializations? So how how did you structure this?

Rodrigue Schäfer: It's it's a generic thing. So the harness stays the same, the model, the primary coding model stays the same, the rever model stays the same, right? But what we're looking at is to create a little bit more of a long-term memory in the future where the model understands what it has done with a code base and keeps that information for next time, right? So this way it might get better over time on a particular code base. But we don't have like a specialized agent for, I don't know, iOS apps or a specialized agent for something else. And that's not currently what we're doing. And I'm also not sure if if that's the future, right? Might be. I mean honestly, if if training on a very specific problem space would be available. I think you could achieve better results with that kind of specialization, but it also requires a lot of effort, right?

Sebastian: So then maybe to make it a little bit more concrete, so basically the the agent is tagged in the Jira ticket, right? It then gets the repo. And then the repo probably contains some contextual information, right? So like agents MD or Claude MD or whatever, context MD. So the the contextual information. and then the agent will just based on this given contextual information within the repo and probably skills library, MCP libraries will then pretty much yeah, take it away and and implement. Is that the setup?

Rodrigue Schäfer: Yeah, that's that's largely correct. so with the repos themselves, some have more context than others, right? we obviously also have

André: show.

Rodrigue Schäfer: repos that do not have any agents md file or cloud md file and do not have a lot of context aside besides the code itself, right? So the quality differs depending on how much context the repo has. The biggest problem we have right now is that the repo does usually not have context about its dependencies. So it doesn't know how the full API of some other service looks like that they're calling right necessarily. So maybe of course you can see the function call but you don't have the full picture. And this is especially if you're talking about let's say APIs that the repo does not use right now, but that would be needed for the feature that you want to implement, then it this this information is completely missing. And that's something we're tackling now to create that kind of context hub where The agent will be able to to understand a lot about the architecture, about the dependencies in this microservice landscape, and then be able to make decisions on what what it needs to do.

André: Yeah, I agree with Sebastian, probably one of the most advanced, agentic workflows we have, we have seen or we have heard about. so very, impressive. it's not available. it's not publicly available, right? and there are no plans asking for a friend.

Rodrigue Schäfer: it's not publicly available, right? But we have been talking about it internally.

Sebastian: Okay, so then we should keep an eye on on the public repos of delivery hero. amazing. I would have tons of questions. Unfortunately, we're

André: Same here.

Sebastian: nearing the end of the episode, right? So I'd say let's conclude the main segment with that. and go to the two recurring segments towards the end of the episodes, of which the first one is the so called reality check, where we always ask our guests for what the fuck moments or wow moments they have recently experienced with LLMs or agentic systems. Do you have anything in this category to share with our listeners, Rodriguez?

Rodrigue Schäfer: Yeah, but I won't name any model or vendor because we have deals with them. I don't want to embarrass anyone. But I had stations where a multimodal model insisted on not being able to understand images or videos, right? It it was definitely multimodal, but somehow I don't know why. It it just didn't know that it has that capability. There was a very weird thing. And sometimes even this this happened when it like a couple prompts before it actually, you know, did that and was able to do it. So that these are these are situations that are a little bit strange. and these happened in this case this happened really at the very early stages of multimodal models. So we're talking, what was that? I think that was actually could have been last year. or yeah. It's it's it's not

Sebastian: So not very recently, basically then. Yeah.

Rodrigue Schäfer: not very long ago.

Sebastian: I see. Yeah. And is there anything that excites you specifically right now? So something where you say, okay, this is the next big thing?

Rodrigue Schäfer: Yeah. Yeah, I mean I I I think that agents that can act on your behalf are an awesome idea and s to some extent, but I also see a lot of risks in that. So I was not one of the people who kind of bought a Mac Mini and put open claw on it, right? But

André: You

Rodrigue Schäfer: I think it's it's still a very, very interesting direction to essentially have some sort of personal assistant that is helping you with your day-to-day stuff and also what what what Facebook just or Meta just announced, right, with Muse goes in that direction. So I think that that is a a big deal in in the future. There are a lot of yeah also risks around privacy here, risks around identity theft in the end, right? Or so I'm not one hundred percent sure if I would do that. currently I'm also when I'm looking at, you know, it's things that you want to do on your that that deal with your private life, such as you wanna do your tax statement, right? And you wanna do that with AI. If you do it with a public with a subscription, then your data might end up in the training mass, right? And this can be quite confidential, in fact. So That's something we're we're we're what what I'm looking at here is more like I hope that with the new hardware that's coming out, that we'll be able to run strong models locally without having that kind of risk, right? And with agents, I think running them locally without having to call any external subscription. API is also yeah, helping you a lot with containing the risks that come with with these kind of systems.

André: Absolutely. I don't want to go back to the main section, but have you actually also thought about local inference for your HeroGen agent?

Rodrigue Schäfer: I mean for HeroGen would not really be possible because quality is still a very important part. And usually when you switch to local, you either need to invest massively into hardware, right? To to run any model that comes close to what the frontal models are doing. and yeah, no, we we haven't done that yet. I know that one of our subsidiaries has tried that out, so they bought a A bigger machine from AMD to to do that. It wasn't worthwhile for them at this point. I think these things change, right? As hardware gets

André: KM.

Rodrigue Schäfer: cheaper, hopefully, right? Let's say the memory crunch goes away. then we might see this more often, and people will start making similar decisions as when when the cloud started And people start to move to the cloud, right? We I think the dynamics are maybe a little bit similar in a way. you have this yeah, third party where you can offload your your hardware in this case to a

André: Workload. Yeah.

Rodrigue Schäfer: workload to, and it deals with that. But if that is too expensive, then you will consider doing it on premise again, right? And this kind of tension between Too expensive to run it with a vendor and the hassle of doing it yourself, this needs to balance out in some way. given the the current situation with hardware, with also cheap models coming, Out of China and cheap models in the open weight space, right? I think there will be pressure on the bigger players on the frontal labs to keep prices somewhat acceptable, which in turn will lead to people not going the on-premise route because on premise always means at least if you do it in a company, right? You need to have people who manage the hardware cycle, who maintain

André: Absolutely.

Rodrigue Schäfer: that thing, right? And this is At scale, this might be worth it, but

André: Yeah, this is actually the reason why I was asking. You probably have to scale, but I fully understand the feedback on quality, the feedback on actually also the nice, how you say that, between availability and the pressure it puts on frontier models to keep prices at a reasonable level. So I think that maybe it's enough to stay in that sweet spot.

Rodrigue Schäfer: Yeah, I mean you see it, right? Both OpenAI and Antropic push prices down recently with their new models.

André: Yeah, yeah. True.

Rodrigue Schäfer: So yeah, it seems to go in that direction. in our case, yeah, if we would do that, I think we would probably would have to to hire a department of system engineers to keep that up and running. Or alternatively, you can also use the cloud, right? So you can host that. on Google or on AWS. And then what you have is basically you pay for a very expensive instance that is not used 24-7 because your engineers are also sometimes out of office. And and that if you would need to make that calculation if that's actually worth it, right? What's more expensive? This with the waste it gives you, or paying by usage with A vendor like Entropic or OpenAI. Or Gemini.

Sebastian: Yeah, and I think I think you're you're pointing at exactly the the right or important points, right? It's the the price wars basically that that pretty much have begun. And this is where I see a little bit of difference in the to to the hyperscaler sector. I mean there are also there's also price competition, but the switching costs costs are much lower if you purely consume inference, right? Because then you can pretty much and there's also there are many

André: Yeah.

Sebastian: non-frontier lab providers of inference, right? Where switching costs are are virtually zero. However, the frontier labs are still they are ahead and they that that's a race. They need to stay ahead in order to stay relevant on the inference level, right? And on the other hand, they're also trying to integrate more deeply with their like the the open AI dots very recently and they tried different approaches, right? Cloud co-work and whatnot. so I think It's very interesting to see this, but I I'm fully with you that right now for most companies, probably like if they can utilize subscriptions, then it's anyway heavily subsidized and it's a no brainer basically. if not, then still it's right now the price point and also the the quality level that you can get and the the ease of use is worth a lot. So it's it's hard to make a case for for going entirely on prem.

Rodrigue Schäfer: Yeah, I would only do it if if privacy or sovereignty would be really key to the business. Then then of course, yeah. But if if not, you also see you have this high upfront cost to buying the hardware which is which which you put you need to put somewhere, you need to have the personal and so on. There's just I don't know, a a lot of hassle and a lot of yeah, you need to get it as well. Yeah, true.

Sebastian: And y you need to get it. Getting the hardware is also not not that easy right now. All right. Then we have a final segment, which is prediction. So do you have a prediction for our listeners?

Rodrigue Schäfer: Prediction, yeah. So I mean, as we're working on this end-to-end software factory at delivery here, I think we're we're as I said, right. We we know others are also doing that. We know there are several vendors that are providing tooling for that. I think end of next year we'll see that this works end-to-end, right? So that this is really a ticket that goes to production in a safe way. that is also checked for anomalies, wrote back if things go wrong, we'll see agents that that can handle a certain percentage of operational incidents automatically, right? And we actually need that. If we can't do that, we're we're blocked because incidents typically scale with the number of deployments that you do if the change failure rate keeps. is is at at a similar level, right? So and if that handling of these incidents is still primarily done by software engineers, then we have a huge bottleneck. And a bottleneck that's not about just capacity, but it's really about your your yeah, resil or actually surviving in terms

André: resilience. Yeah.

Rodrigue Schäfer: of your business. Because if you can't handle incidents anymore, it's probably game over for most.

André: Okay, noted, end of next year, we need to reach out and ask you whether your prediction became true or not.

Rodrigue Schäfer: Okay.

Sebastian: Yeah.

Rodrigue Schäfer: I'll I'll take that challenge.

Sebastian: All right. Thanks a lot, Rodrigue. Was great, very insightful. and yes, thanks for being our guest.

Rodrigue Schäfer: Thanks a lot for being here.