“I don't have money, but I do have a very particular set of ~skills~ connectors” — Ashley Jeffs - podcast episode cover

“I don't have money, but I do have a very particular set of ~skills~ connectors” — Ashley Jeffs

Jun 07, 20241 hr 18 minEp. 66
--:--
--:--
Listen in podcast apps:
Metacast
Spotify
Youtube
RSS

Episode description

Transcript

Jonathan HallJonathan Hall

This show is sponsored by you, the listener. Stick around till after the news to hear more about that. This is Cup of Go for our June 7, 2024. Keep up to date with the important happenings in the Go community at about 15 minutes per week. I'm Jonathan Hall.

Shay NehmadShay Nehmad

And I'm Shay Nehmad, and I'm really hoping nothing important in the Go community is gonna happen between the 6th 7th Yeah. Because we're actually recording to this, on 6th.

Jonathan HallJonathan Hall

Recording a date early because I'm traveling. I'm in a different time zone right now.

Shay NehmadShay Nehmad

And a different mic.

Jonathan HallJonathan Hall

And a different mic.

Shay NehmadShay Nehmad

I I catch that all the time in other podcasts where people switch up their setup, and I'm, like,

Ashley Jeffs

that's not your voice. Do you have

Shay NehmadShay Nehmad

a podcast that you listen to for long enough that you recognize, you know, the sounds in the room and what

Jonathan HallJonathan Hall

Yeah. And then a few of the especially if they do a live recording, you know, on location somewhere, it's very different sound.

Shay NehmadShay Nehmad

What's your number one podcast? This is not a show. We're gonna get into there's super interesting news. But what what's your number 1 podcast? Just hours.

Jonathan HallJonathan Hall

I don't know. I've I've actually mixed up a lot lately. The one I'm listening to right now is called Your First Million. And it's about a cup it's a couple of, entrepreneurs talking about and interviewing other entrepreneurs to make money.

Shay NehmadShay Nehmad

Cool. Your first million. That's such a American sounding It

Jonathan HallJonathan Hall

sounds a lot more egotistical than I think it really is. It's actually kind of a cool shot. It sounds really like like, startup bro type thing. I don't I don't think it really is. Anyway, let's talk about Go.

Shay NehmadShay Nehmad

Yeah. Let's get into Go.

Jonathan HallJonathan Hall

So last week, we mentioned that Go 122.4 and 121.11 were about to be released. They have been. So, this included 2, security fixes. The first one was mishandling of corrupt central directory record in zip files. So if you're using zip files, be sure to upgrade and the other is related to net IP unexpected behavior from the is method.

Is that right? Is that is or LS? My thought doesn't tell me the difference, for IPV 4 and IPV 6 addresses. So if you use IP addresses Or zip files. Then you should be upgrading.

Shay NehmadShay Nehmad

Sounds like you do. I know of a of a tool that uses a zip as a Go dependency, the GitHub, CLI. Because the bug I fixed was in zip files. So I don't know. Maybe we should really I really was on top of upgrading the CLI when my version came out. And ever since then, every time we're on GH and the they command that it's like, oh, upgrade you with brewing. Like, no, I don't think I will. So I might.

Jonathan HallJonathan Hall

But, yeah,

Shay NehmadShay Nehmad

you know, new security releases always good to upgrade. You should stay on the latest patch always. Right? The there's no reason to not upgrade patches in, Go Miners as you need to worry about a little bit, right, when you upgrade from

Jonathan HallJonathan Hall

Yeah. Right.

Shay NehmadShay Nehmad

Like, the latest upgrade, broke the in it, imports and whatever. But also edge cases. Right? It's pretty stable. It's a pretty stable language.

Jonathan HallJonathan Hall

It is.

Shay NehmadShay Nehmad

But you don't need us to remind like, convince you that, oh, zip file. I need to check if I have zip just upgrade. Nothing bad's gonna happen. It's like arguments with my kid. Like, give it up. But I don't I'm not using it right now. So what does it matter? Just give it up.

Jonathan HallJonathan Hall

Let's talk let's talk about a couple of proposals.

Shay NehmadShay Nehmad

Yeah. So for a future, you know, patch release that includes a security fix, although this might be, you know, something that may wait for a minor version, There's a proposal here from Neil, Damien Neil, to add some safer file open functions to OS. You know, there's a big class of vulnerabilities around files. I think the most common one is, you wanna serve files from your HP server, and then someone asks for the dotdot backslash.backslash.backslash.backslash passwords cat file and your etc password, I mean. And, you know, you travel outside the intended location that you wanted to serve the files from.

This is called directory traversal. And if you're, into security or want to get started, just find any place that lets you download files and start typing these, paths. One of them is gonna work. I promise that. To try and, you know, circumvent these cases, there's a proposal here for new functions in the OS, package, which just on the surface, I'm I'm starting with a minus 100 points.

Like, I don't like expanding even more the the OS library. It seems to me like it's pretty complete, and any other functionality you wanna add above it or, you know, alongside it or inside it should be in a in a different pack because you wanna keep it, like, small. And the reason to to opt for these functions is because operating systems offer them. Right? So the always package basically wants to provide an interface that allows you to interact with, you know, win API or, you know, modern Linux functions, and there is.

You don't only have open to open a file descriptor in Linux, you have open at. And if you're if you're familiar with win API, ntcreatefile has root directory just to prevent these specific cases. Right? So the operating systems offer these functionalities, and this proposal, you know, is just for, hey, let's expose it in the Go library as well. This is really, really well documented and people seem to like, the proposal is really well written and people seem to like it.

Even though I started a minus a 100 points, I ended up deciding to upvote it as well. It is important to note that you don't have to wait for this proposal. If this sounds like something super relevant to code you're writing right now, there is safe open. It's like a package, from Google, just called, safe open that has like open beneath. And you basically say, only open files that are beneath this path and then it's, like, immune to directory traversal, you know, like, exploitations, I should say.

I'm wondering what do you what do you make of this? Like, should we do stuff like this if it's that justified? Like, OS supports it. It's for security.

Jonathan HallJonathan Hall

Yeah. I do think it's justified. I also agree with your sort of knee jerk reaction that you why do we need to add more variant to those in our library? But I think this is justified.

Shay NehmadShay Nehmad

The thing that's killing me is, alright, someone's gonna do open, and then someone's gonna oh, it's open. They've written Go, like, 5 years, and they try to open a file, then someone's like, oh, no. It's, open in now. Like Mhmm. Why would you change it? Even if it's justified, it feels like a frustrating transitionary period. Right?

Jonathan HallJonathan Hall

Definitely. And and I don't I don't think we're gonna be in a place where we can, like, deprecate with GoVet, for example, the old ones because they're still definitely relevant for certain certain cases. So it's not nearly as clear cut as just stop using the old one and use the new ones now, which is gonna kinda be unfortunate, but that that's reality.

Shay NehmadShay Nehmad

Yeah. And and by the way, it's not like operating systems offer everything out of the box. Like, Windows for, I think, for delete file does not have delete file at or delete file beneath it. So it is going to have to be a pure go implementation using the, you know, other capabilities that the operating system, or, you know, Windows file system API, exposes today. And, you know, I ended up being on, sure, let's go ahead with this.

But I really hope that the documentation is gonna be great and the errors are gonna be really, really good, you know, and the error text is gonna be very specific and there are gonna be different error types for, you know, traversal attempted and root directory access attempted and what happens if you pass in a a symbolic link that actually points to the, open end. Should it work or shouldn't it work? Like, every time you touch file systems, it looks simple on the surface. Oh, if it's in the directory, no problem. But when you actually get into it, you can either go with whatever the person who wrote the c function in the Linux, you know, kernel decided, that's the truth, or tried to make sense.

Sometimes these line up, sometimes they don't, but it's another, case of someone taking a really, really good package. In this case, it's Google Safe Open and offering it in the standard library. And I think there's a big difference between what I did, a few months ago where I said, let's take a really good package called yaml and suggest it in the standard library, which is a huge dependency, really big and doesn't really have justification other than, you know, maybe if we offer JSON, why why not the ammo? Where, you know, there's a ton of real world examples of people accidentally using open file, and exposing themselves to direct reach reversal. But it's gonna be a tricky, proposal and it's gonna be a tricky implementation.

I think if you're into, like, operating systems, and I don't know, files and security, this is something you should really jump on. Both the discussion and the implementation. And it's still, like, open right now. It's still active. So, you know, it's it's all open. I'm I'm not sure, how the team even feels about this one other than adding it to the active column, which is a good sign.

Ashley Jeffs

Mhmm.

Shay NehmadShay Nehmad

So link in the show notes if you wanna dive into it.

Jonathan HallJonathan Hall

So if that one wasn't full of enough, nuance, let's talk about another proposal.

Shay NehmadShay Nehmad

Yeah. This one's technical, but I I think it's interesting as well.

Jonathan HallJonathan Hall

Yeah. So the, you may recall that, presumably in Go 123, unless something drastic happens, they have to revert it. We're going to have the ability to range over function. So you can create a function with a certain signature that that can be fed into a for loop, a for range loop to iterate over the return values of this function. And so there's a new proposal. How shall we say?

Shay NehmadShay Nehmad

I think it like categorizing it as a new proposal is not as relevant as just as imagine someone in your, work Slack. There's there's a channel called big refactor whatever, and someone or it'd be huge ass new feature, and someone's like, wait, new thread. We need to discuss this specific sub feature.

Jonathan HallJonathan Hall

Because That's really weird.

Shay NehmadShay Nehmad

The whole Range Rover fun thing is a really big issue. I'm not sure even most people understand it or like it or or they're not sure where they use it. And then people get into it, get, you know, I've been writing with it for a while, and it's working really well. The discussion here is about a specific implementation detail of, let's imagine you're, ranging over the seek, right, the sequence Yeah. Yeah. Which is, let's, range over a map, Right? And get they have keys and values.

Jonathan HallJonathan Hall

Right. So your your function returns

Shay NehmadShay Nehmad

2 keys.

Jonathan HallJonathan Hall

The value. And then you call it multiple times, and you get multiple copies of your multiple instances of key value pairs.

Shay NehmadShay Nehmad

And I remember discussing this on the show and being like, damn. They say other than the fact that the type is called seek 2 with the with the number 2 in it, which is just fugly, there's I had no issues with it. Like, I couldn't think of a problem. But, as stuff happens in software development, the team has been using this for a while. Right?

And they've come up with a case where it doesn't work so well, where it's not key value, which you just imagine, oh, for kv equals whatever. That's what I've been doing in all the other languages. What happens when it's for value and error? And it's been very unergonomic to write it.

Jonathan HallJonathan Hall

And the reason this is a problem is because by convention, and this is already true with the slices of maps, you can emit the second variable, right? But if you only want the keys in your map, you just say for key, both equals range map, and you just get the keys and you don't get the values.

Shay NehmadShay Nehmad

But for functions, obviously, that doesn't work because you have to do let's say you wanna ignore the error, which, by the way, in a in a function that you iterate over kinda makes sense, you know, because you expect the iterator to finish if it's, EOF.

Jonathan HallJonathan Hall

It's very context specific, I think. In some cases, you might wanna ignore the error. In some cases, you don't.

Shay NehmadShay Nehmad

And it's it's mostly it's it's mostly frustrating having to write, you know, I explicitly ignore in some places. The way you ignore the return values in go is you assign them to underscore. Right? So in your range expression, you assign the arrow to an underscore. Now that doesn't sound so bad and honestly, it it really isn't.

I think they're really workshopping all the little tiny frictions that they are going into and they're not afraid to rework it, which in my opinion is good overall, like, for the success of this feature. But it it does end up with, I don't know, a 100 comments discussion on whether we should or shouldn't omit, allow to omit this, this, you know, variable, which you can ignore anyway. Right? You can assign it to underscore. This entire discussion is about whether you can drop the underscore and just do like for value in whatever.

And let's imagine this whatever, this function is you iterating over, lines from a buffer. Right? You iterate over lines from your WebSocket. So you write a function that reads a line from the WebSocket or, you know, the next message or whatever. I imagine like a Kafka iterator or something like that.

And you don't care about errors at all in this case. Because you, I don't know, handle it in some other level. That makes a lot of sense. Let's say if we're talking about Kafka, you have one layer that reads from Kafka, deals with all the parsing error, deals whatever, and then you just wrap this whole thing with a message iterator that you just wanted to work. You don't wanna worry about anything.

You just just wanna worry about the mess you just wanna iterate over the messages. But you still need to be able to return error in case something horrible happened, But then you wanna ignore it, let's say, in one in the test. Because in the test, you don't care about the error. Or maybe the other way around, by the way. Maybe you need the error in test, but in production, you're like, ah, whatever.

It's good enough. This proposal is all about allowing you to just iterate over a function that returns 2 values and ignoring the second one. The main case being ignoring er. I don't know if I'm for this one, by the way, overall. Even though I I've been praising the whole, process, like, the workshopping it of it is beautiful. Yeah. I don't love the fact that you can omit, like, the the value and key value Yeah.

Jonathan HallJonathan Hall

I think I I would be in favor of requiring the explicit underscore. Oh, honestly, I'd be in favor of that in all cases, even for maps and slices that we have today. Of course, they're not gonna make that change. But

Shay NehmadShay Nehmad

Yeah. Because that's gonna piss off

Jonathan HallJonathan Hall

a lot of people. Yeah. I I I think more explicit is better. Less

Shay NehmadShay Nehmad

I've never I've never written, like, you know, slightly more explicit code, and have been like, oh, I really regret that time invested. It just never happened to me.

Jonathan HallJonathan Hall

Exactly. And there have been times when I've done 4 k comma blank equals whatever, and then my linter or whatever tells me I have to remove that. I'm like, okay. I guess I will since you told me to, but I wish I could leave it there because it it's more it's more explicit. Yeah. I'm in favor of this

Shay NehmadShay Nehmad

I'm gonna do it, but I don't like it.

Jonathan HallJonathan Hall

Right.

Shay NehmadShay Nehmad

This this whole episode, I don't know if this is just because we're both kinda, tired out of our our minds, but this whole episode just sounds like quotes from my daughter. I'm gonna do it, but not because I'm happy about it. Yeah. So this proposal is new. Again, it's part of a whole big thing.

So it's not, new in the same sense that we just discussed with the, you know, safer OS stuff. Right? It's already doing implementation of other thing. We changed we wanted to change one thing to another thing. Most people here, by the way, agree with dropping error, which is surprising to me because I'm not in favor.

There is the I don't wanna say unfortunate because it's I don't think it is unfortunate. But there is the thing to know that Russ, who I think is leading this feature for sure, is the one who opened this discussion and clearly has an opinion. So I think unless something drastic will happen, you know, he's the person, in the trenches. Right? So I trust his opinion a lot more than my $2 analysis here on the show. So I got a cool link on our channel.

Jonathan HallJonathan Hall

I like cool links.

Shay NehmadShay Nehmad

Our channel, the ad break is really soon, so I'm gonna plug it, quickly. Capago, kebab case on the go for Slack, has been home for really interesting links and and discussions and and jokes and japes. And, Peter and Links. And Peter, has been listening to the show and actually been listening to me, which is interesting. Funny decision.

And I've been working at work, on fixing flaky tests. Actually, since we've, discussed this, we already implemented a flaky test, like, detection pipeline that works automatically and sends messages, and then we got some responses from some teams. Cool, so you're gonna fix them? So, you know, some teams are really overloaded with the features right now, so we're actually fixing some tests. It's a really interesting, like, cultural discussion as well.

And just in time, 2 days ago, Uber, released a flaky test overall at Uber. They have a hue if you don't know, Uber has a really big engineering culture thing. Like, they have a lot of open source. They have a lot of, blog posts. I think they're really big into engineering branding. You know what I mean?

Jonathan HallJonathan Hall

Yeah. Yeah.

Shay NehmadShay Nehmad

I assume they even have an Uber, like, developer podcast, just assuming. But maybe we need to find it. So they, started tackling, flaky tests. Just, you know, for our listeners who maybe aren't familiar with the term, what are flaky tests, John?

Jonathan HallJonathan Hall

Well, they're tests that that don't they aren't consistent. Yeah. Is that is that a fair way to put it? They aren't consistent?

Shay NehmadShay Nehmad

I I would categorize them as the devil, but, you know, to each their own. So it's basically tests that fail randomly, and then let's break down randomly into okay. So they break down if you run them in the incorrect order. They break down statistically because they rely on on the some DB residue or something that has to do with time or something that has to do with undefined behavior. Right?

Let's say, in it, order in Go up until a recent version was undefined. So if your test relied on that, it is flake. Even if it is passing deterministically, it can be flake. But the the fact that a test can be flaky is not the part that matters. The part that does matter is you're working on a super critical feature at 5 AM before a demo.

Right? You've been working all night and you're trying to push your branch with a critical fix and the test fails and you're debugging it, and then you rerun and it passes and you wanna die. This is a really bad developer experience. Yeah. And they went all out, man.

This blog post is is amazing. It really shows the difference between small scale and big scale enterprise y, software development, where, like, they have a goal. They developed a whole subsystem in Uber. They're calling it Testopedia that shares you know, they share the structure where, basically, they monitor, all the tests, and every test has a a life cycle, Right? Where it can be stable, which is the good state.

It can be unsound, which is the service marks it as, oh, this is, flaky basically, and then a human can either delete it or disable it, and the system can decide to automatically delete it as well if it's too unsound for too long. So if a flaky test is flaky and the team doesn't get to it, the system automatically deletes it and, you know, the moment something gets into unsound, a Jira ticket is fired off. It's a really interesting system. They even had to worry about, like, the scale of it, right, analyzing the test data. This is just something that, you know, we have a lot of tests at Orca where I work, but this is not the scale we're looking at.

Like, this is pretty amazing. There's some really interesting Go specific stuff here as well. I think the way they share their data model is really good. If you're DevOps, you should read this. If you're engineering enablement, you should read this.

I got this link for from, like, 7, different people, but, thanks a lot, Peter, for, sending it to me. You you were only 7th because, I checked the the Kapago Slack slightly less than the work Slack. That's the only reason. But it's a really, really good you know, there's something about a really good blog post that you read. It has a lot of charts, has a lot of code.

Everything's written super well that you feel like the person who wrote it was like, I just want to give this to you. I wanna gift this to you. It's not like AI generated slop top five list whatever. And I'm really interested in your thoughts about their future plans. So they built this whole system and now they're working on developing LLMs to, like, generative AI to both resolve flaky tests automatically, like suggest a fix, and find the, categorization, like explain why this test is flaky, not just the fact that it is flaky.

In your experience, you know, dealing with these tests, do you think that's even feasible, or is every flaky test a unicorn?

Jonathan HallJonathan Hall

I think in some cases, it's feasible, for some I mean, I actually have a few flaky tests I deal with on a regular basis. I I have a pretty strong idea why each one happens, that they've been happening long enough, but I've I've built that knowledge. Mhmm. When I'm faced with a brand new flaky test, often it does require an investigation to to determine that. And I'm I imagine an LLM could could help in many cases to help, you know, pinpoint that.

You know, this is a data race or this is a, a timing bug or this is, you know, test execution order, whatever.

Shay NehmadShay Nehmad

I'm not sure. I just have a feeling, like, it's slightly too complicated for just shoving it to do chat gpt and asking it why it's flaky. But maybe if they have their own, like if they're at the point of developing and training their own models, then maybe. But I'm not sure.

Jonathan HallJonathan Hall

I don't know. We'll see. I mean, it wouldn't have to it wouldn't have to solve even a large percentage of them to be helpful. If it if it could point you to the right answer for 10%, that's that's the big value.

Shay NehmadShay Nehmad

The the problem I have is if the other 90% is, like, pointing you towards a solution that ends up wasting your time because it looks correct, but it's not really. That would be helpful. Anyway, their numbers at the end are also staggering. They're like, quote, in the Go monorepo, we are steadily detecting around a 1,000 flaky tests. And I'm like,

Ashley Jeffs

what? That's horrible.

Shay NehmadShay Nehmad

And then you finish reading it, and it says, quote, out of 600,000 in total. That's so many tests.

Jonathan HallJonathan Hall

A lot of tests.

Shay NehmadShay Nehmad

This is a texti company. Right? Anyway, so thanks a lot for sharing this link. It was super interesting to me. I also it was, you know, how you know a link is really good? The time between you took you to read it and the time you sent it in work Slack. For me it was almost instant. So thanks a lot, Peter. Keep keep them coming, man. We really like your contributions in the channel.

Jonathan HallJonathan Hall

One last story that was, brought to us by the channel, the same channel.

Shay NehmadShay Nehmad

Sounds like a cult, the channel. The greater good.

Jonathan HallJonathan Hall

So I I found a different article talking about the same story, though. But the the headline is Red Panda Acquires Bentos expand its end to end streaming data platform. Bentos, as you may recall, is a data streaming platform. We've interviewed the creator of it on the show before. But, you know, the the biggest surprise to me here was that Red Panda is an actual thing and not just a made up, you know, logo. You know what a red panda looks like?

Shay NehmadShay Nehmad

The animal? Yeah. I've seen one up close. It almost bit my, my nose off

Jonathan HallJonathan Hall

Wow.

Shay NehmadShay Nehmad

In a zoo so one time. They're really cute.

Jonathan HallJonathan Hall

Like, I would have thought it was a Firefox mascot if, somebody hadn't told me otherwise. But, yeah, they look really cute.

Shay NehmadShay Nehmad

They're also, like low maintenance zoo animals. My mother-in-law told me because she used to work in a zoo.

Jonathan HallJonathan Hall

I'll keep that in mind next time I open a low maintenance zoo.

Shay NehmadShay Nehmad

Yeah. I can't imagine you're running high maintenance one. So what's up with this article, man? This sounds like huge news. Do you know any way anything about it? Can you can you grok it, figure it out?

Jonathan HallJonathan Hall

Yeah. So, like, I don't know. One of the most interesting things here, other than the fact that Red Pandas are real and they're cute, is that as a result of this announcement of this acquisition that Red Panda is acquiring Bentos, a new team has come out to fork Bentos. It's it's like Terraform all over again.

Shay NehmadShay Nehmad

Terraform and OpenDoFu. Right? Yeah. I think I I tried to read about it. There's a lot there's a lot of nuance here. It's not like Yeah. All of Penthouse got bought, and then everything got forked. We need to dive deeper here.

Jonathan HallJonathan Hall

I'm kinda con well, I honestly, I find it kinda confusing. I I I wonder if if so we can get an expert. Oh, you know what? I think I have actually Jeff's on my speed dialer here. Maybe we could get him on the show.

Shay NehmadShay Nehmad

Do the speed dial mobile. Welcome to our ad break. We have, Ashley on hold here, so I'm gonna rush really fast. If you wanna help sponsor the show like the rest of our beautiful, beautiful, members on Patreon, we would love for you to do it. This show is a hobby, but it's pretty expensive.

We have to pay for editing and hosting fees and stuff like that. So if you like the show, if it's useful for you at your work or university, supporting us, $8 a month is, really nice. To find the link to our Patreon or maybe to our swag store where you can hook yourself up with some hoodies, cups, whatever, to find our Slack channel where, we just talked about the, link Peter shared. You can find everything at cupago.dev. That is cupago.dev.

If all these fancy communication, platforms are not your thing and you're not into Slack, you can also email us at [email protected]. Another way to support the show that we really appreciate, we don't pay to advertise anything. Well, I don't know. I pay for Internet to share people the link to the show. I don't wanna say anything anything.

But we don't pay for ads as you would think about them. This show is only spread by word-of-mouth. So share the show with a friend, a colleague, co student, whatever, and leave us a review on Spotify, Apple Podcasts, or wherever you listen to the show. It helps us, you know, get through the rankings and, get into some random listeners' hand. We've gotten some really good feedback about the show recently on LinkedIn and on Slack, and I even got some face to face at work.

And I'm really hoping to, you know, run into some, listeners in Amsterdam, where we're gonna have our, I'm gonna go to DevOps Days, and we're gonna have a meet up on, June 19th. So we're really excited about the direction the show is taking, and your support is a big part of that. Thanks a lot. And I think we're just gonna jump into the interview. See you next week.

Ashley Jeffs

Listen, I told you not to call me at this number again. Next time, there's gonna be consequences.

Shay NehmadShay Nehmad

This is an emergency, Ash. We know this is the never call you on this number line, but the people need to know about red pandas. Wait, that's not the oh, red panda. Oh, oh, wait.

Ashley Jeffs

There's only 1.

Shay NehmadShay Nehmad

Yeah. Hello, Ash.

Ashley Jeffs

Hello.

Jonathan HallJonathan Hall

Thanks for taking my call Hello, Willow. Even though this is this is the wrong number, The red phone. When when we're done

Shay NehmadShay Nehmad

with the call, can you, return this briefcase with this phone to the president? No. So welcome, Ashley Jeffs, also known as Jafail, online. For the people who don't know you, can you briefly present who you are?

Ashley Jeffs

So I'm that guy what did the, blobfish, Bentos open source project thing. You may if you're in the ghost space, you might also know me from my project called Gabs, and there's another even less popular one called Tunny. I've got a few few few fingers in various open source pies in the Go space, all with rather silly logos. But obviously, the more successful one is is Bentos, which is an open source stream processor, and that's all written in Go. Yummy. Yum. Yum.

Shay NehmadShay Nehmad

Yum. Mhmm.

Ashley Jeffs

And that's pretty much I do lots of YouTube stuff. People might know me from that because I've got I actually have content that's not related to my various open source projects, but it's related to code sometimes. So people might know me from that, but probably fewer. Cool. But you can find me on there and Twitter and all the other places.

Jonathan HallJonathan Hall

We'll have links

Ashley Jeffs

here to all

Jonathan HallJonathan Hall

that in the show notes.

Shay NehmadShay Nehmad

And if you're a long time listener, you may remember we had Ash and, Mihai on the show, like, I don't know what it was a year ago.

Jonathan HallJonathan Hall

Probably. Let me look it up.

Shay NehmadShay Nehmad

That I think about it. Damn, that was that that was long ago, man.

Ashley Jeffs

Are you gonna sound bite all the stupid stuff I said?

Shay NehmadShay Nehmad

That's a really good idea. We might ask, Filippo to edit all that stuff in. So a lot has changed since, that last recording we did, since that last interview, and we need to talk about it. What do you think about my new tattoo?

Ashley Jeffs

It's not a blobfish, so I'm not interested.

Shay NehmadShay Nehmad

So the big news this week, obviously, is everybody on both sides of the aisle, agree that your code is great, and, Ventas got acquired by Red Panda, now it's called Red Panda Connect. Like, there's a thing called Red Panda Connect, and there's still the Bentos open source library. And we wanna talk about that and try to understand what happened. We wanna congratulate you, by the way, 1st and foremost.

Ashley Jeffs

Thank you.

Shay NehmadShay Nehmad

Dude, that's great. That's a really, really good move. Red Panda, by the way, in my opinion, this is just my opinion, really good stewards of open source stuff, really cool guys. So, you know, this would have been a different tone if this was, like, IBM. But, not if we have listeners than IBM. Not not, like

Ashley Jeffs

They're they're actually I would say they're not doing a bad job with Ceramic. I actually think the changes

Shay NehmadShay Nehmad

I'm trying to think about a company that I really wanna, like, name and shame in in terms of so then I thought about it a bit some more, then let's steer away from that specific part of the conversation. I don't know. If Saudi Aramco Nukes Tech.

Jonathan HallJonathan Hall

Yeah. Exactly. Seal Club Inc, if they brush them I wouldn't have been No.

Ashley Jeffs

They're doing really well with their open source libraries.

Shay NehmadShay Nehmad

They they actually built the pprof. I'm sure. So anyway, congrats about that. We

Jonathan HallJonathan Hall

We really

Shay NehmadShay Nehmad

wanna get into it and what happens in the future. But just in case someone, you know, wants to understand this news but doesn't know what Ventas really is. And they we don't wanna send it back to old episodes. Can you give, like, a one minute summary of what this open source project is and why is it so big?

Ashley Jeffs

I can't promise 1 minute. Basically, what we had so for about 10 years, I've been working on this open source project. It's a stream processor. If you don't really know what that is, it's basically something that reads from message queues and things like Kafka and your NATs and RabbitMQ, that kind of stuff. And what I built is a sort of engine for doing all that stuff with the decorative config.

So rather than having to write code and all that stuff, you can just write some structured YAML. And just this emphasis on operational simplicity. So making sure it's really easy for people to build those configs and maintain them, but also when you deploy this streaming processor, it doesn't do any nasty stuff. And that's basically, ingrained in me from my time working on services like that and seeing how badly the operations people suffered when they had to deploy services like that. So things like data loss, back pressure, recovery, metrics, observability, all that stuff.

And it's very, very hard in the stream processing space. As the project grew, the number of connectors we had, got bigger and bigger and bigger, and the community got bigger and bigger and bigger. It was all organic. It was all just me living on my own off various. So I've I've gone through the rounds of monetization strategies.

I've tried about it's been, like, 5 different phases, which we can go into in detail if you want, of me trying to monetize it essentially without VC funding. I ruled that out a few years ago that, basically, I didn't wanna do that for a number of reasons. And the project was getting to the point where it was too much for me to be the only person working on it full time. And we have contributors, but they're also doing their own thing, trying to live off, you know, their own means and figure stuff out on their own. So we didn't really have the ability to expand from that.

So I was trying to find lots of ways of bootstrapping some sort of company without sacrificing family life and the ability to watch TV and video games. And I've been pretty open over the years of how that's been going, to very varying levels of success. And, eventually, it got to the point where, okay, I needed to make a decision, and the timing was right. It wasn't exactly when I expected to do this, but I was basically This was on my list of options at the end of the journey, and it was it was coming to that point anyway. So what's happened now is I've sold the project.

So there's been an acquisition of the company behind Benthos. And what's happened is so it used to be fully MIT licensed, all the connections and the engine. We already had the plan in the open source community just in general to separate the core stream processing engine from the connectors themselves. And the idea is that you would decentralize so people can have any suite of connectors that they want, because they all have different dependency trees and different runtime, weirdnesses. And it was just becoming really cumbersome having all of them baked into this one open source project.

So we wanted to split those out anyway. So what the acquisition has done is it fast tracked the splitting of we've got an MIT licensed core engine, which is the plug in engines, the stream processing engine, all the important stuff, all the stuff that makes Bentos, Bentos. That's still called Pentos. Red pounder Bentos, if if you wanna, fully adhere to trademarks, but you're not gonna get sued just for saying Bentos. But the other bit is called Red Panda Connect, and that's essentially the distribution of, essentially, using Bentos as a library to build all of these plug ins.

We've relicensed most of them to Apache 2, and then there's 2 that we've enterprise licensed as part of the monetization strategy. So, basically, the acquisition purpose is to try and find a way of sustaining a team of engineers to work on this project and not sacrifice their lives doing it. And, hopefully, that's gonna be healthier for the project, but who knows? It's only been a week. Who knows what's gonna happen?

Shay NehmadShay Nehmad

Pretty fresh.

Jonathan HallJonathan Hall

Yeah. So are you gonna stay on the team? Will you be still working there, or are you going to the Bahamas now and you're done?

Ashley Jeffs

No. I'm staying I'm staying on it. I don't have to, but I'm staying I'm sticking around. I mean, my my intention now is to build out enough people who so there there's, like, lots of different groups of people that I can I can create, I can spawn? So there's obviously people who just are able to support Red Panda Connect users. Red Panda Connect is a lot, so I'm I'm starting to shorten that to, rednecked. Just the the red at the beginning and the necked at the end. Redneck. Alright.

Jonathan HallJonathan Hall

We need we need Jeff Foxworthy on here now.

Shay NehmadShay Nehmad

Yeah. I can hear the the radio, ads already. Right? Like, banjo twang. Welcome, y'all.

Ashley Jeffs

Shout out to all my, southern US friends.

Shay NehmadShay Nehmad

Do you know I I have to do do this. Do you know that, in old Unix I'm not sure if it's Unix or Linux, but really old distributions. Redneck was a language. Like, you could install it in English, like Spanish, French, Hebrew, like whatever, Russian, and redneck. Not as a joke. 1 of the engineers was a redneck and it'll be like, yep. Nope. Because in the I'll look that up.

Jonathan HallJonathan Hall

There's a screenshot here I found. In 1998, Red Hat had redneck as a language option.

Shay NehmadShay Nehmad

Yeah. Yeah. Red.

Ashley Jeffs

Oh, okay.

Shay NehmadShay Nehmad

It could happen.

Ashley Jeffs

Damn. That means I can't I can't trademark it. But, yeah, anyway, so I'm I'm trying to I'm I'm building out people who can help support. Because right now, it's the developers who work on the Bentos ecosystem, and they they are also in the questions channels, the Discords, the Slacks, answering all the questions. And most of the time, you're you're you're basically consulting for free.

Right? Because people are coming in and saying like, how do I do x to y and do this blah blah blah and use case blah. And it's like, that's that's it's great that we can help those people, but we're not gonna be able to do that forever because the community is getting bigger. So we're not we're not gonna be able to just answer those questions.

Shay NehmadShay Nehmad

We're not

Ashley Jeffs

gonna spend half an hour every morning answering some random consulting questions. So the idea is we're gonna expand the number of people who can help, just generally help, and understand the the service. Then we're expanding the people who can work on it. Because right now, there's lots of people who've contributed features on top of the engine. So people who are adding, like, fields to connectors and documentation and a new connector here and there.

There's, like, a small pool of people who contribute quite a few bits consistently, but the the number of people who really get down and dirty in the core stream processing engine, It's basically just me and maybe Mihai to a to a slight degree, but if you look at the the real nasty, like, gnarly bits, that really takes a lot of brainpower to get all the factors of stream processing right. Imagine, like, delivering guarantees, vertical scaling, back pressure, and, you know, all the other bits, you you know, usability, error handling, all those things. To get all of that right for some of these bits is really difficult. So I obviously can't be the only person on the planet who knows how to do that stuff because it's just gonna degrade over time. Like, the the project will degrade if I disappear from it.

So the idea is I want to onboard more people with enough time to get stuck into that stuff, which is already happening. So it's not just me who has to work on that stuff in theory. And then it's just better for everybody. Right? And then, you know, obviously, more people working on the connectors, more people more people consulting.

So it's it's just a general, endeavor to expand the community. Because, I mean, as well, I'm doing all the graphics, all the documentation, all the videos, all the training materials, all of that stuff was just me, which is great fun, and it it means that all the content can be equally ironic and just generally silly because, you know, I'm not a professional, so I'm just doing all this stuff fun, which has been great, but that can't that can't last forever because you can't you can't just have one person doing all those things forever.

Shay NehmadShay Nehmad

I really love the the the, like, student listening to the show being, like, the guy who just sold his 10 year project to a reputable streaming company. Like, no. I'm not a professional.

Ashley Jeffs

My family are like, did you did Ash, did you sell the blobfish? Is that what's happened?

Jonathan HallJonathan Hall

So

Ashley Jeffs

They they didn't get

Shay NehmadShay Nehmad

any Yeah.

Jonathan HallJonathan Hall

So I'm curious. Before this acquisition, how much of your time was dedicated to this? Was this full time for you?

Ashley Jeffs

Yeah. Well okay. So full time is so the the public face of what's happening, right, is a fraction of my time spent on the project as a whole because what I'm having to do to survive is spin all these plates, not just for my current income, but also, like, trying to figure out what am I gonna do in, like, a year? And what what Mhmm. What am I gonna live off in a year's time?

And trying to get to that point where I can hire more people is the all the the main goal. So I'm you know, I was dedicating a lot of my time to just that. You know, that it was getting to the point where I would almost spend all my time some weeks working on that stuff. Because I don't, like, I don't know how much of your audience are solo entrepreneurs, but just doing stuff like procurement, like getting getting a company. You have a company that is desperate to throw money out.

Mhmm. And they want to they wanna have a support contract, and then it goes to procurement. And it's like, oh, no. We can't. We aren't giving you money. You're just some guy in the UK. Mhmm. No. We can't do that. And, you know, I'm I'm spending my time reading through, like, 60 page contracts and talking to random people through email, trying to figure all this stuff out. And it it means that the amount of time I was spending on the project was getting smaller and smaller.

Shay NehmadShay Nehmad

Just because of

Ashley Jeffs

that? Working on Benfoss Studio, which is like a complete closed source, spin off thing, which is basically a visual UI for it, which I like, that was bay that was my big plan for the neck. For for essentially this year and last year was to try and monetize that. But, again, it's it's the procurement that ends up killing me.

Jonathan HallJonathan Hall

So now that the acquisition is done, are you employed by Red Panda? Or okay.

Ashley Jeffs

Yeah.

Jonathan HallJonathan Hall

And

Ashley Jeffs

Yeah. So I'm

Shay NehmadShay Nehmad

badge and everything? Company badge, company uniform?

Ashley Jeffs

Company badge yet. I got a laptop.

Jonathan HallJonathan Hall

Oh, there you go.

Shay NehmadShay Nehmad

Damn. I'll

Ashley Jeffs

just fight my way into a physical office. It's everything. But, yeah, I've I've got a a official title. I'm Connect technical

Jonathan HallJonathan Hall

lead. Alright.

Ashley Jeffs

So I'm I'm running the the project to to continue, you know, working on this thing, expanding the team, expanding the resources. And then, like, I'm still gonna be doing the pull request features, all that stuff.

Jonathan HallJonathan Hall

So from one day to the next, the main thing that changed was who pays your salary. Yeah. But but over the coming months, you're building a bigger team to help offload a lot of those tasks.

Ashley Jeffs

But but also all that time I was spending on trying to, like, essentially solve next year's income Mhmm. I no longer have to spend any time on any

Shay NehmadShay Nehmad

of that.

Ashley Jeffs

I've now got a there's a sales team that I can hand stuff over to. There's, you know, there's sales engineers that I can liaise with, and, you know, I don't have to worry about any of that stuff now. That's great. So, I mean, that's that's freed up a huge amount of my time. Obviously, I've just joined this company, and I'm getting all of them up to speed.

So that's taking up, you know, some of my time. But, already, I'm I'm spending more time on pull requests. And I'm not like, there was there was this horrible sensation. I've generally always enjoyed working on the project and especially being free to just do whatever the hell I want. Right?

Make whatever decisions I want, has been a wild ride, but it was getting to the point where because I was stressing about, it doesn't look like this is gonna work, and I was worrying about, oh, no. What if I have to because I'm not I I decided against VC funding just because I don't think I'm gonna enjoy that job, if I'm, like, the CEO of, like, a VC funded start up. And it was gonna eventually lead to tough decisions that I wasn't gonna wanna make around licensing and all those things. And the the worry that I had over the last, like, year or 2 is I'm merging pull requests, and I know that I have to support what I'm merging. That's always the contract.

Right? Because I'm I'm the face of the project. People are coming to me when they have questions, and ultimately ends with me. If there's nobody else volunteering their time in the in the chat channels, then it will land on me eventually. And anything that I build to try and live off, I'm also supporting this whole bag of this massive burden of connectors, all these different technologies.

And what I was finding is I I used to be able to fit all of it in my head. So somebody would come with a feature. I'd merge it. I'd understand it, and it would just be there in my mind. And it was getting to the point where that wasn't the case anymore.

So I was merging, you know, like, MongoDB connectors or, Google Cloud connectors, and people come along and say, oh, how how does this thing work? And I wouldn't even be able to tell them that that component exists. Like, I wouldn't even know what's in my own thing that I'm running. And it's it's stressful, you know, like, because I don't want to stop people's poor request from being merged. Right?

I don't want to, like, prevent because people have already done that work. They've they've put the effort in. So I don't wanna block them, but I I don't have enough time to do the merging. I mean, even when I do, I'm, like, freaking out about it. Whereas now, I have a team of people who can help me with all that stuff.

So me merging pull requests is just about is it good code quality? Does it fit the, you know, the overall purpose of the project? Is it gonna help people? If that's all yeses, then merge, and that's it. Mhmm. As simple as it gets.

Jonathan HallJonathan Hall

Awesome. That's nice. We're laughing because, the the fire was just scared. Yeah. The thumbs up and

Shay NehmadShay Nehmad

Yeah. And do you see a difference or I guess, what differences do you see between stewarding, you know, a herd of open source cats developers, in trying to direct this sort of open source chaotic energy versus, you know, a team, in a company that, like, has a purpose. I'm trying to abbreviate the the title as well. So your rednecktied, your Red Petna Connect technical leadership is probably a lot more I get that you've only been there a week. Maybe that's an unfair question.

But you plan for it to be slightly more structured, I guess, than

Ashley Jeffs

there's gonna be aspects of it. Yeah. So there there's obviously gonna be, like, enterprise features that we're gonna wanna have, to unblock customers and, you know, the the usual business y stuff. Those are obviously gonna be things that we're gonna try and fit in on a deadline. There's also gonna be open source features that we're I mean, same same applies.

You know, things that we're gonna wanna need is nice to have as we're gonna use. So we wanna get those in on a certain timeline. To be honest, that doesn't really change from my personal perspective because I was always doing that. I was always setting deadlines and then slapping the keyboard until it was done. But getting other people on board to be honest, it was at the point in the last couple years where I felt as though I would be able to say, hey, Mihai, can you do this thing?

And as long as it's something that's good like, obviously, it's gotta be good for the project. It's gonna be fun to work on. You go and do it. I mean, he's doing it for free. Right? So, you know, there's obviously within reason, you you need

Shay NehmadShay Nehmad

to do

Ashley Jeffs

the stuff. But if it was something that was good for the project, there's been times where that's that's been the relation we've had. So it's it's kind of like having employees light.

Shay NehmadShay Nehmad

Mhmm.

Ashley Jeffs

And as long as you're respectful and everybody is kind of on the same page, then, you know, that's that's kind of what you're doing. So I guess you're kind of like it it's the training wheels version of running a team where you're using, you know, motivation and mission to get that sort of stuff done. Whereas now I can just say, money next week or now.

Jonathan HallJonathan Hall

So I I wanna talk a little bit about the acquisition. So Red Panda bought Bentos. Did you approach them? Did they approach you?

Ashley Jeffs

I've known Alex for, like, 5 years. There there was other suitors. It was kinda funny because any anybody following my movements for, like, 2 2 months ago, I think it was, I was at I was at Kafka Summit hanging out with the WarpStream guys at their booth. And then, you know, I'm, like, spending a week in San Francisco. And everybody anybody that I'm interacting with online can just track my movements to see what's happening.

So it's like you can keep it you can keep it a secret to an extent, but, I mean, if people want that info, it is it is actually just there. It's it's not really particularly easy for me to I've I've had a good relationship with Senadia for a long time as well. Mhmm. They've been they've been OGs. I mean, so my main monetization has has just been nothing for a couple years, and I've been living off sponsors, and they're by far my biggest sponsor.

They they basically kept me alive when I had my kid, because I didn't have time to to really chase up support contracts, and they just happened to appear at the right time. I was like, what do you need? So, yeah, I mean, I've obviously had conversations with lots of companies kind of the same the same ilk where they're all partners in that sphere. And, I mean, I didn't I didn't really have, like, a a a strict agenda or a timeline thing. Basically, I can't really go into details specifically of what happened, but stuff suddenly moved very, very fast, and it became apparent of like, oh, okay.

This is gonna happen then. I guess this is gonna happen right now.

Jonathan HallJonathan Hall

You don't need to answer this. I mean, I'm gonna speculate. I'm pretty sure I'm right on this. The reason you chose Red Panda is because of their logo. It's an animal. And that is, with the Bentos eat ethos of animal logos.

Ashley Jeffs

They kind they kind of fit. I've already so there's obviously been loads of fans making, mashups of the 2 logos. In terms of in terms of mascot to, to blobfish ratio, it's actually pretty good, I would say. Definitely good. Better than better than any of the other candidates because, I mean, cute mascots is hard to get anyway. Obviously, the red panda is very cute.

Shay NehmadShay Nehmad

Yeah. We wouldn't know because we knocked it out on the park, like, the first first try, but I assume for other people it's hard.

Ashley Jeffs

Same. I just slapped a plodfish together. It's been the same for, like, 10 years. But, yeah, it's it's a good brand. It fits fits very well with my brand vision, I think is what they say. The brand ecosystem. The cine I keep saying the cinematic universe of blobfish. It kinda fits.

Shay NehmadShay Nehmad

The VCU.

Ashley Jeffs

They cut yeah. They they're compatible.

Shay NehmadShay Nehmad

And, you know, the the acquisition basically means, you know, some people have been worried about, wait, acquisition, does that mean all the open source work is going away? You mentioned that the core is saying MIT license, which basically, you know, if you don't wanna get into the legalese means open source and you don't have to worry about it too much because it's not the Red Hat license where it's copy left. And some of the plugins are enterprise, which makes sense, you know, Red Panda have to sell something. And the rest of it is Apache 2. Now Apache 2 is kind of a new kid on the block.

It's, the Elastic, search, license. Right?

Ashley Jeffs

No. That is is Apache v 2 is just basic I mean, it's basically MIT with a bit more legalese around it. So, I mean, you can do whatever you want. AWS could take all of that if they wanted to.

Shay NehmadShay Nehmad

Oh, really?

Ashley Jeffs

Don't AWS. Yep. Apache 2 is,

Shay NehmadShay Nehmad

the people from, AWS on the on the on the podcast. You know

Ashley Jeffs

what I mean? At least Tell them no. Don't a actually, no. Go ahead. Go to town. Take take all the connectors, I think. Yeah. So the the the move so, basically, the move from MIT to Apache 2, I don't know why. I was fine with it because it's basically the same license. MIT is obviously more is is less wording, so you you have even fewer protections, like, for what you're sharing.

But the the ethos is, I don't care. I'm just putting this out there. I don't really care what happens to it. But Apache 2 has a bit more legalese around copyright, that sort of stuff. I have no idea what the implications there.

I think it's to do with, you know, there are smarter people than me who understand the commercial side, figuring out that that's that's better for insurance or whatever. I don't know. And then the enterprise stuff, basically, it we just need, like, a bit of enterprise stuff. We're obviously gonna be adding cool features that are sort of, like, tacked onto Red Panda Connect, just to make it work cooler. So, I mean, an example is we've already got, a new config section specifically for Red Panda Connect where you can forward your logs directly to a topic, which is kinda cool.

People have been asking for that for a while. And there's obviously gonna be features that we're gonna put on on top just to make them all work. Cooler together, I think, is the official term. But, yeah, the the idea is just one connector or to the, enterprise. There's 2 it's the Splunk and the Snowflake ones. And, you know, basically, the mentality there is you're already paying for Splunk. If you're using Splunk, you're paying for it. And if you're using Snowflake, you're paying for it.

Shay NehmadShay Nehmad

You're paying for it multiple times over because your data's getting leaked.

Ashley Jeffs

Yeah. Yeah. Exactly. Of the week. Or maybe they're making money from that. I don't know. Maybe you get a kickback if you take it away. I don't know what joke. Don't you hear me?

Shay NehmadShay Nehmad

Insurance companies are the one pocketing all the the difference. Trust me. Cyber insurance.

Ashley Jeffs

Guess. Yeah. But, yeah, that that's the mentality. Obviously, we're we're already working on expanding them. I mean, peep people were, obviously hyper aware on the day of, well, you've just relicensed enterprise on something that was MIT, which is a 100% correct.

The goal was to show people what the separation was gonna be for the announcement, but, obviously, we're not gonna leave it as it is. Let's just leave the enterprise license. We're we're gonna expand it and and basically redo them and, you know, beef them up a little bit, stuff that we couldn't really do or have the resource. Because, I mean, I don't I can't maintain a Splunk connector on my own. It's like an open source thing.

It's currently a template, and it's just basically using the HTTP component under the hood. And it's like, I can't I can't like, if anybody asked me a question

Shay NehmadShay Nehmad

on that, I would have to go and fish out, the with the Splunk server.

Ashley Jeffs

Right? Yeah. Exactly. Yeah. Because I can't I can't run And there's no there's there's not really a lot of tooling for I think there is, like, a a VM you can run now. But I I remember at the time somebody was asking me about it. I was like, I can't I can't do anything for you. Sorry. But, yeah, obviously, now we've got the resources. But, yeah, so there's there's 2 two components that have been enterprise licensed.

Then what we've done is because it used to be that you would import all of the connectors sometimes as, like, a full suite. If you just wanna have, like, a custom build that has everything in it, all the plug ins, they used to be like a macro. And, obviously, now that there's licensed components, that's a bit dubious for people. So what I've done is I've separated out packages that are all the Apache licensed connectors or all including the enterprise ones, and you can pick 1. And I've got explicit licenses now.

So, at least now you're not like, now you can be a 100% sure if you're making a build, whether it's fully open source or not. I mean, I've I've never run a multi license project before, so it was a bit of a learning experience for me. But, yeah, we're trying to make it as easy as possible to to have custom build with whatever you want in there.

Shay NehmadShay Nehmad

What's the use case for, like, I get why you would want all the processors because I don't see them as, like, super heavy and you oh, suddenly, I have protobuf. Suddenly, I just I wanna start playing on the blah blank, whatever. But why would you need all the inputs and outputs? Like, I can't imagine a system that gets I don't want to imagine a system that has Cassandra, Discord, the Twitter, SQL, Mongo, and HDFS as the inputs.

Jonathan HallJonathan Hall

I just, like, please tell me my website.

Ashley Jeffs

So the I think the the the main reason is, so I've got a bit of insight into this. But, basically, when when you're a company that's big enough, you've got teams all over the world that have all grown independently. So they all they all use different tools and services. Some of them came in through acquisition. So they, you know, they they made all these decisions before they were part of the company.

And you need to have the ability for all of these teams to kind of collaborate. So these sorts of tools that connect all of these different technologies together are really important, and they're so important that you end up having a team internally that is in charge of it. So they will have their own, like, custom plug ins that they'll write for it, and they'll have their own, like, framework for how you're supposed to deploy this thing and, best practices, all that kind of stuff, how to do it securely, all these things that you should you should be doing. And, you know, we use this deployment mechanism, and you do it with this and blah blah blah. Here's some templates.

They'll have their own internal documentation, all this stuff. And for them, they just wanna have this, like, mega binary that is gonna solve all the use cases that they maintain so they only have one build. Mhmm. And they they won't necessarily even know what connectors they're gonna need when they're when they're shipping this thing out. So it's almost like they are, they're they're almost like a a Benthos distribution that's just internal.

And I know a lot of organizations that do that. Like, when when they eventually get to that size where it becomes this massive problem to solve, it it becomes really important that if you're giving people this tool to solve it themselves, you're giving them, you know, everything, which is it does seem like a lot because the binary size is like a 100 meg. A 100 megabytes. In the Java world, that's not a big deal, but in in the Go world, that's massive.

Shay NehmadShay Nehmad

Yeah. Wow. With all the dependencies on, like, I assume the core engine is, like, you know, kinda tiny.

Ashley Jeffs

Yeah. Yeah. I think I think it is. I think it's pretty small.

Shay NehmadShay Nehmad

So you mentioned It

Ashley Jeffs

is all the client libraries. Specifically, AWS is pretty big.

Shay NehmadShay Nehmad

I saw you have, Azure. I remember I really hope you don't have to ever do as part of, you know, a client to deal whatever the, Microsoft Graph API as an input. Because let me tell you, man. They just took the entire c sharp codebase, and I don't know what they did do it. They, like, transpiled it into Go, and their Go package is, like, 2 and a half gigs if you add it to your build.

Something crazy. I remember it literally crashed our GitHub, pipelines last time we tried to, integrate it. And we ended up, like, not using the the SDK. We just, like, copied the files out of it. And they were, like, yeah. We're not really gonna maintain it, so sorry on GitHub. Speaking of stuff that's not gonna get maintained no. I'm just kidding. What happened? So so you're talking about Snowflake and Splunk being, like, 2 connectors.

It talks about growing a team. Are there any other exciting features on the horizon now that you have I assume

Ashley Jeffs

Oh, yeah.

Shay NehmadShay Nehmad

Internal Red Panda integrations. I don't know. Is the studio, like, getting a front, like, you know, it's gonna be front and center, the UI stuff? Like, what's going on? What's the what what are Bento's, fans, should look forward to now that you they don't have to worry about you, you know, ending up on the street?

Ashley Jeffs

There's, so console is another product we got. It's open source, Red Panda console. That is already able to give you a an editor with autocomplete for Pentos configs.

Shay NehmadShay Nehmad

Damn.

Ashley Jeffs

So they've been they've been working on that at the same time that, you know, we were working on the, repo split. There's also tooling around. We're probably gonna take Studio and just, like, break it up into bits because there's lots of problems that I solved with Studio that will probably fit in different places better than than just having one central thing. But, yeah, I mean, we're definitely gonna start moving, some of that good stuff out into better kind of, like, tools that we can give people. But, I mean, the the console, editing experience is is is like, people have been asking me for autocomplete for years, and I I basically just put my hands up and said, I'm I can't do it.

I'm just not capable. I I don't know what tools I would need to do. Like, I I built a visual editor before I could muster the front end magic to get autocomplete to work. And yet the, you know, the console team, they they had it pretty much immediately. Like, they they were like, yeah.

Give us the bits and tap tap tap. They did some stuff on a laptop, which looked sus, and then suddenly they've got autocomplete in console. So, yeah, I mean, there's there's obviously gonna be, like, integrations into Red Panda, but there's also gonna be stuff. Because, I mean, you don't need console with, you can just use that. But, obviously, most most of the functionality that we're gonna be talking about is is stuff that's gonna be cool with Red Panda.

But the stuff that the open source community is gonna be interested in specifically is just more support on the core engine. I mean, I wouldn't be surprised if at some point I'll be doing performance improvements, stuff like that that I just haven't had time for. Lots of more connectors. Pull requests are finally gonna be free to to get merged. The the pull requests will flow once more Mhmm.

And just more more hands on deck, more stuff, more bits. Is that specific enough for you? There's gonna be stuff and bits.

Shay NehmadShay Nehmad

I'm really excited when you said performance because the the the moment you say, okay. My thing is stable enough. And when I say my thing, I mean, your business and both, like, both your business and your project are stable enough that I can, like, I just wanna come to work one day and profile it and make it better. Yeah. That that means that it's actually providing so much value for so many people that humanity has, like, okay, little funny man, sit in your box and make it better.

Although, I don't know if that analogy is apt because I don't know if you're actually little. Like, all your I

Ashley Jeffs

think blog posts. I'm pretty small. I'm pretty small. You need to meet me at a summit. I think you'll tower over me.

Shay NehmadShay Nehmad

I'm a pretty tall guy. Yeah.

Ashley Jeffs

I like it that way. I don't like being in

Shay NehmadShay Nehmad

the middle. Tall Jonathan is, but I'm gonna find out in 2 weeks. Oh, come

Ashley Jeffs

to Amsterdam. Okay.

Shay NehmadShay Nehmad

Let's hang out. Well, actually I'm

Ashley Jeffs

actually not far.

Jonathan HallJonathan Hall

No. Not too far. Oh. What?

Shay NehmadShay Nehmad

Off the record, we're gonna we're gonna course you to to come. What's wrong?

Ashley Jeffs

What do you

Shay NehmadShay Nehmad

One last question about the, you know, Red Panda stuff. You, said you knew these guys, like, before, and they actually sponsored a project and and things like that. Like, where did you meet them? Just through Bentos, or, you know, did you go to high school with these guys?

Ashley Jeffs

No. I didn't go to school. The, the I mean, you you end up just kind of, like, mingling through, like, blogs and stuff. So I I remember when Red Panda was announced, publicly. It was it was when it was, like, fresh, and my immediate reaction was, I can finally run Kafka integration tests.

Because up until that point, it was like zookeeper, and it was an absolute nightmare to get it running in a in a, like, test framework without having, like, a a dedicated server for it, which I was not gonna do. And then Red Panda comes along, and it's like you could just run it. Like, you just run it quick, and it's it's low memory and all that stuff. So I immediately jumped on that as, oh my god. It's amazing.

And then I think, basically, they had some some users who were already using Benthos, and they obviously talked about that. And then Alex reached out to me, and it's the whole, you know, we'll do a blog together and post on our website, post on your website, and, you know, mix. And it's it's a good way of getting people to kind of discover new tools because up until that point, I I had basically no connections. That was about 5 years ago. I had no connections with anybody else in I said, that was probably 2020, I think, actually.

Not that long ago.

Shay NehmadShay Nehmad

I said,

Ashley Jeffs

oh, no. That's almost 5 years ago. Yeah. But back when I was a little and back when I was a little kiddo, yeah, I I didn't really have any, like, interaction with the the the bigger, more established tools because, you know, I was kind of like this organic developer driven. I didn't really have any, like, corporate presence.

Like, no nobody at Kafka Summit was talking about this blobfish project by this kid. So it was all it was all kind of, like, developer driven. But I think we we kind of, like, matched in our general marketing pitch because, you know, the whole pitch with Benfoss is it's just way simpler than the other tools, both to use and to run. And the the term that I was kind of using and still use to an extent today is the the operational simplicity. Is you've got this thing, you're running it, and it's it's simple in how it behaves.

Fewer hours on pages in the middle of the night, that sort of stuff. And they're obviously very, very similar because it's, you know, it's Kafka without the hassle. It's basically Red Panda. It's better performance and better behavior and, you know, easier to deploy and manage. So, yeah, we we just kind of started interacting based on that.

And, you know, it's it's the same with all these other, like, companies that, you know, you end up you end up interacting online, have a call just to say, like, hey. What's up? And then, you know, sometimes it's through VCs as well. Venture capitalists like to connect you with people. So that's that's like a whole, avenue for for getting contacts.

Shay NehmadShay Nehmad

I heard someone say that's the only thing they're good for.

Ashley Jeffs

No. Not me.

Jonathan HallJonathan Hall

No. No. No.

Ashley Jeffs

I wouldn't say that.

Jonathan HallJonathan Hall

I wanna ask one question that might be a little bit sensitive. The day after the announcement, warp stream announced that they had forked Pentos. I don't wanna go into too many details there that their announcement was, I don't know, a little bit adversarial. What are your thoughts? And I'll just leave it open like that.

Ashley Jeffs

So the like I said, that month was a bit wild. Obviously, I was hanging out with WarpStream. I still I I like all the WarpStream people, personally. They're in a difficult spot because so I think, essentially, the big the big thing that happened is people discovered that Kafka Connect is bad. It's like a it's like a product to sell.

So, like, lots of companies now are dropping support for it, and that used to be their sort of connector ecosystem. That was what they were selling to connect to other, tools. And it's difficult to justify how costly it is to maintain and run, for these businesses. So the the sort of Kafka ecosystem, you I used to have no presence there. And I didn't really have the resources to kind of, like, reach out because it's very it's very established, very corporate, lots of consultants kind of dictating and thought leaders who, you know, they're they're obviously very much tied to the JVM space.

So I kind of carved out my own space, and I wasn't really privy to any of that stuff. But in the last year or so, I think because there seems to be this sort of collapse in trust on Kafka Connect, which I think is a bit it's a bit over the top. Kafka Connect is is obviously fine for a lot of things. It's just difficult to to justify, for some of these companies. They're obviously looking at other tools, and, obviously, Benfils fits that bill really well.

And there's there's a lot of crossover. Obviously, there's lots of stuff that Calf Connect Connect does that Benfos doesn't, but there's there's enough overlap there that you can kind of prioritize it. So a lot of these companies were seeing it at the same time, and they're all they're all kind of, like, dabbling with it. So, WarpStream can you know, they're they're they're playing around with the idea of running Benfos configs in their product, and I I think it's scary because they're obviously, like, serious competitors. So they they obviously knew that, some sort of deal was happening, just because of all the signs.

It's pretty obvious. So I think they they kind of had it queued up ready to go. Because, I mean, they they announced that, like, the day after. Yeah. So the day after the the acquisition was public, they're on Hacker News talking about their fork. Mhmm. I'm not anti fork. We I've specifically structured the repos right now to enable people to have forks of the connectors.

Shay NehmadShay Nehmad

Mhmm.

Ashley Jeffs

And the beauty of it is there's an MIT licensed core engine. And if you fork all the connectors and put whatever license you want on it and support it however you want, it will compile with that engine, and other people can pull components from either. So they they could mix and match any vendor's connectors. That was always the plan from from the technical perspective, but what they've chosen to do is do, like, a monolithic fork of the whole thing. So they they have a monolithic repo right now, which is just everything in one thing, which isn't decentralized.

So, I mean, I'm not sure what their plan is. But right now, basically, my main gripe is that that's not good for the community. Because, obviously, if the whole point of this fork is is for the community anti Red Panda, then, you know, at least make it decentralized so people can share plug ins. Because right now, they can't share plug ins between the 2 forks. But that's there's no there's no technical reason why that couldn't be the case, and it would rely on MIT license codes.

So there's not really any reason for it other than just where.

Shay NehmadShay Nehmad

I think the we had a Ohad on the show from env0 and I think from sort of the other side of the of the fort. Right? They, Infzero and some other companies, were the people who forked Terraform into, OpenTofu. Now this is obviously not the same because, Terraform was monolithically open source and then monolithically source available, and this is a lot more open sourcey than that. Like, Red Panda Connect is a lot more open source than, Terraform is right now, for sure.

But I think the the reaction, makes sense on both sides, and it it I think it does hurt the community in the short term, these sorts of decisions. Even Open TOEFL and and Terraform, it's like, we had all these people working on this beautiful thing together, and now they're separated.

Ashley Jeffs

Yeah.

Shay NehmadShay Nehmad

But I think, like, with Open Tofu, we're seeing, you know, in a few we still need to let it cook for, like, a year or 2 until we finally see see the results. But overall, I think it sort of stresses out, the core developers, like the actual people, like you said, in the gnarly bits to make sure that their side of the fork is really good. Basically, competition 101. Yeah.

Ashley Jeffs

Competition.

Shay NehmadShay Nehmad

It's and in the long term, it's not like these two projects are gonna live side by side for 15 years now, and, you know, we're we're gonna invest a ton of time into both of them. It's gonna get lost. It's not it's not the case. Right? So I think I sort of agree with you that it's kinda mad just because I saw the, announcement.

I was happy for you. Then I read the blog post, like, oh, no. Is Ash a shill for has he been a shill for Red Panda for the last 10 years even though Red Panda's only been out for 6? Like, but I I overall, I really hope that this will end up being they do a fork when the Red Panda Connects project ends up being slightly better for it, you know, as an educational experience. And I hope, both companies end up succeeding in their own niche.

Ashley Jeffs

Yeah. Exactly. It's it's difficult because it's like you've got the competition side, The fragmenting of the sort of ecosystem around it was probably gonna happen anyway because I think that there was, like what I predicted was gonna happen was people who were just they they cannot tolerate, a company running a project, especially one that has to make money and is part of the capitalist system. So it's like they they have a a problem with that, and, you know, I I was expecting people to have kind of, like, community forks. And they're they're kind of there to keep the project honest.

Shay NehmadShay Nehmad

You know

Ashley Jeffs

what I mean? It's like it's not really that we're gonna diverge. We're not gonna do a bunch of changes that means we've completely diverged from your other project, but it's more just we're gonna keep this thing here, and it's gonna be known in the community as, like, you know, the the project x. It's just some extra thing that people are aware of. And the idea is that if Red Panda ever became bad stewards, then this is the thing that people could just flock to.

Whereas, because there's there's a big competition aspect between these two companies, I think it's it's it's a bit sad that they're kind of like they're the only dominating forces right now in the Forks space. I was kinda hoping it would be a community fork that made technical sense, rather than kind of like a panic fork. Like, it it just feel a little bit like it was like a panic maneuver, that's not necessarily well thought through. I mean, they've got load like, it it's obviously the first week. It's the 1st week of the acquisition, but it's also the 1st week of their fork.

So, I mean, they might they might change things so that there is more, shared code base, but I I don't know. I I don't really know what to to expect at this point.

Shay NehmadShay Nehmad

They might not knock you out of the park with features.

Ashley Jeffs

Well, yeah. Like, obviously, I can I can just keep slapping on features and pull requests and stuff, which is obviously the intention anyway? But the the thing I would personally be scared of, as the you know, this is kind of like my baby is is the idea of just willy nilly stripping out stuff from the engine in the name of, you know, benchmark numbers and just rolling out features from, like, a competitive standpoint and basically racing to the bottom of you you essentially eliminating guarantees. Because there's there's stuff in the Benthos engine that I know is critical for the 2 AM pages. So, like, the delivery guarantees that Bentos has is completely unnecessary for a product that you're putting out there and giving to people.

Because if you have weak delivery guarantees, well, you're just one of many products that people run that have weak delivery guarantees where the user won't realize. Because it's not an you know, it's a 1 in a 3 year event where you'll be hit by that stuff. And, obviously, the core engine of Bentos is specifically designed to be the one that doesn't have that. And, you know, this is a it's a technical decision. It's very, very difficult to get that balance right between, you know, back pressures, delivery guarantees, speed, scaling, all that kind of stuff.

So my worry is, you know, fork city happens. Everybody's got a fork. They're all stripping the engine out in weird, incompatible ways, and then we end up with this, like, race to the bottom because that that's what Bentos wasn't. Like, Bentos wasn't the tool that was kind of pressured to, sacrifice those things in the name of, you know, delivering all these things. I'm optimistic because I don't think the engine really needs to change that much.

It's quite stable now, but that does that does worry me. But, you know, I'll just leave and, you know, 10:10 years on, I'll just be doing something else when I say I don't know. Mhmm. I'm hoping to make myself, unnecessary in the project anyways.

Jonathan HallJonathan Hall

Yeah. Right.

Ashley Jeffs

Maybe it'll work out.

Shay NehmadShay Nehmad

So, Ash, you're actually, the first repeat interviewee that we have, that's gonna get both questions, stumper questions. Last year our stumper question was, what feature would you reap out from Go, and what, feature would you steal from another language? When we ended up with a full feature list on both columns, we were like, okay. We need to rephrase this. And I'm actually interested to know when you started learning Go, what was difficult or surprising for you?

Go was your wasn't your first language. Right?

Ashley Jeffs

No. So I was I was a c plus plus kid. Did a bit of c sharp, and then went back to c plus plus, and then Go. And that was pretty much it from that point onwards. It was about I think it was about 2016. No. It's actually must have been early. No. It was probably 2014, Maybe 2013 I started using Go. You can look at because it so that library I've got, Tunny, that's pretty much I made that immediately when I started learning Go.

Because the one of the first things I noticed was, hang on a minute. How is there no thread pool? Like, how is there no goroutine pool here? Because I was just expecting that in the standard library, and I was very surprised, so I made one. And it wasn't until I made it that I realized, oh, actually, this is very unnecessary because, you know, with with channels and coroutines, it's not it's it's not difficult to reimplement that when you need it.

And you normally, you know, you normally want to reimplement it.

Shay NehmadShay Nehmad

1st commit, March 23, 2015, but the copyright is 2014.

Ashley Jeffs

That's probably I probably,

Shay NehmadShay Nehmad

Copied it

Ashley Jeffs

over from somewhere else. Rebased it or something? Yeah. Yeah. 2014 is probably about maybe the end of 2014. Let's find this out. We need this right now.

Jonathan HallJonathan Hall

This is not this

Shay NehmadShay Nehmad

is not that good of a code, I have to say, like, first, first commit from 2015. So, when you came in to go, what was, like, difficult for you to to grok and and wrap your head around? Or maybe what was surprising for you?

Ashley Jeffs

The thing I didn't like was, lack of some slash enums slash union types, whatever people want. Like, depending on your language is what you'll know it has. But it was it was that because, I wanted to do JSON parsing, and I found it really weird that it it has to pull things out into, you know, generic maps and generic arrays rather than, like, a some type where you can make it more explicit because that's just what I was used to. And then I found it really weird that, like, if you've got a generic map, why is there no syntax for just walking it? That blew my mind because it's like, this seems like everybody's gonna hit this.

Right? Everybody's gonna have a JSON API. Everyone's gonna have, like, a poorly defined structure because that that was my world. I I just assumed everybody had that problem. So I was I was shocked and devastated when I found out there's no, like, there's nothing there to kind of help you with that. But, obviously, like I discovered, you could just, you know, you can just slap a library together. It's not it's not a big deal. You can just do that yourself. Just get get it. Pull yourself up by

Shay NehmadShay Nehmad

the bootstraps.

Ashley Jeffs

Yeah. Make yourself a library for it.

Shay NehmadShay Nehmad

Write another for

Ashley Jeffs

it. Generics.

Jonathan HallJonathan Hall

I think a lot of that's finally coming in 1.23.

Shay NehmadShay Nehmad

Yeah. If you're, the new ASH joining, you know, to go right now 10 years later, you'll get the Range Rover funk and all this, generics nonsense, and you can build

Ashley Jeffs

it up. Nonsense.

Shay NehmadShay Nehmad

I don't know. Russ Russ needs to keep himself busy somehow. You know? You need another connector on top of your 220 connectors, and Russ needs a seek 3 iteration.

Ashley Jeffs

I'm moving on to Zig.

Jonathan HallJonathan Hall

There we go. That'll solve all the problems. Yep.

Shay NehmadShay Nehmad

Ash, thanks all for coming, man. Again, Thanks

Jonathan HallJonathan Hall

for taking our call.

Shay NehmadShay Nehmad

Yeah. For taking our call.

Ashley Jeffs

Never again.

Shay NehmadShay Nehmad

Not on this number.

Jonathan HallJonathan Hall

I'll I'll burn this on after we hang up.

Ashley Jeffs

I've only had this SIM for a week.

Shay NehmadShay Nehmad

Again, I really wanna congratulate you on this, you know, we can dive into the technical details and and what happened, what's gonna happen, whatever. But I think a congratulation is in order. You mentioned somewhere doing this interview, an end of a journey, and I really, really feel, this marks a big, stone in your journey. I don't know if it's the end of it, but it's a really, really cool one. And I think it's a really big success story for open source and people who work on open source and, you know, have some sponsors and and are thinking about long term what they can do, can really look into your just tireless work on this well, I don't know if tireless, but very

Ashley Jeffs

I was very lazy at times.

Shay NehmadShay Nehmad

Yeah. Yeah. But I mean, like, relentless, dude. Working on the same project with the same, tone for many, many years, and having just this recognition. By the way, both the fork people and red red panda people and just random people on Hacker News, every nobody's like, yeah, but maybe the engine's not good. Everyone no. Asher's code is pretty great. Right?

Ashley Jeffs

And the logo's good. Everybody unanimously loves the logo.

Shay NehmadShay Nehmad

Yeah. For sure.

Ashley Jeffs

More so than the engine.

Shay NehmadShay Nehmad

Well, it's less gnarly. You know what I mean? It's cuter. So we just really wanna congratulate you, man.

Ashley Jeffs

Well, thank you for having me on. It's been great.

Shay NehmadShay Nehmad

Do it again next week

Jonathan HallJonathan Hall

to solve a business.

Shay NehmadShay Nehmad

Yeah. Next next question.

Ashley Jeffs

No. Thank you. I think I'm done.

Shay NehmadShay Nehmad

Alright. Thanks a lot, Ash.

Transcript source: Provided by creator in RSS feed: download file
For the best experience, listen in Metacast app for iOS or Android
Open in Metacast