Well, welcome to another episode of the Ruby Rovers podcast. I'm your host today, Valentino Stole, and we're joined by a very special guest, Scott Werner. Scott, do you want to introduce yourself? Tell us why you're Ruby famous? Here? Ruby famous? I wish someday. Thanks. Thanks for having me, Valentino. It's super exciting to be here and be a
part of this. Yeah, so I'm here. I'm the CEO and founder and co founder of sub Layer, where we're building a self assembling AI agent framework in Ruby and kind of supporting tools and infrastructure for uh to make something like that possible. And yeah, basically been programming with AI and Ruby for and we were just talking earlier basically exclusively trying to get AI to write all my code for me and me actually write the code and primarily Ruby. That's
awesome. So just so word clear, you're talking about sub Layer is a project. Sorry, Yes, the sub layer is the GM. Sub Layer's the GYM, and so yeah, it's uh, you know it said. You know, I've been describing it as a self assembling AI agent framework, and kind of what I mean by that is that each of the each of the components that we've built are designed so that it's easy for an l l
M to generate new additional components that you might need. So the the the framework is very minimal, and it's really about kind of defining the the conventions, the interfaces and then kind of letting you lose to to expand on that for whatever you want your application. I'm super curious here, like, uh, where do you where do you start something like that? Like how did you get involved than this? Right? Like, uh, how do you
just stop coding? Uh? It was tough, you know, I think I think a lot of us were really inspired when we saw chat GPT for Katt. We saw GPT four come out and people using chat GPT and you know, posting videos of I use this pro throw this prompt into chat GPT. I copy and pasted the code into x code and I have a running app and I've never coded before, and uh, you know I saw that and immediately it was like, oh, well, those people are going to
want a way to add functionality and modify that software over time. And uh kind of realized that a lot of the stuff that we do in XP around like TDD and and the different refactory as we do lend themselves really well to working with LM. So breaking things out into small pieces, the smaller requests, the simpler request you have for the LLLM that you the more likely get
you something back. That's that's good, you know. And also kind of had this, I had this conversation with people like, you know, you would be mad at me if I gave you a thousand nine PR, but at least you could come over to my desk and yell at me. Right if you've got a thousand nine PR from chatchypt that would be a big problem. And so kind of with that line of thinking is where we kind of
started our journey. That's really funny. And I just think of the costs associated with that too, for something that you don't want right right, that's really cool. So I've definitely checked out cub lair. I've used it to generate specs because I don't like write writing specs. I'm not a test driven person, but I do drive tests, so I'm more of a backwards test
driven developer whatever you would call that, and so I use it. Yeah, it was a little script that I run sub layer and I say, hey, generator, you know our spec test for this Ruby file and for the most part it does pretty good. You know. Yeah, so I love the kind of syntax for just like, hey, here's a generator uses this prompt to do it. It's super cool. Ah. Thanks, So why don't you give us like the high level like how does this all work?
Right? Yeah, a little you know a little bit of background like how and why it works is uh, you know, we started calling it promptable architecture, but the core idea was that they were going to be patterns, styles, ways that you build your application that lend itself well to l
o ms taking a short request and giving you something back. And you know, some of the things we started out with things we kind of already know of, you know, clear interfaces, single responsibility principle, uh, loose coupling, and kind of came about those through you know, reasoning of you know, it's going to be much easier to add code to your app if you can just drop that new file in rather than have to update a whole bunch of other different files, and you know, other things like having these
conventions that you can throw into the prompt and the LLLM will just believe you that that's your convention and follow it. And so that's that's been a lot of kind of our trial and error figuring out what are these what are these
patterns? How do we how are we going to structure our applications differently to take advantage of this, And so, you know, one of the main things around the sublayer gem is, you know, every single component is very loosely coupled that you know, you could have one hundred generators, you could have a thousand generator and they won't affect each other, and so you can you know, one of the one of the things that we've been playing with
is like maybe you know, don't repeat yourself or duplication in your code base. Isn't that bad if it takes you know, in the dots you can see that you can generate a generator in like, you know, ten seconds, Maybe it's not that bad if you know you and I both work on the same story. If you know, it's us just describing it and getting
the code and it doesn't affect anything else in the code base. So so there's there's promptable architecture and so like that's been kind of our drive for this GEM, and one of the things that we've found works really well with that is that you really only need to give the LM one example to generate something new and so generate a variation off of that, which is where our kind of main tool blueprints comes in and basically what it does, which I think
is you know, has been pretty common experience from a lot of the developers
that I've talked to. Nine times out of ten, when you're working on a new user story, there's somewhere else in the code base that you do something similar, or there's a pattern you probably want to reuse, so you go, you look at that file, you repurpose some of the code, maybe copy and paste, and you know, keep the code based very similar to what you're doing, which we kind of realized is a manual, human version of RAG, which you know, I made a joke to somebody that
like, my development style is retrieval, augmented generation, very good at googling and looking around the code base, and you know, it was kind of mostly joking, but then kind of took a step back and was like, actually, maybe there's something to that that's really funny. I never thought of it like that, but yeah, we are just we're just doing RAG.
Right and so real, so you can actually kind of operationalize that and the you know, one of the things that I think for you join a new team and what takes so long to ramp up is that experience of, oh, yeah, when we when we run into that problem, we do this pattern over here, that's in you know, this file on you know this line. But like you only get that from months years of working on that
team. That intuition whereas you index these things, you put them in a vector database and you query them, you can find them and uh, you know, I we built that. We originally actually on the sublayer GEM thought we were going to build a whole bunch of different you know, components and
generators and stuff. But uh, on our product planning meeting one Monday, we'd like, you know, we basically thought we were going to do about this much work this week and had the idea of maybe we'll just generate one
of these generators. And then a whole work weeks worth of work was done in a day, and it was like, take a step back, what did we just kind of like discover so like you know, building, we've found like building in that method building this method of like very loosely coupled components. Small components can be very powerful and quick. I didn't realize that you have a list of of pre defined blueprints on the site. That's really cool.
Yeah, that's super cool. I was gonna say that, you know that this mentality is makes so much sense, right, Like, I feel like I've been at places before where they have had just like generators in general, like reils generators right where Okay, you just like want to start up a new rails app, like you use a generator, Like you want to create a new model that has all of the conventions, use the rails generator
for that. Like it definitely has helped, like in a consultancy for example, Like you know where you have all this boilerplate stuff that you just want to build off of. It's like, well, you use the pre existing stuff and then at least you have the shell there and it's all hooked up and wired correctly, and then you can build off of that. But this is like the next step after that, right, like because then you can then start filling out the stuff in that boiler plate right to adapt to whatever
it may be, which is really cool. And so like how does it how well does it work? Right? Like, I think that's like maybe what a lot of people hesitate to jump into all this with, right, Like you know, okay, so you have all this source material for it
to use, Like, isn't it just making mistakes? Though? So it can Actually one of the things that I've kind of realized and been trying to figure out how to how to describe, right, But it's almost like by intentionally designing your classes and interfaces a certain way, you can almost like target
the hallucinations to where you want them. Which you know it makes mistakes, but they a lot of times are mistakes in the style that you would probably want anyway, which I know it doesn't really make sense, but like so in the actually I think, what do you what do you mean by that?
So like, well, you know, one of the things, if you you know, go ahead and generate something on the doc site, you'll see that we have kind of this idea of an l M output adapter and right now it says, you know, there's a symbol for a single string. So we're basically the simplest generator you can make is here's a string.
I'm gonna send a prompt to an l M and I want this specific string back, and like you said, like your your example is the string could be our spectats for you know, some Ruby code that I've got, But if you ask it to build a generator that maybe gives you a list of a list of refactors to make in that Ruby file, it might hallucinate that it needs an L and output generator that generates a list And while that might not exist, it'll still be in the conventions and the style of that of
the library, which might exist but also might point to Okay, well, here's the next feature we need to build. We need to flesh out what it means to get a list back. And so having those spots where there's nothing different from the base example to the new example you're trying to make, but the place is where it's different, you can basically design your code so that those are the pieces that change. Output adapter might be a little more
complex. The thing that I think you run into a lot is it just generates a new prompt for what you're trying to do and doesn't touch anything else in the rest of the code. And so I'm thinking about it like I'm targeting the hallucination to that prompt or to the to the initializer rather than anything else in the in the code base or in the in the file. Gotcha, Yeah, that makes sense. I have so many questions around the blueprints aspect of it, and I mean I guess, like where does where does
somebody even get started? Like is blueprints a good place to get started with the general code generation aspects of it? Or is that like a first test the waters with sub layer and get used to the generator's aspect of it? Like where do you suggest somebody? Like that's hey, I've got a team of people, we have a conventions in place we want to start adopting this
process. Like where do you start? So, yeah, I guess one place you can start We just you saw on the site we just rolled out some new functionality where you can sign up you can start to put some code examples in privately and start to generate from them to see how it how it acts, and that that could be a great place to kind of start to start to play with UH promptability, which we've got a whole bunch of features we're going to roll out, kind of being able to score or point out
areas where you know where it might make the code more promptable, and so so that's one aspect if you if you're curious about playing with with your own code. And then on the sub layer docks, we actually and I'm sorry this is very meta, but we've actually created this this interactive docks product which we're going to be rolling out as well, where live in our docks, where you can come and go to any of the components we have generators and
actions right now and see it in action. Just write in what you're trying to do and it will give it give you reasonable There might be some things depending on what you're trying to do, might not be implemented yet. But one of the things that we're doing, if you go to the running there's a there's a guide on the left about running local, running a local l M with Lamaphile, And what Andrew on our team does is actually in the demo of setting it up, go to the dock site, generate the code
for the demo, runs it on the video. So I'm sorry we're doing I was on earlier because I had looked at blueprints a while ago and I hadn't seen some player's latest stuff and this is so cool. Yeah, you just go on to the site and you can like, uh, you have just generate this stuff. It's so cool. Sorry if we're inflating your bill
here, but that's maybe why it's there. Yeah, so I'm curious, like, uh, you know, definitely local inference is uh, you know, a hot topic and people are trying to you know, it's getting faster and MacBooks are running it quicker. Is that something that you recommend people start with? Though? Like, is it fast enough to like make make like
the development flow smooth. It's it's not yet. I would probably recommend starting with something like, uh, you know, I think for for some some of the stuff that we're doing, you know, five dollars in uh in open Ai API credits can kind of help you get started. I think Google's has a free tier or even Gemini one point five flash, which is super
fast. You know, we're we have we're using you know, some of the some of the models on our site, and you know you can also just go and play with way play with stuff in with private blueprints for the local models. I I kind of do think that's where it's where it's probably going for for at least a lot of stuff. The problem I think the The prohibitive thing about that is that you need a very powerful laptop to really be able to run the local models fast enough and powerful enough to actually get
good results back. Yeah, four thousand and five thousand dollars a book pro or five dollars into open API or opening API credits. I remember I first saw the you know, the first Open models, and I go and I, you know, install on my Raspberry Pie is I'm thinking, oh, yeah, you know, like I'll just throw everything a chain of Rasberry pies. And it turns out you can't really like spread imprints like that. It's
not it's not really there yet. And it's also super slow on and anything like even on like a nice m one, right, So it's kind of funny like you sit there and you're like watching reminds me of the old days of like waiting for an image to load on dial up or something, right, like you're waiting for the sentence to complete. Yeah, so I think it also, you know, the risky run is that it gives you the
wrong impression, like if you're if you're just getting started. It's definitely good to throw five dollars at some of the like frontier models like cloud three point five GPT four to oh or or Gemini, just to see what's possible, because you can get the wrong impression if you're like like this one of the thing just like went off the rails and started talking to talking about something unrelated, right, right, Yeah, So I'm interested, like, have you
found like certain models performed better for like certain tasks like related to code generation, or are the ones that are specifically better at generating Ruby code or like genering certain kinds of Ruby code, or even just like are there certain tasks with code generation like generating the specs for X or something like that where some models just do a better job than others, or is it kind of like
just use any one of the top ones. I my opinion and my approach is kind of really just used the top ones because they're really those are going to be the slowest, most expensive, worst models we're ever going to have. Again, you know, it's just getting better and it's just scaling up, so like getting you know, getting used to what's what's possible, and you know, even if you run to the wrong run into an issue, like it's more about the UX and like what is this, you know,
what does this imply? If you can do this. I was telling talking to somebody about you know, remember remember when everybody's like, oh ai, image generation is never gonna work because it keeps getting people seven fingers, and like, I can't remember the last time that was, you know, part of the conversation, but it was all about, like, here's a great way to generate an image. Ignore the hands for it for now. That
reminds me. I saw recently, Uh, some some university has the the third thumb or you're gonna actually like you use your feet to like you like to get like another thumb like prosthetic on your hand so that you can like, you know, hold stuff while you're like doing something else with one hand. They showed peeling a banana with one hand. You know that guy, That guy's clearly aerated. Oh my gosh, it's so funny. But I
guess it's more to your point. And actually one of the things we've been trying to figure out the right way to present this and maybe it'll come out
as something interactive on the blueprint site. But we've found that different attributes are different, like uh, different different attributes of code quality that we've talked about for a long time are actually kind of quantifiable now where one of the experiments we were running was you know, having having really descriptive good names and then generating off of that blueprint that you know, GPT three point five did really well. When you made those names worse or less descriptive, it had a
harder time figuring it out, but GPT four could. And so like you know, that kind of goes the follows a lot of the scaling for the other models as well, where if you give it an easier problem to solve, it can solve it. But as soon as that problem gets harder, you need a more powerful model, which I've been testing out with some people, like we might be able to put a dollar value on technical debt because it costs you a dollar per thousand tokens versus you know, ten cents per
thousand tokens. That's really funny now, sourcing the jobs we don't want to do to AI, it's all a cost per token. At this point, it's always been. That's that's an interesting idea. Like, you know, definitely refactoring makes a lot of sense. I have a special GPT I made like that, you know, it has a bunch of resources that I've collected over the years at its disposal to like help me refractor stuff specific to Ruby,
and it works great. And having that extra knowledge base like is awesome and I'd use it all the time just for like rubber duck sessions, right, which I know the next step would be to then it actually changed the
stuff, which I haven't spent the time to do. But like now that you know you've made it so easy with sub layer, I'm definitely going to revisit it because that's definitely a task where like you could just say, hey, like generate a pull request for me, like you know, and all that get up APIs are there, like it could use it as a disposal Like have you played like with tool usage in that way yet As far as like the sub layer connection, uh yeah, And actually I guess even to
that that task is is kind of the where we see things going that. Yeah, we we're kind of looking at it, like you know, the
those kind of automations. Once you have that LM and you have you know, an easy way to take get some data, send it to an l LM in a specific way and then get something structured back, you can build things with that and you can build things with that very quickly, like that, you know, take this, take these refactoring or these you know, these suggestions, make a pull request or or you know, kind of anything
you can think of. We've been we've been toying with UH with one idea where it kind of sits in our UH sits in our video chat for our planning meeting in the beginning of the week that UH watches the video, check manages the transcript, analyze the transcript, and then anytime, you know, you know, when somebody asked a question about a story and then you get an answer, and then you forget to put it in the story, and then like we you know, a few days later, we decide, here,
you can just build an automation to kind of pull that out and put that in the story for you. And like that's a super quick task that doesn't take that much work anymore, right, And and so very long winded way of saying, we've explored the tool usage, but I think we're looking
at it a little differently then how the providers are offering it. You know, we we'd absolutely rely on function calling, but only for UH, only for like the structured output aspects of it, and relying on you know, things that we're good at and things that we can do deterministically and reliably, which is you know, call APIs and uh yeah, pull make make requests and make make get requests and get make books requests and it files locally that
kind of thing. But it matters to like get the parameters that you're going to send to those APIs and then we can do that deterministically. But like have the have the l ll M do what it's good at, and have us do what we're good at. Yeah, that makes a lot of sense.
Uh do you see it more of like moving in the way that kind of GitHub is pushing with like configuration steps or is it like I'm curious if you've played with like the opposite of like the object oriented aspect of like okay, here's a prompt and you know it can have all these extra functions and stuff wrapped into the code aspect versus like you know, the configuration outline where right where it seems get hubs going that way to fit better into their CI
structure to be honest, right where you just have this YAML file and then it has prompts and can perform actions and stuff in it in that way, like where is that flexibility more advantageous in like putting it in your code versus like in something like a configuration file. That's a good question, you know.
I we were definitely coming at it from a different perspective. You know, I think a lot of the multifunction calling the giftub workspace are very kind of conversation driven, and so you have those functions that are structured ways for it to get in more information to do the next step in the conversation, which can be powerful, can be very powerful, and I mean we've seen
a great bunch of examples of the power that is possible. But I think from maybe this is just you know, the thinking, wishful thinking about software engine the future of software engineering, that you know, a lot of the principles that we've found wild building, while not exactly right when working with an
l M, still matter. And so you know, one one thing that I'm nervous about and haven't why I haven't really gone the route of like multiple different functions for it to choose from, is uh kind of the same reason why you know, it's it's bad to have multiple it's bad to have a dozen parameters in your function. Right, there was a there's a quote that's something like if you have if you're if your function has ten parameters, you
probably forgot one that. Like a lot of those things while folks see and kind of intuition and not like backed by any science. Really, I don't know, still feel right to me. I still feel like you want to know that, like this is going to happen, This is going to happen, and this is going to happen, even if there's some you know, fuzzy non deterministic thing happening here. Yeah, I agree with that. Uh
huh. I'm personally against the step by steps with with LLLM usage in general, mostly because like you kind of want observability into that and and kind of like a way to intercept those steps. Like as an example, I see like the usage over time starts to become more like reactive than it does like
input output. So like especially once you start using agents more or things that are like can communicate with each other, it becomes like an observability thing, right, Like you end up having things that like are watching you know what is happening and can help redirect right if things start going wrong or interrupt, and when you have that step by step process, it starts to make it harder to change that, right because it's it's got the code flow and like
okay, well you tell it to go to a different point in the code flow. Like then it's like instruction jumping, Like what are we back to
see ag in? Right? Like assembly, Like I use Ruby so that I don't have to know assembly, right, like I think like Aaron Patterson is known notoriously known for his joke like you know, I do assembly so you don't have to write like uh, and so I see you know, the the prompt usage kind of going in that direction too, right, Like people write natural language and it does stuff for them, right, Like that's
the whole driving force of all of this. It's like, you know, we had like a product manager come in and make some changes to get better outputs, right, and like then our end users see the result of that and like very specific you know tweaks like general people can make, right, Like I work at doc Simony and it's like a social network for physicians and they go on the chat shept and they use chat shept to like solve problems
for themselves, right, Like they'll go and they'll you know, generate like a denial appeal letter for some insurance provider, and they just like then they use it, right, and so they use natural language to make that happen, and like obviously like they have some issues like that they have. There's a lot of handholding there, right, and and so like the whole point
is like to help drive people to use the natural language. And then s as developers are like to introduce like the advances of like extending it, right, like okay, chat, chap, all these models like yeah, they give you text Like, well, because of that, you can have to generate any number of things because we're coders, right, Like we can start pipelining and obfuscating that you know, input output that the alms give us to mutate it in different ways, right, And the users don't have to see
that, right, but they could still give it the same inputs that us as developers would even give it. Right. That's like that's where I see, like, all right, we got to get away from this, like all right, step by step instruction Like that's a programmer like mentality, right, Like we're starting the design evolution is going to be really interesting to watch for developers. I think I'm curious. I'm curious, like what what your thoughts are on that aspect, Like where do you see like like the design
driving force of a lot of this stuff moving? You know. So my previous startup was actually a a prototyping tool for voice apps, and so this was like Rata's Google, Google Home and Alexa were coming out. We built a tool for uh, you know, prototyping out different conversation flows and that
same thing, a step by step back and forth kind of conversation. And you know, one of the things that we kind of realized is that conversation is kind of just like one one modality, and we got acquired by Adobe built that into XD because we, you know, firmly believe that you know,
you you want multiple modalities for inputting information to a machine. Sometimes it's a voice because you're in a you know, your your hands are busy and you can't actually press a button, or you know, you're cooking or something. But other times you're on the subway and like asking asking your phone for your bank account information, it's probably not a great idea, but like pressing the buttons on your phone's probably fine. And so all that being said,
I think I think where things are going. I don't think we're going to throw out like the last However, many decades of UX research that we have into like interfacing with machines, that there are going to be a lot of times where you have buttons and forms too, you know that people can understand and intuitively grasp right like can you can you imagine going to Facebook and having to like write a sequel career or asque like the Facebook bot to add somebody
as a friend versus you know, just clicking a friend button. So I think it's going to be I think it's going to be a mix, even if behind the scenes there's going to be you know, this non deterministic natural language API that that way that we've been using, but like from a user standpoint, not requiring them to understand the right way to prompt that makes sense. Yeah, that's interesting. I mean prompt I hate to say prompt engineering, but trying to get get the right thing to do what you want is
still frustrating to me. It is, like it's so it reminds me of like when I first started to learn how to code, to be honest, which is a little funny. Likes so much fun to bring that excitement back, right, it really is. Yeah, And uh, I don't know, are there other tools that you use for that, Like, you know, prompt is aspect that you'd recommend to people. I radical, I definitely kind of out out there on the prompt design stuff. I you know,
you we've got an open source server for blueprints. We've got sub layers open source so you can see the prompts that we're using. We're very minimal most of the prompt design for us, which it's maybe it's prompt designed, but it's all about like what code are we going to give it and like making the problem easy. And so we haven't really used many tools because the there's
maybe ten lines of prompt that we rely on. But I think I think thinking about it like programmers is helpful that you know, if you have you know, for you know, we're talk a little bit about this of just like what what makes code promptable that if you have to say, like I need this input, this input, this input, then do this other thing and then you know, do that, it's probably bad code. It's also
probably not very promptible code. And like taking that further, if you're asking the LLM to do this and this and this and that, it's probably going to screw up somewhere, and so like break those apart the same way that you you know, you would much rather not have a thousand line method. You break it out into individual pieces that you can individually test are individually reliable,
and then piece those together. Yeah, that's kind of my approach to I do use like sometimes we'll use like, you know, the models against each other to build and iterate on, which is helpful, like how would you improve this? And then go to the other one and say, well, what do you think about that? Like how would you improve this from?
It's like they often will, like you know it do better if you use the like a competing like model against uh the other, which is a little funny, and especially if you tell it like oh, like you're in a competition to like make the best prompt, right, Yeah, I mean I haven't tried to like take the deep dive to be like this is a matter of life or death, right like, uh, and you can also win a million dollars, right, like just shove all of the you know,
prompt tricks, which is kind of funny, like those are going away, like it's getting harder and harder to make those improvements work. But yeah, I don't know if they're they're doing that training, but but yeah a little funny. Yeah, yeah, that brings me to my next point, like, uh, you know, blueprint seems like it's like ripe for fine tuning, right, Like you have these pre existing conditions and pre existing formatting. Uh, you know, it seems like this would make a great flow
to fine tune the models to give the outputs in more desirable formats. Like do you even see that as like being a meaningful improvement or is it are the better like just you pay to use better models of this one? I mean, the my my thinking right now is the is you know, until we until we get to the you know, the end of this curve of
you know, the costs dropping. I think they drop by the quarter every eighteen months or something like that, where you know, it keeps getting cheaper and new models come out, which I think until until that stops or slows
down to really just just keep going. And you know, I wrote a post a little while ago titled waste Inferences, which basically kind of that call or just like you know, we can actually build very simple applications and do very simple things even if yeah, it costs a little bit more to you know, use GPT four oh or Cloud three point five today, but six months from now it's going to cost half as much, and then six months
from that it's going to cost half as much as that. And so like if you if you over optimize today, that's time you could be spent. Time you could be spending finding new patterns for when when these things cost nothing. I don't if you remember. I mean I remember early on you used to get charged. Some people still get charged for bandwidth, but like the like entry tier for any kind of web hosting was also a bandwidth charge, and like if somebody linked to an image on your site could go bankrupt.
But like I haven't seen anybody you know, really talk about three. But like the costs are so minuscule now that it's not even like called out funny you mentioned that. I remember being on a call there was like some kind of like a I don't know, like presentation or something that a lot of
the Open Eye engineers that were giving to a bunch of companies. I was like enough to join in, and you know a lot of people were asking like, you know, it is fine tuning like worth it for these models, like and they're you know, how do you get even like your pre existing machine learning teams on board to start using this stuff right, And what they were basically saying bluntly was like, you know, the models are getting so good that fine tuning is really not going to be worth it over the
long run in general, and that like basically it would be better for your like machine learning team even to get used to using the large language models then it would be to have them like you know, do like machine learning to like for a recommendation system or something like that, like over time, Like you would be more advantageous to like basically get your foot in the door to the models and work through prompting than it would be to like get trained data
on some kind of like you know, PyTorch or something like that for whatever you're trying to do, like because over the long run, you still have to maintain all that, like you have to worry about the same things like is it being effective doing its job right and then also improving those aspects of
it, whereas the models themselves, like they already are improving themselves. And you get and you can make it, like you are, right, like where you can get it to improve itself like iteratively, and so it's interesting to see, you know, it's I'm kind of torn on it though, because I do I have worked with some fine tuning aspects to like get it to conform to very specific outputs as an example, it and it does like perform much better for those very like the you know, Obie's book is great
like describing the narrow Path, which is like the perfect example of like how like all of this works really right, It's like the smaller and narrower the scope you can make the task, like, the better that it performs, and like, you know, the problem with it now is like GBT four is like such a vast like knowledge base, right that like it starts to perform poorly the greater the tasks that you ask of it, or the more
that you ask of it. So sure it can like start to distill and do a pretty good great job most of the time, but like the more you start to like ask for it, like as an example, like you
know, what which of these categories does this content fall into? Right, It's gonna make a lot of mistakes because you're asking it to like match up too many things at once, right, But if you're just like, well is it this category yes or no, it's like gonna get almost one hundred percent of the time, right, And so like it's more think thinking like those aspects of it, which is kind of funny because like the pricing model is like perfect for them, right, Like the more that you use it,
uh, you know, the more money they make. But also like you know, the the more that you need to use it, the times wise, right, starts to increase just by the nature of the design, which is kind of funny. I just realize I put a disclaimer on that
waste not being ab I that's funny. But I guess the one thing I would say on that, and like even to to your question, and you know, maybe started could have started the I said this a lot earlier that like a lot of these these these things are so new, right that like a lot of what we're doing is a lot of what we're doing. There's a lot of you know, papers coming out around a lot of these techniques. But I think everybody's just really you know, they have the theories of
why these things work. They can get it to work, and that might work for them, it might work for this. There might be pieces of what people are doing that are right or wrong. I don't think anybody knows for sure, and like especially how uncertain the future is that I think. I think it's you know, these opinions are mostly mine from the way that
we're approaching it, and I could be completely wrong. I've seen successes and I have seen like evidence that we're right about some things, but that doesn't mean that there are a whole bunch of other techniques and patterns and ways to
do it. And so like that's the's you know, at this also kind of terrifying that, like, you know, but it's also super exciting, right, Like it's so fun to see, you know, to like expand the way that you're thinking about it when you see somebody try something that you wouldn't have thought to try or tried and failed and then see them succeed with it, and you know, expand your understanding of these things. So if something you know, for anybody listening, like if something interests you and like
you, it's super easy to try out. So like don't let don't let somebody say saying like this doesn't work or this does work stop you from just trying it, because it's usually like a two cent API call to find out if your idea there's something there. Yeah, for sure. Now it's never been a greater time to experiment because you can just prove people wrong so easily or or prove yourself right. You know, like it's it's very easy to
test that out, which is honestly great. With all the new tools that are out there, you know, perfect time to plug the Ruby AI builders discord. You know, people just like drop in the wildest stuff and you're like, oh, I could do that, like, you know, definitely come join us. Yeah, it was a couple of weeks ago and somebody found that it could you could send it Base sixty four and coded information and it would be fun. And then so that was like a really fun thing
on a Friday night night. How does how is it able to do this? That's awesome? Yeah, I wonder if you can like squirt around some of the token issues compression. I don't know, I wonder if it makes yeah, it any slower at responding. I haven't noticed. So I wanted to briefly touch on you know, for those that don't know you've started hosting a what what seems to be a repeat event in New York City, Uh, the Ruby AI meet up? How did you get started with that?
Like? What what prompted it? Like, you know, I attended the last one. I thought it was really really great to meet new Rubias like in the AI space, but even non Rubius were there, which was pretty cool to see. Yeah, how did you get involved in that? I've gotta you know, I've got a backstory of course, so you know, I one of the we got asked a lot, you know, as we
were building this startup why Ruby? And I am as you can maybe if you can see some of the books here, a little bit of a software historian and really love looking and digging deep into like the reason for, uh, why we do what we do and kind of realized that, you know, every time there's a new platform shift or new kind of thing we can do with computers or as things change, there's like there's kind of a pendulum swing, and you you know, you kind of see like object oriented programming
and the guy kind of coming out of small talk and then you know that's more of the informal kind of like fuzzy reasoning kind of approach, which then you know, you got it got more formalized through like the eighties and nineties when with like C plus plus and Java, and then you know, database back web applications became a thing, and we found that the dynamic language is
Ruby, Python. JavaScript made it possible to test out a whole bunch of ideas and find the pattern and find what we how what it means to have like these dynamic web applications. And then through the twenty tens or so, it's become more formalized with Typescript and Rust, and we're not really trying out all these different variations we've got. We know what we want to build and we want that to be stable and scalable. But now with all lems,
nobody knows right. And so my thinking was, you know, I need a dynamic language where it's going to get out of the way and I can test out and interface a DSL and idea super quickly. And started thinking back to like when I was getting into Ruby, how there's just like this explosion of different gems, people trying things out, Like why the Lucky Stiff was putting things out like rapid fire, and you know, something like camping comes
about which inspires Sinatra. I don't know if this is direct, but in my mind this is this is my head canon that camping inspires Sinatra, which inspires Flask and express, which is like kind of what a lot of things are built on now and we kind of are in that period now where like we want to get bring people together and kind of like what I said before, just like I don't know, you know, maybe I'm right about some things, I'm wrong about some things, but like we need to get together
and share those ideas, bounce those ideas off each other, get into arguments, and like sho show each other cool stuff that were building. And so that's where the Happy Hour came about. It was like I feel like, I feel like Ruby is right for this. I feel like there is uh, there is you know, from talking to people one on one, there
is definitely that feeling of like where did that magic go? So I was like, ah, let's see if we can let's see if we can bring some rubies together in New York and uh, you know, test that test that theory out and so we did and it was it was a huge I feel like it was a huge success. Like you know, got a chance
to meet you. We had people coming from all over the East Coast to come come to it and yeah, got uh you know, we've sponsors for the next one coming up in July July twenty fourth test, double infield fire hydrant. Saw the you know, saw the excitement, saw the energy, and uh, yeah, we're trying to make the next one even bigger, bringing more people together. I think another head canon. I don't know if
this is exactly right. The story you hear about like GitHub starting was you know, the you know, happened at a booth at a sports bar after after a meet up where they were like, you know, we should try this, and I don't know, I wasn't there. I don't know the conversation, but like that's the story you hear, and I wanted to try to, Yeah, see if we can recreate some of that magic. Yeah that's awesome. You know, I appreciate it, like I know a lot
of people appreciate it. Like, yeah, that was the first Ruby event
I've been to in a Quinton quite a while. And just because it was close and you know, like there wasn't you know it, it reminded me of the Unconference tracks right where you know, they're just like come hang out at a conference like and not go to any talk, and it like removes all of the barriers to like meet people, right like, which is kind of funny because you go to the pay for the conference so that you can listen to the talks, but then they end up having like you know,
somebody's just like it ends up being like a lightning talks but like side side tracks, right, which is kind of funny. But it has that kind of like vibe to it, right where like people are just like talking about like all of their excitement, which is like all this stuff is so exciting, and where do you start and like what are people working on? Like
you know, you don't know until and like that. The meetups was like definitely the culmination of everybody just being like, here's what we're working on. Like this is exciting, you know, and it was super cool to see. So I'm looking forward to going again. Yeah yeah, actually I guess I can. I don't have the the access to the comments, but send this to you link to the link to the event with anybody. Oh awesome, yep, we'll drop it in here. Yeah, come join us.
It was so much fun and yeah, like you said people travel for it. You know, it's really cool to see. Yeah, it's funny. You know you don't realize like how yeah, how tough the last few years have been being you know, remote and hybrid that like you don't really get a chance to like as much, I guess anymore, just like meet and hang out. And yeah, got a lot of feedback of just like like that same you know, your same comment, like I haven't been to a
Roovie event in a long time. I haven't, you know, I haven't seen this many people at a Ruby event for years. Like there's definitely you know, yeah, something we've lost by like going mostly remote and hybrid, but being in person kind of brings back for me anyway. Yeah, yeah, I remember my first Reels comp or was it in Maryland? And uh yeah, it's definitely even intimidating in a larger setting and having smaller groups, you know, it definitely is easier to to chat chat with people and social
ats, I think. But you know, one thing, I think we we had this moment when we first met. It was like I know your name, but like I know you're from the Avatar and discord right right, uh bridge that gap a little bit yeah. I mean to be honest, if extras had like a you know, print your avatar, like I feel like a lot it would make like, you know, these social connections easier to see. Uh, just slap that on. That's funny. So you
where are you going next with this? Like where where's sub layer heading? Where's blueprints? Like where do you where do you see like your next phase in all of this? I think the next thing and you know, with the topic of this is Ruby as a sleeping giant for AI application development. Like I think it's you know, we've kind of laid the foundation here with the framework, new version of trying to get the new version out last night.
But I had a couple of things I had to do. Lay the foundation with the framework, lay the foundation with blueprints, and then really kind of show how easy it is to show and like spread the word of how easy it is to build these l empowered applications. That changes a lot of you know, changes the way when do we think about what's possible? Right, I think there's still a lot of it's hard to it's hard to go from a place where you know a project in twenty eighteen even you're like,
okay, that'll be about a team of like fifteen people. It'll take you know, a year or two to do, turning into like a ten cent
API call like that. Mind it's like it's it's wild right now. I'm sure you've you've experienced it with just like some of the things these models can do, and so really like our next steps are bringing more and more of that to uh to the forefront of just like what all what are all of these things that you can do that take you half a day to automate, where like previously probably just ignored it because it was so costly and so boring
and mind numbing. And so building out the framework more bringing bringing more use cases to the forefront, and then you know, expanding blueprints to make that even faster so that more and more of these examples that you have, the
more things they can do and more things they can generate. And then so trying to experiment with the interactive docks piece getting ready to roll out kind of making it possible for anybody with API docs to uh have what we have in our docks uh see if see if there's interest there, So even if you're not even if you're not on the forefront of like trying to turn million dollar projects into tens on API calls. You can still see some benefit today.
You know. What I would love to see is uh, just being able to give like a GitHub link to like lines or something like that and say, like do something with this. I think that could make a really cool like example of hey, like here are the lines to some code, like you know, change this or something or make it say something silly. I don't know, Yeah, I don't know. Yeah, like there's but there's
so much stuff you could do like that, Like it's so easy. Yeah, I would love to see more, like more ease of use, right, Like it's it's already easy, but like still people don't you know know where to start, uh, which is which is a little bizarre at the same time, right, like, uh, it is as easy as just
typing, right. I hate to like distill it down to just that, but like, uh, you know, at this point you can just go to like you know, Microsoft dot com and like use chat GPT for free, you know, like it is that easy to get started, which unfortunately you got to buy into that, right right, Yeah, I mean I think that's that's the thing, and I think there's you know, one of the things that I've been trying to figure out is how to I've been trying
with that that example of like you know you're already doing rat rag is just you googling for like the docs, finding the dots and using it and using
it in your task. That Like, I feel like there are a lot of like very big, scary, like technical terms that you know, when you're getting ready to do a side project, it's like I'm gonna have to like read all these dogs to figure out what these things mean, when like what they really mean is like put an example in the prompt or like in context learning, it's just like give the they give some examples in the prompt right and right, and it figures out how to do it like that.
I think like that simplification I really want to try to find or work with people to help find because I've been talking about this with with like rails, where there used to be like, oh, you have to know all this stuff about like database normalization and third normal form and and this and that, but it's like it really boils down to, oh, you mean like customers have many purchases, got it right? So I think I think there are that I think is one of the things that's going to happen need to happen
next. So just like the like easier description like distill this down, there's you know, ah, there's very deep uh you know, deep explanations to take it further and make it more and more powerful. But to like get started, it's just like you mean, you like just copy and paste that into it, right, Yeah, the dummies guide to prompting, you know. But to be honest, like I never understood why like they came up with the phrases they did, like you know, zero shot or a few
shot, Like what does all that mean? Like there was it's almost like nonsensical to like say that when you just mean examples, right or not examples?
Like, yeah, I think that's what's one of the things that you've been talking about with you know, Andre and I have been talking Andre from Lang Chain RB I've been talking about is that, you know a lot of this stuff is coming out of research and coming out of academia and like those things, those those terms do matter for like those those domains, but from you know, an application engineering side of things that are more like you know,
applied we can have our own descriptions of these and why they work, and that are separate from you know, the science of the theory and more of the practice. Right was the you in In theory, there's no difference between theory and practice, but in practice there is. Well, is there anything else you wanted to cover to today before we jump in the picks here?
Yeah? Well, I you know, if if anybody listening is going to Madison Ruby, I'm going to be giving a talk, uh going more into using l MS in uh, a little bit of the philosophy on you
know, how to how to deal with l and generated code. The talks called going post l you know, won't give any spoilers away, but you know, leaning very heavily, which Obi talks about Postel's law in his book, but you know, being building your systems so that they can be liberal and what they expect liberal and what they accept and conservative and what they send, uh, just a little bit more liberal and what you what you accept than than what you're used to. And then you know, there's a lot
of things that Ruby can do easily any language to do anything right, but like the meta programming things that you can do in Ruby to change your application at run time. Ah, I think are under explored, so be going into going into that a lot at the talk. That's awesome. Yeah, I share your your sentiment there. I I've definitely tried to have the lll M generate Ruby methods and run them and then reuse them. It's it's a lot of fun, a lot very dangerous. But yeah, definitely under explored
for sure. So that's cool. Yeah, people go check out Madison Ruby if you're not going. Uh, I'm excited to watch that talk remotely at least. Yeah, hear what you got. So if people want to reach out to you or find you on the web, you know where can they do that? Sub layer dot com. Uh primary Scott at sublayer dot com if you'd like to email me uh at Scott wernerd dot or at Twitter Scott
Wernerd at Twitter, and let's see we've all. You know. I'm also like we talked about in the Ruby A I Builders discord, uh, you know, sharing you know, we uh discussion the big the big discussions right now are on that that arc A g I challenge which we've all been kind of racking our brains about. And then you know the link here on the
sub layer site. We have a we have an access to our discord, which is a little bit more you know, about our releases and maybe some less Ruby specific stuff and more kind of the thinking on what is how do we how do we think about these these AI tools and use them for building applications just in general. Yeah, I've been enjoying that channel as well. It's super interesting the stuff gets that gets posted, and you would think it would get shared in more places, to be honest, A lot of the
yeah, a lot of the papers I get shared. Maybe people just aren't reading as much as I am. I don't know. I don't think so, but it feels like an unlimited amount of stuff to read. There's so much to read. Wake up every morning, where did I go? Yeah? I feel like the paper I read last week is just like already like you know, dated, Like like how is that possible? All right, Well, let's jump in the picks. We've been talking about so much great
stuff. Picks our segment where we just pick literally anything. It could be code if you want to, It doesn't have to be, but it's typically what I pick, and if you need a minute, I can go first. So I've decided to to fine tune a large language model for Ruby, mostly just for the experiment than anything, but I'm calling it Ruby lang dot
AI. I'm gonna start building it and open and just like learning how all this stuff works and how it might be used to source all of the awesome open source Ruby that's out there that is designed incredibly and make use of that and take advantage of it and make a a new language model that is very Ruby centric. Uh. So I'm gonna see how it goes. I'm hoping that I'm successful in it, but I may not be. There may just be a bunch of lessons learned, but we'll see. You can follow my
progress Ruby lane that ai. Yeah, I mean I saw that you posted about it either yesterday or the day before, and yeah, excited to see where that goes. Yeah. You know, I got this stupid massive GPU server at home and I was just using it for imprints and I'm like, uh, you know, I was just wasting away over here like running imprints on it, and so I thought like I should fine tune something, and this seems like a perfect use case. So here we go. That's great.
Yeah, so I guess I've got I've got two things. One you know, the I talked a little bit around the like the formal informal kind of pendulum swing. You know. One of the places where I got that from was an AVD Grim talk the Soul of Software from Ah it was a long it was a long time ago at this point, but it's there are
a couple versions of it up on YouTube. He goes into more of the kind of the informal mindset and how there are those two splits of software and how neither one is particularly absolutely right, but more it's each each one gives
you different different things depending on what you need. So like that was that was very impactful for me, especially now given the you know, the approach we're taking and the other thing read a lot of substacks and there's one called strange loop Cannon which he had to post a little while back about what lllms can't do, which you know is also a trap a lot of times, because as soon as you say it's not possible to do this, everybody on
Twitter tries to prove you wrong. And in his post he had one about it can't do Conway's Game of Life, and I think like within a week somebody had proven him wrong. He had tried fine tuning all these different things, couldn't figure it out, and the collective wisdom of Twitter came up with a solution, I think. But then he has he has one recently. It's very good. I highly recommend just the sub stack in general. That one is very good. And then the latest one, seeing like a network
is very was very very good as well. That's awesome. Well, I appreciate you coming on Scott and talking about all this code generation stuff and the love of Rubai and you know, it definitely is a sleeping giant. And I think we're just going to see more and more of why that is. And you know, we'll have to have you on again after you're you know,
you start creating many companies menalize all of this movie code generation. Uh, you know, I could definitely foresee a future where you're just like, oh, you know, create a Reil sap that does this, and it just does it. It seems seems easy enough, simple. Yeah. No, thank you for having me. It's been a lot of fun. It's been awesome, uh, you know, chatting and catching up and excited to
see you in person a month. Yeah, totally all right, Well, until next time, folks, I'm out here, and come come visit us next time.