How'd you like to listen to dot net rocks with no ads? Easy? Become a patron for just five dollars a month. You get access to a private RSS feed where all the shows have no ads. Twenty dollars a month, we'll get you that and a special dot net Rocks patron mug. Sign up now at Patreon dot dot NetRocks dot com. Hey, welcome back to dot net rocks. I'm
Carl Franklin and I'materi Comp. You are here for your dot net keeeking out, listening, pleasure and all that stuff, all the things, all the things anything you want to announce today, buddy.
No, Well let me I told you earlier. But yeah, I know we got a puppy. Okay, because you know, having a grandchild wasn't enough, we also need a puppy. Yeah, so I've managed to tie this while she's at yoga. Got to have the puppy asleep.
So wait, Manute, I thought you were swearing off dogs after Zach.
Listen, I already had the perfect dog was seventeen years. Don't need it down a dog. I didn't need that dog, to be clear. But sometimes sometimes you're out numbered one to one, you know, Oh that's great that it's really got to be her dog this time. So I'm doing the only logical thing I can do, which is good to Australia for three weeks. Okay, I'm leaving on Thursday, so I'm literally I'm gonna go away for three weeks and hopefully.
What kind of puppy is it?
It's a it's a mix of Ausi Shepherd and border collie and some poodle and some burnies, a little bit of everything.
It's Oh my gosh, it's a cute little.
Dog, no two ways about it.
But you know, does it have an identity complex?
It's got springs and its legs is when it's gone.
Wow.
Okay, man, we're in the early stage needle teeth and pee on the floor and bit by bit figuring it out. I already took a tick off for her because she was out in the lawn.
What's her name, Jojo short for Josephine Lily. Oh she doesn't look like a deep fried potato wedge then.
No, not like some dogs, you know. Oh, she's a cute little dog but cool. And she's got that French twying tour so that hits the crazy name, right, anyway I got, I got worse problems like, yeah, it makes her. If it makes she who must be obeyed happy, then I'm happy to do fantastic.
All right, Well, let's jump into it with a better NO framework.
All right, what do you got?
All right? Well, in honor of our AI guest today, Vishwaesle, who's been on the show several times before, many times, I found a trending repo on GitHub. It's called krillin ai k r I l l I n AI or a one, depending on your point of reference. It's an AI audio and video translation and dubbing tool. So let me just read the paragraph here now, just does a caveat. I have not downloaded it. I don't know if it's any good, but it is trending, so that tells me
that people kind of like it. Crillin ai is an all in one solution for effortless video localization and enhancement. This minimalist yet powerful tool handles everything from translation, dubbing to voice cloning. I see there's no the commas don't work here. Translation comma dubbing to voice cloning comma. I would say, everything from translation and dubbing to voice cloning. Formatting seamlessly converting videos between landscape in portrait modes for
optimal display across all content platforms. YouTube TikTok, Billy Billy, Duian, we chat channel, Red Note Kawai show. With its end to end workflow, crillion Ai transforms raw footage into polished, platform ready content and just a few clicks. So that's interesting. You know, translation a localization anyway is one thing in the world of software and web apps that we know very well. But dubbing and you know, translating video that has audio in it.
Well, the voice cloning one is the craziest one where I take the voice of the person speaking and then change their language, but it still.
Sounds like them change the language.
Yeah, that's nuts, but it's very jen Ai.
Yeah, it's nuts, but you know that's the stuff that we get to see. This is the world we live in now, so world we live in. So that's what I got. Richard, who's talking to us.
Today, grabbed a comment off of show nineteen forty four just a few weeks ago with our friend Jody Bershall, doctor Jody Bernshall. Yeah's got a few comments on the show, and this one's from Joshua hillar Up, one of our regulars. I'm pretty sure already has a copy of Means to go By, But you know, Joshua, thank you. And he said, I'm thinking about the non deterministing aspects of LMS, and what's frustrating is that there's been many decades of computer
science research on software correctness. It just never seems to get used in the industry because it's too hard to learn. If you did that in air quotes, too hard to learn. LLLMS work really well at covering that hard work, and I don't think it's as hard as people claim, at least in the beginning. My hope is assuming enough money put into developing these languages and tools to make the use of that research. Now there's such an obvious use
case for it. And we probably should have talked about this more with Jody too, Joshua, because she does come from that I hate to say old school, but because that would be like five years ago form of machine learning where quality and accuracy, you know, validation was a huge part of the job.
That's right. I remember talking to Seth Warez and he said, ninety percent of my time is cleaning the data.
Yeah, to get to a meaningful quality. So there is certainly some pressure on that, but we're we're bringing a lot of people in without a lot of experience the space, and they are not concerned about these things and they should be.
M H.
So as usual, Josha, thank you so much for your great comment, and a coffee of music. Cobi is on its way to you. And if you'd like a copy music, go bay rte a comment on the website at dot netrongs dot com or on the facebooks. We publish every show there and if you comment there and I've read on the show, we'll send you a copy music O bay music to.
Code by dot net. That is how you can find it. It's still going strong. Track twenty two is available now and of course you get the whole collection MP three Flak or Wave. This is episode nineteen forty seven, and so I can just give a little brief summary here about what happened that year. It was a year of significant global transformations. Of course, India and Pakistan gained independence from British rule, and the Marshall Plan was initiated to
help rebuild war torn Europe. In the United States, Jackie Robinson broke baseball's color barrier, and Chuck Yeger broke the sound barrier. Also, the Truman Doctrine, which pledged US support for nations threatened by communism, was established, which signaled the beginning of the Cold War. You got anything to add?
Nineteen forty seven is the year the invention of the transistors. That's bartying Britain and Shockley with the first point content transi should have been talked about for years, but semiconductor making pure material semi conducts were hard. A lot of that came out of the war. And so what's funny is reading some of the documentation at the time and they weren't sure if it was actually going to be
all that useful. Interesting, you know, because vacuum tubes did such a good job and this thing did not have the potential the vacuum two did because the materials weren't quite good enough. Yet they had no idea what was about to happen. Huh No, but they you know, Shockley would go on with Fairshire Semiconductor to help make the originalized sees with Robert Noise and all those like. All
of that comes from there. But there's the beginning and one other fallout of the war raytheon's radar range, the first microwave oven.
Wow, yeah, radar range, that's what they called it.
They called it. They called it the radar range literally because some of the early earlier technicians working on radar noticed that the chocolate bars in their pockets would melt when they got too close to the magnetrons.
Okay, so how safe was this thing? It doesn't sound like it was very safe.
It's it can be is superheating the water, so it will you will get water burns if you get too close to it.
Yeah, right, And but radiation poisoning. Were they worried about that?
No? No, still, there's no I you know, ionizing radiation is more complicated than this. This is just microwaves, just like a bright light, and it can burn you because it vibrates the water molecules, and those water moleoscules, once they get to a certain temperature, are going to damage the tissues they're in.
Okay, all right, let's bring Vishwaz back on dot net rocks for the empteenth time.
It's at number seventeen, by the way I counted. Hey, well that's umpteen, isn't it. That's umpteenth. Yeah, we're ext the umpteeth category.
Bishwaslele serves as co founder and CEO at p wind dot AI, A noted the industry speaker and author. Mister Lealey is the Microsoft Regional Director for Washington, DC. Welcome back, Vishwas. So it's new in the world of AI in vishwas Land.
Well, i've been. We talked to you last year, very focused still on trying to grow this Jennai startup idea that we started working.
Yeah, you left your longtime company to do this startup.
Yes, I left. I was there for thirty years the previous company. How's it going last sixteen years? Is the CTO? Well, it's going. It's going well. I mean, as you can expect as a startup. World is all about good days and bad days. But it has been very exciting a lot of learning and we are growing.
That's cool.
Now. If I recall when we talked last time, you were focused on generating responses to requests for proposal for the RFPs. That's right, right, which in government land can be absolutely massive, like hundreds of pages.
That's right, that's right, and that's what we have been focused on. We call ourselves a thoughtful copilot. And I'll explain the word thoughtful in a second. We call ourselves thoughtful copilot for proposal writers. So what that means is we will take your information. You're you know in the years past you have responded to RFP documents that you've done.
You have competency documents, you have some fancy documents like spars and for those who don't know this already, se parts are documents that government agencies give you to validate your experience. So, needless to say, you have a collection of documents and now a new RFP comes along. And these RFP documents can be, as you said, Richard, can be really complex, can be fifteen hundred pages long with
different sections in them. So for example, they might be a section which describes how should you respond to the RFP, maybe a section which talks about what would be the evaluation criteria for somebody looking at your RFB. And then there may be another section which talks about all the things that you must do.
So that's all the boring stuff.
Our job is other boring.
You Usually the stuff that you want to read last, you have to read it first.
Yes, yes, and all this is part of something called the Federal Acquisition regulation, which something came out many many years ago, has different sections related to different parts of solicitation. So what we specialize in is, now that we have your knowledge repository built out, we deeply understand the solicitations that comes in, and then we help you write a draft quickly solely based on your information, so that we can give you this first draft quickly and then you
can build on it. And there's a very interesting saying in the proposal world that proposals are often one at the end Richard that you know, you leave yourself enough time to do some creative thinking, then you goos to your competition, you do some innovative thinking. But the sad reality of proposal writing is you never reach the end. You spend all your time in the middle, sure, just trying to get a good proposal out. So our value
proposition is very simple. Can we accelerate the initial stages so that we give you more polishing and thinking time. And that's ultimately what improves your probability of winning And which is the name of per company, peven dot Ai.
The probability of winning. Mm hmm, that's really good.
Well, part of this again, we're calling the previous show was being able to respond to more RFP so you have more chances at work opportunities. The thing, of course, has scared me is you're committing to stuff inside this RFP yep. And so if the tool is already the text for it, and it creates a commitment for you, you can't deliver on like you could be in serious trouble.
I think that's why you said creates a rough draft, right, Yes, you can't just like say, okay, here it is, send it off.
See. Yeah, And that's that's the thing.
Yeah, you certainly have to know what you're saying.
Yeah, no, Carl, that's a very good, good catch there. That's why I said draft. We're trying to improve the quality of drafts so that you know, you can focus on other things. But at the end of the day, make no mistake, you're putting your name down on the proposal.
Sure, so you should read it.
You should read it.
So it's kind of like having a you know, a manager under you or a secretary or somebody who knows your business say hey, you can just say look through this RFP and give me a rough draft and it might have all the accurate information from your side, but you know, you're the one that's got to commit to what you're agreeing to. And so I guess my question here is can you go back. I can have a conversation with this bot if you will, and say, you know, why did you why did you commit me to doing X?
You know, and have it give you an answer from your data.
That's a that's a good point. We have an intermediate stage, and this is the pattern patent should say that we filed. We have a provisional patent. Yeah, so what's It's not like the bot is just or the copilot in this case of our proposal, copilot is just going to look at your documents and just spit out one hundred page document that it thinks that you should be doing. That's a recipe for disaster because you go pick up things
that you don't want it to do. And there's a very important threshold where you really have to save time to people. If people say, hey, I have to first read your silly forty documents that you generated and then I have to rewrite it. Thank you, I'm not going to use this tool. So there's a very important threshold that you have to cross. So in our model. The patent that we have filed is called object based writing.
Oh wow, okay, and what that means is and what that means is that once let's say the r FP aries and we have gone in and tried to analyze it, and we create something called the flight plan. And why are we calling it flight plan because we are the copilot for proposal writers and in the aviation industry, obviously there's a very important concept called flight plan that is
used for communication. So what we do is we try to deeply using AI models, try to deeply understand the RFP, understand the evaluation criteria, understand the instructions, and we manifest that into a UI pattern called the flight plan. So you go in and then you intera to the flight plan, and this is where you go and say that I want you to use this architecture. And let's just take
a concrete example that our listeners can follow along. Let's say the RFP is about a records management solution, right, and you may have done ten different types of records management solution, maybe use SharePoint for that, maybe used some purpose built records management solution things for that.
By the way, I'm sorry, yep, yep.
So if you had to do that. So once the flight plan comes in, what we do is we say, hey, we think that you have used these are three or four architectural patterns to go after this kind of work. Here's a suggestion, but please you need to update that with maybe, like you said written Carl, SharePoint is not the right solution. You want to do something else, so replace that, And in that flight plan you tell us that you know, I'm wanting to undertake this task using
this architectural pattern. And then you also in the flight plan you get to tell people do you really understand the pain points? Do you really understand the motivation of the buyer? Why did they issue the RFP? And do you why do you why do you think you're differentiated than other competitors? If all your differentiation is or we have all the certifications in place, well, every other competitor is going to say that. So we force you to think critically about the response and at a higher level.
So is it kind of like an outline that you can interact with? It seems like it is. You you have a model for a conversation with the AI. Yes, so yes to this, no to this? What's about that?
Yes? And you know, so you're setting think of this as that you're setting the vision for the response that you want to generate. Right, you're talking about your competitors, you're talking about your win themes, you're talking about your pain points, you're talking about your solutioning strategy, and on and and on. There are lots of data structures that we capture, and then once we have that information, then we try to in a thoughtful manner, try to infuse
what you gave us as a direction. Right, if your capture team, if your salespeople have done a good job and they weren't their salary, they probably on the golf course realized that the key motivation for this RFP was the pain point this organization is going through is the last three upgrades were not there, and that key point AI does not know about or would never know about.
And this will be your capture team. And then you take that information, you have all of the data structures, you have your knowledge repository, and I'll talk more about it. We do a lot of work, and I very much agree with what you were saying Card earlier about sets comment. We do a lot of collation, classification, tagging of content
and cleaning of content. So now have you have your content, We have your vision for how you want to respond, and then we start drafting your content in a thoughtful manner. And that drafting is done using best practices. And this is where we are really fortunate to have Shipley. I talked about this last time. Shipley is a fifty year old company which is which taught people how to write persuasive proposals. That's what they do. They teach classes, they
generate content. So now we have this information, we use their best practices. And one example would be when you're responding to an our RFP, be sure about customer centricity. We talk about customer's problem, but speak to those problems maybe in your terms without sort of taking ownership of the problem. And as if you're teaching them right about active voice, passive voice, and then very important those are all very important. And then you start with the problem understanding.
Then you say where you have done this kind of work, you do you have really good past performance about this. So they are best practices of writing that come in. And this is why it takes us minutes and hours to generate the draft, right because you know, I was joking with you last year when I was the show that if you went to Chad Gypt and said, CHADGEPT, this is a very important question. I'm going to ask you, please think about it for five minutes before you respond.
Chad GPT does not know what to do with the four minutes and fifty nine seconds, right, because it just gave you the answer right away. Right in our case, that's how it works. In our case, what we are doing is, Oh, you have asked us to draft this information.
Let us try to Oh you told us this here, okay, and your knowledge repository saying this and one other thing that the comment that you read just a moment to good Richard that if you tell the language model that please take these ten things that I'm telling you and infuse them and generate some content. Language model, maybe listen to the first or second requirement and maybe the last requirement and then forget everything in the middle.
Right.
So this is where the thoughtful copilot comes in. We go step by step manner to generate the content.
I love it, but I appreciate that this is I got to imagine when you've an experienced RFP person, you're probably comparing the new RFP to past ones you've done.
It's got to be a certain amount of cut and paste there at least referencing material, which means you tend to try and solve the same problem over and over again, rather than this customer centric like do you understand this particular problem this time, which may or may not be explicitly stated, and then make sure you're tuning the material
to that, even if it is from past material. This is where the language model actually is super advantageous, because you can train it on the past data and it will generate original texts yep, presumably with the impetus of the new customer problem attached to it. This is stuff that's hard for people to do, but I think the software be pretty good at it.
That's right. This is a very important point I in my previous role we talked about. I was there for a long time and I didn't write proposals, but I often served as a technical smme for a certain section. Right, Hey, this is the section we need to write about. How should we write about and what would happen there is? I would tend to talk about the projects that I focused on, you know, even if they were not the
best fit. Right, because organizational memory influences how you respond versus This is where the superpower of the language model is that you have one hundred different past performance to look for, and maybe the the ones I picked because of my bias because that's what I've worked on and I'm very proud of those, Maybe those are not the best ones. Maybe you should put something else that has better engagement to the problem at hand.
Right, So.
I can see the advantage of this because it's very specific. But then again, you have to feed it your data. What about the stuff that's built into office You know that that knows like all of your documents and your spreadsheets and your your files and your PDFs and all that stuff. Is it too broad? Is that knowledge too broad to feed an RFP into and say come out with something meaningful.
Yes, that's a that's a great question. We we don't get that question now much much much more, But previously we used to get this question, why should I use your copilot? We have the M through sixty five co pilot, right, And the answer to that question is, if I can even take a step back, you have general purpose Shared chat GPT, claude.
What have you?
Right, and think of them as outside your firewall. Great tools. I love them. I'm sure all of you have two or three or more subscriptions because they are great at synthesizing information, summarizing information. Just you're trying to write something about a topic, it's great to collaborate with generate some content. Of course, you have to think you ultimately you own the content, and there can be inaccuracies, of course. So
that's outside the firewall tools. And I say that with some hesitation, because only two weeks ago or three weeks ago, chat gpt announced that, hey, in addition to outside stuff, you can also point to a Google drive or something like that and we'll start ingesting your content. So let's just keep that aside for a moment. Let's just talk about Claude and chat GPT. Great tools, but outside the
firewall tools. And then you have the office productivity tools like and three sixty five Copilot an equivalent that are built into, built into your enterprise. And they're great, right. They know who you talk to a lot on email, they know who the subject matter experts are. They have a full access to your one edrive, what have you. So they're great. But again they are general purpose, and two important differences right. You can use them to write
an email to a customer whose irate. You can use it to plan a party, a going away party for your co worker. They do well, they will get all these things right. You don't have to worry about copying and pasting information because they have access to all the data. There is security built in. They only get to information that you have access to. All all great things with those productivity tools. But when we talk about a domain specific coal pilot like ours, there are more things that
need to be done. So I talked about having very specific documents. We go to a customer and say, can you please give us your past RFP response documents, and maybe they have two thousands of those documents. We look at those documents and we break them up into logical boundaries, We classify them, we auto tag them, things like that. We are very opinionated about what we are trying to decipher from your proposal library content.
It's a specific data set rather than everything in your enterprise exactly. You might not you know, you might not want that email to your wife about the new dog. In your response to the you.
Don't want that and no, Caul, that's absolutely right. And there's one other sort of technical issue. Right, the general purpose tools don't know exactly how to chunk these documents, and often they chunk it at some page boundary or some fixed word boundary, and we cannot do that. We chunk them based on the logical sections the structure of an RFP document, so that when we provide that as a context to the language model, we provide the most
precise context. And it's all about context, right. Language models can get confused easily, so we provide So that's one big difference, right, all of the upfront work. If you ask me where where we've done our most R and D. We are fifty people strong now, so you're growing quite a bit. So fifty to fifty two or fifty three people and many data scientists awesome, awesome team of engineers, mlops, DevOps, whatnot, all running on measure and of course data science engineers.
So one part of research is ingestion. How can we be opinionated optimized ingestion? That's one part. And then also how do you glean some metadata from these documents? That's also important, right, because semantic search can break down if you're trying to find very factual information. So how do you extract the metadata things like that. That's one part of R and D. The other part of R and
D is we talked about the flight plan. We talked about how you get so much data from the user about win themes and pain points and solutioning and all of that. Right, how do you effectively generate prompts dynamically so that you can generate quality content that adheres to proposal writing best practices. So those two things together is why there is a domain specific copilot, and people try to do that. Initially, Hey, let me just upload the RFP to the to the M three sixty five copilot
and say can you generate a response? And then in some cases the response is limited. We are generating a response which can be one hundred and sixty hundred and fifty pages long. So, first of all, you can't do that with a general purpose copilot. And secondly, how do you without having to write all the prompts yourself? In our system, the users never have to write a prompt
because the prompts are changing. The best practices for prompts before the reasoning models came along, and the best practices for prompting after using models are very different and you can't expect your end users to keep learning new prompting techniques, so we dynamically generate the prompts based on what is in your knowledge repository. Right, So those are the two things. Sorry for the long answer, but those are the two things that are different.
No, that's okay. Yeah, that's really great. And this seems like a good place to take a break, so we'll be right back after these very important messages. Stay tuned. Did you know you can lift and shift your dot net framework apps to virtual machines in the cloud. Use the elastic beanstock service to easily migrate to Amazon EC two with little or no changes. Find out more aws dot Amazon dot com, slash elastic Beanstock, and we're back.
It's dot net Rox. I'm Carl Franklin, that's Richie Campbell hey, and that's vishwas Lele our friend. And just as a reminder, if you don't want to hear these ads before, during, and after the show, you can become a patron a Patreon for five dollars a month. Go to Patreon dot dot netroocks dot com. We'll give you a feed that has no ads. You still have to hear me apologizing
for them. Okay, well, you know the thing, as you were talking, it just kind of occurs to me that what you're doing is you're creating a company around a specific domain, right that where you know how to ingest the documents, you know how to query, and you fine tuned it. It's not like somebody is going to come to you and say maybe they have, but as anybody come to you and said, hey, you did such a great job for this domain, can you make me a
company around this domain? And that's essentially what you've done with just the starting with the idea of their domain. And I imagine that's what a lot of our listeners are trying to do with their own businesses and their own domains, isn't it.
So that's an excellent question, Carl, That's an excellent question. Let me my thinking has evolved over this question. I think you had asked a similar question last year when we talked, and my thinking has evolved. So, Number one, I think the patterns that we have figured out in terms of breaking down these documents in a domain specific manner, generating these prompts dynamically, those patterns generally are applicable to
any business problem. So that's right. But what I've also come to realize is in order to get true value from jen AAI for any business process, any complex and proposal writing is a complex business process. There are hundreds and thousands of business processes. You really have to do a lot of domain specific work in order to provide value to the customers. It's not like you could take
these patterns. Well, you have to Let's say you went to the financial sector, you went to the insurance sector, You'd have to do a lot of domain analysis to figure out what are the key entities, what should be the metadata. You'd have to do a lot of work to figure out the best practices, think about the prompt generation engine. So it's just not a matter of taking
these patterns. Maybe the patterns supply at a higher level, but it is the work that has gone into realizing these patterns is where we have spent the last year in.
Right, because it seems like there's a lot of people that are using RAG and it's kind of like what you're doing here, But I don't think there from what I understand, look at it, like as Richard likes to say, it's in a squirt bottle. You know, we just want to spray some RAG on our documents and be able to query them and get what we want out of them. And what you're talking about here is a lot more involved, Like it's not just a RAG tool that you're using.
It's everything from the original document, how to break them down, how to chunk them, how to put them in there, the things to look for, the way to respond like it's U And so I imagine you're we don't really look in terms of accuracy because you're not saying you know how many widgets are in BIN forty five right now, You're not. You're really genuine generating a text response which you ultimately have control over.
But accuracy is very important part for us. By the way, Carl, responsible they is an important tenet for us, and it manifests for us in many ways. And I'm really glad that we tried to adhere or architect to these tenets from the very beginning. And let me give you a couple of examples. Right, so, every time we generate a piece of text for you, the responsible AI tenant says that you should try to be transparent with the user. So you have to tell them exactly which document, which
paragraph did you source the content from. That's one. And then ultimately we said, it's a shared responsibility model. It's about the complementarity between the AI models and the human beings. So how do you help the proposal writers get you an accurate response sooner? So we generate something called the hallucination report every time we generate a piece of text.
It and what the haalucination report is that every time we write out some text, we try to extract any assertions that you've made in that text or the AI has made on your behalf, based on your knowledge repository. And we said, you know, you're saying that you've done these five thousand no cloud migration, but we really can't find anything like this in our knowledge repository, right, which is.
A great thing for it to say when it can't find something, rather than make something up.
Yeah, that's right. And we know that the sales team tends to sometimes take a creative license with some things. Maybe they combine to projects. Right, So it's not saying this is wrong, it is saying that it will it will serve you well to take this statement. We can't find an explicit reference to the statement in or repository.
Can you come back and sort of take a look, and maybe you can approve that risk and say, no, we are taking these two projects in and we're saying that overall we have done these kinds of migrations for this aggregated agency, and that's how we are making this assertion. That's fine, you can approve that risk.
Yeah, it may be an accurate inference or it may be completely wacky.
But at least if you have the references, you can go evacuate it yourself if you want, Yeah.
You can. You can go back and read. And the third thing.
That we also do and argue the interpretation.
Third thing that we also do is something called the completion report. Right, So the RFP document was forty to fifty pages long. It had instructions, it had evaluation criteria, it had many many requirements, and it's a very important aspect of responding that you're compliant. You're answering all the questions that you were asked to do.
Yeah, you didn't miss any you didn't miss any.
So we again try to generate a compliance matrix for you and say, look, these are the things that were asked in the RFP. But again, this is a very important part you have to go and validate that yourself, because language models can be inaccurate at times. So it's a very much an assistive technology, and we are not quite getting it to autonomous anytime soon. That press up button proposal comes out that you're ready to submit.
But it also occurs to me that again this is something it's good at that it could lead you to cerve responses or lack of response to a requirement, the same way that a generative AI image recognized er may point a radiologist to a particular spot on a picture and say this looks anomalous yep, and encourage them to
look in the right corners. Given all the guidance you're providing here, which I really appreciate, right that the flight plan approached the prompt generation to make sure it's accurate and focus on the right material with the references. Does the language model actually matter? Like you talked about open AI last year, Now there's been many more models released. I mean cloud was always around, but deep seeks appeared like does any of those make a difference given this strict set of guidance.
That's a great question, So let me answer that using two important points. Right turns out that these rf peace can also have sensitive information, and you need to have things like CMMC two standards where you are explicitly telling the government that this information is not just public RFP. So think about you're going after some RFP that talks about some kind of things weapon system and even the RFPs can have what is known as a controlled unclassified information.
So in order to support and show that you're handling that information correctly, you need to show to the auditors where your packets have traversed in that system. And then ultimately where did you send this data and did let's say you sent it to the external system like language model. Did that language model itself have a security classification and it is giving you guarantees about not logging that data beyond the metadata and things like that.
Right, yeah, and the sovereignty where did that they actually go?
Yes?
Now, I mean this makes a strong case for running as a local model.
Well, that's true, we use but you know the same time, these RFPs are very complex, so we want the best comprehension that is available.
You want a trillion parameters, you need want trills.
I need a trillion parameters, right, So what we do is platforms like asure opening. I give you that guarantee and you can actually show to people how your network, how do your packets from all the way from incoming RFP to how the response generation traversed a certain network path that the auditors can say, yep, we certify, we're okay with that, that these packets are fine. So that's one. So it's not just a matter of hey, let me just go send this data packet to somewhere else unless
they have a classification. So that's one one important consideration for us.
So just narrow is your choices of models based on whether or not they're going to clear see see two.
Yes, that's one. And secondly, Richard, the other thing that we've realized is that these models have a personality. So just because they speak English doesn't mean you can just swap.
Now anthropomorphic of you Vishwa's goodness, Yeah, you've pushed.
Richard's answer promorphic button there.
Yes, so these models have a personality against say saying that, And so I find this interesting when people say, hey, these are models are all English pays, so you can just switch one and see which one works. When you are generating hundreds and hundreds of pages of prompts. As we do, it is really important for us to understand the personality of the model, and we can change the model, and in fact, we change the models all the time. When we went from four to four turbo and so on,
and now we're using oh one and three. Every time we change a model, we have to do extensive testing to make sure our prompting layer works well. But at least these models are a certain family. Part of these models, just switching to us different models.
You've all described you're only talking about open ai models there so far.
Yes, we're talking about OpenAI models, so that's the other part. So those two important considerations have gone in into deciding which models we go against. We're thankful to open ai and Azure open Ai that their models have generally kept up. It's not like they're not they're the ones who came out with the reasoning models. Of course, reasoning models are available on other platforms as well now, but they've kept up with the advancements that are happening. So we're pretty
happy with where we are. But we constantly run tests to see how other models would do.
Now and you mentioned already the difference between using a model that doesn't have a reasoning feature, and I'm doing air quotes yep, because I read the Anthropic paper where they talked about the fact that the whole reasoning behavior is a post back or behavior that the result to the prompt generated and then analyzed after the fact by the tool to generate the quote unquote reasoning. Right.
So the idea here is that I'm trying to generate a response to a certain section. Sure, and either I can just in a non reasoning model, I would just say, hey, generate an answer, and we'll just go through a certain chain and just start generating a response immediately. But the reasoning model, it just tries to develop a plan and say, Okay, you're asking to generate this response. I could use this
example versus this example. Let's try with this example, go halfway through and realize that there's not enough meat in this Let's just go back and switch the example. That's what reasoning is all about, right.
Yeah, that's what it implies. But what the Anthropic paper says is that they're actually doing that after the fact. They already have the statement they want to say, and then they take their statement apart and fill in the pieces to show how they quote reasoned it.
Yes, more like justification than reasoning.
You showed me the equation. I knew the answer, and now I write out quote my work even though I've already got the answer. I'm just trying to fill it in to make you feel better because you asked me to show your work.
Yeah.
No, that's the anthropic paper, no question about it. But we are adding on top of that reasoning layer. We say, hey, which past performance do you want to use? How should you be writing about it? Things like that.
Yeah, and I'm not going to say anything bad about the fact that it goes and poll's references because you need those. That's all the quality stuff for there.
Yep, that's fishs. Last time we talked, we've got on the subject of how you think jen Ai and AI in general is going to affect the lives of average developers like us. Has your thoughts on that changed in the last year.
My thoughts have not changed. I think developers who will greatly benefit from tools like that. It is not a panacea. You can't just say hey, I'm going to give you these things and go code me something. I think it puts developers who have been thoughtful, who have worked on the craft who understand the edge case as well, who are able to critically look at the code that the lllms are generating. They're in a very good position because they can really get that acceleration at the same time
they're critically analyzing what's being generated for them. I think it's a real challenge for people who are beginning because it can very superficially tell you that it is working, but it may not actually be working exactly the way you want it to, or.
The AI might give you a solution which is more complex, and you're building and rolling your own thing when there's already something available that does it, and it's not going to tell you that. And I've had this experience and you go back and you say, hey, can I just use X? And it says, oh, yes, absolutely.
Yes, Yeah.
Well why didn't you tell me that before? Are you idiot? Right?
Yeah?
So to your point, somebody who doesn't know about you know, tool X or tool why isn't going to find out that it gave you the answer that you wanted, which is how do I build this? It didn't say you didn't say how do I solve this problem? And if you did that, you might have gotten a different answer.
Well, that's that's very important. It's also important to do it incrementally and iteratively so that, yeah, you don't just get two hundred lines of code, you have no idea and you run a test and you think it's fine. But if you do it incrementally, all of the basics of getting this right. Can I refactor this, can I get can can this piece be optimized? Using this? Yeah, it's accelerating that immensely, and so.
You know, it's funny you're describing exactly what you say p wind does for RFP that experienced RFP builder respond. Builders can use this tool to accelerate the behavior. But it is an interaction the tools pulling resources from past work and suggesting options that then you could iterate on incrementally, piece by piece, so that you know, when we talked a year ago this was about making RFPs faster. Now you're really talking about higher quality and better tuned to the demands of the new RFP.
Definitely, definitely, and the quality. We are reaching a point to where people say I can spot judge it PT generated text from a mile right, people see that I don't know if there's.
Well AI slop is now in the vernacular for a reason. There's a lot of bad generative AI slop out there.
I can definitely tell when images are AI created because they're so slick, you know, and people have made.
A half he even fingers.
Yeah, people have made a I don't know, hobby or maybe a career out of generating. You know, the most peaceful, sublime picturesque spot with a house by a brook and you know this and that, and they just post it on Facebook and say a beautiful place right with no contacts, with no you know, people are like, wow, where is that? Well, it's amazing, right, Yeah, it just doesn't exist. And I could spot those pictures a mile away.
Let me tell you, as a conference organizer, I know when you use chat GPT to write your abstract, abstract at your abstract because they are all the same.
Yeah.
I got a thousand abstracts in and four hundred of them start with in a world. Yeah right, like come on.
Yeah and b. Because we're so focused on the quality of writing, we explicitly look for these things where language models are trying to quote unquote write the perfect text because you know, you've given them ten things to write and they find the best way possible to pack that information and sort of present it. The problem is it is very onerous on a reader when you're packing it so much information right, and then overuse of cliches, making
statements without sort of providing background information. These are all common sense of LLLM generated writing, and we take explicit steps in our engine to either correct this mistake or constantly remind the model to slow it down to make it readable, to improve the quality. These are all things that are works in progress. A lot more work to be done there, but those are the kinds of things that we're thinking about. Goes back to all of these things are needed to have a real impact on solving
a business problem. When you think about this, it's not just a matter of taking this and saying, Okay, I'm going to go start generating financial reports tomorrow. There'll be a lot of work that needed to go in.
As a programmer, you kind of think of yourself as a senior programmer slash manager, somebody who can write code but chooses to offload the boring stuff to a junior programmer. And it's all about the prompts, you know, It's all
about what you ask it to do. And I've had the experience of chatchypt anyway where I said, you know, I'm trying to do this, and it might be about programming, it might not, and it will say, oh, well there's currently no way to do this and blah blah blah, this is not possible, and or maybe it gives you some suggestions that are just stupid, like is it turned on? You know, that kind of stuff, And then you can say, well, is any can you check the internet to see if
anyone else is having this problem? And that is the most powerful prompt you know. If you're not satisfied with the answer it gives you just say, hey, goog google it or whatever you know, and it will come back with sometimes with yes, there's a somebody's having this product.
It seems like this is a common problem and this is what this people, these people did to overcome it, right, absolutely, So it's it's all about the prompts and it's all about thinking in that sort of manager senior programmer to junior programmer mindset.
Yep. But back to you. You know the p win tool. You're mostly writing prompts with the tool to then run them to get results for the operator, that's right, that's right, And because there is a repetitive piece to this of setting the restrictions and making sure the context is correct. Like I got to imagine there's a couple of big paragraphs at the top of every prompt just to make sure it stays in the constraint, and then specifics for the section you're in in the RFP.
YEP, there's some systems prompts that go in and every time to sort of keep.
And it occurred to me, you know, thinking about this problem space and thinking about in our reactions to AI SLOP and you know, detechnics and so forth. It's like, if you think this pattern matching software is awesome, try a human because humans are really good at finding patterns, especially, and that's what they see. There's certain patterns to what a lot of low quality l M tools generate that you immediately perceived. And it's now starting to shift our
minds where it's now repulsive. Yeah, and we're in a funny place, but it's there's some good advice in here visual so I really appreciate it. Like a year later, you're speaking very differently about the product you're making and the tool and the way to use these tools like it shows that it's the evolution that's going on here.
Yeah, I know, not Richard and call that's that's certainly true. Learned a lot in this process of awesome technology. There's no question about it. I think we'll be it's the next super cycle, as Gartner Le likes to call. They said, you know, every tech super cyclist twenty years. The last one started in two thousand, but digital and the Internet and we are at the We're at the intersection of the previous one and launching into the gen AI.
Yeah. Yeah, this has been I felt like this is a fundamental shift in U acts.
It's a fundamental shift.
The way we interact with our equipment is about to change. But I think this generation isn't going to be twenty years. It's probably gonna be more like five or six or ten.
True's that's that's so.
Much has happened in the last.
A lot happened in the first few years of the twenty first century as Internet came out of the dot com boom, and you know, we shifted to mobile.
It's not like we've been going slow so far, no, but it does seem to be accelerating.
I think, Yeah, I know, it seems to be accelerating. But also I would say that there is some platauing, which you may see counterintuitive when I say that plat doing because GBT five was not released. They were going to release GPT five. Right, Just a bigger and bigger model with more data is not leading to better results.
Yeah, the last few years we've pretty much consumed the whole Internet into tokenization. There isn't more Internet to grab, so that that's not an exponential function, Like we're kind of addicted to the exponential function, but it only existed because a small group of people worked extremely hard to increase the density of silicon substrates. That's the only thing they you know, most everything else doesn't work that way,
and this doesn't because there's only so much data. And it's very clear that bringing in more data generated by AI for this is like taking a photocopy of a photocopy. It's degenerative, it's not useful data.
Well, I can't wait to talk to you next year to see what's new and see what else you've learned. Yeah, it's fantastic. Thank you, Vishwaz. It's always very talking to you.
Always a pleasure to be on this show. Thank you for inviting me again.
Well, thank you and thanks for listening, and we'll talk to you next time on dot net rocks. Dot net Rocks is brought to you by Franklin's Net and produced by Pop Studios, a full service audio, video and post production facility located physically in New London, Connecticut, and of course in the cloud online at pwop dot com.
Visit our website at d O T N E t R O c k S dot com for RSS feeds, downloads, mobile apps, comments, and access to the full archives going back to show number one, recorded in September two thousand and two.
And make sure you check out our sponsors. They keep us in business. Now go write some code. See you next time you got and Daddy see a summer time that means his home, then my Texas in line vegetable