Model Context Protocol Deep Dive

And these few things will happen, some of which are driven by an LLM, then something will pop out the other end or I'll email this person. So they kind of start developing those individual assistants. And then I see them start to kind of have this light bulb moment of the layer on top of those individual assistants or tools, which I think we could generally call that agentic layer, which is now saying, well, I can interact with unstructured data. I can interact with structured data. I can interact with these automations.

Maybe I can interact with other systems via API. How do I start tying those things together in interesting workflows and in various ways? That's sort of what I'm seeing. I don't know if you've seen that pattern as well.

There's a website, ModelContextProtocol.io, which talks about the specification and all of that. There is this origin with Anthropic, but I think from the beginning, Anthropic hoped that this would be of a widely adopted protocol. Maybe we should talk about the need for this first. We've talked a little bit about tool calling on the show before, Chris. I don't know if you remember some of those discussions, but there's very often this I think we even talked about this cringe moment of people talking about an AI model renting you a car or something like that and interacting with kayak.com.

Well, the AI model does not do this. Something else happens below the hood of that, which I think has been for some time maybe called tool calling or function calling, which is essentially the steps would be the LLM. You would give the LLM context for maybe the schema of the kayak.com API. You would then ask a query and have the LLM generate the appropriate maybe JSON body to call the kayak.com API. And then you would just use regular good old fashioned code, Python requests or what have you, to actually execute the API call to the external tool get a response.

Does that flow generally? Am I misspeaking anything?

But there's tools that do that. So what if I just leverage those tools via API and I could ask my agent, quote unquote, to give me a time series forecast of X. There's a tool under the hood that interacts with the database, pulls the data. Then there's a tool that makes the time series forecast, and boom, you tie these two together. It looks like your AI system is doing the forecast, but really you're calling these external tools under the hood.

The thing is everybody has different tools. Everybody has different databases. Everybody has different APIs. Every product has different APIs. Everybody has different function code.

And so similar, I think, to in the early days of the web, everybody was sort of posting things and creating their own directories of content on the web, their own formats of things. There was no protocol necessarily that everyone followed in terms of their usage of the internet or the web. But then there were protocols that were developed, right? Like HTTP. And now it's common practice.

Like I have a web browser, right? And I can go to any website and I expect certain things to be served back to me from the web server that gives me the website. And those things should be in specific formats for my browser to interpret them and me to get the information. And so there's a protocol or a standard in place for each of those web servers. So when I visit Netflix or whatever, it's sending back similarly, let's say, structured or formatted or configured data to me as if I go to Amazon.com and I'm searching for products.

This was not the case until recently with these tool calls. An AI model, everybody was on their own to integrate tools into their AI models using whatever custom code, whatever custom prompts, whatever custom stuff, which means, Chris, if you have created a tool calling agent and I now want to use some of your tools that you've developed, and I have my own tool calling agent, I may have to modify my agent to use your tools, or you may have to modify your tools to be compatible with my agentic framework. And that's kind of the situation that we've been in for some time, I guess is painful, reasonably painful.

And and I think this is yet another step of this field that we have been in and that is, you know, raging forward, maturing naturally in the way that it needs to go. You have you know, now we have a protocol that operates as that standardized glue that we that can be adopted, and everyone knows what to expect. Just a great analogy with HTTP, as you pointed out, in terms of being you have a standard format, standard serialization, and you can plug right into it. So, yeah, this was good news from my standpoint.

Who's creating MCP servers? What kind of model can interact with an MCP server? What, you know, payloads go back and forth? And so, so it it may be worth just kind of digging into a couple of those things. So first off, it may be good to dig in a little bit to what kind of how an MCP system operates.

And then that may make sense of some of the other things that we talk about in terms of models that can interact with MCP servers. So there's a series of great posts, which we'll link in the show notes. I encourage all of you to take a look at those mini posts. We'll of course post the main blog posts for Anthropic, but also the Protocol website, but a few of these blog posts as well. So one of the ones that I found that I really liked was from Phil Schmidt, Model Context Protocol and Overview.

And this one is really useful. And I think it's useful because it helps you form a mental model for the various components and how they interact in one of these systems. So you might be in your car listening to this. I'll kind of talk through these main components because you might not be looking at the article. But in the system that's using MCP, there are hosts, there are clients, and then there are servers.

So the host would be an application that the end user application. So let's say this is my code editor, and my code editor under the hood somehow is going to use MTP stuff. I'm going to be coding, I'm going to be vibe coding, right? And ask for various things, and it's

And then the servers are those external programs or tools or resources that you reach out to from the client. And those expose, like I say, tools, resources, prompts over this sort of standardized API. So it's a client server architecture. The client lives within the host, which is the end user application. And then there's the server, which you could think of again as this MCP server.

Now we should talk a little bit about what the MCP server does, but you could think about the client as invoking tools, making requests for resources, making requests for prompts or prompt formats or templates. The MCP server is exposing those tools, resources, and prompts. Does that make sense, Chris?

So the tools are maybe things like we already talked about with the Kayak API or calling into a database. They are functions that can perform certain actions like calling a weather API to get the current weather, or like I mentioned, you know, booking cars, or there's MCP servers that will help you perform GitHub actions right on your code base. So these are the tools or the functions that are that are exposed. So that's thing one that an MCP server can expose. Thing two would be resources, which you could think of as datasets or data sources that an LLM can access.

So things that you want to expose either as configuration contacts or datasets to the application. And then the third would be prompts. And these would be kind of predefined templates that the agent can use to operate in an optimal way. So let's say that your tools in your MCP server are related to kind of question and answer and knowledge discovery. You might have some preconfigured question and answer prompts that the LLM could use that you know would be optimized for a certain scope of work or something.

You could think about it like that. So these are tools, resources, and prompts that are exposed in the MCP server. Does that make sense?

I was thinking back to the good old microservices days, Chris, which you may fondly or not fondly A

No, I can't because that's not exposed as part of the MCP service, but maybe I can do GitHub related stuff, or maybe I can do, you know, database related stuff or or whatever that is. Alright, Chris. So we've talked a little bit about MCP clients and MCP servers. There's certainly much more that is available, to talk about and dig into in the protocol itself. And, you know, we've scratched a little bit of the surface here.

We're not gonna go through the whole protocol on the podcast. Maybe that's a relief to to our listeners, but there is a whole protocol there. I think it would be good to talk though about kind of two additional things, which immediately popped into my mind when I saw Anthropic releasing MCP and talking about it is, number one, how do I create an MCP server or where do I get access to MCP servers to tie into my own AI system? And then secondly, well, what if I don't use Anthropic models? Can I use MCP?

Those were two immediate questions from my end. I don't know, Chris, if you've there's various GitHub repos that are popping up and also examples of various MCP servers. Have you seen any that are interesting to you?

There's ones that can control your browser, integration with Zapier, all sorts of things. So people have already created many, many of these MCP servers. And again, when you're creating this MCP server, basically you just have to create essentially a you could think of it like a web server that has various routes on it, but these are specific routes that expose specific sorts of things, these tools, resources, and prompts over a certain protocol. And there is communication back, for example, back and forth of JSON, for example, over server sent events. But again, there's a specific protocol that's followed.

Now you can look through all of the specific details of the protocol if you want to say create a model context protocol server for your tool. And I actually wanted to do this. So we have an internal tool that we use for doing text to SQL. It's very frequent. I often call it the kind of dashboard killer app.

It's like everyone's created tons of dashboards that no one uses in their life. And wouldn't it be better if you could just connect to your database and ask natural language questions? So we have a whole API around this and you can add your database schema information and all sorts of fun things and do natural language query. And so there's, I don't know, six or seven different endpoints in this kind of simple little API that does structured data interaction. So I'm like, Okay, cool.

We've written that with FastAPI, which is awesome. So it's a web server. It has certain endpoints that allow you to do the SQL generation or allow you to modify database information or allow you to do various elements of this operation. And we utilize that as a tool internally via tool calling. So I thought, well, what would it take for me to convert that into an MCP server that I could plug into an agent?

Well, you could kind of do that more or less from scratch just following the protocol, but people have already started coming up with some really great tooling. So there's a thing called FastAPI MCP. If you just search for that, this is a Python framework that essentially works with FastAPI and basically converts your FastAPI web server into an MPC server or MCP server. It works. From my experience, I just added a few lines of code to my FastAPI endpoint, wrapped my FastAPI application in this framework, and then ran the application, which is, again, this FastAPI application.

That was immediately discoverable as an MCP server, meaning that I could, if I had an AI system, which we'll talk about that bit here in a second, if I had an AI system that could interact with MCP servers, my service now, the Text to SQL system that we use, would be available to that agent to use as a potential tool that's plugged into a database that we would connect it to. Does that make sense?

That's good. I love and MCP has shot up that very, very quickly. So yeah, in the world that I'm in, playing it, kind of combining MCP as a protocol that works at the edge as well as in the data center is a big deal for me.

And it doesn't mean that all of that is unauthenticated or you can't apply security of any type. What it does mean is that if you are for example, in the example that I gave, so I've now converted our text to SQL engine into an MCP server. I can plug in a database connection to that, connect to a database. But depending on how I set up the connection to the database, there could be potential problematic vulnerabilities there. And if I don't have any authentication on my MCP server, and I put that on on the public internet, anyone could use it.

So there's two levels of kind of security or authentication or threat protection that's relevant here. One is the actual connection level authentication to the MCP server. And the other is, well, I can still create a tool that's vulnerable, right? Or has more agency than it should. Yeah.

And it still brings value because you still have that standardization and the various vendors, whether they be commercial or open source, can provide interfaces to that to make it easier. So it becomes a much more pluggable and yet not tightly integrated, which is a good thing, architecture. And I think MCP really gives us that that same capability now in this space. And so it's like I said, it it it really is pushing it up the maturity, you know, you know, for up the maturity level from from we're all writing custom glue code to now, hey. I I'm gonna standardize on MCP and away we go.

You would you would definitely want to know what you're running, you know, what's included, where you're running it, how authentication is set up, etcetera, etcetera. Similarly, if you're connecting to someone else's MCP server, like Chris, you're running one and I wanna connect to it, depending on the use case that I'm working with, I may very much want to know what what data does your MCP server have access to? How are you logging, caching, storing information, etcetera, etcetera? You know, is it multitenant? Is it single tenant?

Etcetera, etcetera. So you can bring some of those intuitions that you have from working in the world that we all work in, which involves a lot of, you know, client server interactions and bring that into into this world. Okay, Chris. We've talked about MCP in general. We've talked about creating MCP servers or the development of them.

There's one kind of glaring thing here, which is Anthropic released or announced this model context protocol, and certainly others have picked up on it. And you see OpenAI also now supporting MCP where before they had this kind of their version of tool calling in the API. So there's a more general question here, which is, well, I'm using LaMa 3.1 or DeepSeek. Can I use model context protocol? And more generally, as models proliferate, which they are, and people really think about being model agnostic, meaning they're building systems where they wanna switch in and out models, do I have to use Anthropic or now OpenAI to use MCP.

So the answer to this question, at least as far as what we've discovered in our own work is as of now, sort of yes and no. But in the future, definitely there will be flexibility to many things. So what I mean by that is Anthropic has a kind of head start in a sense, in the same way that OpenAI has released certain things like various agent protocols or tool calling or stuff. It was something they released, something they had been working towards and they had maybe an advantage initially. Anthropic obviously has been working towards this.

Their models, their desktop application, etcetera, supports it well. Others are playing a little bit of catch up and that would include open models. If you think about something like a LAMA 3.1 or Quinn 2.5 or whatever model you're using, those open models, there's nothing preventing them from generating model context protocol aligned things. Agreed. But they haven't necessarily been trained as part of their training dataset to generate those things.

Meaning you can have an open model that generates what you need for model context protocol interactions, but you're probably going to have to load the prompt of that open model with many, many examples of model context protocol and information about it for it to be able to generate that, which is totally fine. You can do that. We've done that internally and I've talked to others who have and there's blog posts about it, etcetera. In that sense, that's why I say yes and no. There's nothing preventing you from doing this with open models right now or models other than Anthropic.

You might just have to kind of load that context window with many, many examples that are MCP related and aligned for you to generate consistent output for MCP servers. But what will happen similar to what happened with tool calling. So if you remember, you know, tool calling was released. Everybody The progression, I kind of see it this way. It's like people found out There've been a lot of cases of this.

People found out that models generally can follow instructions. And so at a certain point, people developed prompt formats like Alpaca, ChatML, etcetera, that had a generalized form of instruction following. And those generally got more standardized and now all training sets, well, not all training sets, but many training sets for kind of the main families of models like LAMA and others include instruction following examples. Then people started doing tool calling. And then people started developing tool calling specific examples to include in their data sets that they're using for models, including tool calling formats, which are in Hermes and other datasets now.

And so now many models do have tool calling examples in their training datasets. Now we're going to have the exact same progression with MCP. People can do MCP right now with open models if they perform in a certain way. It will become more efficient though as MCP examples are then included in training datasets for open and other closed models moving forward. So it's kind of now and not yet situation.

And based on, you know, things like, you know, providing examples that make it easy, there'll be a certain amount of kind of all the things competing and probably something that will kinda shake out as more popular than the others in the alarm because this is what we see over and over in software. And there'll also be a point where any that are genuine contenders, you'll have servers that support both MCP and all those top contenders with examples of each until it becomes clear kind of what the world is is going to go do. So I I think, yeah, I I think Anthropic was smart to do this, and they got a a leg up. And they put out a high quality protocol with a lot of great examples and SDKs right off the bat. And that was a smart thing to do to try to kind of win the marketplace very early in the game.

So it'll be interesting to see how that but I think the the key point that I'm trying to make is and that you're making clearly is that the the world has changed in that way, in a small way, in terms of, you know, everyone's gonna now have to level up into having this kind of AI specific middleware that ties the model into all the resources in the resourcing and tooling and and prompting that it needs. So I'm I'm very happy to see it come into place, and we'll see some shakeout in the months to come.

But but the ones that I were really was, using from the Python world, if people want to explore those and and look at those a little bit more, The one, if you're a FastAPI user, then I would definitely recommend you look at FastAPI dash m c p.

You can go and look at that. Well, the MCP inspector can help you check if you're connect to your MCP server, validate which tools are listed, execute example interactions, see what's successful, see what's returned from the MCP server, all of those sorts of things. So very useful little tool that is actually also linked in the fastapi MCP documentation as well. And, Chris, you you had mentioned a a Rust client. I'm sure there's a lot of other ones that are out there.

I am intrigued kind of generally, you you you've been exploring this Rust world quite a bit. Would love to hear any resources that you've been exploring there. People might be interested.

And it is it advertises itself as a minimal ML framework for Rust, but it's really caught my attention because as I'm often, you know, advocating for edge context and edge, you know, use cases where we're getting AI out of the data center strictly and and and doing interesting things out there in the world that may be agentic, may be physical as we go forward. Candle is an interesting thing. And if we're lucky, we might have an episode at some point in the future where we can kinda dive into that in some detail. But if that if edge and and high performance minimalist things are interesting to you in this context, go check out Candle at Hugging Face.