The API design centers on effective communication and it's not just communication between developers but it's communication that combines product thinking, Business and Technology. All in what?
Hey, everyone. My name is Henry Surya, we Robin And you're listening to the tekhelet Juno, the show will be bringing you the greatest technical leaders practitioners and thought leaders in the industry, who discuss about their Journey ideas and practices that we all can learn and apply to build a highly performing technical team and to make an impact in your personal work. So let's dive into our Journal. Hello to all of you.
My listeners. I'm happy to be back here again with another new episode of the tekhelet journal podcast. Thank you for spending your time with me today, listening to this episode. If this is your first time listening to technology, you know, I would invite you to subscribe and follow this show on your podcast app and social media on LinkedIn, Twitter, and Instagram, and if you have been regularly listening, and enjoying this podcast, and love the type of content that I'm producing.
Will you support the show by subscribing as a patron at technology, you know, dot Dev, slash Patron, and support my journey to produce great episode. Every week building, web apis has become ubiquitous whenever we build Tech products, almost everything is nowadays available on the internet, and it's just one API call a way to integrate, disparate software systems in order to make them work together, to perform a series of tasks or activities.
Think about your mobile apps software as a Service electronic payments or even smart, home devices that you use frequently. They are all connected seamlessly through, apis, that power the integration. And the importance of the apis is not all about choosing the underlying Technologies or protocols such as whether we should choose rest API, graphql, grp C and so on, there is something more beyond that. My guest for today's episode is James higginbottom.
James is the author of principles of web API. Design, and an executive API consultant in this episode James explained, why it is extremely important to design, apis, properly and share the five key important principles of API design such as not to design it. In isolation importance, of API, documentation, Etc. James also recommended the API design first approach, which is a rapid and lightweight outcome based API design, process to design and deliver apis success.
Really, including the addr process, aligned Define design, refine establishing API boundaries, and how it relates to domain-driven design or DDD towards the end. James also shared some recommendation for API testing strategies and also some API testing anti patterns that we all should avoid. I really enjoyed my conversation with James understanding the importance of a good design principles for building successful apis including his First approach and the addr process.
And if you also find this episode useful, please share it to someone, you know, who would also benefit from this. And you can also leave a rating or review on your podcast app and share comments on the social media on what you enjoy from this episode. I always love reading your reviews and comments and they are one of the best ways to help me spread this podcast to more people. And it is my hope that they can also benefit from this podcast. So let's get our episode started.
It right, after our sponsor message. Are you looking for a new cool swag? Tackle, it Journal. Now offers you some swags that you can purchase online. These wax are printed on demand based on your preference and will be delivered safely to you all over the world where shipping is available. Check out all the cool tracks available by visiting package, you know, dot, f / shop and don't forget to break yourself. Once you receive any of those tracks. Hello everyone. Welcome.
Back to another episode of the technology on our podcast today. I have with me, a guest named James higginbottom. He's actually an expert in API design. He has been a software developer and architect for more than 25 years. So that's really a long time and mostly helping people to develop deploy apis apps. Also helping companies even to go through Transformations based on these API. So James just also published a book titled principles of web API design.
It's something that you can look for. You need to build a web API for your applications, for your team's or for your company. All this will be covered in this episode. So thank you so much for coming to the show James looking forward for this composition. Yeah. Thanks Henry. I appreciate you. Having me and look forward to our discussion. So James, maybe in the beginning, you can introduce yourself for those who haven't
known you yet. And then maybe you can highlight starting points or major highlights in your career. Yeah. So for those that are not familiar, I've been Consulting for a number of years, ranging from startups and software as a service company. To Enterprise it. A lot of what I do is helping organizations to establish grow and mature their API, programs, as part of a larger digital transformation initiative.
Part of that is coming in and working and I 101 or team basis to help organizations to grow their expertise. And that also, oftentimes includes training. So to doing live in person or remote workshops on API design techniques interesting enough. I've got started in software quite a few years ago. You're talking about turning points in the career. My grandfather, gifted me. A Commodore 64 computer when I was about eight years old.
He said, I want to get you this because I think computers are going to be big someday, and I'd like for my grandson to learn how to use them. I took to it and turned it. Into a career is been an absolute amazing opportunity to go from kind of the early client-server days through
client networking. We were figuring out how to network systems across Either a private Network or over the internet and then eventually into web development and that's also led into web API design and helping organizations. Think about how to treat their web apis, not just as a technical asset. But really, as a product that's owned and matures over time. Thanks for sharing your story. Interesting enough. This is not the first time, I guess actually shared about Commodore.
So it seems like that change a lot of people's career. I personally haven't seen it. So, looking forward one day maybe to see the real product. Of comodo, right. Let's talk about your book itself. Principles of web API design. What do you think are some of the important things that led to to writing this book? A lot of this book comes from practical application that I've spent with organizations and myself. Personally, designing, apis. I come from kind of a blend.
As I mentioned, in my career of through the older School, way of doing things where we used to build or design and build everything up front, which we don't want to do. We learned other techniques for it. But I've also seen a lot of opportunity to think about API design and new and interesting ways. I've always referenced an old partner in co-author of mine. Keep Casey who helped me put together a self published version of what turned into this book. When we put it together years ago.
He always said that API design represents the effort required to design the third user interface or referred to an API design as the third user interface. It's the UI for developers. What I've really taken from that is API design. Enters on effective communication and it's not just communication between developers but it's communication that combines product thinking, Business, and Technology all-in-one. And so, it's always been a real interest to me.
So getting an opportunity to put a full book together after having years in the field working with teams, coaching them API design and different techniques that you can use to get there effectively and quickly without bogging down. The delivery process was always been a passion of mine. So that's been Focus of this book. So first of all, why is it important to actually design? An AP are. So you mentioned that your co-author of the book told you about.
It is actually one third of the user interface. Why is it so important to design a pi properly? I think it really does go to communication. We don't often times think of our API design, as part of the communication effort. We think of our documentation, the web portals developer, portals marketing materials and other things. But our design itself communicates. The words that we choose the patterns that we apply, they all communicate to us.
And I think as developers we all encounter or have encountered web apis, or even just helper libraries in our favorite programming languages. There were either a pleasure to use or really difficult to use in really frustrated us. So I think taking the time to consider your API design and sintering it on an outside in thinking process thinking about
the outcomes that are desired. And we're trying to deliver with that API, really contributes to that effective communication, and also enables great documentation as a result because it makes it easier to document the API in the communicate, how to do things. You don't spend as much time saying, here are the little nuances or caveat sore things that are kind of non-standard that I put in here.
Instead, the API design becomes very intuitive for the person, trying to use it to build a solution to solve a problem to automate something. So I think there's a lot of important steps. Be placed in effort, centered around API design first, and IMS also assume when you say about all these communications, it didn't just happen for a lot of people who build apis, maybe also based on your consulting or expertise in people, doing this stuff.
So, why do you think those people do not actually empathize on getting the API or communicated with so many people who are connecting to it? I think four years API design was really pushed to the technology teams over the years.
we've had to make decisions about Frameworks and helper, libraries even entire programming languages platforms, Cloud infrastructure, anything along the way, a lot of those decisions tend to center around technologists, but as our organizations start to think about how we expose web, apis to allow others to integrate with us, whether it's Corp to Corp, just business-to-business kind of Integrations, or if it's more of allowing consumers to access the room data or to automate,
Oceans, whatever the case is. We really have to spend more time. Emphasizing the API design. So I think up to this point. It's just been kind of pushed off to the side. And now, we're really seeing not only Tech leadership software, architect senior leaders, any developer on a team, now, responsible for producing delivering an API, either for internal use, or maybe external use. But we're also seeing product
owners, get involved as well. So now, what we have is this opportunity, Think about apis as products and that is requiring us to take a step back and to rethink the approaches. We have to the way we approach apis from the design perspective. From the coding perspective, slow down and meet those parties that have input subject matter expertise to bring them along with us in the API design. And not just to forge ahead with the code.
But rather to take the opportunity to turn it into something that centers around more product oriented experience. So, you mentioned that product owners, sometimes get to be involved. But is this only for the case where you are building an API, as a product or is it also applicable for people who build like, back-end apis, where you just connect from your mobile application as well. Maybe your web front-ends. So any thought about that, we're starting to see more and more
product owners get involved. Whether this is an API for a software-as-a-service and apis, are product offering that an organization is putting together and releasing. Or if it's just internally facing, it's still taking a bit of time because I think from the product owner perspective, their response beliefs and what they're held accountable for tends to still be related to either a partner to partner integration or a web or mobile app.
So we tend to still focus on the wire frames in the features and the function of the application, but it's starting to become more and more apparent that through these digital transformation efforts. Anything we do, whether it's meant to have a human Ur face in some way Voice Web mobile whatever it is, or whether it's automation that there's an opportunity to consider the API as part of the product offering.
And so that's resulting in a lot of product owners starting to go beyond their initial focus of just the product as its seeing inexperienced and more of the holistic view from front to back and that includes the apis that help to power those Solutions swell, so let's go to some principles that you at clean. In the book, The First principle is that you set API should never be designed in isolation. What do you mean by that? That's really what I've been talking about here.
And that's why I've spent a good deal of our interview so far, discussing it because we have oftentimes required. Our technical teams are delivery teams to shape the apis themselves. While there are quite a few developers out there that have designed well-thought-out apis before I think there's an opportunity. Opportunity to expand our influence and to incorporate other parts of our business in the process.
So in this case, we're going to have, as I said, subject matter experts from product owners to perhaps even security teams or the office of the sea. So that is looking at the security aspects of an application and what kind of security concerns the are with an API, you know, personally, identifiable information and non-public information other bits. Of information. We may not work with leaked. There are different subject matter, experts that should be consulted.
When we think about the apis too, often. We've released web apis out and either not secure them or didn't think about the design from those different perspectives from security from product ownership, form, the user experience, and developer experience. So the first and foundational principle is, we should never be designing apis in isolation. We should be looking for those other team members to come with us.
We also You have to kind of slow down as developers and if necessary educate team members that may not be as familiar with some of the technical details of HTTP. And other things, just kind of help meet them where they're at, and part rep with them and move forward with the API design, with them involved, rather than just going off in a corner and building something. Because there's a lot of opportunity to make apis. Better to make them more secure
and to make them longer lasting. Yeah, which I think also led to the second principle, which is you mention about the design should start. An outcome-based of Focus, right? So when you gather all these people around, I'm sure there's a common outcome that you want to get out from this API. Is that correct?
There is when I mean by the outcome based focus is not just what these team members are including or wanting to factor into the API design, but there's also the aspect of who's going to be using the API and what they're trying to do. So, part of what I do, is coach on a technique called jobs to be done, which started with Clayton.
Ensign who wrote Crossing Chasm and several other business books, his perspective was whether you're building a start-up or whether you have a large Global organization, anything that you do service or product is solving a got to be done and the best way to be successful at it in a find something that is going to do well is to understand the customer, listen to the voice of the customer, understand their problems and what their desired outcomes are if we take those
things into consideration. In the piece in the middle, is the job to be done. If we understand what the situation is and what they really want the situation to be the in state, then what we put in the middle is that job to be done. So when we think about our API design, we want to think about the same thing.
There's a tool called job stories richer and similar to user stories, but they start with the when which is that situation, the I want to, which is the job to be done and the, so I can the outcome that they desire, when I talk to teens of The API designer when I'm designing my own apis. That's where I like to start. That means that all those team members that I mentioned from principle.
Number one, that are subject matter, experts and have input to provide or going to be contributing to that understanding. And I might require our product owners to take a step back and start talking to the target audience. Get feedback. Just like we would any other product and use that to help Drive literary kids. I needs to deliver and what that will kind of outcomes we can produce as result of using that API either as a standalone or until a Raishin, with others is
very interesting. You use this concept that is normally used for building products. So like when we want to gather user stories or requirements, we use this jobs to be done framework, but you mentioned that you actually use this to actually come up with the API design, what you call Job story, you mentioned, but when I want to, so I can. So it's really interesting that you actually bring all these the best practice from product development of product, specification, to actually web
API specification as well. The next one in your principal is a Actually, you mentioned that you need to select API design elements that matches the knee. So I'm not very sure about this principle. Maybe you can help to explain about it. Yeah, I think this really stems from the idea that there's been a lot of discussion over the years. Do I use a rest API style. Do I use graphql? What about grp? See?
I've shared discussions in the past and spoken at conferences, where I kind of dispel the myth that there's any one particular API style that's best. So when we think about the design elements of API, we need to think about how people are going to be using the API. What's the context in which they're going to be using it? Is it primarily from a web? Browsing device? Phone tablet, laptop. Whatnot. Is it a voice-based? Is it that we're going to be integrating with other systems?
And so we need some events. So we might think of things like web hooks or server-sent events or websockets, maybe Kafka streams. If you're sharing messages within an Organization across teams across systems, all those things need to come in together to match the needs of the consumer. So as we're taking a product based approach, part of what we need to be thinking about is not only what the API design should look like, but how or what a pi style, or Styles, more than one style.
Should we apply to make it the best match for our consumers? So just because an API producer a team that's designing and building. AP, I fell in love with graphql. Love it. If Their audience that's going to be using API is not familiar with it. They have two choices, one. They proceed with graphql and they invest in educating, the audience on how to transition to
graphql for those. That may be more familiar with rest or other styles or to set aside their desire as a provider as a single team and think about holistically. All of the different things are going to be consuming this API and determine if a different style will be better match. Or maybe it's offering Choice offering both rest and graphql and maybe even some event capabilities with web hooks for callbacks when event notification and reactive style
of interaction makes more sense. So, it's just the idea that we want to recognize that. There's no one single solution that there's no one thing that is best. But if we factor in our target audience, then we'll be able to determine what the best fit is at least for today, and we can continue to listen. To them and then offer new Styles as the needs arise that keeps us from being a little too focused on ourselves and a little bit more focused on the variety of developers around the world.
That may be one in consumer API and would prefer to do it in the way that they would, like, rather than the way that you might want to mandate. I know that you mentioned as no best API styles of API Technologies, but for those techies, who listen to this, I'm sure they will have the same question. Is it rest? Is it job? You see? How is it graphql, right? The new Oh, darling in the API world. But from your view, maybe you
can give us some summary. When do you think we should choose each of those popular Technologies? I found that rest or just basing a web API on HTTP. Typically using the rest architectural style is a great approach because the tooling exists in so many languages. It's so easy. I can use a variety of developer tools to assist me in the development process in the design processes in. Assuming the API and trying
things out. I just see that as a popular option for a lot of organizations that said I'm seeing some organization started to go down and explore graphql, a little bit. They're finding that it's a really great fit for the response shaping aspect of it, even though there's other aspects that tends to be where people gravitate toward the idea that I can craft a query, that looks a lot like a database query. I can be very specific about the elements that I want.
So if you have a very deep resource structure, Where you have lots of nested resources and you don't want to have to do query upon query, to go, deeper and deeper, and you don't want to implement your own version of that on top of rest. Oftentimes the teams will choose to use rest as a foundation
complement it with some graphql. The maybe, if they need some high-performance service to service communication, they'll decide to go grp, see, speeds up the development process with the code generation that's built in. And also, some of the performance improvements, the grp see takes advantages of from http. To some of the compression of headers and other things that are built into it and the bi-directional communication, it gets teams a lot further.
So again, it's not one size fits all but I do see the majority of apis that I work with tend to start with rest and then they look at other styles to complement it and then we see things like web sockets and webhooks used for venting quite frequently. And I think you mentioned, also when you explain about it, you can mix and match.
So not only choose just one style, but you can actually start with, providing rest and maybe some job PC on top of Of it, maybe graphql for those people who really wants to shape the response of the web API. So thanks for sharing that summary. So let's move on to the next principle, which is I think a very very interesting because you mentioned also in the beginning that API documentation is probably the most important user interface for developers.
I've seen in my whole career. It's a documentation seems to vary, some are really good. Some even has the capability for you to test it on the documentation itself, but some are just like, okay, what? The end points where the parameters and all that. So maybe you can explain a little bit. Why do you think the documentation is the most important UI? You mentioned it as the UI for developers. The reason being is there's different paths that we all take
to work with an API. So as a developer that's looking at a brand new API. The reference documentation is very important. I think all of us know whether it's a helper library for our favorite programming language, or whether it's a web API is allowing us to take advantage of a software as a service solution. Or It is the reference documentation, is where we spend
the bulk of our time. However, there's additional documentation on top, that's really important getting started guides that help frame up and position. The API is to what it does, what it's capable of doing cookbooks or, or step-by-step guides that show you how to do common, tasks are really important as well. So we sometimes forget, particularly when we're designing our own API that, just because we've spent a lot of time thinking about it design.
It implementing it deploying. It troubleshooting getting it ready and push it into production and we've Managed IT production for some time and we have people successfully using, it doesn't necessarily mean that next developer then encounters. Your apis going to know exactly what your apis meant to do. What you've designed it for. What it's good at. Maybe what it doesn't focus on or that you would recommend complementary apis to pick up where you leave off and how to
get started. Those type of aspects cannot be be underemphasized when we think about an API. And the last thing we want to do is developers is document. I don't know why a lot of us have developed this habit that done is when the code is done, and the tests are passing, and I can get a pushed out to a cloud resource in its up and running.
But if you don't document it, then those developers that are trying to understand how to use it or get very frustrated, just like you probably get frustrated with a particular website, that's poorly designed or, you know, a word processor spreadsheet. This is not super intuitive. A mail app that just doesn't work the way you want it to
work. I think all of us have probably used over the last 18, 24 months, a variety of video chat and collaborative messaging platforms from slack, two teams to WebEx to everything else that's out there. And we have our preferences for what we like to use in what we don't that user experience or that frustration that we have is oftentimes for web apis rooted in great or terrible lacking API
documentation. So taking the time to document Even if it means asking for outside help getting a technical writer. That's really good. At asking you the right questions and getting the copy so that people can understand it. And it's approachable will make your API very popular because it stops the frustration cycle. And it helps people get things done. Some of the most popular web apis. We know of the software service apis that we know of, that's what they did.
So even if you're building an internal API, you want other people to use your API internally, right? Little bit of documentation. If you're not standing up a whole portal, just take the open API spec or whatever it is. You're using to capture your documentation, spend a little time, put a few examples or walk people through some basic use cases or usage examples. Help people get started. I have some empathy for the people that are seeing your API
for the first time. Maybe they're having a bad day and now they have to get something done. And this API is going to help them. Give them a good day. Not a bad day. So spend some time documenting or get some help if you need to get an assist. From somebody contract somebody and get somebody else in your organization, might be better at it to give you some tips. Whatever it is and really focus on the documentation they can occur over time. It doesn't have to all be done immediately.
But over time you should be improving as a start. Simple, grow it, improve it. Don't let it be. The last thing on your to-do list when you release a new version of the API or make an improvement constantly look for ways to improve it. Take feedback from troubleshooting sessions, or support sessions with developers trying to use your API, take what? Struggling with back and try to improve the documentation and just continue to do that. It'll go a long way to making a
successful web API. Yeah, so from my experience as well, it's not just the tooling actually to build these API documentation becoming more varied and becomes easy as well. So like you mentioned open EPS specification, what used to be known, Swagger, I think, and also tools like Postman, so they evolve into building ecosystem or building tools for people, collaborating building API or using other people's API. Why? So I think look for tools as well? Not just like building by
yourself. I think that's also a key here. Absolutely. Yeah, the last principle that you mention in the book is actually this is very interesting. Definitely, you mentioned, apis are forever, but it tends to stick. So you need to plan accordingly. Yeah, I think many of us, in the API, space is dealt with web, apis, can understand this one. And I know that Verner vogel's from his days at AWS, really kind of solidified. This one through various posts
that he's made over the years. Regarding the challenges and the opportunities and the successes that AWS has had with their apis, that as really codified, the idea that apis are forever. I mean, if you think of the S3 API from AWS just that API alone, imagine if you woke up tomorrow and they broke something. They changed in API signature. They renamed something. They drop an operation that you needed and that you were dependent on you woke up and
things just stopped working. We just can't do that when we think about Web apis. We can't just think about a code base that we can refactor and change things with and as long as it all starts to compile and work again. We're okay. We're talking about distributed systems many systems of which are outside the control of other people using it. So those that are consuming your web API are out of control of what you do is the provider and vice versa. In that case.
We have to have a strategy for how we manage versioning how we approach it being sure that we're not breaking people and also making sure that we're not creating brand new versions of apis. Every week, or every month, it's great to enhance an API with non breaking changes. You'll additive changes those types of things are great. But removing things renaming things.
Those things that would break someone who's using that operation or using that particular field in a response or using a particular feature that this part of your web API. And you just decide one day on tired of maintaining it, or I'm tired of the way. I named this. I'm going to refactor the code that refactoring ends up changing your web API because you haven't separated the let it get caught. Tracked from your implementation
details. And so you're unable to make it still work the way it was designed even though you've changed things internally. If you start to introduce breaking changes, then you're going to introduce opportunities for customer Charter. So, if yours a software-as-a-service, you have a web API and people are integrating with your web API to automate or integrate your software as a service capabilities.
With other things that they use other software as a service has and they're doing things to help them and automate their lives and you make a change. They now have to go make a coat. X. Well, they now have an opportunity or a question. A choice ahead of them. Do I make the code change? Or do I go find someone else? Who treats customers that are integrating with their web API with more respect and prevent breaking changes from coming in and then me waking up to a bunch of problems.
So the way that we treat our apis, the way we version and the way we manage change is really important and we have to plan for that their strategies that outline in the book for that and different techniques you can think about than a Ford you Unity's to make breaking changes before you're committed to those designs before this designs become Frozen where you can only add new things, but you can't go back and change those things. You've already put out there. There's ways to do it.
But keeping that in mind is absolutely important too often. We just plunge headlong into the code. We think we've got it all figured out. We put it out there. It doesn't work. Or doesn't meet the needs or expectations yet, people start using the API in different ways. And then we want to break it to introduce the right way of doing things because we don't like the way it turned out. End up causing too many problems and we want to avoid that. So I myself personally have
experienced. For example, there's an API deprecation announcement or emails. You have to move to the new API by this. So personally I find sometimes it's very annoying because what is working? Well in Michael Bay's, suddenly you have to change because of this, I think it's very important that you have this mindset that API that you design and publish them to stay forever.
I don't change it too often which brings us to a good Segway. To the next discussion, you mentioned API design first approach for many this doesn't come intuitive because normally they will just start from product requirements code and showcase, especially when you integrate with the body between services. So you mentioned here that important Concepts that I learned throughout my career as well. That API design first approach tend to be the best practice
here. So maybe you can elaborate on. Why do you think it's the most important thing? I think, when we think about building software, we went from sitting down and Something about a design and spending a little time designing out our software. Now granted a few decades ago. We spent too much time there and not enough time actually producing or delivering. I think the pendulum swung the other way where we said we'll code is most important thing.
Our code is our documentation. We can just kind of evolved our design as we go and figure things out. Yeah, we can do that in small bursts. The small increments inside our code base. Many of you listening may have experienced success in that way, and I don't discount that. But you also have to recognize as we said with all these different principles that we can't design in isolation that we have to think outcome-based.
Do we have to think about how are consumers, want to interact with our apis, how we're going to document them and that once they're out there and we have the first integration with our API. It's out there forever. We don't want to be tasked with having to keep up with your API changes because you keep changing your mind on how your
design should look. And I've got to keep up with it because you're deprecating a version and forcing me to write more code and adjust my code to make things work.
So, there needs to be a balance between what we do with upfront design, and how we deliver the book, outlines an API design first approached, the idea that we need to spend a little thought in incorporate this input from other team members and to think about an outcome-based focus of what our API is going to do. If we spend a little time doing that, then we don't have to experience what many of us have
experienced. And I know I have is writing code that never makes it into production because we completely missed it. We met too many assumptions that were incorrect. They're weak jump too fast to write the code and we didn't ask any clarifying questions. So, what we delivered missed the mark and now we have a few days to a couple of weeks to try to make something work to fix the things we need to fix. And so we, then start saying, well, that's going back on the technical backlog.
Well, that's going on the backlog while fix it on the next version. I've experienced that way too many times. It was frustrating. So finding that balance with web apis in particular because they are forever principle. Number five, apis are forever to think about. How do we A lightweight. Rapid API design first approach that will get us closer to the right Mark, prevent us from writing or designing an API. That's incorrectly or misaligned with what really is needed and
still get feedback quickly. So that we're not doing this totally in isolation in our own corners and it's too costly or too late to make those changes. So that's what the aligned Define design refine or addr processes all about. Yeah, so maybe you can go and explain He's a DDR process. It's a line defined design. Refine. The aligned is how do we ensure that? The API that we design is alignment?
With the outcomes that are desired by the developers are going to using the API by the end users that will be indirectly using that API maybe through a web or mobile app or anything else gaining alignment by starting with the job stories. Detailing those job stories out to the next level where we have step-by-step. We understand the stepwise process is involved to achieve the outcomes that we found in the job.
Stories. And then defining our API, by looking at those finding our API boundaries and the API operations were going to need independent of any one particular API style. So I had mentioned earlier that, you know, we can mix and match things, rest, and graph, qmg, our PC. So if we align first and foremost to make sure that all of our assumptions are dealt with that were not writing code, that has assumptions built into
that. Are incorrect or not designing an API, like why's it have incorrect assumptions when we understand what the needs are with the outcomes. Are we break those down to the next level, understand, step by step? What we need to do. Then the API design will evolve along with it. So we go through the Align phase. We going to Define where we turn that step-by-step process into apis that have operations to fulfill those step-by-step processes that produce those outcomes.
We found in the alliance phase and then we apply one or more apis. Styles to design the API. So we might apply a rest approach for the majority of our operations, maybe offer a graphql. All using the output of the Define phase an API profile that defines with the API supposed to do a high-level protocol agnostic. And in, once we designed it now, we have this high level design that we can quickly. Push into an open API, spec or graphql. Sdl, schema definition language or a gr.
PC IDL, interface definition language. We can capture it. And then socialize, we can use those artifacts to socialize it. We can generate mock implementations of our API using tools that can take in one of those definitions and create like an in-memory mock implementation of it, just so that you can kind of try it out without it really being completely coated up yet. Now allows us to refine the design. I go back and rework that. So the aligned Define design
refine is the four major phases. We go through, there's different steps outlined in the book that take you step-by-step. Up how to go through the Align phase and make sure that everybody is in alignment, from business product, owners to the tech teams. And then how to define the API, find those boundaries, find those operations, then, apply the apis Styles. You want to create the design and then to circle around gain feedback and do that in a rapid
fashion. So you spend a little bit more time up front, but you save a lot of time on the back end where it's more costly to change code and it can be very frustrating to have to take some well written code. Well, designed code. And make Um last minute changes in really mess it all up because we misunderstood something the beginning of tries to encourage communication throughout its
really applying the agile. Manifesto principles right into this design process without it being too front-loaded or too, heavy weight. So that's what the book outlines and that's what I really recommend. It allows us to have that design first approach so we can talk about the design. Look at how we would integrate with it. Make sure it's going to hit the mark and we haven't missed anything. And then we can go write our code and do what we do best there. In the different delivery
phases. So I also find this kind of approach very useful whenever you build both apis independently, and also starting from scratch, for example, so especially the Tema dispersed geographically, for example, different time zones, but you need to collaborate through the apis for those services. Having gone through all these aligned defined design even providing mocks testing for people to integrate with the AP art.
So it's speed up the development so that they can all do in parallel at certain point, you Because it in a staging or maybe some kind of testing phase where you can ensure that. Okay, actually, these two Services can talk to each other. Thanks for sharing this addr concept. I think it's really useful. Another thing. I pick up when you mention about this addr is you mentioned about API boundaries. What do you mean by API
boundaries? Is it something that is related to bounded contacts and all these DDD? It is here. So people that were looking at the add or processing techniques that we train on will recognize some of them from the DDD. Base what I wanted to do. And what I have tried to prevent is isolating those that are not familiar with DD or that are only practicing some of the techniques and preventing them from being able to take
advantage of this process. So it is designed to be able to incorporate many of the concepts of tdd. We use event storming and courage of in storming during the Align phase to try the surface things. If it's a domain that's a little bit newer.
For some people bringing in subject matter experts from the domain and having collaborative sessions to explore and understand it. It is really valuable, as well as what you mentioned, what the API boundaries, the idea is that I've seen a few apis over the years that may have hundreds and hundreds of operations. Those are just so unwieldy to
understand and get started with. So unless you just really surround all that, really large API with a lot of copious documentation that guides the reader to where they need to be. Unless you do that. Then it's really hard to get started is really not approachable to have to take on an API. That's really large. So the Is to bring in boundaries where we can segment parts of the API. That doesn't necessarily mean that you have more than one API product.
It means that you're finding different apis or different operations that are cohesive. So I go back to the fundamentals of software development, High cohesion. Loose coupling loose coupling says we don't want to tie into the internals of our systems into the implementation very much. Like encapsulation. We want to hide the internal details from the external API. We want to loosely well in just couple to the API itself not to the internal implementation details.
Hi could weejun is about having related functionality grouped together in the same code base, rather than scattered all over the code base, where it's hopping between module two module, what we call Spaghetti code or big ball of mud were things. You just all over the place and there's no Rhyme or Reason for it. All those techniques that it exists in PD are really beneficial whether the team is early in their DD Journey, whether they're not on a journey at all. Only. Just want to design.
Only great software or whether they're DDD the experts, taking the time to evaluate where those boundaries are for your apis will enable you to be more efficient and decompose a very largely scoped product or surface area of H operations into smaller, bounded areas that are more digestible and easier to manage and easier to decompose for delivery by the
API provider. So we bring out some of those DD Concepts in throughout the process, but if you're not practicing DDD, no need to be concerned. It doesn't require it. TD for you to be able to take advantage of it. Maybe for those people who are interested in the DDD if we can go just slightly deeper. He's a pi resource like the term resource from rest here correspond, very closely with the entities. They do. What we typically do though is we try to get teams to look at the Aggregates.
The Aggregates tend to be where the operations and commands and queries that you send in tend to Cluster around. They tend to focus more on transactional boundaries and the Nation of multiple entities together. So the resources typically represent the entities, but the apis tend to scope to the aggregate. So we use events storming as a great way to find that we can
lay out a particular process. Start with our events and the start to expand and find the commands that generate those events and then identify Aggregates and detail out the event storming process. Often times. Those Aggregates are really good hints of where API boundaries are at, we can look at what we've discovered.
And then reassess. So we can always make some changes so that we have better cohesion if we've identified multiple Aggregates, but maybe they belong together in a web API because we have to keep in mind a lot of web apis because they're going over a network must be more coarse grained than our DDD code bases that tend to be more granular and can operate
in process. So when you go to a distributed architecture where we have web apis, we don't want to have too much Network. Traffic won't be too chatty. So we have to start looking at how What are we interacting over the network, which is different architectural style, than how we might, architect a standalone code base, and process that may expose that API. And how we coded up internally
looking at DDD. We really want to focus first on the Aggregates and then allow that aggregate to help Drive the API operations. And then behind the scenes, we continue to apply d. D d, Ed smaller and smaller levels as we realized the API during delivery. That's why explaining all this and clarifying. How How does these two concepts actually help each other all aligned with each other? So thanks for sharing that maybe
the last point about API. So you kind of like design, your kind of like develop it, you kind of like put the documentation but how about the testing strategies? Because at the end of the day people just want to integrate with your EPA. First, of course, they will need to go through the testing strategies. Maybe you can elaborate some of the testing strategies that you commonly see throughout your Consulting.
Yeah. Yeah, one of the anti patterns that I see is that teams just In off with building apis because they're often times have a front end to them. They'll use their UI testing sweets, whether it's selenium or something like it to drive front end test Suites to build those and they'll assume that their apis tested because they tested
their friend. You are the problem with that is that often times our front-end application whether its web interface or mobile or whatnot, they're doing front and validation client, side validation. So a row Aeneas request. We'll never make it to the back end server because on our client is catchy and time. So you're testing the client that it's validating the input before it goes to the server, but you're not really getting a chance to test the server. So just relying strictly on UI
based testing to exercise. Our API is just not sufficient enough. It's a good start the integration from front and the back end making sure everything's working but it's not the complete story. So the other things that I recommend is one is looking at Contract testing. This can be automated. There's a lot of tools out there. Now we're starting to merge that you, if you capture your API contractor, you had scription and say, open API spec what you
speak of swagger. As you mentioned earlier, if we use that, there are some tools that will read that description which is machine readable and we'll turn it into a starting point for a suite of automated tests. There's some commercial tools like that is, if you open source, libraries and others that can help make sure that the API is doing what it says. It will do that. Your code matches. What the spec says. That's captured an opening Prospect.
The other thing you could do is integration or acceptance. Testing. I prefer to really just focus on acceptance testing which takes the job stories and the activity steps from our line, phase of the addr process and turns those into executable code. So those that are familiar with behavior, driven design bdd things, a cucumber Frameworks like that are probably familiar with the wind and and so one
kind of those types of formats. You can likewise take the work that you did upfront during the design the alive. Unfazed and turn that into acceptance tests that can verify that all your apis. When combined together can produce the outcomes that you initially defined in the job
stories during the Align phase. So you're taking all the work that you do upfront and that's going to in turn drive your testing to make sure that your API delivers on what you initially identified in the aligned phase is what the API needs to do. So doing acceptance, testing around our API in that way is really valuable.
Some teams will do it with you. Seeing tools, like you mentioned in like Postman and others that have collections that are built up that exercise each step and maybe script the transition from one call to the next pulling data along the way. So the output of one called turns into the input of the next call, so they can make those orchestrated calls and make sure that things end up in the final outcome state that we want, or they might write it by hand or some combination to do that.
So those are just kind of the high level ideas I have there but there's more that's going to detail about the book, but it's really important to make sure. Have a solid testing strategy. And if you do this design first approach, then you have all the artifacts. You need to help inform you of what kind of test you need to write. You just need the right them. So it really makes a lot of sense to build a pull all that together.
Thanks for sharing some and Tibetans like, you know, testing from just the UI. So I think here also it's applicable to apply the test pyramid principle right? Where you don't just have all the UI based testing, but you have lesser integration or acceptance test part of the you iPod. Thanks for sharing that so James. He's been really great to talk about all these apis. I learned a lot all these principles and the tools and also the concept that you mentioned. But like job stories, addr.
Unfortunately, we have to end this conversation. But before I end this conversation, normally I would ask all my guests to share the three technical leadership wisdom. So this is mainly for you to actually advise other people based on your experience or your knowledge. So what will be your three
technical leadership wisdom. Yeah. I think the first one and it's one that I try to really incorporate into my workshops is Be curious not only of new technology, but of our software development history. I think we tend to always strive to see the new and interesting and that shiny new things. But if we go backwards a little bit sometimes and look at what people have done in the past and learn from those that came before us.
There are a lot of lessons, we can learn that we can then apply, as we move forward with software development. There's a lot of wisdom out there and sometimes we just ignore it because we feel like, it's maybe two older or not relevant today, but a lot of it still is relevant even though Though, maybe the technologies have changed. We're doing were distributed computing than we used to.
There's a lot of history out there that I think people should be students of and so I try to incorporate that to my workshops a little bit as I go just so that people can gain that Insight if they haven't been exposed to it previously.
The second thing is to always evaluate new technologies and light of what we've learned from that history, oftentimes will see new things, come up development team puts them together and you open source project, put something together and they think to the first one to have ever done it in reality. Have it. In fact, when story is that, I've had a client that ended up re-implementing a message,
broker over HTTP. So they effectively got pretty close to AWS has sqs, and they didn't even know it. They didn't know those Technologies existed, you know, it's no fault of theirs. All of them on that team had never really had a need for message-oriented middleware or traditional message. Brokers think rabbitmq, things like that. They just never encountered it. And so they ended up re-implementing the same thing themselves, but it was a little less reliable.
So we I evaluated that and consider how we could learn from technology has been around, will be the best fit for them. The third thing I would say is passing what you learn to the Next Generation. I think there's been several studies that have indicated that a half-life of a developer in our industry tends to be around five years.
So that means that we're not really always doing a great job of helping the newer generation become more familiar, not only with our history and knowing how to evaluate new technologies, but just the wisdom that we've gained in learning how to code. So software development can be A very solitary role at times. We can put our headphones on and get into that zone and really have a good time. Writing code. That might be our passion and I enjoy that as well.
But it's important to step out and help others, and listen, and learn from others as well. So passing on what you learned and then learning from other people. That's the third one. Thanks also for spending your time. He had to pass on what you know to all those software developers who are still learning or maybe developing web apis. So, thanks games again, for people who want to To follow you online or get to know more about your work. Where can they find you?
My company website? You can reach me in find articles that I've written. Another insights is at launch. Any.com., That's Lau in ch a in why.com., That's my consultancy. I also have a newsletter called API developer weekly. So you go to API developer weekly.com. It's a weekly hand curated newsletter, where I focus on a lot of API related articles and non API articles whenever I find something interesting. Doing. So I just send it out once a week.
It'll keep you up to date on what's Happening and interesting developments case, studies other things to get written by other people. I share that out every week. So make sure to put that in the show notes. So, thanks again. James. I wish you good luck with all the consultancy and helping people to build a proper API. Thanks Henry. Appreciate it. Thank you for listening to this episode and for staying right till the end.
If you highly enjoyed, please share it with your friends and colleagues who you think would also benefit from listening to this episode. And if you're new to the podcast, make sure to subscribe and leave me your valuable review and feedback. It really, really helps me a lot in order to grow these podcasts better. You can also find the full show notes of this conversation on the episode page at technology. No, the death website.
Including the full transcript interesting quotes, and links to the resources and mentions from the conversation. And lastly make sure to subscribe to the show's mailing list on technology. No, the deaf to get notified for any future episodes. Stay tuned for the next technique Journal episode. And until then. Goodbye.