Karsten: Welcome to the Open Source Way. This is our podcast series,SAP's podcast series, about the difference that open source can be. And in each episode, we'll talkwith experts about open source and why they do it the open source way. I'm still Karsten Hohage and inthis episode, I'm going to talk to Simon Heimler, or Simon Heimler,about Open Resource Discovery and in the larger sense, that meanswe're talking about metadata, right? Hello, Simon. Simon: Hello, Karsten. Nice to be here.
Karsten: Nice to have you here. Thanks for being here. Now, Simon joined SAP in 2020, ifI’m correctly informed, and before that worked in many differentcompanies of different sizes also being a freelance web developer.
Usually he did, as he says, full-stackdevelopment with JavaScript, TypeScript and also down to softwarearchitecture doing a lot of work also in low-code/no-code platforms, andthen gradually always getting more into data, metadata related aspectsand into data- and metadata-driven development, which he says is awesome. Simon before we start into this beingawesome do you know the game "Simon Says"? Simon: I know now. Yes. Karsten: Okay, an Englishcountry game, Simon Says.
It kind of resembles our"Alle Vögel fliegen hoch", all birds fly up in the air. Somebody's giving commands and you haveto follow the command if they do it correctly and you don't have to follow thecommand if they don't do it correctly and correctly, is to start with “Simon says”. And interestingly, isn't that somehowjust like metadata: if the metadata are correct they're telling the other softwareartifacts what to do and how to behave just like Simon tells everyone what to do.
Now, uh, this is bullshit, right? Simon: No, it's actuallyquite a nice example. So, let’s take the simplest one. A file extension is metadata, right? Or the media type of a file. And if it’s not there, yourcomputer wouldn’t even know what program to open it with. So, without that, you need to makea manual informed choice and go hunting for the right decision. Karsten: The “open” command doesn'twork if it isn't preceded by “Simon says”, or a postceded by the extension. Right.
Now, but we're not here to talk about kidsgames or actually very little kids games. We are in an open source podcast. Uh, Simon, have you alwaysbeen an open source guy? Actually, I started open sourcing,I think 14 years ago, so that's where my GitHub commits start. And also on my free timeI do open source projects. I have several I work on. One of them best going is actuallya, browser extension to search bookmarks, tabs, and history.
It's like a Visual Studio command line, Simon: just for your browser. and that one is actually,quite used, I think, now. Karsten: And how's that called? Simon: search bookmarks, history. and tabs. So, if you look for that in thebrowser stores, you will find it. It's available for all the big browsers.
Karsten: Okay, if, you don't mind,we'll probably share it along with the podcast information sounds likea useful tool, if I understood you correctly, it's not like, I mean, I canas a standard search through my browser history, but this combines the searchin history tabs and whatever, right? Simon: Yeah right, and you can organizeyour bookmarks with tags and so on. Karsten: Okay. Sounds cool. I might even check it out if I understandhow to install it or plug it in, whatever.
But this is also not what we're hereto discuss, but that it's interesting. Of course, 14 years ago, it waspretty early as far as some other discussions I can compare to isconcerned with doing open source, our actual Topic of the day, though, isopen resource discovery, in short, ORD. can you explain what that is? Simon: Okay, yeah. So this is basically aprotocol or a standard. And this is always a little bitabstract because most people don't interact with this directly, right?
What it does is, it allows applicationsor services to self-describe, to expose the metadata to otherparties that are interested in. So for example, if I'm a developerand I want to integrate against an application or service, there'sa lot I need to know, right? So, what is standpoint,what APIs does it have? Where do I find the API documentation? Can I get all that in amachine-readable form? And even better, I might wantto have the full overview of all the systems in my landscape.
So, I just don't wantto ask a single system. I want to see what is available tome as a system landscape, right? And this is what thestandard allows you to do. So, it's about decentralized metadatadiscovery and putting out a big picture of what developers can do with their systems. Karsten: And that isnecessary exactly why? I mean, it's among other things aboutAPIs and APIs, shouldn't they be documented and declare themselves anyway? Simon: Yeah. And there are already standards out there.
So, for example if you build a rest API,you probably know open API and you might have an open API file describing theCPI, but now where would I find this? OpenAPI file, right? So do I need to go tosome documentation page? Uh, where have you stored it? Um, I don't know. And Open Resource Discovery is actuallyhelping with that particular problem. So it's let's say well-known entry point. It's like a index page of allthe available things, and it will include the OpenAPI file.
So, it doesn't replace that standard. It's basically helping you tofind it and to get it, right. So it's like a metadata crawler protocolfor everything you have to offer. Karsten: is OpenAPI required to workwith ORD or is that, was that just one example and ORD would abstract alsoover different underlying standards? Simon: Yeah, it's just one example. So, ORD supports a lot ofdifferent protocols, also events. We are also going intomore other personas.
Like we can describe businessobjects for the business users, which think more in that terms. We describe data products forpeople who come from data science. Maybe not developer personas,but actual data analytics people. And for all of that, we justhave some shared high-level information and that's it. So we don't go into details there. There's already standards to do that. Karsten: And uh, in the first place,that sounds like a case for ORD and then historically seen how did this come about?
And how did this come about thatthis is, as I take it, correct me if I'm wrong, an SAP-driven project? Simon: Yeah. So, at SAP, we have also historically alot of different products and acquisitions and services, and it's actually startedas a metadata publishing protocol. So, many people wouldactually know api.sap.com. This is the SAP API portal. You can find epis, events, butalso other types of resources here.
And we basically took that data modeland the idea and made it a protocol so we can automate the publishing andwe can also use it for other purposes. So now it. doesn't only power API Hub, it actuallypowers the BTP and the SAP BTP. It powers many use cases thatuse the data even at runtime. So, we basically took it from a staticpublishing protocol to something that works for many different use cases,and also works on real customer landscapes, not just a static view.
Karsten: Now, the two thingsyou've mentioned is, it started in what was known as API Hub, whichis now Business Accelerator Hub. It is now doing its job all over theBTP, but this is still within SAP and we're talking open source here. Since when is it open source? Simon: Yeah. That's a really good point. So, we open sourced it actuallya while ago, October, 2023. And the reason why we did it.
So yes, it's currently used atSAP, but we built the standard in a way that it can also be usedand understood by other people. So, and it may be SAP partners orcustomers first, but it's really built in a way that even companies that arenot affiliated with SAP can also use it. So, we carefully split our SAP concerns from the general concerns. Karsten: Okay. So, it is now open source, it isnot limited to SAP applications. Did you mention under whichlicense it's open source?
Simon: Uh, no, it's Apache 2 license. So, it's a permissive license,which also gives you, I think ease of mind when using it. Karsten: Okay. And then next thing, isit like truly open source? Everyone can contribute asthey like or does SAP have some special veto rights or whatever? Simon: Hmm. Yeah, that's a really interestingquestion, because right now SAP owns the standard, eventhough we made it open source. So it's open source,but not open governance.
And that's a reallyinteresting distinction, right? So, think of it like um,yeah, the history of OpenAPI. So, it started as Swagger which was ownedand developed by one company, right? And then at some point intime, it got moved into an open governance organization. We have many companiestogether worked on it, right? But with Open Resource Discovery,we are at the state where we only have SAP working on it right now. And so, that's why we can't put it intoan open governance organization yet.
But if you find the right partners,we are very open to do that. We are actually talking to companiesabout that possibility right now. Karsten: I think I've always had anotion somehow that there is a difference between open source and open governance. But I think this is the firstepisode where this is this clearly expressed in these terms, you know.
We've discussed this kind of thing overand over in different episodes, but this is the first time I think somebodysays open source versus open governance. Thanks for that. Uh, made me understand something more. So, from this being mainly SAP-drivenand being SAP-governed what would be the path from closed SAP-governedproject to an open governance project?
Simon: So first of all, you needcollaborators, you need people who want to work on this on equal groundand be open to collaboratively own and develop the standard, right? So if it's only one person,one company that doesn't work. And once you do that, you need tomake a decision what governance foundation you put it under. Maybe it's not technically needed,but I think every successful bigger open source project has at least somecommunity or governance guideline where this is put down into rules.
And for standards in particular, theyare usually donated to an open source foundation like Linux foundation,CNCF, Oasis – that's the typical candidates for such standards. Karsten: Okay. And then I guess it also depends on theresonance out there in the community. Right. uh, are you getting anyinterest in already so far? Simon: Yeah. So far, we weren't pushing it toomuch because we are still also working on it as SAP-internally,but we have a few customers and partners who expressed interest.
And we are actually in touchwith big other companies about potential for collaboration on it. Yes, but it's nothing I can committo or state concrete names here yet. Karsten: Okay, no concrete names,but when you say “big”, I think you mean really big, right? Simon: Yes. Really big. Yeah. Actually such standards are usually mostinteresting for the really big companies.
Yeah. Karsten: Okay, then everyone can goahead and guess the names if from an SAP perspective, we say reallybig, who we could possibly mean. Doesn't leave all that much,but we didn't say any names. Let's turn away from that… and, and, and turn back to open resource discoveriesORD’s capabilities. We talked a bit about it before, maybeyou want to elaborate a little more in the first place, it was doing interfaces. What else, did you say? Data products? Uh, you said systems even?
So what does a requester learn from ORD,about which kind of artifacts again? Simon: Yeah, I would say it really dependson the persona who needs to understand it. So, an architect, for example,might be most interested in what kinds of systems I have. So that's not exactly ORD territory,that's usually handled by service discovery systems or inventory systems. But they are interested in theirrelationships with each other.
and ORD can be used to expressdependencies between systems and how they relate to each other up to thepoint where you maybe even can make some deductions about data lineage andwhat is integrated with each other. So that's maybe the architect persona,the developer persona as well, right? The developers are usuallyinterested in interfaces. So, every integration against the systemshould really go up through defined contracts, and that's APIs and events.
And we also have a more genericcapability concept, which said you can use to express like I can domaster data integration about the following master data objects, right? So that's actually meant to be verygeneric and extensible because we don't know what kind of capabilities youwant to tell the world about, right? Yeah, and then the data products Iguess right now a hot topic and we are, this is still in better in ORDbecause we are figuring it out on the go, like many other companies right now.
And yeah, I guess that's themain things we have in there. But in general, we could sayevery metadata that is interested to outside consumers that theapplication or service knows best because it's the one implementingit, make sense to put into ORD. Karsten: And how do you steer apart thesedifferent interests when you said, uh, depends on the persona? Is there meta meta data or arethese different templates or how do I have to imagine that? Simon: it's all the same, uh, interface.
We just sit together with an SAP. We do have a governance. So, we sit together with thedifferent stakeholders, adopters, but also consumers, what they need. And we try to find an interface thatcovers the need of all the personas. And then, actually, what theyare interested in, they will do this in the aggregator or in acatalog UI by querying, right? So there might be views or queriesthat are typical for one, persona. Yeah. Karsten: Okay. So, so this is decided at, atcall time, basically, right?
Uh, you, don't provide already. This is interesting for onlyarchitects or something. Okay. Uh, cool. Um, by the way, uh, my metadata todaysays, uh, "Has cold equals yes". Uh, so if I sound a bitdifferent, uh, from the usual episodes, please excuse that. I hope my snorkeling is alwayson the other track while, um, it's Simon's turn to talk. And of course the benefit here is westill do these recordings remotely, uh, and that's why I don't have to worryabout Simon catching my virus here.
Overall, as you said, you were leadingdiscussions with different personas, what they need, uh, and then they haveto, on their side, basically put this into catalog interfaces or whatever– even within SAP, how do we ensure that ORD is correctly adopted becauseit doesn't seem to be that simple. Simon: Yeah, indeed. It's not. So, it's a lot ofinformation you can expose. And also we have guidelines atSAP how this is done correctly.
So, because our consumers want torely on some things like certain level of quality or completenessuh, correctness of the metadata. And the data product / data meshsphere, there's this interesting term federated computational governance. Um, and the data product data meshsphere, there's this interesting term federated computational governance. It sounds really complicated, butit's a really interesting concept. Let me break it down.
So federated means validationshould happen, or at least in our understanding, it shouldhappen very early at the sources. So we also have metadata validator, Whichevery LoB, every application or service can implement in their own CI/CD pipelineswhere they can check the metadata. And if there's any issue, they canfix it before they merge it, let's say into their main branch, right?
So, you can put it right to thedevelopers, where the code change happens, it can already be validated and it'sdecentralized, so we don't validate. Ok, so, sorry, we also validate centrally,but it's more like a quality gateway. So, we don't want. Issues to just appear once youpublish, that's really bad, right? That's the last point in time whenyou want to detect a mistake, when you publish the documentation, right? So you want to find it early.
And it's computational because we wantto automate as much of that as possible. And the metadata and the welldefined structure of the metadata already helps us with that. We have many SAP productstandards and guidelines. And through the metadata, we cansometimes detect if those guidelines have been met, and we do that. So, we have, we call it compliance levels. So we can say, yeah, thisresource, this implementation, matches a certain standard, right? And those standards can be multiple.
There could be higher standards,lower standards, data products, land. It's also sometimes calledthe medallion architecture so you have bronze, silver, gold. So, we have different standards,but we want to validate compliance automatically against it. And the governance aspect isthat the rules, how to do that, this is decided by a central teamtogether with the stakeholder. So, consumers have a say in thatbecause they're interested in those rules to be enforced, for example.
Karsten: And, uh, these experiences thatwe are making internally with ORD and the necessary quality measures that we'retaking and everything, are we also sharing that with the community or are we mostlyonly sharing the ORD definitions itself? Simon: Yeah. So right now, we have shared theOOD and with it, we already hint at this concept of compliance level. It's called policy level in ORD. We also shared some of theSAP rules because as an SAP customer, this is what you get.
This is what you can expect. And there is more to come. So we are thinking about open sourcingsome of our tech or tools to do this federated computational governance aswell, but it's not rocket science, so it's mostly an automation mindset anda governance mindset that drives it Karsten: Okay, and I guess much of thatwhat else do we share and how much do we collaborate on also depends on how thecommunity develops, how the mentioned big players act here and so on, right?
Simon: Right but so far we are mostlytalking about the specifications because sharing that across companieshas an interoperability benefit, right? So, if many companies share the samemetadata standards, you can build a shared ecosystem You get network effects. You might be able to build marketplacesbased on shared standards, right? Think of it like USB-Cstandardization, right?
Once we've got a common standard here,Suddenly we have a whole ecosystem of USB devices and gadgets and soon, and they work with everything. So that's really nice. That's what shared standards do. Karsten: Okay, then, as we'retalking about shared standards, and as I'm a simple mind that isvery often driven by just, small similarities, a short question here. I've encountered So many acronymsbeginning with O's these days. Uh, open, of course. But let me ask you anyway.
Are there any similarities,relationships between ORD, which we're talking about right now, so open resource discovery, andsay, for instance, OCM, ODM. OCM is what again? Open Component Model. And ODM is open data…? Simon: One domain model. Karsten: Uh, exactly. Yes. uh, see, I get mixed up. All these O's. Simon: Yeah. But like with those acronyms,it's very different things. But I think some of the comparisonsthere are connections or relations. So Open Component Modelis also standard by SAP.
And open standard but it caresabout a whole different perspective. So with it, we also describedapplications and services, but we describe their software part, right? So, it's not what consumersare interested in. It's what people who monitor anddeploy the apps are interested in. Yeah. So, I would say they are verycomplimentary, but solve a very different problem from a very different end user.
Karsten: Okay. So, the idea having, a standardthat mostly provides information about software artifacts is similar. But the approaches, what do I need,the type of information from ORD, or from OCM, which I think is forthe delivery pipeline mostly, like lifecycle aspects and so on, isn't it? They're just different scenarios. What, what I'm, okay. Okay. Simon: But it's probablythe same mindset here.
So, if you want to standardize it sowe can share that we are also open and transparent in how this works. Karsten: Let's put it this way: the samemindset is expressed by the O in the beginning, and, then the following letterssay, no, it's really something different. Simon: Yeah, probably yeah. And ODM is another one. So ODM did change its concept over time. With ODM 2. 0. there has been a blog article aboutthat on the SAP blogs and actually now part of the new ODM is now part of ORD.
So I talked about describingbusiness object in ORD. Yeah. ODM is one such scenario. So we can use ORD to publish thevocabulary of ODM and then create all the nice relationships like this ODM entity,business partner, what APIs and events are there, so I can actually interact with? Is there a data productabout business partners?
So it helps business users thatspeak the domain language and not a technical language to find thingsand to find the relationships between the things, even across SAP products. So ODM is our shared language,I would say, shared vocabulary. Karsten: Okay. 'ODM is your shared language'. When you started, I was about toask, would that mean ODM is, has become a subset of ORD or is therejust a partial overlap, or how would one hierarchically sort that? It's its own thing.
So it is more than just the taxonomy. It also comes with an ID concept. uh, it's about how to join data, uh,easily and correctly, but, describing the ODM taxonomy that happens now via ORD... Karsten: mmh hmm Simon: ...and it's just,uh, it's basically published as yet another ORD content.
Karsten: Okay. Okay. If people would like to read up moredetails on all that, because I feel if I try to venture further into understandingthe differences here, I may start asking really stupid questions and I thinkanyone who has a use case, a touch point, or whatever with things like ORD, ODM,OCM, et cetera, will now have a first understanding of what ORD is all about. Where do these people with that firstunderstanding go to find out more?
Simon: Yeah, the best placewould be the ORD GitHub page. So, it's deployed as a GitHub page. It's actually a nice page where you willfind some human readable documentation. There's also a small videowhere, it's explained and an introduction, which is like a primer. So, it will walk you to the basics. There's also an ORD reference application,which implements the ORD protocol so you can try it out, how it looks like,how it works, if you implemented it.
So that's where I wouldrecommend people to go first. Karsten: All right GitHub soundsstraightforward enough, which leads us and if there are other sources uh,that we think of later, we'll of course again, publish them with the podcast'smetadata information or the text page, uh, that we have in the internet. Finally, as we are reaching ourtarget time that we usually have, what are your two, three, possibly fourthings that you would want everyone to remember from this episode?
Some people call the key takeaways. Simon: Mm-Hmm. Okay. So maybe one is that ORD isprobably a behind the scenes technology for most developers. So I don't expect that, um, it's relevant to everyone, right? So ideally this is handledby your framework or someone implemented it for you. Once you have the metadata and you makeit accessible, it's the developers, again, who benefit from it right?
And I think it's very good to have,aggregated documentation, aggregated machine-readable metadata that helpsdevelopers that helps uh, uh, I know it's no AI, everything, but it helpsalso train or guide AIs better. It helps let's say a lot ofuse cases with automation. So that's basically anadvantage we get out and ORD is, yeah, let's say to that end. Karsten: Okay, here's one, makes sense. Another. Simon: Yeah. Okay. So, from my experience, creatingsuch standards is really a long game.
So, you need to, yeah, the standardalready existed for quite some time. Even before I joined SAP, therewas already something in the works. Until this is adopted, it takes a while,and then there's a point of I would say. A tipping point where people adopted,if there's enough consumers and providers that speak the same standard,then there's a point where suddenly the value gets exponential, right? So, this is the platform effect.
So it takes a while, but onceit gets rolling it can deliver really great benefits to everyone. But you need get past that pointfirst, and that takes a while. Karsten: Alright. Yeah, I guess, we'll first have to startseeing about that because you said it. opens first only in 2023, right? Simon: Yeah. And I think it will take somemore time to get wider adoption Karsten: Yeah. I mean, we have just started basicallyto make it an open standard, right? So we'll give it some time there. all right.
thanks Simon. Thanks for being here. Simon: Yeah. Thanks for having me, Karsten Sonice to be part of the podcast. Karsten: And everybody else out there. Thanks for listeningto the Open Source Way. If you enjoyed this episode, share it. Don't miss the next one. We used to say we publish everylast Wednesday of the month but we have become a bit random here. I hope you have us subscribed, so youget notified when there is a new episode
out, you'll find us at on sap.compodcasts and in most places where you find your other podcasts, like Apple,Spotify, but of course, also in all those open source podcast clients out there. Thanks again and bye bye. Bye Simon. Simon: Bye