Welcome to
"The Open Source Way". This is our podcast series, SAP's podcast series, about the difference that open source can be. And in each episode, we'll talk with experts about open source and why they do it the open-source way. I'm your host, Karsten Hohage, and in this episode, I'm going to talk to Max Mehl from Deutsche Bahn - so the German railway system - and Sebastian Wolf from SAP about the concept of Software Bills of Materials or SBOMs. Hey, Max and Sebastian. Nice to have you here.
Yeah,
thank you. It's a pleasure.
Hey,
Karsten. Nice to see you again.
Yep, it's
again with Sebastian. If you go through past episodes, you will find him at least twice, I think, or three times before. Max is with, let's say, something like the Open Source Program Office at Deutsche Bahn. As I said, German railways, or rather the DB Systel. That's the IT and communications part of Deutsche Bahn, I assume. And Sebastian is with the Open Source Program Office at SAP, so that sounds pretty similar.
The difference between the two is that Sebastian is a developer by trade, or at least by history, and Max by degree is a political scientist. Max, unplanned question: How did you end up at the IT department of Deutsche Bahn as a political scientist?
That's a
good question, and I get that asked very often. Actually, when I decided for what I want to study, I had two main interests. Which one was informatics and the other one was political things, basically. And I was wondering which is the one that I can train myself in, and which one is harder to do. And so, I decided to go to the university and study political science.
But actually, during an internship that I had to make during the course, I, yeah, got to see, got to know the Free Software Foundation Europe, where I was also later employed . Made my internship there, and was also very interested in technical things, anyway. So, this is how I got into the open-source free software world, and I stick there. And I find it fascinating that the overlap of policy making, political things, and technology is becoming larger and larger.
Good for
you, good for you. At least I think that. I'm totally tempted to now take a sharp turn and have 45 minutes of political discussions here. But I don't think either of our companies will like that. So, let's maybe take a look at, you guys know each other from before, as far as I know. You met at, what, the Bitkom Open Source Forum, and you also talked at an SAP event, Max. Is that right?
Yes,
that's right. Sebastian and I, at least first in reality, met at the Bitkom Open Source Forum last year, I guess . And but also virtually I think beforehand. I think all things about open-source licensing, the reuse initiative and other topics.
That's
right.
If I
remember correctly.
That's
absolutely right. It was really a pleasure to meet you in person, especially also after those long Covid times, yeah.
Absolutely,
yeah.
Okay,
but we're not really here to discuss the history of your long friendship, or however we want to call this. Everyone probably knows what a BOM is, a Bill of Materials. So, that's the list of things that are contained in something. Everyone at least has an idea what a Software Bill of Materials might be. So, Max, what did you exactly point out at this SAP OSPOlogy event? Why we have to worry about, or think about SBOMs more?
Yeah
basically, what you said was part of my job, that I was given at this event. I was asked by the organizers to give a short introduction into SBOM . And the background is that we aimed for a baseline understanding of SBOM, because it was part of subsequent discussions that we had on this day, be it on the... for political topics or also some technical ones. So, it was there to avoid larger knowledge gaps and also misunderstandings.
But apart from just the basic introduction to SBOM, that you basically just also gave here, Karsten. I was saying to the people that SBOMs are basically just a file with a defined format. And unlike other advertisements that you can see around, SBOMs are not a tool, not a solution, not a product that you can buy, not a project that you can just install, and definitely not a universal remedy for all problems that you might have when managing security or open source.
SBOMs, in my opinion, are only a means to an end . And it very much depends on the exact usage scenario that you depend on, and also the implementation, and very individual use cases and risk assessments that you can make as an organization or also individual. And this is also where the largest challenge
is. I talked about some stumbling blocks, and I think the largest one is the exact implementation of SBOM generation and also consumption into the very unique software development life cycles that an organization might have. And my final call to action to the audience was to not wait for the perfect solution to cover all the possible edge-cases and use cases that you might have. But instead get started with it.
Talking with other parts of the company, with the engineers, with everyone affected or integrated into the supply chain that you have as an organization. And yeah, get started and get real life experiences.
Okay,
I see. But what, again, is the particular relevance to open source? I mean, Sebastian, we do have a system, the PPMS, that takes care of things like that internally at SAP. What's the specific relevance to open source?
Yeah
so, open source is pretty much everywhere when it comes to SBOM handling. And there are two main dimensions to it, right? The first and probably most important one is that there are a lot of open-source component usages, as we also see in this internal system. So, and it's probably not only the case at SAP but across the whole industry, virtually all software products either contain open-source software these days or touch it to a certain extent.
So, we need to handle these open-source software component usages. And why is that the case? That's pretty simple. Virtually all open-source software components come with certain legal obligations. So, it's not a, let's say, optional thing, but it's a mandatory thing from a legal perspective, by law. So, we need, for example, to give credit to the original authors, we need to check which license obligations we have, what we can do with the software, and so on and so
on. So, to be able to fulfill all of these requirements and to manage them properly you need to get a full list of components including, and that's very important, including all the metadata. And this pretty much is this SBOM. So, it's both a combination between commercial and open-source components usages. The second dimension is, of course, that you can use open-source tools and services to create an SBOM, to do this management of this metadata.
So, there's for example, this Open Source Review Toolkit, as one of the most prominent examples probably. But of course, there are a lot of other ones in the community that can be used from the smaller to the bigger ones. And yeah, that's also used to fulfill all these tasks from a legal perspective.
And if I
may add to that, I think it's even beyond just legal requirements, or requirements caused by the open-source licenses. I think SBOMs also help with improving open-source management in general or support it. For instance, one use case is, if we as an organization had a common standard and place where to put SBOMs, and how to generate it, and also how to ingest it; we could have a really good oversight in, basically, how open source is used in this organization
. And that could help answer questions like: What is actually the most used open-source component, or dependency, that I have in my whole supply chain of the whole organization? Or I could answer questions like: Which technologies are most relevant for me? Like, where do I have to train my developers in? Where do I have to invest, perhaps? So in general, it can help allocate resources
. That could also be open-source sponsoring, or investment into technologies, or I want that my developers contribute their time in meaningful projects and not those only for edge cases. So in general, SBOMs are a tool to help with open-source management in general.
Now with
that last statement you took it a bit also to the side of being the basis of analytics. But is it not still mainly a part of, let's say, as we've had another episode that was called like that, building trust in the software supply chain because everything is properly declared?
Yeah
so, that's certainly one aspect, I would say . So, but there's much more to it. You can use it, of course, to manage your whole software supply chain, as we do it with all this metadata inside of SAP already since decades, I would say. But it's, as I said, also to manage license compliance, also when it comes to export control restrictions. So, there are additional metadata when it comes to cryptography and similar things that are under a certain jurisdiction not allowed to export.
And many, many things more. So yes, it's one aspect, but we certainly go beyond that in several areas.
Okay,
understood. But don't we, for all of this that we talked about so far, don't we have, or doesn't the world have SPDX and CycloneDX for that?
You're
right. Like SPDX and CycloneDX are basically the most prominent standards for SBOMs in general. But, as I said in the beginning, SBOMs basically are just a file, a well-defined file with a format. So, these are machine readable, in part also human readable, and yeah, but in the end, they are also only files and they come with some reasoning behind it, and of course a lot of discussions
. Like for instance, which information is actually important for managing the software supply chain, as Sebastian said, but also for general open-source management. But, as I also said in the beginning, I think the main challenge here is, how organizations can create SBOMs for all the software that they develop and also ingest from externally
. And how to integrate it into the software development life cycles, how do developers work with it, and how do I store then SBOMs, and how do I analyze them, and with which questions in the back of my mind? And both standards don't really answer these very hard questions . Probably because they're very individual and depending on the unique organization, and on the unique questions, and the risk assessments, and challenges that an organization faces.
Yeah,
maybe I could add as well a bit, that there are also additional challenges inside of each company that are not necessarily already in the standards. We had another podcast episode about Software Bill of Delivery . That comes later in the software life cycle, of course. And that's of course, also an important aspect in the company. How do I actually deliver my software, that has all these, let's say, different components? So, I have my freight papers, if you might call it like that.
How do I bring that then to the customer ? And then we need to have concepts like products, services, or other components that I also need to manage inside my company. And that highly depends, of course, on the business processes inside of this company. So yeah, the standards themselves are extensible. There are also developments that have targets to include such concepts that are necessary in the industry. So, the standardization is planned or ongoing.
But of course, we still have a long way to go here. And that's part of the challenge every one of us needs to solve in the different companies, of course.
Okay,
maybe two things to clarify quickly. Max, if I understood you right, we could actually do everything with SPDX or CycloneDX, but the question is how exactly? And does everybody do it the same way, or at least in a way that could communicate with each other, right?
Yes,
that's right. Perhaps you cannot cover every edge case that you might think of, but I think the basic functionality is there to manage the risk and the topics that Sebastian and I have been talking about. But yeah, in general also one of the arguments that I often bring in when I talk to people is when I say: Well, there are so many fields that I have to fill with these SBOMs, about so many different metadata. And my main argument is to not focus on filling all the possible fields that you
need. But instead focus on the minimum requirements that you need in order to answer the questions that you would like to have answered, or the risk, or the opportunities that you've identified. And then, of course, what you said already, it's not only important that I generate this information, but that if I have a supplier, for instance, that I also asked these suppliers to give me this information properly.
And then of course, the whole process of, did they actually fill in this information correctly starts. But I think this is the main challenge for an organization, especially for larger ones. What do I actually want to know and need to know and what is optional in the end? Because perhaps it's down the road, or it's even unanswerable in the long run.
Okay.
And other return questions to Sebastian. You mentioned the Software Bill of Delivery as opposed to the Software Bill of Materials. Would this SBOD rather be a part of an SBOM, or vice versa, or neither? Or if I understood you correctly, first you have an SBOM and further down the process you have a Software Bill of Delivery ? Or how is that?
Yeah, it's
more like this. Of course, there are some kind of interdependencies between both concepts and, yeah, Software Bill of Delivery is also rather new, I would say. But, as I already mentioned, it's probably simply a little later in the software life cycle. So, if we introduce these metaphors again . Software Bill of Material is rather: How do I package my piece of software or service? What's included ? So, the freight papers in the end. So, what's in the lorry and so on.
And especially these days when you need to have services running and you need to be able to reinstantiate them properly in the exact same status as they were before, you need to have also some kind of an operating manual, if you might take that . How to deploy, how to deliver them in the environment you want to run them. And that's where this comes into play. And of course, for all the details I would recommend then the other podcast episode, yeah, about this topic.
All right.
Okay, then maybe let's leave it at SBOM is freight papers, as you just put it yourself, right? Now, last time I mention Software Bill of Deliveries . I know there is actually a project that tries to establish a standard there. How about SBOM? Are we talking about a specific initiative here, an open-source project with a GitHub repo or something? Or is this just a discussion that we need to lead?
Yeah, we
certainly need to lead this discussion. And there are also certainly a lot of projects going on as I mentioned. One aspect is this open-source solutions that address the challenges, but in essence, it's simply that legal requirements already came up because of the licenses which I mentioned. Or they will come up much, much more in the near future for security reasons, for auditability reasons.
And security is actually a big, big driver for these new regulations that come up from governments in the end. And the United States and the European Union are pretty much at the forefront here. There's this so-called US executive order on cybersecurity, which mandates the availability of SBOMs for software products that you want to introduce in the market. So, we need to have them. And the same is planned for the European Union.
There's this so-called Cyber Resilience Act everybody has with this cyber. So yeah, but it's currently being developed. And yeah, it seems to be that it will be final before the elections next year. And it also contains that we need to get our software bill of material right. We need to get the information to secure the software supply chain. That's the main aspect of it. And yeah, that drives also community engagement. You said specific initiatives.
We have pretty new open-source foundation called the Open Source Security Foundation. Within this foundation, there's the so-called SBOM Everywhere initiative because it's part of the software, or one aspect is the secure software supply chain. And there are also a lot of discussions among companies, for example, especially also in Germany, in associations like the Bitkom as we met Max,
right? But of course, many things are also happening decentrally in the affected companies because we had that the problems need to be solved and they potentially need to be solved pretty soon as we're facing these legal requirements, not only from the licenses but now also from governments and supernatural institutions.
Yeah.
And I think you mentioned Sebastian already a number of organizations or foundations where Aspen is the main topic. And I think there are many other fora where SBOM also plays a significant role, but which are quite focused on a specific community or interest.
And also, SBOMs are, they're discussed very specifically for this focus of the community, like, for instance, a security related community would rather speak about SBOMs for security reasons, of course, to answer questions there with a managing risk in the supply chain. But a community focused more on open-source licensing and best practices for OSPOs will see other use cases in SBOMs and talk about it perhaps also a little bit differently.
Rather than the space about the open-source opportunities, like how can we better manage open source and direct our developers and contributions to technologies that are interesting to this organization? And this makes it also a little bit difficult for the whole discussions to be focused and to be streamlined into the further development of SBOMs, because there are so many different aspects to it.
Little
side note there. I think I just noticed that you are not coming from an engineering background from university, Max. You actually said Fora, where everybody else would say forums. Anyway, just happen to notice.
These are
the lawyer groups in which I'm part of.
Oh, okay.
I see, I see.
Yeah, a
very special community, to put it bluntly.
Apart from
that, I think I got an understanding already that there is a need for discussion, as Max, you put it before, we probably need to agree on a minimal set of what SBOMs should contain, to which everyone should then hopefully comply. What else, maybe Sebastian, is the need, are the requirements, the shortcomings of current approaches.
Yeah. So,
these specifications are of course under active development. So, if we mention SPDX or CycloneDX and they are extensible and of course let's say then a moving target, if we say they can't solve something they might solve it in the near future. But as Max already said, the challenges are not only in the specifications themselves as in many, many other cases. It's not technical issues. We have problems or challenges to solve on an organizational scale.
So, structures in companies that have existed for years or even decades, yeah, they suddenly need to take care of new requirements. And that's always an interesting thing to see, especially for all of us, which we are working in bigger companies, so to say. So that affects software procurement, IT, legal software delivery and many, many more. And in all these areas, more things need to be documented because they need to be audited.
And as all companies are different, there is no silver bullet to that. There are maybe best practices, as Max mentioned, and you can exchange them. But finally, the solutions need to be tailored and need to be adapted to the company culture and the organizational structure based on these blueprints, best practices and how you may call it. So yeah, that's, uh, as usual in the software industry. Yeah. The coding, when you have all the things clear, simple.
Maybe the hard work is before and afterwards, I would say sometimes.
I think
Sebastian said everything that I would have said, and we already have. He mentioned all these different interests and also before, and I think that that's a big challenge for these specifications for SPDX and CycloneDX, that there are so many specific needs that the stakeholders would like to implement in, in SBOMs and the specifications.
And I think it's very important to weigh between these interests and to avoid a further bloating, and also breaking the existing standards and workflows that have been established to keep the core intact of these specifications and these processes is already important enough, because companies that implement SBOMs into their processes already struggle with implementing just the core of SBOMs, and not all these extra features that are implemented.
On the other hand, of course, it's important to think further in going more into the direction of this SBOD, the deployment, the reproducibility and all these different cases that you can have, and not only SBOM , but also hardware and operations and so on. So it's very difficult. And I don't have the silver bullet here. And it just shows how SBOMs are such a great tool to tackle many different challenges and problems that we have.
Okay. In
summary: Great tool, but many differences challenging. What's the designated place to discuss? Is there one yet?
Well,
there are some. There are the beginnings, and we also need to see how things evolve here a bit. So, in the end, I would say, have a look at different foundations, such as the Open Source Security Foundation and the Security Tooling Working Group. And we discussed that also in events like the OSPOlogy event. So, just organize, discuss about it, either virtually or in person . Because many, many people have ideas how to solve a certain thing.
And then as, also in the open-source world, if many people work together, things will converge and we will have a better solution and overview in the end than we had if we did it just ourselves, I would say.
Okay and
do you think that will be enough? Or would someone have to go in the lead and say, here's the place to agree on an SBOM standard now.
Well,
it also will converge a bit, I would say, in the end. So, we have this interesting situation that there are at least two major technical standards in the end, so SPDX and CycloneDX . And we've seen that several times in the industry for various problems and solutions. And yeah, it's hard to foresee if there needs to be one, or if there will be one. Mm don't know. Max, what do you think?
It's also
very hard for me. We discussed that very long. Also internally, which is the standard that we would like to adopt primarily. And I think the feature set is a little bit different, but also the working mode of the communities is perhaps a little bit different. Like I personally know more about the community of SPDX, and I value it because it's very open and transparent and people can just join. I'm not sure about CycloneDX, to be honest, and I know that a lot of development is going
on. Like for instance, SPDX is working on a new major version that changes a lot. It's still in the process, but this is also an argument that I would make as go upstream. Do not only complain about shortcomings of one or both of these standards, but actually go upstream and contribute. And I very much liked what Sebastian said about the open-source spirit here. That also applies to these standards. And these specifications, of course.
Discuss in smaller circles like these OSPOlogy events or in Bitkom or also security foundations. But in the end, it has to go directly upstream, and we have to learn from real life experiences that we made in our organizations to adapt to the actual needs of these organizations, and of the users and implementers. Which standard will win, or whether we actually only have will only have one standard or, in the end, three or so I don't know. I really don't
know. It really much depends on how organizations play with it. And in the end, the more practical approach.
Okay,
Max, you mentioned something about internal discussions, so let's take it away from the whole world there for a second. Or maybe Sebastian first. What are we doing at SAP to tackle the SBOM needs?
Y
eah, we have a long history of SBOM management at SAP - I don't know if they called it SBOM back in the day when they started it - because we need to solve challenges like license compliance, export control, security vulnerability management and so on. And there are several proprietary internal solutions to tackle them. But we, of course, also use open-source tools like Eclipse Software 360. And yeah, this diversity, heterogeneity create also issues.
And we also tackle that by, let's say, creating a unified solution. And there's also a nice video about it, maybe we add it later to the notes. So, that's a different story. And yeah, all this is part of the challenges and the solutions we have at SAP. So, we are heavily working on that in several teams and in areas. And I'm curious how things will go in the next months and years to come, especially with the Cyber Resilience Act and the US Executive Order.
And
I am also curious how that is at Deutsche Bahn, Max.
Yeah,
that's a good question. And first of all, let me again say that I'm with DB Systel. So, I cannot speak for whole Deutsche Bahn. And this also shows a challenge that we have in the organization that we traditionally are a little bit federated, sometimes for very good reasons, and it has its upsides. But of course, with the standardization, it's of course a little bit trickier than within DB Systel.
Still, we of course, the main software part of Deutsche Bahn where a lot of software is being made so SBOMs here is very well managed. I would say we have an interdisciplinary team of people, and this is what I see as a big advantage that we include security engineers, open-source specialists, project managers and many other stakeholders.
If we have a look at the whole Deutsche Bahn, it's interesting to learn that, of course, there are other parts of the company that are more concerned about hardware. For instance, I have a colleague. He builds the next generation of high-speed trains in Germany and he's of course interested in SBOMs, but also in hardware BOMs and in operational BOMs and other parts.
So, he wants to have a very good oversight of which screws and which elements and under which configurations things are running in this new generation of trains. And here it becomes very tricky. And of course, also very interesting, because we have to think broader than just software and think about processes, workflows, but also tools that can manage perhaps all these use cases if that's possible. So, here's an extra challenge that we
have. But yeah, it's an ongoing journey and we are definitely not at the end of this journey.
Okay, that
makes me hope, by the way, that it's not so much a challenge - the difference between hardware BOMs and software BOMs - but that one can learn from the other and vice versa, maybe. But let's not get into this, because that's more part of the discussions that we're discussing about . The needed ones about SBOMs, I guess . Now that we've taken a quick look inside SAP and Deutsche Bahn, just in a quick summary, either of you: What's the relevance in particular to open source
again? Why are we discussing this in this podcast?
Yeah well,
to sum it up: We all use open-source software components in various products and services. So, we need to analyze, declare these components with respect to legal requirements. And this makes SBOM a part of managing any software component on solution. May it be completely open source or combination when we use open-source components in commercial solutions. So, it's in everyone's interest to have standards, blueprints available, how to handle these SBOMs. And that's where we
are. And why is it important to have a look at open source in the area of SBOM management and creation and stuff.
And with
this topic that we're discussing today I have the feeling that the usual before last question is more important than ever. Where would you want people who have an opinion about SBOMs and who are willing to discuss, where should they go? Where should they join the effort, the discussion?
Yeah
so, I already mentioned the Open Source Security Foundation. That's one place to be, I would say. But if you want to learn a bit, have a look at the websites of both SPDX and CycloneDX. They have a very decent overview, what's all about, about the nitty gritty details, and also learning offerings. So, from the very beginning to detailed questions. So yeah, that's the two things I would recommend.
Okay. And
apart from join the discussion there, what are your three to four main things - your key takeaways - that you want everyone to remember? If you want, you can just take turns. Maybe Max you start.
What I
already said in the beginning, SBOMs themselves are not a solution to anything. They are a means to an end, and an organization has to know which questions, and which challenges it faces and what it wants to have answered. And SBOM is not a universal tool or a cure for all these kinds of issues, it requires a lot of thinking of processes. And on the other hand, SBOMs are very diverse, and probably especially here now, at this stage, we'll never be fully complete or correct. So, we have to know
that. But SBOMs are bringing us a little bit closer to know the truth about the supply chain and to be better at managing open source and more than ever, I would say talk to people. This is the open-source spirit. Include people that have interest in SBOMs and that need to be involved in this process because it's so
big. Talk to the engineers, talk to different parts of the company in the supply chain, talk to other parts of the company who have more interest in not only software, but perhaps other things that have to be in an inventory in some way, and of course, contribute your experiences in smaller discussions, but also upstream with SPDX, CycloneDX and other specifications.
Okay. How
about yours, Sebastian?
Yeah, try
to keep it short . SBOMs will become, if they are not already, a vital part of the software development life cycle. These new regulations in many jurisdictions - US, European Union - make the proper generation and management of SBOMs simply mandatory for software vendors. And last but not least, of course open source plays a vital role in both the generation and management of SBOMs
. Because open source is in the freight papers of virtually all software products, and there are a lot of solutions for SBOM generation and management that are open source.
Oh my God,
you've just said the evil word: mandatory.
Yeah.
Anyway.
No, that's, it's good that you mentioned that. And with that, thank you very much, Max. Thank you very much, Sebastian, for being our guest today. It was nice to have you both here.
Yeah.
Thank you.
Yeah.
Thank you for the invitation, Karsten.
Thank you
very much for having us. Thanks.
It's been
a pleasure.
You are
more than welcome. And thank you out there for listening to The Open Source Way. If you enjoyed this episode, please share it. Don't miss our next episode. We usually publish every last Wednesday of the month. But we may, in December, publish early on the 20th because 27th would fall into this most lazy time of the year. In any case, you will find us on openSAP and in most of those places that you find your other podcasts
. Be that the mainstream ones Apple, Spotify, or the open-source podcast clients, et cetera. Thanks again and bye bye.
Bye bye.
Bye bye.
Thank you.