Hello and welcome to another wonderful episode of the VCD Roundtable. This is episode 48, and I atleast have full control of my computer, microphone, and camera. Unlike some other people (in the chat), I just actually solved it on my own. So with this episode, we are going to look into Orchestration and Automation, which is at least when it comes to Cloud Director, far more a topic for Matthias. So before we get to Matthias, Toby, any opening words? No, absolutely not from my side.
Hello and welcome to episode 48. Yeah, let's hand it over to Matthias and listen what he is referring to. Hi guys, this is Matthias. Yeah, I'm not in control of my mic anymore, because my MacBook Pro refuses to accept the microphone I configured. It insists of using my iPhone for whatever reason. But yeah, nothing I can automate. Cloud Director Orchestrator and Automation. Automation has multiple approaches. For example, we use automation to onboard new customers.
Onboard new customers, for example, is not like a... We're not automating it to have it more or to have it onboarded faster. So from my point of view, Automation is not always to speed things up. On the other hand, a big part of Automation is to have configuration consistency and there's standardized implementation of something. We all are human beings, so we are pretty aware that you just fat finger something.
Maybe you select the wrong allocation model for the OVDC and then you configure everything. And then it's like, "Oh, I selected the wrong one." So delete everything. So that's a big part of Automation as well, to have standardized configuration. I don't know how many guys of you have already played with Automation on Cloud Director. So we have two main pillars to automate Cloud Director.
First, we can just use the APIs and build a workflow to do certain steps, one after the other, to achieve a goal. On the other hand, we can use Cloud Director Event Broker Services to trigger based on certain events and external systems to kick off an automation process. So these are the two main pillars we have. From my point of view, and I have a big grin on my face because Yves and Toby both told me, "Orchestrator is dead" years ago. It's still there.
And it will be there in the future and forever. So I'm a big "Orchestrator" fan. On the other hand, I have to admit there is a great PowerCLI implementation or a cmdlet for the Cloud Director API as well. So everybody, feel free to use PowerCLI as well. I'm more the "Orchestrator" guy, but that's just my personal opinion. Most important thing is to automate stuff. So PowerCLI and Orchestrator can both be used to access the Cloud Director API and automate things.
I still stick with the example: tenant onboarding. Why am I using this example? Because tenant onboarding includes many, many steps. So first, guys, what do we need? We need to select the PVDC. We need to create an OVDC, select an allocation model, choose resources. We deploy an Edge gateway. We need an OVDC network. We need IP space allocation, maybe some default firewall rules in the Edge gateway. Some users create things like this.
So a ton of stuff which needs to be done to onboard a new tenant. That can be done in an automated fashion. So all these steps I've mentioned can either be built in PowerCLI or Orchestrator. Before we dig more into that topic, I need to make you guys aware that Cloud Director currently offers two APIs. We still have the old standard REST-based API. We still have the old standard REST API. And the new one is called the Cloud API. I have to admit, the Cloud API is the new stuff, its cooler.
I'm refusing to say the better API. It's an easier API compared to the standard one. But the Cloud API still misses a lot of important stuff. Everything about automating and managing virtual machines and vApps is still not in the Cloud API. And I don't know when it will be part of the Cloud API. So what we need to think, if we cover automation, we have the two main aspects I think we're going to cover is PowerCLI and Orchestrator.
Power CLI is shipped with its cmdlets for Cloud Director, and they can be used if you need to implement something and you don't have a cmdlet, use PowerShell to implement the REST API call. And on the Orchestrator side, it's the exact same thing. So you could implement the plugin in Orchestrator and use the plugin. Honestly, it is nice, though. I have to admit, it's helpful. It's nice. You can solve a few very basic tasks really fast with the plugin.
It has an implicit authentication, which you can use in JavaScript code. So that's pretty nice and straightforward. But I honestly tend to implement all the Cloud Director calls via REST. The plugin has not all the methods and API calls implemented, so you are dependent on the development team of the plugin if a certain feature is available via the plugin. If not, you need to build it in REST anyway. So that's the reason I prefer the REST approach in Orchestrator.
And what I start with, I build a template. So a few templates that I can use over and over and over, because I don't want to build and structure every single workflow to rebuild every single REST API call over and over and over. So it's easy to just grab a template and change a few lines of code, the method, the URL, or some inputs, outputs needed, and then use the same workflow over and over and over.
So it's also a very standardized approach how to build workflows for automating, in our case, Cloud Director. But I use the same mechanism if I automate NSX or other stuff. With Cloud Director, as long as we use the Cloud API, we have a great Swagger documentation of the API built into the Cloud Director product. On the top right, just click the question-- I think it's the question mark. And then you have a menu entry saying API Explorer. And then just navigate over there.
Please be aware, Cloud Director is still a multi-tenant product. So you have an API documentation for the tenants and for the provider. And they are different. Always keep that in mind. Oh, what just pops up in my mind is: There is a very cool mechanism, guys. If you as a CSP would like to offer your tenants API access to Cloud Director, there is a new session to generate user tokens. So it's somewhere in the UI. On the top right, where the username is, you can go to User Preferences, I think.
And there is a section or a menu entry saying "generate access tokens." So you can use that access-based mechanism for the API for your tenants as well. It's pretty nice. Also for service accounts and other stuff. So it gets even more secure. Nevertheless, so the Cloud API documentation is great. It's easy to use. It's built into the product. You have all the URLs and like a Swagger documentation is, you have all the tryout, functionality, copy, paste, etc., all that kind of stuff.
If you have or if you need a REST API call, which is not part of the Cloud API, you need to move over to the standard API. Yes, there is a documentation. I would phrase it, it's not as easy to use as the Swagger documentation of the Cloud API. It's a website in VMware code and best of luck to find something. Honestly speaking, I prefer if I need to deal with the standard API and I have no clue how a certain API call works.
I prefer using the developer tools of my favorite browser and just grab the call, the UI fires in the background against the API because the Cloud Director UI does or is just consuming the REST API in the background. So there is no secret behind that. Use developer tools, grab the REST API call from the backend, which is monitored by the developer tools and then you have the URL, maybe some data model.
So you can go to the header section, validate which header it is, which API version it is and then check the payload. If it's a put or a post, grab the data model. Sometimes you will find some hidden options in the data model if you're using the developer tools. So even with the Cloud API, not every single parameter of the data model is specified. And then you just use this stuff, you have sniffed and used a different data model.
It also helps which parameters are mandatory, which are optional because I, for example, sometimes struggle with, "oh, I need parameter ABC or where do I get this information from?" Maybe it might not be available. So it's easier to be like, "oh, it's just optional. I just leave it empty or remove it from the data model as well." And then you can do it by REST. I think that's a pretty straightforward approach for automating Cloud Director and also basic tasks.
And then you just move on for NSX and other products. If you have Cloud Director, please automate NSX via Cloud Director. I think that's something very, it's always like, oh, yeah, thank you. I saw it in your face, right? So it is important to say, even if you automate stuff, stick with the rules. Cloud Director is the leading system. If you've Cloud Director and your tenants are consuming NSX, automate on an NSX via Cloud Director.
Except, unless you're automating something in NSX, which is meant to be a managed service for your customers. So it would be configured in NSX manually anyway. One of the features which just comes up is, for example, Distributed IDS/IPS. That's a managed service from a CSP perspective. Okie doke. So that's basic automation. The even more interesting part is, yes, Yves, you want to share some thoughts. You need to unmute yourself.
Yes, I think the big question for me is, on one end you said, you don't know when the Cloud API is going to be finished. I think the answer to that is very, very straightforward. It's never going to be finished, because that requires additional development investments into VCD. And VMware was very clear that besides patches or anything else, they are not going to invest a penny into VCD anymore.
I would not, to be honest, expect anything really coming back into that one, especially looking at how long it took them to get the API to where it currently is. And I would not expect much to the other ones. They promised that we will get similar API in the new systems, but they did not say similar to what. So whether that is the old-old one, the Cloud API or the current Automation API, which they also sometimes refer to.
So the question is, if someone didn't start so far on their automation journey, what would be your advice at the moment to automate anything around Cloud Director? Would you still advise them to go directly REST? Do you advise them to use any of the existing packages? Or maybe use something more like an infrastructure as code approach?
Yeah. So I still think the plugin is great, but, and there's a big but, if you start using the REST approach, you're more flexible because all the other products support the REST API as well. NSX, the vCenter server, SDDC Manager, even Operations manager, Operations for Logs, or it's a bit more hidden, but they have a REST API and they all use the same mechanism. If you are familiar dealing with one REST API, you're familiar dealing with all the other REST APIs as well.
The only big difference between all the APIs is authentication, right? So you once need to figure out how to build Cloud Director authentication via REST, or you need to figure out how to build the authentication for NSX because they're different. But at least speaking for VMware by Broadcom, the documentation, how to authenticate against the REST API is properly written for every single product I've dealt with so far. Toby?
Just to add here one thing, especially for those who really begin to start now with Automation or playing around Automation, let's call it that way, because if you really have already a use case that you would like to automate something, then maybe it is not the best idea.
But if you're really starting now from scratch and say, hey, I would like to have people inside and look and play around with it, I would heavily recommend to play around with the REST API of the Operations Manager, especially for the future, more and more becomes the centralized API for the whole VCF game. So if you now really start with the REST API, then I would highly recommend that you play around with the API of the Operations Manager. That's a good mentioning. Thank you.
So if you're familiar with REST, you're good to go with all the products. And it's not related to only VMware by Broadcom. But I have to admit, I don't know anything about the documentation quality of REST APIs of other vendors. So I'm not commenting. So that's very useful. You can even add Orchestrator into the service library of Cloud Director, and then you can use a VCD UI to kick off or to start VRO... it's Aria Automation Orchestrator -- a VRO workflow.
And if you build the input form properly in VRO, it's identically displayed in Cloud Director. And if you're really brave, you play a bit with the actions in Orchestrator to populate drop-down menus and stuff, and then you can add filters based on organization ID, which might make sense to have an input form of a workflow, a multi-tenant based on Cloud Director, because Cloud Director knows the organization ID, OVDC user ID, and stuff like this.
And the combination with roles and rights enables the visibility of certain objects like an OVDC network. So that's this part. Let's move to the dark side of Automation -- event-based stuff. I think that's even a more challenging area to automate infrastructure behavior, because that's exactly what we're talking about. So within Cloud Director, if you go--so that's the provider context. Within the provider context in the administration section, there is a menu entry called Extensibility.
That's where you add the message broker to the system. And on the second tab, you can enable events. And then you can select--you can either trigger that Cloud Director should send all events to the event bus, or messaging bus, or to certain events, and you can choose if it's blocking or non-blocking. So non-blocking means Cloud Director sends an event and continues with its workflow, or if it's blocking, Cloud Director fires the event, triggers the event, and waits for the response.
So the response could be successful, failed, or cancelled. And it sits there until it reaches the default time out. And that's five days. So you can wait for an external system. That might make sense if you ramp up a new virtual machine, and you need to grab an IP address from an external IPAM.
It might make sense to wait for it, because you need to inject it into the boot process of the guest... for example And of course, if you use the event-based automation, you need to implement a RabbitMQ message bus in the backend. There are some hacks to access the MQTT, so the internal Cloud Director message bus, but I'm not the biggest fan of it, because I didn't understand it. So it's a bit more of a hack.
So I don't preferr that option, because I leave the appliances as they are, because from a support perspective. So I'm not changing anything in the appliance. Just attach the message bus system, and then you attach Orchestrator, on the other hand, to the message bus. Do some configurations to route messages. So I have a nice blog article out there. It's like zillions of years old, but still accurate. And then you just consume the messages, and based on the message, you start the workflow.
Hint, or today I call it a pro tip. If you deal with messages, there is something out there called the VMware Cloud Director Notification Package. If I have it in my head correctly, I think it was built by Christophe Decanini initially. I think, so he's a great guy. So that notification package enables something very specific, which is, you can collect a message from the message bus. That's Orchestrator out of the box behavior.
But you have a message runner workflow, and the message runner workflow converts the message body into Orchestrator usable objects. Which is very helpful, because otherwise you need to start dealing with XML. And dealing with XML in Orchestrator with the Rhino JavaScript engine is not even a nightmare. It's like, I don't know. No, it's not working. Yep, oh, Yves volunteers. So from now on Yves is doing all the XML stuff for you guys. I'm not touching Orchestrator So the notification package.
I have something to write my code now. I tried it and it can't deal with it because the Rhino engine in the JavaScript engine version in Orchestrator is way too old. And does not understand the XML header fields and stuff. Which tells me a lot about the product. It's a great product. It's old. It works. But coming back to one sentence before, are we sure that AMQP still is supported in 10.6? Because MQTT should now be finalized and working. From what I have heard and what I have tested.
I don't get the access method and RabbitMQ is still supported. Does Hock charge an extra license for RabbitMQ now that he owns it? Please not. Just asking. I don't know. I didn't read anything. So another very very common use case for the Cloud Director Notification package is or are the blocking tasks. If you're using blocking tasks, the message runner workflow converts automatically. Or no, let's phrase it differently. The message contains the blocking task ID.
And the Cloud Direct Notification package, the message runner workflow converts that ID into a vCloud blocking task object. So that can be triggered via the plugin. So these are some rare cases which are just the plugin is so useful. I once built a whole rest mechanism with blocking task ID to resume or cancel whatever. But honestly with the plugin, it's just two lines of code. And then you just trigger the blocking task. It's successful, continue, it's failed, it's aborted.
And you can also implement based on that behavior some error handling in the workflow. Some rollback or whatever is needed. So that's the event based automation approach out of Cloud Director. And the Notification package also enables you to filter the messages based on organizations, based on users, based on event types or whatever is needed. Is part of that notification package. The only thing is, the message runner workflow is not working anymore out of the box. Second pro tip.
The methods of the date time object have changed. You just need to go into the code, it's around line 140, and just change three or four methods. And then it's all good. You just need to alter the workflow and then it's working again. It tells the error message tells you exactly what's not working. Just validate that the methods are wrong, use the correct methods and you're good to go. That's the second pro tip. Yeah, so that's... So maybe one last idea based on a statement of Toby.
Because Toby said, and that's something you always should keep in mind. If you're new to the automation game and you want to start with automating stuff, I have two tips. First, never ever just sit in front of the computer and tell yourself, "I'm now going to automate something." It's not working. It's not working, because you don't have a goal. You cannot achieve the goal. You will never have a result.
Always come up with the use case and implement the use case and please start with very simple ones. And that's my second point. Never ever start, if you're new to the game, start automating virtual machines or vApps in Cloud Director. That also refers back to the vCenter server. Please do not start automating virtual machines. Because these are from the amount of methods and properties they have, they are by far the most complex objects. Just start with simple tasks. Let's call it that way.
Yes, exactly. Toby, you're completely right. If you're struggling in the beginning, what could be an easy thing to automate just to play around with it? Create an Edge Gateway in an existing org. The OVDC itself is a very complex object as well because of the allocation model. That's the only reason. And I have another, I today have a ton of pro clips.
If you would like to build an OVDC with Flex as an allocation model, the REST API is the only way to go forward, because the plugin is not aware of that allocation model. It's that simple. It was just introduced about five years ago. Yeah, so it's like it never went to school. Good. I think that is a good starting summary around Automation Orchestration. If you have any questions in any of these areas, feel free to ping the master of old-school automation called Matthias. No, his name is Sascha.
Well, Sascha does the fancy new stuff. And also, if you have any other topic suggestions, feel free to drop them somewhere in the show notes, no matter which platform you are using, so that we get that. Do we know already the topic for the next recording in two weeks? Okay, I see silence. Okay, I see silence. So that means you should better follow us on social media and see what we are going to announce, what is going to be Episode 49.
And then we better have a good idea of what's going to be Episode 50. But nevertheless, I would say thank you all for listening in. You can find this also on any of the podcast platforms and subscribe over there. Any closing words, Toby? Not from my side. Start with automation as soon as possible and have a happy day. Yeah, guys, go into automation. It's just cool. It's better than just clicking and watching progress bars moving and stuff.
And then you free up time to learn new topics and do even cooler stuff. The first steps are pretty hard to take, but get used to it, go into it. It's not that difficult once you get used to the mechanism. I think, and I will end with a side take, I think it was William Lam who said, "Before I do it once, manually, I automate it." A good man, a wise man. Thanks for that closing statement. See you all on one of the next episodes and automate the world. Thanks for tuning in, guys. Bye.