Paul Cornish: I can quickly pull back hundreds of pounds worth of licenses by checking whether a common area phone's been signed in this year or not. And now I can start reusing those licenses. I'm doing, you know, a new deployment next week in a new location. Uh, so I can see that actually there's 150 licenses that are not being used.
I can bring those licenses and I can apply them to my new site.
Tom Arbuthnot: Welcome back to the Teams Insider Podcast. Really great one this week. We've got Paul Cornish, who is UC architect at Merlin. Merlin are responsible for a number of entertainment and theme park facilities all around the world. 120 sites, really interesting journey story to Lync and then Skype for Business and then Teams Phone globally.
We talk about PSDN connectivity, we talk about different phone devices, we talk about DECT, uh, two and a half thousand phones on Teams SIP Gateway as well. Really interesting journey, so thanks to Paul for taking the time to talk us through it. And also many thanks to Pure IP who are the sponsor of this podcast.
Pure IP are an Operator Connect certified provider for Teams. They also offer direct routing as a service and managed SBC. So if you need Teams telephony, be sure to give them a look. Many thanks to Pure IP. And again, many thanks to Paul. On with the show. Hey everybody, welcome back to the podcast. Got an exciting one this week.
A really unusual deployment scenario. So not your typical knowledge workers. I've got Paul Cornish, who's UC Architect at Merlin. You might not be familiar with Merlin, but they run a whole load of theme parks and attractions. Really big organization, really interesting organization as well. Paul, hi, do you want to just give yourself a bit of an intro?
Paul Cornish: Yeah, Paul Cornish, as Tom just said, I work for Merlin Entertainments. My background is I worked for Microsoft. For a while, and then, uh, BBC Studios, and then obviously doing Exchange initially, and then moving to the, the UC world of, uh, of SharePoint migrations, Exchange migrations, and, and Lync, and, and, and Teams.
Tom Arbuthnot: And you came from that, that kind of previous world that I'm more typically familiar with, the kind of enterprise deployment type world, obviously BBC might have been a little bit more interesting in certain scenarios, but, uh, Merlin, very, very different.
Paul Cornish: Yes. Uh, yeah. So as I said, the BBC World Wide World was, was just deployment for office workers and Merlin yeah, we have, you know, we have a raft of different attractions, about 140 different attractions around the world. So that, as you can imagine, lots of different telephony. Scenarios, including Analog, you know, DECT, um, some parks that are a mile and a half wide, some attractions are in malls. So, yeah, it's, uh, when we embarked on this in the early days of the Skype days, yeah, it was some great challenges.
And how we, and obviously this journey over the last few years, how we solve those challenges, how we, how we, how we, Did them solve those changes with Skype and then how we then migrated from Skype to the, the, to the Teams environment.
Tom Arbuthnot: Yeah. So take us back to the kind of the, the, the Skype days. So you did a lot of Skype before.
You've gone pretty much all Teams now, but you've got, you've got Skype, Skype deployed, uh, all these in a lot of these different kind of complex scenarios, didn't you?
Paul Cornish: Yeah. So, so the way it started out is, is. We opened up a new site, actually an IT site in London, and my boss said to me, you know, I don't really want to put a phone system in here.
What can we do? Knowing very well I did OCS R2 and started off with Lync at the BBC, what can we do here? I said, well, let's get a SIP provider into our data center and look at how, you know, look at how we can achieve that. And then how can we quickly deploy to the site? So we did a little RFP, uh, went out to a few providers, um, and we decided to work with Pure IP in the end, um, and rolled that out into our data centers, um, uh, and then got the site up within a few, a few weeks, uh, like from an, it was really a knowledge worker environment there.
So we did that. And then once that was proven that, you know, we've got the SIP trunks into our data center, we've got, we've got this site up and running, we can start potentially doing some, some new sites and migrating existing sites within Merlin. And we did, we took one of our more complex sites, um, was actually had a, that site actually had a telephony issue.
Uh, so we, we started looking at, you know, can we do, could, could we do Lync? And it was actually Lync at the time. Um, so we, we migrated that site to Lync, not actually having a desktop client out to our users yet, so we actually just deployed phones, and that was just the start of the journey, you know, how we, how we migrate the site.
So yeah, we integrated using a Ribbon Gateway, Sonos at the time, um, complications involved in that, it wasn't ISDN, it was CAS circuits. So we put those upstream and got those solutions solved, migrated the, you know, the, the, the VT terminal, which was the Avaya at the time and how we, how we, you know, progress that migration, deployed that.
And yeah, we, we, we achieved that. So once we'd achieved that, and that included 200 analog lines as well. So once we'd achieved that, Uh, we knew we could probably do most of our migrations at all our sites within, within Merlin.
Tom Arbuthnot: That's, that's pretty awesome. And there's a jump there moving from traditional telephony to link and then Skype business server and running your own SBCs with Ribbon and terminating those lines and kind of jumping forward a bit.
And you've now moved. Most, if not all of that to cloud, is that right?
Paul Cornish: Yeah, so we took that journey and then we probably migrated about 90 of our sites to Enterprise Voice Skype. So there was various journeys there, some of the sites we did remotely and we could do them really quickly with just buying Buying telephone numbers from Pure IP.
Some scenarios we had to put a different telephony provider in Korea, um, South Korea, um, uh, in Japan as well. So we basically built up a big Skype environment, quite a complex environment, as you probably know, Tom, from having three sites of Skype deployments, and Edge deployments, and SIP trunks. And then really, Pure IP, you know, the concept of direct routing came about with Teams. So Pure IP had a great offering where we just, um, they put a direct routing into their sites. So we had four or five different sites in Europe and Asia and America. And I can literally go to a portal and say that number is now on direct routing rather than, rather than the SIP trunk of my data centers.
Or we could do multiple, multiple ports. So it's really easy just to move to Teams. And then we have the, you know, once you've moved your telephony, it's just about creating accounts, moving the user accounts. Um, or, and also, you know, the analog, we have to, you know, change the way the gateway works, some of the gateways we have to have direct routing into Pure IP with or other telephony providers.
And so, yeah, then it was really about how we move the phones. Um, and that was probably the hardest part of it all. You know, some, some of our sites had, have 250 physical phones on site, VVX's, PolyVVX's. So how do we move that phone from Skype to Teams? And there are some challenges involved there. Microsoft say you have to factory reset the phone.
If you factory reset the phone, we have to go and visit every phone. That's quite a big job in 250, uh, uh, phones in, in some awkward locations in our, in our parks. So we, you know, worked out a way that we could actually factory reset the phone, but keep the PIN number. But, um, and so that was quite good. And then we had to use another script.
Uh, which is out in the public domain to, to basically get the code to sign in the phone. So then it was a matter of just doing signing into 250 phones in an evening. So we could, we could literally migrate a site from Skype to Teams in an evening, uh, and be up and running. So, yeah, it was, uh, some, some, some fairly slick deployments there.
And in new sites, we put in new phones.
Tom Arbuthnot: Yeah, if you've done some quite big sites with that kind of big bang cut over and where you can, as I understand that you've, you've gone to PSTN from the cloud, so be it direct routing as a service like, yeah, either initially to Skype and now with Teams. So you still got some local SBC, I believe?
Paul Cornish: Yeah, we've got local SBC where there's complications. So, for example, let's take one of the larger theme parks. We have a Mitel phone system for the hotels. We have 200 analog telephones still, because the sites are very large and they've got analog lines. So in those scenarios, we put a, we put a Ribbon gateway in, uh, that could be virtual if we've got direct, if we've got SIP trunks, if they're still physical, which is probably only a couple of sites where we still got that, but where, but then really it's a virtual Ribbon gateway.
SIP trunks into the Mitel, SIP trunks into Pure IP or a provider we've got at the time, SIP trunks to the analog gateways, we used Tenor gateways initially from, from Sovereign slash Ribbon. Uh, we started moving to the Audio Codes now because they don't supply those anymore. Um, so yeah, that all to the, you know, the Ribbon gateway is your, is your central hub.
And then we work out the routing where a phone call needs to go to, to Pure IP or whether it needs to go to analog or whether it needs to go to Mitel, we just build that up. So we've got that still in a, in a few sites, but a lot less than we had in the, in the Skype world.
Tom Arbuthnot: That's awesome. And I was talking to you on the prep call about the hotel.
It's a really interesting scenario. Obviously you have hotels with special phone keys. And my initial thought was, Well, they're going to be around forever because they have the special keys, special layouts, but you were saying you're looking at kind of brand new ways of thinking in those scenarios as well.
Paul Cornish: Yeah. So, so you probably, you know, where you stay in hotel rooms a lot, you know, there's less and less there's phones in, in, in hotel rooms. So really it's about how we, we can start removing the, the reliance on, on hotel rooms and, and our phones in hotel rooms. And that's happened that, you know, quite a few of our attractions.
So then it's about how do you, you know, how do you, how can the. Customers still phone down, so that most people have mobile phones now, so we have, we have a card to the wall, which says, bring this number if you need help, or interactive into the TV to contact reception, or there might be scenarios where we put, um, phones in the hallway, so.
If you need to make a quick call, you just go out in the hallway. The new, the new Teams phones have the, you know, the, the, the, the press one button to, to call somewhere so you can, we can deploy that as well. Um, but there are still, there's still sites and still locations. There's still countries where, you know, it is expected to have a phone, a phone in, in.
In the hotel room and that that's still be that'll still be hanging around from from until there is, you know, until they're removed and
Tom Arbuthnot: yeah, it makes sense. I guess a lot of experience these days. Uh, have an associated app as well. So you often you going there for your schedule and your chat and your support and everything else as well.
Paul Cornish: Yeah, and there's more of that happening. You know, there's going to be an interactive tablet when you're in the room and then you just push a button. You'll be able to order your, you know, you can order your, you know, F& B, you can order your meals and you can, uh, and you can contact reception from there.
Tom Arbuthnot: That's awesome. And you've still got scenarios where you need like different hardware or specialist hardware. So like, for example, I think you're using Spectralink DECT for some scenarios.
Paul Cornish: Yeah. So we, so the two scenarios we were using SIP gateway, obviously when we migrated from Skype to Teams and I've got two and a half thousand VVX phones out there.
So, you know, They're still a very good solid piece of hardware and they never, never break.
Tom Arbuthnot: And they're all running off of the Teams SIP Gateway or SBCs?
Paul Cornish: The Teams SIP Gateway, yeah. So as you're aware, Tom, you can buy Audio Code. So if we got some, some bespoke analog, a couple of lifts that are needed, for example, or To be honest, the lifts move over to 4G in most scenarios, but if you've got, I don't know, if you've got a, you know, one analog phone left at a park or a location, we can use the SIP gateways from Audio Codes.
So we deploy the MP series there in some scenarios, so they talk directly to SIP. And same with SpectraLink. SpectraLink, when they move from Skype to Teams, they use a SIP gateway. So we just, We had a site that was on Skype, um, and we just moved their Spectralink over to Sipgate, which is just seamless.
You buy the licenses you require and you can continue using the existing environment. So, they work really well, but we have some challenges that, you know, that some of those are deployed in, in, in Sealife's. Um, so there could be, you know, we, we've got to make sure that phone works as you walk through the tunnel, the sharks going on there overhead.
Tom Arbuthnot: It's quite a unique scenario to deal with.
Paul Cornish: It is a very unique scenario, so you have to make sure your, your, your base points for the, for the, but it works seamlessly in, in, in these attractions, you know, so it's pretty good.
Tom Arbuthnot: That's amazing. And these, this huge project, but these combined efforts, you said, have driven quite a lot of cost saving in terms of PSDN connectivity, in terms of hardware and maintenance.
Paul Cornish: Yes, if you go back to my scenario is, is, you know, in the earlier days, you know, when you moved desk, You moved your, your phone and your phone physically was moved with you. So for the days where you have to do cable moves, you have to move phones, you have to make add and remove changes into your, your PBX, that PBX may have been supported locally, but it may be supported remotely.
You'd have to log a call and say, I'll need to move this telephone from here to here. So those, those OPEX, those, those operational costs completely go when, when you move to your Skype Teams environment. It's all managed in the, in the. In the, in the Teams Admin Center or in Active Directory changes, um, and physical changes and phones, less and less phones deployed now really.
And once you've, you put them out in the, in the, in the attraction, they're normally fairly static or you may need to add one. But the cost savings, yeah, I mean, going back to the early days of a theme park, you know, you could have ISDN lines. And you could have a PBX that's being, being, being, that's supported at the cost of that, that could be three to four, you know, £4, 000 a month.
And then you then move to a global SIP provider with, with some DDIs, you know, you're, you're, you're in the rounds of £100 to £150 a month to, for telephony. So, you know, it's huge cost savings. You apply that across your 130 attractions across, across, across the organization. Yeah. And then there's very large savings that have been made over, over the years.
Yeah.
Tom Arbuthnot: That's amazing. And you talked about the some of the first deployments were like just phone first. So not even soft client first and talk, talk to me about kind of. Teams and collab and UC in video and meeting rooms, like how's, how's that gone? What's, what's the video and room story there?
Paul Cornish: Yeah, so the early days in, in Merlin, they were, you know, as most companies had, they maybe had a Tandberg sitting in the corner on a couple of OSDN lines. There wasn't a lot of cloud. When I started here, Office Communication Server was used by IT only, so there was no IAM mechanism. So what we've done is we've gone across that journey, um, to the point where all our major meeting rooms have standardized equipment.
So I've selected to go with, um, MTR over Windows. Uh, I've selected to go, I think in the early days we had some Polycom that we put when, because they had native connectivity with, with Skype. We moved over to the Logitech platform, so we have a Logitech tap in every room. That may be just a very simple, you know, there might be a little camera over there, but you know, in every room has that tap on the desk with a HDMI cable.
My, my, my, my remit was we have a tap, it's standard, everyone can come into the room. They can know how to join the meeting. They've got a HDMI cable if they just want to plug in and and connect on the screen. So we've got that even in our small rooms with with we've elected to have to have that set up. So every room within Merlin has that set up.
There are a few rooms where, um, I think there's a couple of Android, um, devices out there, but we haven't really gone down that route. We've stuck with the, uh, With the Windows debate, but I've decided to do that. It's a standardization and that may change over time as, but that's where we are with that.
Yeah, so yeah, it's good. So we've got good standard.
Tom Arbuthnot: It's really interesting to understand. Yeah, I know you spend a lot of time. You've been in the space for a long while. We've had lots of conversations around vendor selection and system selection. And again, it's interesting because your use cases are sometimes more unusual.
So finding the right phones or the right, the right room systems is really interesting. Yeah.
Paul Cornish: Yeah, yeah, I mean, the right phone has always been interesting because. Obviously, when Teams came out, we had to decide on what phones to use, so I evaluated the Polly, um, what was it, the CX500?
Tom Arbuthnot: CXs, yeah.
Paul Cornish: Yeah, the CX ranges, and then Yealink.
So, really, only two out at the time, I think Audio, Audio Codes were bringing, bringing some out at the time. Um, so I evaluated both of them, and what I realized is, is, is, at the time, is, is, the Polly only had these big touchscreens, and, and you can't really put, uh, you know, a touchscreen in a doughnut shack, because it, it, you know, It doesn't work.
It's overkill for what is needed. So we wanted to try and go with a fairly cheap touch phone. So we went with the Yealinks. The majority of our sites have Yealinks. And, um, just been evaluating the CX350s, which is, which are very good. Um, and they're, they're touch phones. I think Poly are now bringing out touch phones.
So. You know, I, I'm giving the sites a choice. You want a Polycom phone or do you want Poly phone or do you want Yaelink so that they can, they can choose that. But yeah, ven vendor choice is really difficult. Difficult because, you know, there's, it's always changing. You know, you can, you just need to go to the, uh, the Teams blog and you can see there's like 20 different UC vendors.
Yeah. And you know, when I was making the choice at the time, it was really Cres Crestron, uh, Yaelink. We got some Neat boards as well. They're very good. We got some Neat boards in some Oh, nice. Some offices, they, they give us a very good. Uh, white screen experience, uh, whiteboard experience, um, so then we just put them in and wheel them into something else, so we've got three or four of those around, so yeah, so yeah, it's good, but it's great, but staying on top of it is very difficult to looking at, you know, what devices are out there, but standardization for me works.
You know, it works for the business as well. When they know when they go in a room, they can hit a button and it's the same button every time.
Tom Arbuthnot: And what is, as much as you can go into it, what does Project and Ops look like? Because I know you've driven a lot of this in house, so you've obviously worked with partners at times, but like largely project driven in house, is that right?
Paul Cornish: Yeah, largely. We worked with Modality on the early days of Skype. Um, Skype days, but then, yeah, we, we haven't worked with any partners sitting there. We work with New Era Tech for our meeting rooms, so they, they can plan out our larger meeting rooms, but they know our tech, they know, they know that, you know, you're going to be putting in a tap in that room, so that works really well.
So we work with them, they're really good vendors to work with from that point of view. Um, obviously Pure IP from, um, From started with our SIP supplier, but they also when we moved to the Teams deployments, we asked Pure IP to supply the hardware as well. So we've got a standardized support from from that point of view.
They support the SIP trunks into the hardware. They support the hardware, the maintenance and they monitoring, monitor those as well. So Podcast, If one of my SIP trunks go down, I'll get an immediate message to me and my operational team to see that. But yeah, from an ops point of view, you know, Teams is really, really good.
Also created some Power BI reports so people can see the telephone numbers that are provided to us.
Tom Arbuthnot: Yeah, you've done some pretty flash stuff in house in terms of extracting data and building your own reports.
Paul Cornish: Yeah, that's fine and so on. I've got Power BI's that we use for devices so we can see, um, we can pull out from graphs and various information about our meeting rooms and devices, but also create some reports on licensing so I can see, I run a report every Monday, I can see how many people have left the organization.
Whether their enterprise voice license is still applied to them so I can remove that license. I can see when, if you start bringing in, um, sign in from Azure, I can see what common area phones that have been created that are not used anymore because we were large, you know, two and a half thousand common area phones, 2000 common area phones out there.
You know, phones are going to be removed from areas and they're going to be added to areas. So I can see that when there's an account
Tom Arbuthnot: You mean not everybody carefully raises an IT ticket when they change a site globally?
Paul Cornish: No, no, that's not the case. So you can, I can quickly claw back, you know, you know, I can quickly claw back hundreds of pounds worth of licenses by, by checking whether a common area phone's been signed in this year or not.
And I can start reusing those licenses. I'm doing, you know, a new deployment. Next week in a new location. So I can see that actually there's 150 licenses that are not being used. I can bring those licenses and I can apply them to my new site. So, uh, so yeah, that report is an all encompassing, you know, who's got what telephone numbers, who's got who's.
Who's been deployed without a voice policy, for example, and who's been deployed with a license that has not been enterprise voice enabled. So you can quite keep, you can keep on top of those licenses and, and, and claw that back. So that's, that's a quite good little Power BI report that I share with everyone as well.
So it's published in a, in a team, so you can see that as well.
Tom Arbuthnot: And that's a mix of dipping into graph and dipping into PowerShell and kind of putting data together.
Paul Cornish: I write my own, um, um, you know, Microsoft SharePoint list, for example, our sites list. I could bring information from there. When new sites were created, they automatically added that.
It brings in all, um, all the users as well so we can see their information. So yeah, it brings that in and then it gives you a good overview of your Teams environment. I'm sure there's a lot of providers out there that do some sort of the same sort of thing, but it's quite good for me to bring that information in.
Tom Arbuthnot: It's cool to know it can be done in house as well and you can tune it to exactly the way you want it.
Power BI is an incredibly powerful tool for that if you know how to drive it as well.
Paul Cornish: Yeah, and it's got a little button. You can say, Oh, I want to go into a certain brand and then it can, it breaks down to all these brands. You can click on that brand and that gives you a list of all the people in that site and their telephone numbers assigned, what numbers are free and what their policies are, their voice policies.
You can quickly The operational guys can quickly say that this guy in Japan is making a phone call that's not working. You can say, Oh, actually, their locale is UK, or they've got the wrong voice policy, or you can quickly see that information.
Tom Arbuthnot: Awesome. Do you get asked internally about, like, different types of usage?
Like, like, are people using it? How are they using it? Does that come up?
Paul Cornish: Yeah. So, yeah, we use a product called, uh, Clover. Um, so that, that was on premise in our Skype environment. It's now in the cloud. Um, and that can create executive reports for us. I can create, um, reports on usage, um, for example, I can, you know, we can, we can do a piece of work where I can take a site and we can look at that site and we can see that there are maybe 100, 100 enterprise voice licenses used, but actually, Jim Smith never makes a phone call.
So, actually, you can go to them. Do you, do you really need this? You know, this license cost? Uh, we have to pay for that and we can call that back if we need to. So yeah, we can use that. Um, yeah, create some executive reports. We can create, um, call quality reports, but I tend to not, you know, this obviously in the early days, you know, Tom, you know, from, from the Skype days when there's quads on the circuits and stuff, now everything's on direct routing over the internet, you know, normally, normally the quality report is actually someone's home internet or whatever and so forth.
And actually the stuff you get in TAC is, is, is, is, is actually very good.
Tom Arbuthnot: Yeah, yeah. I hear the same thing for most people. Well, a combination of with with working externally, a bunch of qualities out of our control and the codecs have got so good these days, it really has to be very poor network to fall over.
Paul Cornish: I would go to say that no one's ever reported anything to me in the last year. I used to get loads in Skype
days,
but I was reported, this call was bad. I don't get any of that come to me anymore.
Tom Arbuthnot: It's amazing, isn't it? Cool. It
Paul Cornish: is, it is, yeah.
Tom Arbuthnot: I mean, you've done so much in these projects, any kind of takeaways for people in terms of approaching these complex sites or complex scenarios?
Have you got a methodology there? How do you approach it?
Paul Cornish: I think you've just got to look at, there's a lot online nowadays and people write really great blogs on how to approach it. I think you've really got, there's some scenarios where we've migrated a site and we've said, right, let's do the upstream gateway and ISDN lines and blah, blah, blah.
And there's other times you're like, Do you know what? Let's just do the big bang. We know Pure IP can port these numbers. The porting process is pretty slick now. So I know that I can reliably take a site down for a few hours and actually the numbers will port. So actually I know I can port to direct routing.
So does this site have anything complicated? No. So, you know, okay, I know I've got 150 phones or whatever, but actually I know I can port really quickly and I can migrate those phones with three or four people in, in, in an afternoon or in, in a day. So I think you sometimes, sometimes you could be, you could think it's more complicated, you know, put the upstream gateways, put all this, when actually, sometimes it's just a big bang approach.
Just, just, it is just the best approach. Um, and there's a lot of tools out there. I know, I, I, you know, I know the supplier, the phone suppliers do a lot of. Management tools, but actually a lot that you can, you can do yourself, um, a lot of reporting. You can, you can do yourself. You could build your own report.
So, yeah, I think, uh, I think, uh, yeah, I think there are suppliers out there. There are the modalities still out there and so forth. And I think if you don't have good in house people on the knowledge that using these providers is still a, is still the way forward. Yeah. And I think it's a, it's a, Yeah.
Standardization is, is, is the best for me. I get asked, can I go and put this 70 quid, uh, video conferencing unit in the room? I'm like, you can, but actually if you spend a little bit more, I can support it for you. My team can support it for you. It, you know, standardization is, is, is definitely the way forward.
Tom Arbuthnot: Yeah. Selling the benefits of the standardization rather than I'm not just making you spend more for spending more, like you're fitting into a global service model here.
Paul Cornish: Yeah, precisely. Precisely.
Tom Arbuthnot: Amazing. Well, Paul, thanks so much. It's such an interesting journey and it's great to talk to somebody who's doing the more complex stuff and at your, your pace, it's just been amazing.
So thanks for taking the time and appreciate you sharing all the detail as well. Really great.
Paul Cornish: Ah lovely. Good to talk to you, Tom. Thank you