Welcome to the Azure Security Podcast, where we discuss topics relating to security, privacy, reliability and compliance on the Microsoft Cloud Platform. Hey everybody, welcome to episode 87. This episode is actually one of those special episodes you do once in a while, where we're going to talk about something that is released into general availability or some new feature that's coming out.
And this time around, we're going to talk about two new cryptographic controls or two updated cryptographic controls, I should say, in both Azure SQL Database and SQL Server. We're going to talk about the pros and cons of each of them and basically what the changes are. And this week we have two program managers who basically own these features. We've got Peter. Peter's been on the podcast twice now. And we also have Merrick.
And so Peter is the PM for Always Encrypted and Merrick is the program manager for Transparent Data Encryption. So before we get stuck into the actual, you know, what's changed, gentlemen, why don't you introduce yourself? Merrick, why don't you kick things off? So hello everybody. My name is Merrick and I'm a PM in the Azure SQL Security Team.
I have worked with SQL since SQL 2000, mostly in the storage engine, access method, metadata, releasing features that probably most of you know, like snapshot isolation, online index and recently from the metadata or from the security standpoint, the Microsoft Entra ID that for your information, that was the former Azure Active Directory that was changed. The name was changed.
And there was the implementation of the Entra across the whole Azure SQL, finally adding with the SQL Server 2022 that allows the connection to Azure SQL databases from SQL Server. And now I'm part of the Transparent Data Encryption Team, TDE. And I would like to let you know about the latest innovations, features that were released in this area, in the TDE area. The TDE is the encryption data addressed and it protects user data.
So it protects the whole database, protects the transaction locks, the GoDR, the tempDB. It doesn't protect masterDB because we allow customers to use masterDB as this key part of the database or server that has to be used. The TDE has two levels. The first one is the database encryption deck that is available to encrypt the whole database. And then there is the deck protection, the TDE protector that allows this.
And we have two options for this deck protector, either using the system manage key or the customer manage key. In the latest development that we made and released, the focus was on the customer managed key that is using AKV, Azure Key Vault, and offers the policy for customers to manage the key, to allow permissions to the key, to rotate the key, to use the key across the tenant. And that's kind of the main focus.
And the reason for this is the following, that more customers, especially in the financial sector, in the money market sector, in the banking, they want to have access to their keys in terms of they want to own the key. They want to give the proper access. They want to make sure that the policies are really defined by them. So many customers just decide to use the customer managed key and created the Azure Key Vault.
There's one more important part that we need to know when we talk about the underlying storage. We have two types of storage that Azure offers today, which is the remote storage and there is a local storage. Now the remote storage is encrypted, which means that the data there, the data is at rest, is encrypted and protecting through accessing this data.
So when we put on the top the TDE for certain additions like basic, standard, general purpose or hyperscale, we can see so-called double encryption. One is coming from the Azure storage and one is coming from TDE. However, if the customer is using a local storage and this is that applies to the premium and business critical, this storage is not protected.
So if for some reason the customer doesn't choose to use TDE because it's possible to disable it, there is a big danger that there's a fact that the data is not protected and as a data of rest can be accessed by potentially by malicious groups to do this. So that's why we always encourage customers to use the TDE and ideally to use the AKV because that gives all these options to manage the key and to be the owner of the key.
So in the latest releases, what we have added to the general, to the TDE is the key rotation. As you know, when the key is created for CMK, for customer managed key, it has a special version. Now, the AKV offers an opportunity to change these versions periodically. And the reason for this is again protection that the customer protects this key and whoever has access to the past key will not be able to use the database because the key was rotated, the new version was created.
So this key versioning or the key rotation was added to the entire TDE, but specifically it was also added to another granularity that we offer, which means we offer a possibility to add the CMK at the database level. In the past, the TDE had a CMK defined at the server level, which means all databases underneath use the same key, use the same CMK key.
However, with the recent release that was available in GA, general availability in September and is ready to go, we offer this other granularity, which means we allow to specify the key at the database level. And I'll come in a moment why this is so important. And then not only this, we allow to rotate these keys and we allow to create these keys in another tenant. So this particular feature is very interesting for ISVs that provide the elastic pools.
The scenario that you have around this is the following. You have in the elastic pool, you have typically one server, but there are multiple databases, many databases, and they belong to different customers. Potential security issue is that customers may be worried that they would share the same key with other customers if this is at the server level.
So if the customer chooses to use the database level and on the top of this chooses that I want to create my key, my AKV in my tenant, that is a different tenant that serves the server itself, that allows a customer to get a full control. So customer can create a key there, its own key can set up the rotation. Obviously, there's a setup needed between these two, between the server in one tenant in server tenant and then the setup in the customer tenant.
And this is done through the multi-tenant application with a federated identity. But once this is done, customer accesses its own databases using its own key and it cannot be accessed by other owners, by other databases, because this is a different key. So in summary, first of all, if I can close with a few sentences, make sure that your database is always protected with the TDE and there are ways to check it.
You can use the DMVs like sysdm-encryption or sysdb-loginfo, where you can see if the database is encrypted. This is very important. And the second one, choose the right level if the server level granularity for CMK is the right for you or if the database level CMK with all these benefits that I mentioned just fits nicely to you, to your need and to your scenario. So that's kind of the summary from my end. From my perspective, some people understand this.
So one of the big changes there is the fact that you can now encrypt using TDE at the database level but have different keys per database. That's what you're saying. So historically, we only did it per database server. So if you had 27 databases underneath that, then they'd all be encrypted with the same TDE data encryption key, the DEC. But what we now offer is the ability to have a different set of encryption keys that are used at a per database level.
So you may have a customer on database one and a customer on database two and a customer on database three. And now they have their own encryption key. Is that the case? Yes. And again, this is specifically important for the ISVs with the elastic pools who offer this type of services to customers. Perfect. All right. So now we've got TDE out of the way. Let's switch our attention to always encrypted. As I mentioned earlier, Peter, we've already had Peter on the podcast a couple of times.
Nice to see you back, Peter. Do you want to just give a brief introduction about yourself and then go through the new changes to always encrypted? Sure. Hi, everyone. My name is Peter. I'm a program manager just like Birik in the Azure Data Platform Security Team. So that's the team that manages all the security features in Azure SQL and SQL Server. And I'm the PM for two features, Ledger, which we're not going to talk about. And I'm also the PM for always encrypted.
And yeah, that's what we're going to talk about today and what new investments we have done the past couple of months. Maybe a short introduction, what always encrypted is. So the motivation of creating always encrypted is actually to enable our customers to store confidential information in the cloud. So it's not like TDE where we completely going to encrypt a database. It's really encryption on column level.
For example, like DBAs, hackers, malware, they will never be able to see that confidential information. So we started our journey with always encrypted back in 2015. And the first version was released in SQL Server 2016 and is still available in the later versions. It is also available in Azure SQL DB and managed instance. And on top of that, last year, we also released always encrypted in Cosmos DB.
So like I said, it really protects your data in use from malicious DBAs, operating system administrators and malware. So it's only the application side that is considered as trusted. Anything outside the application like the network, the database engine is completely is considered as non-trusted. So it's actually the client driver on the application side that is going to encrypt that sensitive information, send it in ciphertext over the network to the database engine.
The database engine returns that information in ciphertext and then the client driver again is going to decrypt that information. That was the first version called always encrypted. Was a good version. However, there were some limitations related to flexibility and usability. For example, if you wanted to use an encrypted column in a where clause, that was possible, but you could only use equality, larger than smaller than for example, that was not even possible.
And on top of that, you had to use deterministic encryption. So which is not as strong as randomized. So what we did, we came up with a newer version, which is called always encrypted with secure enclaves that was introduced in SQL Server 2019 and is also available in Azure SQL database. And so what we're doing, so when processing SQL queries, the database engine delegates computation on encrypted data to a secure enclave.
The enclave then decrypts the data and performs computation on plain text. And this can be done completely safely because the enclave is, yeah, you should consider it as a black box and where nobody has actually access to the not the operating system, again, not the DBAs. Nobody can get inside that enclave. Now by leveraging that secure enclave, always encrypted can now support rich confidential queries.
We can now use larger than smaller than we can use like we can use ordered by on an encrypted column, we can even create an index on an encrypted column. And on top of that, an extra advantage that you have by using these enclaves is in place encryption. So we don't have to move the data outside of the database to do your initial encryption.
We can just use the enclave for that compared to the older version, always encrypted where we had to extract the data outside of the database to the application, do the encryption and then push it back in. So these were the two flavors that we had. So always encrypted and now always encrypted with secure enclaves. Now if you wanted to use secure enclaves in Azure SQL database, you have to configure a specific type of database called the DC series.
And the DC series, when you configure that we're going to configure a specific hardware under the covers, which contains an Intel software guarded extension CPU or SGX that is used to secure that enclave. So we're literally using hardware under the covers for that. The problem between brackets was that when you configured DC series, you had the limitation up to a maximum of eight V cores.
And yeah, I'm happy to announce Michael that today we're releasing the general availability for SGX enclaves or the DC series up to 40 V cores instead of eight. This allows our customers that have a high demand of CPU and memory. And on top of that, they need to secure their sensitive information. They can now use up to 40 V cores. So that is one improvement that we have done on the secure enclaves. Another improvement that we have made is another type of enclave.
So next to the Intel SGX enclaves, we also now have virtualization based security or VBS enclaves that is general availability as of today. Like I said, it doesn't use any hardware. So there is no hardware dependency. You can use it in whatever database you configure, like DTU, V-Core, serverless. You can go up to 128 V cores. And it is also available in all the Azure regions. So these are the two improvements that we have done. The VBS enclaves, I think, is critically important.
Actually, they're both really important. But the VBS one, I think, is important because you can basically choose any computer under the covers, right? You're not limited to essentially virtual machines under the covers that use specific Intel CPUs that have SGX enabled. So that's really cool. I think it's really important because it just gives more people so much more flexibility when it comes to compute. That's fantastic. Well, yeah.
I was just going to say that it's very easy to enable a VBS enclave. So it's just a switch in the portal, or you can use PowerShell or even in the SQL Server Management Studio. You now have the option to just, yeah, on creation or even for an existing database. You can easily switch on VBS enclaves. Just takes a couple of seconds to load the enclave, and then we're good to go. And yeah, you can start using always encrypted. Yeah. I think that's important as well, right?
Because if you want to deploy this thing, you want it to be relatively straightforward, right? Not ridiculously complex. Exactly. All right. So let's bring this to a close. Mirricompeter, thanks so much for joining us this week. I just want to kind of wrap things up with just from my perspective, because I certainly talk to customers. Yeah, so TDE, always encrypted. Which do I use when? I mean, why do you have these two solutions? It's really important.
Whenever you look at any security defense, any kind of mitigation, any security control, you always think about what threat are you mitigating? And TDE and always encrypted, even though there is some overlap, they do mitigate different threats. And also they come with some ease of use trade-offs as well. So for example, TDE is really designed to mitigate a stolen hard drive or a stolen volume. This becomes really important with backups.
As Merrick said before, the backup is always encrypted with the same key hierarchy, or uses the same key hierarchy, I should say. So if a backup is stored elsewhere, then it is encrypted. And it's a very important defense for the likes of backups, not just for the volume itself, but also for the backups. Whereas always encrypted is designed to not just provide a solution for encryption of data at rest, it also provides a solution for encryption of data in use.
And as Peter mentioned, the data is essentially always encrypted, hence the name. The only time it's not encrypted is inside the secure enclave, which is a cordoned off portion of memory. I'm not going to go into all the technicalities of that, because that would be up here all day. But essentially it's completely cordoned off. Debuggers don't even see that they're there, that the memory is even there. So you can't actually peer inside it.
And that's the only point where the data is decrypted, and that's where the queries are performed and so on. So they are similar on the surface, but they're actually quite different. And certainly the technologies are very different under the covers as well. So when you're thinking about what it takes to design a secure Azure SQL database solution, you really need to think about both, because they do mitigate different threats. They're not the same technology at all.
So with that, Peter and Merrick, thank you so much for joining me this week. I really appreciate you taking the time. I think these are fantastic improvements, both across TDE and always encrypted. Kind of answering our customers' pain points is always good to see. So again, thank you so much for joining us. And to all our listeners out there, thank you to you as well. We hope you found this podcast useful. Stay safe and we'll see you next time.
Thanks for listening to the Azure Security Podcast. You can find show notes and other resources at our website azsecuritypodcast.net. If you have any questions, please find us on Twitter at Azure Setpod. Background music is from ccmixtor.com and licensed under the Creative Commons license.