Welcome to episode 325 of the Microsoft Cloud IT Pro Podcast recorded live on March 10th, 2023. This is a show about Microsoft 365 and Azure from the perspective of it pros and end users where we discuss a topic or recent news and how it relates to you. Microsoft Loop is finally coming in public preview later this month. So naturally, Ben and Scott share some of their experiences with Loop and in particular loop components to date as well as a few oddities that they've run across using Loop
components. From there, they move into an interesting scenario they recently encountered around moving a hundred plus terabytes of data out of SharePoint and into Azure blob storage and some of the challenges that one might encounter in such a scenario. I have a question, Scott. When can I start playing with Loop ? I don't know. Off the top I can say that Loop is a confounding product to me. The more you make me interact with it,
the more confused I get by it. So example, like real practical example, the other day we were recording the stream for YouTube and we're putting that together. And normally when we do the YouTube streams we go back and forth inside of just teams chat with, with random ideas and kind of, you know, status if we see somebody's dropped out or you know, their video froze or something like that. And there was this need to put in just a bulleted list for things.
And it just so happened that in the tendency that you and I do, teams in like Loop is all lit up and ready to go. So as a, you know, great smart ass, I went in and said, Ooh, rather than creating some rich text here, I'm gonna create a loop component cuz it's gonna be great. So the scenario is you and I are in a teams chat, we're in the same exact teams chat. We both have permission to the chat, we're like in a live meeting, right? We're, we're going through and we're doing it. I wanna.
Put this on top of it. We're in, we're logged into the same tenant too. Yes. Like we're both using IDs in the same tenant. This isn't external guest user or uh, I invited an external user to the meeting. This is you and I logged into teams into the same tenant in a chat in a meeting. Yep. So I went two of us in a chat in a scheduled meeting. This isn't even like a one-off teams chat or anything like that.
I don't know if that would make a difference, but just to like, this is a fairly structured thing and, and kind of a common workflow I would think. So we're in the chat and I create a loop component which is just rich text and it's got a couple bullet points in it and I write up the text that I'm gonna put in there and I hit send the send button in Teams first problem. The loop component does not render for you at all and you can't even see it rendering there.
You get basically a link to click on and go out and view it online at which point you get an access denied message. So you have to request permissions to the loop component that I just sent to you in that teams chat, which in and of itself I'm just gonna say like that is a horrible experience. Like yeah, teams should know that you and I are chatting with each other and that I explicitly created that.
I didn't send you a preexisting loop component so it wasn't like I was sharing a document or a random link that you, you shouldn't have had access to. It's like I created this loop component as a part of this chat. So you had to go open up, you know, the web view of my OneDrive request access. I had to go and know where to go in my OneDrive to respond to that access request or wait for the email to come through from SharePoint that says,
oh you know Ben requested access to your thing. I give you access. I have to tell you in real time, mind you, this is like minutes later as we've been dancing around and trying to figure out what's going on here, I have to tell you that I gave you access to that component. And even then, because teams is relatively static when it comes to chat and things like that, it's not like auto refreshing in the background for messages that were previously sent.
Then you have to click out of the chat in the meeting and come back in, which I'm gonna argue doesn't really work like it works if you're somebody who always chats outside of your team window. Like you and I are in a meeting right now and we're doing video back and forth. So I tend to chat out of the same meeting, uh, the same window that the video's in and the only way to refresh that would be to leave the meeting and then come back like what a horrible broken, awkward experience.
Like it really makes me question if the people who want us to use loop components have ever tried to use them themselves. This is a little odd to me. So I have seen this done and I thought I had done this successfully in the past and I've seen people do this successfully in the past where they've all been in a meeting or in a chat and they've gone and added a loop component and everybody's been able to access this. So I'm also a little curious is, is this,
do I have a setting in my tenant that I missed? Is this a timing thing? Is this like why is it doing this? Because I have, in all fairness, I have seen this work, but to your point, it shouldn't be this complicated is to why it doesn't work. Like. I'm gonna argue if there's a setting in your tendency that you need to set to make that work, it shouldn't, again, that should not even be a consideration if there's something out there that you need to go ahead and crank on a knob like that.
Sorry, like that's just a, a non-starter And I might be a little salty about it cuz we've been doing like planning and a bunch of like product wor work this week and I've been spending a lot of time with customers and thinking about customer asks and things like this. This is just, I, it's one of those ones where I go, did anybody ever try this?
I wonder, I just tested this. So I put a link in, I put a screenshot in the discord of what this looks like and it showed a loop list in draft and it gives me like a message at the top. Some recipients won't be able to view or edit this and this was in a chat with me and someone else in my tenant. I didn't actually send this one to you Scott, it was with somebody else. And I waited a little bit and I finally clicked send and then I got a little
button that said refresh. And now that I refreshed it and I look at access, it looks like they have access. I'm speculating. I wonder if there's a timing thing because it did just work for me, but it was not as quick as you and I did it before, like it was seconds while you were talking where I started it up and maybe waited 20, 30 sec 20 or 30 seconds or so and it did work. Like is it just slow in going in and granting access to it?
Where had we taken longer? Again, to your point, I would argue it should probably be quicker and a little bit more straightforward of ex explaining why it didn't work, but it did just work for me. But it was slow. We. Waited what like minutes the other day cuz we did a whole big dance back and forth of going out to OneDrive and you requesting permissions. So I'm gonna put on my hey I ideation hat thing, right? Yeah, like how, how should that work?
So your argument that it might take a little bit for permissions to propagate, you should tell me that in the client. Like you shouldn't even display the component, put a spinner over for it or something and say like working on it and you need to wait like asynchronously until the other end gives an act and says, Hey, all the permissions have propagated and we're ready to go.
If there's some type of weird asynchronous flow that's going on where like when you click a send button, it's creating the loop component and then you're waiting on another async call to do the permissions later. Uhhuh . Like there could be a weird timing issue there. Like don't make that an asynchronous call, make that a synchronous thing. Create loop component set permissions display, right? Like and then display block. Yeah, just block the display of it.
don't make me have the cognitive load and the overhead to think about that. Especially in a world, uh, where you and I are in the same exact tenancy, same a d tenant, same set of permissions, like all the, like all those things. You're not logged into that tenant right now, are you? Cuz I can't even send you one in the call we're in now. Probably not. I. Wanna go back and try this again anyways.
We can go play with it more. But I think yes, it was one of my, that's only like the second or third time I've interacted with Loop and it has been a showstopper frustrating experience each and every time I've done it that it makes me question sanity as a user, which isn't a good place for any product to be. Yes. I would agree with that, but Scott, you can play with it more now because we've had loop components.
But I did see that the Microsoft Loop app, so now we have a standalone web app for Loop is coming to public preview this month is the current plan. It's gonna start rolling out over this month. Go check your message center because there's a whole bunch of stuff in here. I actually saw it on Twitter before I saw it in the message center and they haven't figured out here. Here's a shocker for you Scott , they haven't figured out the compliance stuff for it yet.
So while it's coming in late March, it is not enabled by default coming late March. They are rolling it out, but it is going to be disabled in tenants by default and you actually need to go over to your cloud policy service. So the, what is it? config.office.com. Yes. config.office.com and go tweak your policies and config.office.com to enable the Loop app so that when it does come, so you can go enable it now, like it's in the policy.
I already went and enabled it so that when it does get deployed to your tenant, you will be able to use it and play with Loop even more because I know that's what you wanted to do. Scott . I really don't know that I want to, it's . It's just another, it's another one of those things. I think it's going to be a, it's gonna be a very SharePointy experience and I often feel the rough edges on SharePointy experiences. Like I always want them to do more.
And I think we talked about this the other the other day too, but uh, we, we did it a little bit more privately. We can do it again here. Uh, Purpose-built tools are purpose-built for a reason. Like they're very good at doing that one thing. So if I look at say, component or a piece of SharePoint, like lists, lists is like, okay, but it carries just all this overhead and baggage of being ultimately
just a SharePoint list, right? It's not really a Microsoft list, it's a SharePoint list and Carries, that's a SharePoint list, all the constraints that come with that. So rebranding aside, like it's got a lot of cruft and it's got a lot of overhead and it's got some weirdness to it because it was designed to live in SharePoint.
I fundamentally believe that, right? Like there, there's constraints in the underlying platform that drive to, you know, how these things come out and, and are manifested for end users. So if I think about like Microsoft lists and then Uhhuh , maybe you go use Airtable, sorry, air tables a hundred million times better like . It, it is, but it's purpose built. Do I get the document integration and Power Automate and all those things?
Like absolutely not. Like I don't get that whole ecosystem, but if I don't need that ecosystem, like a purpose-built tool might actually work a little bit better for me. And the reason this is kind of coming to mind for me is that lots of people compare Loop and especially kind of the standalone loop, like, hey, let's go to loop.microsoft.com as a notion replacement and notion is for all its flexibility, a very purpose built tool.
Like it does have a line where it stops trying to be the everything for everyone kind of tool and it knows its limitations and it just, it wears those words right out there. Like you can find it in the documentation and you just know you go like, oh, this isn't gonna work.
Versus something in SharePoint land and Microsoft 365 Land, quite often the answer is, well like that could work if you did this and this and maybe it'll work if you did this and what if you integrated this thing over here and you did this and you're like, I I I don't want to do that. I just kind of like want to to work and I want to do work, so enable me to do some work. Please. Yeah, and I think I totally agree.
Like I think Loop could have been or could be, it'll be interesting to see when it comes out and we continue to play with it more. But in your, to your point like was SharePoint and OneDrive fundamentally the right place to store all of these files that are going to be your loop components, your loop pages, all of that. Ultimately those are gonna live in SharePoint. Just like you said, Microsoft lists are SharePoint lists, OneDrive document libraries, they are SharePoint document libraries.
And there are absolutely fundamental things that, to be fair, technical deck carried all the way through from the early days of SharePoint. For those of you that have dealt with lists, maybe you haven't dealt with SharePoint lists and specifically, I still run into this all the time, is the whole 5,000 items in a query and in a list is like, I don't know if it was 2003, 2007, but fundamentally it's because all of this stuff lived in a sequel database.
When you would go to query more than 5,000 items from the sequel database and would put locks on the database, that would cause the entire platform to run slow carry it forward 15 years. And we're still dealing with it because of that technical debt. And when you do things like Loop and SharePoint lists and you try to treat a SharePoint list like a database, you inherently run into these things.
To your point, other places are much better to build our li large lists because that's what they were designed for. And when you try to make a SharePoint list a database or even this like are we trying to make SharePoint the storage space for Loop when it should be like, oh, living in blob storage or living in some other backend service that isn't going to introduce some of the constraints.
Now I also get the fact that SharePoint makes sense because you already have all the identity stuff, the permissioning stuff, the sharing stuff. There's a lot of stuff they don't have to build by putting it in SharePoint, but they also do incur that technical debt and those technical limitations that live in SharePoint really because of how it was built 20 years ago. Yeah. . It's interesting. Do you feel overwhelmed by trying to manage your Office 365 environment?
Are you facing unexpected issues that disrupt your company's productivity? Intelligent is here to help much like you take your car to the mechanic that has specialized knowledge on how to best keep your car running Intelligent helps you with your Microsoft Cloud environment because that's their expertise. Intelligent keeps up with the latest updates on the Microsoft Cloud to help keep your business running smoothly and ahead of the curve.
Whether you are a small organization with just a few users up to an organization of several thousand employees, they want to partner with you to implement and administer your Microsoft Cloud technology, visit them at intelligent.com/podcast. That's I N T E L L I G I N k.com/podcast for more information or to schedule a 30 minute call to get started with them today. Remember intelligent focuses on the Microsoft cloud so you can focus on your business.
You also experienced a question going back to using things for what they're built for and challenges that can come with it. What about if you try to copy stuff out of SharePoint with AZ copy? I had a question from a colleague come in and I think it's an interesting scenario. Uh, so first let's kind of start with the question. I have a colleague that's working with a customer and that customer is looking to migrate a couple terabytes of data from SharePoint online to Azure blob storage.
Like they clearly wanna pull some of those files, documents, whatever they happen to be out of SharePoint and push them over to Blob in and of itself. Like, okay, fine. There were some things that didn't even cross my mind in this conversation when I was chatting with a person. So the initial ask, like just to the number that they asked for was higher than this. Well,
like let's peg a number to it, right? Like, so I wanna migrate a hundred terabytes of data, which means I have a tendency with approximately 10,000 users give or take to get like the requisite licensing, right? It would be someplace in that order of magnitude, right? You, you're more than five licensed territory or, or you've bought a, a whole big chunk of storage from Microsoft, but they wanna migrate a a hundred terabytes of data from SharePoint online
to Azure Blob and they're already trying to do it. And the, and the way they're trying to do it is using a storage explorer, which I have to imagine, like, I didn't even ask the question, but I have to imagine what they're doing is either using synced libraries that are already synced through OneDrive for business or they're using the, like the file explorer view and kind of the mm, the, the wonkiness that comes along with like that,
that weird web a not web d a thing that they have going on these days. That thing that should have never been in SharePoint to begin with. Never should have been around to start. I I guess a couple of things here. So one is, uh, storage Explorer as a tool uses AZ copy under the hood for its underlying copy operations.
So you can think about like dragging and dropping a file from say, your desktop into a storage explorer window that's mapped to a storage account in Azure as an operation where Storage Explorer is acting as a nice wrapper and a nice proxy and it's gonna build out and it's gonna say, Ooh, I had a file dropped in here, so I need to call AZ copy CP with this source, this destination and push all those things through.
And if you actually look in the, the logs for Storage Explorer, you can see that it's calling the commands for AZ copy in the background for those kind of core copy operation things, right? The the puts and the GIS and, and all that kind of stuff. So. Just quick question, is there any performance loss by using storage copy versus just AZ copy in the command line?
Or is it really gonna be about the same where Storage Explorer doing that type of stuff as just a friendly cover on top of AZ copy, kinda like the Azure portal is a friendly cover on top of rest calls? Depends on the operations you're doing. Like there's, okay, there are things that Storage Explorer does that are not functionality that's provided by AZ copy really we're talking about like the underlying data movement engine of storage Explorer. Is just AZ copy is.
AZ copy and a standalone, uh, thing for it. So there might be instances where Storage Explorer is great for you, like you need a gooey, you need to browse it, you wanna do something like calculate folder statistics, anything like that. Like awesome, go ahead. You wanna view apples on like ADLs, gen two storage account ton of file or folder like great use storage explorer
for that. Like that's native functionality to it. And then, you know, you wanna copy data, use AZ copy if you have a large data copy to do, you might not want to use Storage Explorer as an extraction layer because it's just adding more overhead at that point, right? It's, it's an orchestrator for you, but ultimately it's orchestrating a single instance of AZ copy.
So if you're trying to do, you know, a large copy operation, and I'm gonna define large in like the hundred million plus file range, like say you're trying to copy half a billion objects, 500 million of those, you might wanna orchestrate that with like multiple running versions of AZ copy or even AZ copy running for multiple machines or things like that.
So Storage Explorer may or may not cut it for you in that scenario, you know, as, as as kind of an orchestration or or or proxy thing that's sitting on top of, on top of easy copy. Got it. So back to our scenario. We've got a a hundred terabytes of data in SharePoint online, not very clear where it sits. Does it sit in, you know, a,
a set of libraries in a single site? Is it multi-site, like whatever, I don't know that it matters, uh, a whole ton, but just from off the cuff from like the question I got, it's like, ooh, it sounds like they're maybe trying to do something unnatural here. So I was just trying to think through like what are their options And back to the, I don't always think about scale thing. I think the way you might, I saw a number that ended in TB and I was like, oh, it's terabytes.
This is easy, right? Like you can totally use AZ copy, you can use AZ copy to do multi petabyte migrations. So like, we're good there we're golden. So my first thought was like, let's just sync it back to OneDrive for business.
Like pull the data down locally in a client and then go ahead and continue to use Storage Explorer or just point AZ copy at that folder where OneDrive has synced locally and you know, don't do it on like OneDrive on like your client on your laptop while you're sitting on the beach. Spin up a VM in Azure that's in Microsoft's wan where it's gonna be able to do a semi performance synchronization through OneDrive for business.
And then certainly like run easy copy from that op uh, from that VM as well. So you're all on the same backbone and, and everything's gonna kind of go a as quick as it can. And you quickly pointed out to me that there's some uh, some potential limitations just around OneDrive for business, whereas I'm like, I don't know, it's just a couple terabytes of data. Who cares? So. This was a really fascinating question.
If there's a customer out there that's trying to do this and they want somebody to help them call me, cause I would love to do work through this, but this is not, again, from your perspective blob storage insignificant from a SharePoint perspective. So here's where my head instantly goes is continuing with our a hundred terabyte example, a hundred terabytes is one and a half. I just did the quick math 104 million megabytes
SharePoint. Naturally, it's probably a lot of office files would be my assumption. I don't know for sure you'd have to go do some looking, but let's say your average file size is one meg, maybe even two megs PowerPoints. Excel could be bigger Word documents could be smaller. You're looking at like 52 million files where my head instantly went when you said sync it with OneDrive. Is OneDrive in the TSH or in the service description document limitations.
And I've seen this, uh, play for optimum performance. Microsoft recommends storing no more than 300,000 files in a single OneDrive team or library site that you're going to sync. And if you are going to use sync, the same performance issue can be seen. If you have more than 3000, 300,000 items across all libraries, you are syncing even if the libraries themselves are smaller.
So if you're talking 52.5 million files and you can never sync more than 300,000 items, assuming 300,000 items are evenly dispersed across all your document libraries and sites and they're not all in one, it's a lot of syncing and unsyncing to get that amount of files onto a local computer where you could even begin to use something like AZ copy to copy them up. I mean maybe you go stand up, what, 300,052 million divided by 300,000.
You're doing it 175 times. No, maybe you're a large enough company. Again, a hundred terabytes, you're looking at probably 10,000 ish licenses, 175 people to do this. If you can get a bunch of people working together, maybe not the end of the world, but now you're trying to coordinate who's thinking what, who's copying what, not missing anything. This is not an insignificant amount of work. Fair enough.
So one of my other thoughts and I and I tend to go to this back from like my old migration days cuz I was used to always going from say like an on-prem SMB file share to SharePoint. You really didn't come back the other way too much, right? But you definitely had this machination, you went up where you, you went a certain direction and I think Microsoft has built out tooling over time like the SharePoint migration tool, uh, thing,
things like that that can help you out. So my other thought was like, well hey let's go to our old friends at an ISV or like a migration vendor like sharegate and just point sharegate at that thing cuz it's got a script ability engine, it has PowerShell support, all those kinds of things. Mm-hmm . So let's say it wouldn't matter a hundred terabytes of data if it was one terabyte per SharePoint site.
That's only a hundred sites of data you have to pull down, script it all out in something like a sharegate and let SHAREGATE just run again. Let it do its thing. So again, let's spin up a VM in Azure that's in the region probably with my storage account and just install sharegate on it, right? Point it at it and let it crank, let it download everything. Once it's all downloaded, then I'm just gonna push it up the other way with AZ copy and let it go. How viable is that?
That's probably where I would go first. Although I was just looking so I couldn't remember. Cuz again, I've always gone the other way with you. Go get a share gate, go get a AvePoint, go get Metal Logics, which is now Quest I believe, which is actually now Dell, right? Dell owns Quest owns Metal Logics. It just keeps going. Yeah. So. Sharegate itself, I'd have to go look at Questa Metal Logics, sharegate, I head handy. Sharegate does not support a local file system as a
Target. It supports it as a source. So you can go to your point from local drives up to SharePoint, but you can't go back down. If you're gonna go to a Target, it's SharePoint on prem online. One of the other third parties may do it Mover, which is now Microsoft Migration Manager maybe used to support it. They were pretty flexible. Though. I know. Uh, SPMT only goes one way. It's it's on-prem up kind of thing. Mover. Is too, cuz they've started pulling functionality out of the old mover.
Do IO when moving it into Migration Manager, which migration manager is different than the SharePoint migration tool. Yes. for those who are listening in audio, like Ben's a big, my head just. Mind. Exploded, mind blown emoji right there. Yeah. I'm. Legitimately trying to think how would I do this? Like there's Logic Apps and Power Automate that let you Connect. That's. The next place I went to. I said, hey, if it's ongoing or even point in time migration,
I don't know that it necessarily matters. Logic Apps, power Automate and Azure Data Factory have connectors. Or at the very least I can just do an HDP call and make everything work through the HDP connector and get it to where it needs to be. You might have some scaling issues in there given number of calls, number of API calls, things like that that you have to consider along the way. But it should be doable.
Should be maybe. So I tried to do this in essence, I have a client that we're doing, we're not doing this, I was going to SharePoint from SharePoint, right or wrong, as this may sound. Their process and the way they did their work was they had certain projects, these were large projects, think like large, relatively speaking, not large compared to what you're used to storing large projects of a folder in SharePoint that would contain four or five, six gigabytes of data, uh, thousands of files.
So much smaller than millions of files in terabytes of data. And their workflow was, once this project reached a certain point, it moved from one SharePoint site to another SharePoint site and then it had another status and moved again. And then eventually the part project got archived off and it moved again.
So we did, I tried to do this with Flow originally where it was doing something similar to that where they could go flag a project, flag a folder, update metadata, whatever it may be to get flow to actually do the move in the copy. It didn't work. The data was too big. Ultimately what we ended up doing is I have a hybrid runbook in Power Automate that actually uses sharegate on a server sitting in Azure. So they trigger a Flow logic app.
The Flow Logic app runs, runs a hybrid worker in Azure Automation runs the PowerShell on the local server to use sharegate to move the data from one site to another site because flow and logic apps couldn't natively handle it, nor it could even just running PowerShell in Azure automation. I think it's really the SharePoint endpoints, it's those APIs you start handling hitting throttling limits.
That same article that talks about, and I completely lost it on all my browser windows when it talks about the limitations of moving, let's see if I can go pull it up here. Uh, moving and copying files across sites. So copy move operations. A single operation has three requirements. No more than a hundred gigabytes of total file size. No more than 30,000 files in each file must be less than 15 gigs.
Now maybe maybe moving and copying down locally or using Azure Data Factory or something like that could work around it. But you're, I think you're still gonna start hitting thresholds and constraints with some of that stuff. That's probably your best bet. The other thought that is kind of in the back of my mind, but again it's still using rest calls and API endpoint that can get throttled would be just to use PowerShell to download the files at a SharePoint.
Just write a huge PowerShell script or write one that you can point at a site and just go aside at a time, run the PowerShell script, download 'em all locally and then use AZ copy to move 'em up. Yeah. I, so, so that was another thought that I had was you could be slightly transactional in nature about this. So AZ copy has things like command line, pr, uh, parameters where you can point it at a say like a folder of files that are sitting locally for you.
But ultimately what it has to do is enumerate those and every upload operation is gonna be kind of a, a separate call and a separate API call. So if you're processing a objects to a certain degree in a linear fashion, you know, you're processing them, you know, in a, in that way versus a highly concurrent model where maybe you're gonna process like 10 million at once kind of thing. Like that might be hard for you to pull off something like Casey copy. It would be a long running operation.
But depending on where the files are hosted in SharePoint, like let's say it is a hundred sites, go into something like Azure Automation and create a runbook in Azure automation where you don't even have to use AZ copy. You could actually use something like just the Azure CLI or Azure PowerShell to
do this as well. Like if you want to be PowerShell driven, like you said, hey, let's go ahead and download through PowerShell using the SharePoint APIs, whether you're doing that with PMP or just native kind of SPO commandlets and then just call an upload operation on the other side and script it out that way, right? Like call this site, get everything in it, download one, upload one, download one, upload one, and then concurrency. You'd have to figure out how to kind of spread that out.
Like how do you get peanut butter all over the place so that you can get the scale and throughput that you want. Like it is kind of an interesting problem. I, I kind of thought about it as easy in my head again, I started with, it's got tb so it, so it's easy in it, right? Like earlier today the, the very first customer conversation that I was working with a customer on was a uh, six petabyte data lake . So I saw 200 terabytes. I was like,
you're my easiest problem today, right? Like yeah, you got nothing going on. . Yeah. See and so you mentioned Azure automation. We may have done a podcast on this a long time ago.
You would something this big, you would have to run it in VMs if you wanted to script it with Azure automation, it would have to be hybrid worker or you'd have to run it just natively a VM and Azure because Azure automation, you would start hitting the fair use limits where if you start consuming too much memory, too much CPU or your script runs for too long, they essentially boot you out of the queue and restart your job length of time.
I've had it happen to me as quick as 30 minutes I've gotten away running something as long as two hours. This amount of files, you'd be way beyond two hours. And because you're dealing with files, I would assume your CPU and memory consumption in whatever's doing the processing and the backend, they boot you out way sooner than two hours. So I don't think you could even do this with Azure Automation successfully, uh,
without doing a hybrid worker, without running it natively on a server. So, gotcha. So at that point you just orchestrated on a server anyway, it's just a power sell script, right? Run it, do the Rone thing, write it and then set it, forget it and let it run and do its thing in the background. I think that's what I would do. I think my approach would be scripted out,
run it on a server. If you wanna run it in parallel to your point, if you have a hundred sites and you are okay spinning up a hundred servers in Azure, just spin up a bunch of servers, dedicate a server to it, it might help with some of the throttling.
I don't know what they watch from the SharePoint side in terms of throttling when you're hitting the api, if you're coming from a hundred different servers, maybe you don't get throttled as quick cause you're coming from a bunch of different IP addresses. Maybe they recognize that it's coming from the same tenant and going to the same blob storage and they're gonna start throttling you. But I think PowerShell on a server to download it then AZ copy to upload it is
probably your best bet. All right. Well I, I mean if anybody else has any good ideas, uh, , let, let us know. This is almost where I want like the bulk export tool from SharePoint. You know, like kind of like we have like Azure import export where I can just send them a disc or have them send me a disc. I almost want you to just be able to send me a disc of my stuff, just do a dump and, and send it out the other side, put it in the mail. And if it gets to me in two weeks, do. They have that?
I don't know. I don't, I don't think, I don't think for SharePoint such a thing exists. There's bulk import in the form of these tools, right? Like, like SPMT and mover.io and some of the other things you mentioned, but I don't think there's a bulk export tool. Like, not even on the compliance side, right? There's nothing that I can say to go in and gimme all my files.
I do believe I was, I was doing some quick Googling on the side, like I did find, like there's CIS tools makes a document downloader. So maybe there's somebody else out there that makes, you know, something similar if that wasn't your jam. But it does look like there are a couple, at least a few kind of paid tools out there that would let you do it. So yeah, I, I don't know. I thought it was an interesting question though. Yeah.
Absolutely. As, and I would be curious if anybody else has any other insight into it. Because one, I've, to your point, I've never gotten this question before about, I have all this data, but I guess it kind of makes sense, like if you're looking to archive stuff, we had some episodes too where you talked about SharePoint as a files server. It's an expensive place to store a bunch of files if you're just storing them there and never touching them again.
Like I can see where this could become a request. As companies create lots of data and they have older files, they don't delete 'em, they wanna archive 'em somewhere. It is way cheaper to store 200, a hundred, 300 whatever, hundreds of terabytes and blob storage than it is SharePoint. SharePoint you're paying, again, I figured it out. I did the math. You get 10 gigabytes for every user. So every user license gives you 10 gigabytes. One terabyte is a hundred user licenses.
So you're either paying for a bunch of licenses if you're going out and buying file storage, it is still 20 cents per month per gigabyte. 20 Cents per month per gigabyte. Eh, it starts to add up. It starts to add up compared to blob storage, especially if you can go into cool or archive. That's fra Is it like fractions of ascent? Yes. Yeah, . Yeah. SharePoint storage is 20 times more than archive blob storage. There could be a valid use case for this.
Maybe Microsoft should go build a feature for archive to blob storage out of SharePoint when you get old files. That would be kind of a cool tool. . I I mean I, I certainly like the idea of it, right? Like, uh, pay me money, . That'd be, that'd be great. Yeah. Huh. Fascinating. All right. I'm curious, like Scott said, if you have feedback, let us know. Twitter, YouTube comments on the blog post, associating the podcast, whatever you want to do. If you're part of the membership,
just give us feedback and Discord and we can share it on a future podcast. But. I was gonna say, if, if you were a a member, you could come along and participate as we record real time. You could see our yes, our made for Radio faces while we're here streaming off into the ether. Yes. Give us live feedback and input on all of our crazy ideas that we come up with. Indeed. All right. Oh, I think that's it for me today.
I thought we'd get through that one in 15 minutes and here we are once again, you know, 35, 40 minutes into it and still debating. Still debating. We do have a question we need to get to it. We do have a question in Discord from one of our members that we have not gotten to that we should put it on the list, Scott, in the next week or two to answer that one cuz we can answer it in Discord.
I've thought about going in and typing it up, but I think it's another interesting one that would be good discussion in a future episode. Next week. First thing, I'm gonna write it down on my Post-It note of things that I need to make Ben do, uh, we'll discuss. We would do it. Listener questions from Discord first. All right, sounds like a plan, Scott. And with that, go enjoy your weekend. Perfect. Thanks Beth. If you enjoyed the podcast, go leave us a five star rating in iTunes.
It helps to get the word out so more IT pros can learn about Office 365 and Azure. If you have any questions you want us to address on the show or feedback about the show, feel free to reach out via our website, Twitter, or Facebook. Thanks again for listening and have a great day.