The Past is now in 3D!

The Past is now in 3D! Steph Strock This podcast is brought to you by Tāmaki Paenga Hira Auckland War Memorial Museum. Jenna Borst

I don't know about you, but putting on a pair of cardboard glasses with a flimsy sheet of red plastic in one side and blue in the other was my first experience with the dazzling and kind of janky world of 3D. But a lot's changed since I first donned some 3D specs to enjoy the ‘haute cinema great’ Spy Kids 3D a couple of decades ago, and those changes are having ripple effects through the world of museums. From life sized printed moa skeletons to illuminating the tiniest cracks in ancient ceramic bowls. We're at a fascinating moment in time where technology is intersecting with the past in ways previously unthinkable, and the applications are only limited by the extent of our imaginations. Kia ora, I'm Jenna Borst, your host for this episode of The Amp, the podcast from Auckland Museum, which amplifies the incredible stories from our collections, our mahi and our place in the Pacific. Today, we're tackling the world of 3D digital technologies and finding out how they're being used to better understand, care, for, protect and present our collections. Where we're going, we don't need roads…just a stable internet connection. Buckle up and join us for a journey to the rapidly approaching future. But first, how does all this 3D stuff work? To break it down a bit, capturing a 3D image is a process that takes the exact shape and sometimes colour of a real, physical object and then creates a detailed digital model on your computer. 3D printing reverses that process, building physical objects, layer by layer from a digital design. Out in the real world, this might look like your dentist scanning your teeth and creating the bits and bobs that keep your smile looking good. Or you might have heard all the wacky things people are 3D printing from the internet. Here at Auckland Museum, we've been making 3D models for about 10 years, which is a long time in technology. Back when we started CDs, DVDs, even iPods were still king, but like the way of the very first entire-room-consuming computers. As time goes by, technology gets smaller, cheaper and more accessible.

Josh Emmitt

Well, it's been used in museums since the 90s in various capacities. So it's used for creating, obviously, 3D objects of our taonga, and that can be used for digital displays, could be used for outreach, so having it on the internet and people can interact with them. Some museums have ventured into 3D printing objects as well. So, it's any kind of range where we'd wanted another copy of the object, more or less, but also for reasons of conservation and being able to have something that you can take out the museum more easily just have an idea of what it looks like in scale. So yeah, whatever your imagination came up with, really. My name is Josh Emmitt, and I'm the curator of archaeology here at the museum.

Jenna Borst You might remember Josh from episode five, Collecting Egypt, not only is he our resident Ancient Egypt expert, he's also carving out ways we can use 3D technologies in our archaeological mahi. Josh Emmitt

So 3D technologies have a long history in archaeology, really going back to almost pre-World War One, where we've got models being made of excavations, actual models for people to explain how their excavations have been, what they looked like and what they found. And in archaeology, we record stratigraphy, which is really the side of the walls of the trenches. And in some cases, we're getting people recording those quite frequently, close to each other. So the idea being, here's an idea of what this looks like in slices, like you're cutting a cake. And what we can do now is we can digitise those and put them back together to get a 3D model of what their excavation looks like, so that people have always had a desire in archaeology to capture 3D.

Jenna Borst 3D imaging is now a pretty standard part of a history pro’s toolkit. It's like having physical props for telling a story, long before Spy Kids 3D was even dreamed of, 3D representations have been used to share glimpses of the past from physical models or casts of places, people and things, we now have non-invasive imaging techniques to use, whether they're based in the museum lab or out in the field. Working alongside Josh to make our 3D models is Richard. Richard Ng

My name is Richard Ng. I'm the senior photographer and media producer at Auckland War Memorial Museum. Jenna Borst Richard's probably got one of the coolest jobs at the museum. He and the rest of the team produce stunning images of the collection from microscopic insect larvae to cars, sharks, airplanes and beyond.

Richard Ng My team and I are responsible for creating the digital surrogates of all our wonderful taonga across natural science departments, documentary heritage and human history, and obviously archiving them and preserving them for generations to come. Jenna Borst

There's a lot of behind the scenes work in capturing images and however many dimensions, two or 3D. If a picture is worth a thousand words, a 3D image? Well, that's maths I'd rather not work out to be honest, it's a complicated business. Richard Ng

So, you can imagine every single corner, every single physical surface in space, there's a dot to it. And with these laser scanners, they can do like 2 million dots per second. That's literally planting all these dots or needles in space 3D space to create this 3D model. Jenna Borst Is this making you think about those toys where it's a bunch of pins in a frame, and if you press your hand (or face) into it, it shows on the other side?

Richard Ng But photogrammetry is vastly different., Jenna Borst Photogrammetry by the way, is a specific type of 3D scanning that uses a lot of photographs taken from different angles to create 3D models. You might have already used a product of photogrammetry without even realizing it when you travel the globe using Google Maps in 3D mode. Here's how photogrammetry works. Richard Ng

So the object doesn't move, but we move the camera, the lighting stays stationary, and in doing so, we capture about 50 or 60 images of that object from different angles, and we process those images in the software called Agi Soft meta shape, and it just computes. It just finds points that are similar and literally stitch them up into the 3D space and create a 3D model for us.

Jenna Borst A great deal of creativity goes into figuring out how to take images of such a vast range of taonga that come in all shapes and sizes, from taxidermy elephants to pocket watches, our collections are so broad, there's no ‘one size fits all’ solution. Richard Ng

So what we learned to date is that 3D scanning is really good for high accuracy matte objects, but 3D scanners, especially laser scanners or structured light scanners. They hate shiny objects, so we have trialled a few times using 3D scans of wet objects like waterlogged taonga, which the scanners hated because of the refraction. So trying to find the right solution for the job is actually photogrammetry was great. So as long as we control the lighting to minimise reflection, we were able to take still photos of that wet taonga and to be able to stitch it afterwards and get a really good model out of it. And that goes with like ceramics or silverware or something that's like, really reflective. So, yeah, scanners don't like that. So we're building up the knowledge base to quickly, very quickly, isolate, identify which technology works best, and you know, and arming ourselves with these options.

Jenna Borst At Tāmaki Paenga Hira, we look after a huge range of items with different textures. To capture the best 3D representation, we have to take the lead from the taonga itself. Richard Ng

We scanned a lot of skeletons and skulls for researchers, and also we've tested a lot of butterflies, wētās and small insects. So, this is where we are exploring the macro realm, where photogrammetry has a weakness in because the smaller the object, the harder it is to get everything in focus. So we need to use macro lenses. And once you start using macro lenses, you lose the depth of field. So we need to combat that. So two of the big challenges that we have to date is actually small objects and shiny objects, and we're trying to work our way through problem solving and coming up with solutions for photogrammetry to be able to capture these challenging objects.

Jenna Borst

You wouldn't think it, but it's actually easier to scan something huge than a tiny little shell. Richard recently scanned one of the largest taonga under our care, Te Toki a Tapiri the great waka taua war canoe in Te Marae Ātea Māori Court. At 25 metres long, the waka stretches across the gallery, taking pride of place. There's a lot to say about Te Toki a Tapiri, in fact, there's a whole episode in the wings. But as part of a recent and intensive revitalisation, the waka was scanned, capturing amazing detail from every angle throughout the project.

Richard Ng

At the very start of the project, we decided that not only that we going to take archival photos of the entire Walker, but we are interesting challenge to create a 3D model of a 25 metre waka. So to start off with, we explored a few three scanning technologies, and we came up with using the Leica 3D scanner. It was very easy to get the scan, because this scanner pretty much self operate by itself. It's quite a dimly lit gallery, so our 2D images, because it's all done archivally, it's lit properly, which you know, the images can only say so much. But with a model, there are cracks, joints, kind of details to the waka that you will not be able to see unless you rotate the model, which adds another dimension, another layer of story, that kind of speaks of that object, which is really interesting. So yeah, constantly, we kind of always amazed at the images that we get to get, literally shedding new light into the taonga, into the objects using 2D I think 3D scanning and 3D models actually adds another dimension to it…literally.

Jenna Borst It's a combination of art and science. Richard Ng

Whenever there's an object that we need to scan or image, my mind would race. It's just like, how to light it, and what do we need to light it for? And what is the ultimate purpose? And it could literally be a 2d image, one shot studio lighting all the way to, you know, soft lighting, 50 shots just to get a section of a model. So the range is huge, and that's where our team, I think we're kind of very grateful that we have these challenges that kind of push us us in what we create, and also upskilling in terms of what we need to do to meet the demands.

Jenna Borst

You might not have noticed it when you've visited the museum before, but behind almost every great display lies a bit of 3D tech. Part of the smoke and mirrors that make up our world-class displays are mounts that keep things safe, attached and looking good. Even if all the conditions are right in the case, a small vibration would cause a lot of trouble for delicate items if they're not properly secured in place. Time to meet some of the people behind the mounts, behind the taonga, the display team. They use 3D tech to make their mahi even more impressive and effective.

Heath King Hi I'm Heath. Jess Wagstaff And I'm Jess. We're from the production services team at Auckland Museum. Jenna Borst There's an art to presenting art, and a lot of things to factor in. When dealing with delicate taonga, Heath King

There's a few considerations the material that the mount is made of, sort of specific areas that might be weak on the object, which is sometimes sort of worked out in coordination with conservation. We'd meet with conservation and sort of assess it from that perspective.

Jess Wagstaff One of the things that we'd talk to conservation about is the material that the object is made from, and like Heath said, any fragile areas we need to avoid, because we want to support the object entirely and keep it secure while also making the mount as unobtrusive as possible. Jenna Borst The display team embraces 3D tech and its day-to-day work. Jess Wagstaff

In the past, we've used the handheld 3d scanners to scan objects and then print them out, using the printed object as the reference for mount making. This is particularly good for items that are fragile, because it means we can work on the mounts without having to handle the objects too much, because often excessive handling can cause more damage than not handling it too often. For example, there was a pumice bowl that was too fragile to be handled repeatedly. And there's also an atua in there who is very, very heavy…and to put him into a mount being made repeatedly, was very, very difficult.

Jenna Borst By working with 3D scans and lighter reproductions, the team can work with more agility and precision. Heath King

Without the 3D scanner there's, there's sort of more steps in terms of, I suppose, the interface with the object, because the object is stored in a safe and clean area, and where you're making the mount as often it's a metal workshop, so there's grinding and lots of stuff going on, so you have to have a pretty well thought out process to do the measuring of the object and the templating and checking and all that. So it makes it more complex and more time consuming, whereas if you can actually take the area, the mount has to fit and scan that and print it exactly to size, because obviously, with the digital technology, it's very exact. So if you make the mount to that, you know it's going to fit without having to sort of work with the object itself,

Jenna Borst Keeping taonga safe and working smarter, not harder. Heath King One of the challenges we had that did involve the 3D scanning and manufacturing law mount making process was in the Volcanoes Gallery, where there's a lava bomb, which had been put on display without a mount that supported it adequately so it was loose to move, so that we thought that would be A good opportunity to use the 3D scanning technology. Jenna Borst

Lava bombs are basically a mass of partially molten rock that get flung out of volcanoes during an eruption. You probably wouldn't want to see one in real life, but you can get an idea from the safety of the museum's Volcanoes gallery. Heath King

And so we removed the lava bomb and scanned it, and then edited the file in CAD software and sent it off to get 3D cut on a CNC machine. And then when we got the pieces back, we assembled the plinth again, and it changed the plinth changed the mounting to be very secure. And yeah, I think it was an improvement to the gallery. Jenna Borst

The team doesn't just make displays and mounts. They're like our in-house makers, and now they've got some pretty cool new tools to help all parts of the museum tell the story of the collections better. Right now, Jess has got a project underway for the Museum's Learning department, whose team help bring our collections to life for school groups. Jess Wagstaff

One thing that I'm currently printing for Learning is shark skin. They have a marine education programme that they're kicking off sometime soon, and so they want a handling object that kids can touch and sort of learn how the texture works. So we often get requests like that. Heath King

Yeah, the 3d printers are actually running a lot of the time. So if you separate the printing and scanning, there's a lot of work that the printers are doing. Like they're a very useful, sort of practical tool that might not be for Mount making, it might be prop making or making brackets for some other display component, potentially or so, they're very useful versatile tool. Jenna Borst

There's a lot you can do with another dimension in the works and a lot that can be discovered. The museum is a centre for looking after and sharing taonga, but a lot of that work takes place beyond our walls, and sometimes that work can be quite unexpected. Ricki-Lee Erickson is the Collection Manager of the Land Vertebrates department. A few years back, she got the call to come and take a look at a very exciting find. Ricki-Lee Erickson

So on the 27th of March 2022 after a massive storm hit Auckland, a couple, Matthew Brown and Ava Peters, made an unexpected discovery when they were just walking along a beach at South Head Kaipara, they found what appeared to be a massive boulder with indentations of what they thought were moa footprints, and they contacted us at the museum, and then we contacted Ngāti Whātua o Kaipara, who's the kaitiaki of that region, and we headed out there to see if these were, in fact, moa footprints or or maybe something else.

Jenna Borst The sandstone block, weighing nearly three tonnes, had fallen two and a half metres from the cliff above the beach where it had been quietly hiding for over a million years. Ricky-Lee Erickson

And when we got out there, we could see immediately that there was no doubt that there were moa footprints. They were sitting vertically in a boulder that had slipped down from from the cliff, so they were almost perfectly poised to be presented as you're walking down the beach. It was quite amazing. Jenna Borst

24-hour surveillance of the footprints was quickly put in place to ensure the protection and safety of this incredible find. Museum staff, iwi representatives, Palaeontologists and geologists worked together to come up with a plan to retrieve the footprints and bring them to a secure location to be cared for by Ngāti Whātua o Kaipara. Moving an incredibly delicate car-sized piece of sandstone is already pretty challenging. Then add to mix the tiny, wee problem of the ocean coming in to sweep away the precious find

Ricky-Lee Erickson We were working against the tides as the bottom of the boulder was actually submerged by the water during high tide, so we quickly had to whittle it away to something that was manageable to carry. Jenna Borst There was no guarantee the move would even work. A scientific find, a million years in the making, was on the line, but luckily, the team had a trick up their sleeve, just in case.

Ricky-Lee Erickson Well, at first, when we were on the beach, we used photogrammetry as an initial way to preserve a digital record of the moa footprints. Jenna Borst Remember, photogrammetry is basically taking a bunch of images of an item and then stitching them together on your computer.

Ricky-Lee Erickson So Daniel Thomas, who was one of our collaborators, he's a palaeontologist, he did that initially to get a good idea of what the prints looked like, as we weren't sure at the time whether the excavation would succeed. Jenna Borst Having taken a 3D record using photogrammetry, the team went ahead with the move, knowing that at least if anything happened, the discovery was safe in the digital realm. Ricky-Lee Erickson

There was no way we could remove the block, and we basically had to shave off the back half of the boulder and also trim the sides down. And it still took six people to carry off the beach. It was very, very heavy, but and it did, ended up splitting during excavation, which ended up probably being a good thing because it was so heavy and it probably might have split later on in a more damaging way. It split just between the top of one footprint and the next one. So it was very, very lucky,

Jenna Borst Finding moa footprints preserved like this in such a fragile medium was incredibly rare. Most tracks or footprints disappear shortly after they've been left destroyed by natural forces. Once the team got the prints back to a facility at nearby Woodhill, they set to work. Ricky-Lee Erickson

He was able to do 3D scanning, which takes a lot more equipment and you need electricity. And he did a much more comprehensive scan of the moa footprints. And this really allowed us to take very detailed measurements of the footprints. The moa footprints on the sandstone that we found they were quite large, so maybe a little bit bigger than a human male hand, quite tall and wide. They are really old, and they, you know, would have degraded a little bit during their fall. So we don't know for sure, you know, how much of that print, you know, could have been, but we do think it is very well preserved, and we do think that it was protected quite well during the fall. It was quite miraculous that they weren't damaged. Despite their claws being so large, we're still looking at a moa that's not that tall. And if you look at birds like emus or even pukeko, they have really, really big claws compared to the rest of their body. So even though we thought, wow, these prints are huge, what we're actually looking at is still quite a very small moa. When we did compare this footprint with current feet bones of moa, there wasn't any that exactly matched our moa is slightly smaller than the North Island giant moa, but also very different from the other three currently recognised species in terms of the width of the footprint is a little bit too big or too small, the toes aren't quite long enough or too short. So we weren't able to actually say with confidence that it's definitely one of these four species, but because it's so old, it could also be an ancestor of one of those species.

Jenna Borst So the great, great, great, great great great great, great grandparents of the moa in our origins gallery, probably with a few 100 more ‘greats’ thrown in there for good measure. Ricky-Lee Erickson

And we know from science that ratites, these big, flightless birds, have extreme morphological radiation. That means that they evolved very rapidly and very dramatically. So it's very likely that a million, a million and a half years ago, we're looking at a completely different species. We were also able to tell that it was approximately 81.4 centimetres up from the ground to the hip height. And it weighed about 13.8 to 29 kilograms. So that's about a weight of a 10-year-old child. And it was moving really, really slowly at 1.7 kilometres per hour, which is much slower than we walk, and slower than emus or ostriches walk as well. So it was strolling very leisurely down the beach about a million years ago.

Jenna Borst Just like Matthew and Ava did, over a million years later, the more enjoyed a gentle stroll along the beach. It's nice to think the beaches of Tāmaki Makaurau have been such a special place for its residents, even eons ago, while the physical footprints are being looked after by Ngāti Whātua o Kaipara, the digital data collected from the discovery has spread its wings. Ricky-Lee Erickson

The project team have put together a scientific paper describing the moa footprints with all of these crazy calculations and these 3D scans and everything's in that paper, and that will be published in a scientific peer reviewed journal, which is amazing, and Ngāti Whātua, are co-author on that as well, which is really cool. And that means that the scientific audience can access information about these more footprints in a peer reviewed, substantiated way. Jenna Borst

The sandstone block may have weighed several tonnes, but the data is weightless. Now people from across the world can learn from these footprints and make new discoveries of their own.

Museums are homes of learning and discoveries. Sharing information is a core part of what we do and how we do that evolves as technology progresses - hello, podcast! While initially, 3D modelling was just another way to look at something and still is, we're now able to analyse these images in ways that help us come up with new ways to conserve taonga and learn more about them. In the case of the moa footprints, the technology enabled the team to digitally preserve the find and gather data that expands our understanding of these great birds. Museums are now in a position where not only the written information about items can be opened up and shared online, but so too can representations of their physical forms. That's where things become a bit more complicated once there's a digital 3D copy of an item, there are all sorts of things you could do. You could print it, expand it, shrink it, twist it, bop it, morph it, chop and change and remix any part of it until it's a completely different thing. When it comes to protecting our taonga, it's not just about their physical safety. It's also a matter of protecting the very ideas they represent, however that looks.

Richard Ng Any content that we create, any 3D models that we create from a taonga is still taonga, so we treat the digital surrogate or the 3D model that we create, just as we carefully handle or treat our physical objects, so the tikanga around that, the releasing of that who gets to view it, how those content and source files get managed and protected, is all the same as if it's a physical, real object. Jenna Borst

The items in our care tell important stories, and they're entrusted to us so we can keep them safe and preserve their knowledge for generations to come. While it's a great thing to share information further afield than our galleries, there are important limits on the things we make available online. Josh Emmitt

So some of the issues that we have really the same as what we have around our cultural imaging permissions for our taonga, and that's the idea of we're effectively making a digital twin of the object, or a clone. So it's no substitute for the real object, in that it was never going to replace the real object. But the idea being, if you make a 3D model of one of our taonga, the question is, why are we making it, and do we share it openly with the world or just with a particular community? What's the reason for doing that? And so the challenge is there ethically, just because we can, should we? Because if we start just doing everything. All of a sudden, we could have people using our models in ways that we don't intend online, or potentially people could one day 3D print taonga and the implications that are quite complex and something that needs to be explored. So right now, with our models, you can only view them. You can't download them or print them or anything like that. But even in house, if we wanted to 3D print an object, we need to ask, is it appropriate to be 3D printing an object.

Jenna Borst As museums like ours embrace cutting edge technologies like these, the way we care for, share and understand taonga is being transformed. The progress being made with 3D modelling is constantly racing ahead. Josh Emmitt

I mean, 3D technologies are moving really quickly. They're now, you know, people using drones for capturing all kinds of things, and it's great. It's great people are creating those models. You can do it on iPhones now, there's all kinds of things. But people really need to think about and by people, I mean people in the museum space and the academic…anyone making a model on their phone, they can do what they want, but we need to be thinking about the data behind it, because the algorithms we have now for reconstruction will not be the best ones we have. So making sure those photos are backed up to the best they can be at the highest resolution, that means that 10 years down the line, they could just be reprocessed with better algorithms. Or maybe we want to process them and compare them to one from 10 years to see the condition of an object, and it's really just thinking, just because we can make a model, should we make a model, and what are the reasons behind it? I'd love to go make a 3D model of everything in the museum, have that for our archives, and make what we can online, but that's at the moment, a huge time investment that we just don't quite have capacity to do one data. But it's more we could do that, but with a lot of objects, should we do that? And I just think we need to be cautious of just because we can, should we? The old Ian Malcolm thing off Jurassic Park? Just because you could, you never asked if you should.

Jenna Borst The future is wide open for new ideas and approaches that will help us care for and understand our taonga. Adopting science makes the work we do a little less Indiana Jones and a bit more Blade Runner. Where will this technology go in the future? Stick along for the journey and let's find out.

That was ‘The Past is now in 3D!’ This episode was written and produced by Steph Strock and Laura Skerritt. Sound Design by Marc Chesterman and Laura Skerritt. The executive producer was Teresa Cowie from Connect Content. Thanks to our guests, Dr Joshi, Richard Ng, Ricki-Lee Erickson, Heath King and Jessica Wagstaff. A special thanks to the C&L Gregory Special Projects fund, whose support opened up the world of photogrammetry at Auckland Museum. For more information about how the museum uses 3D technology, or to check out some of the models online, visit the links in our show notes. If you're enjoying The Amp, please help others to find us by clicking follow, giving us a review or just telling a mate about us.