Digital Pathology and AI in Cancer Grading, T-Cell Imaging & Biomarkers
Aleks: [00:00:00] Welcome. My digital pathology trailblazers. So excited to be here today. And for those who are joining for the first time my name is Dr. Alex Zuraw. I'm a veterinary pathologist by training, and I'm a digital pathology enthusiast. I believe that digital pathology is the gateway to fast diagnosis, and that's what patients have the right to.
So I teach about digital pathology and also bring the right businesses in front of digital pathology trailblazers. It's a mouthful. I know, but I realize I never introduced myself. For those of you who are here for the first time, I'm gonna shorten it for the next time. But I see you joining, and I'm so excited because today.
Is a different time. It's Saturday morning, like, who wakes up on Saturday morning to review digital pathology papers? Me, as you can see, but also there are people online. So whenever you're joining, let me know [00:01:00] where you're tuning in from and what time it is for you today. I see there is people joining and as you join, let me know, just show me in the comments and that you can hear me, you can see me and that you are here on Saturday with me as well.
In the meantime, I'm just gonna start, ah, by the way, of course you are noticing my earrings, right? So you're gonna hear about those earrings later at the end of the live stream a little bit more. And because they're so beautiful and I wanna share them with you, but now without. Without further ado, let's dive first, cause they're so cool today.
I actually enjoyed reading them believe me or not. And I don't enjoy reading all of them. Sometimes it's muscling through them, but today it was a breeze. Oh my goodness. And we have people from Tunisia, so Cool. [00:02:00] Thank you so much for joining. I think, are you're here the first time? Let me know if it's, if you have already been here before and before I tell you some more about other things.
Let's talk about multiple instance learning for. For the detection of lymph node and ovarian metastasis, oma of ovaries, fallopian tubes, and peritoneum. Have you noticed that I now have like purple color of my pen? I decided that clo it's closer to h and e than orange. So let's see how that goes.
This was published in cancers and this is a group from UK Leads. Leads is a hospital. That is leading in digital pathology. These people have been doing digital pathology, I think, since digital pathology started as a discipline. So for this particular paper the background is that surgical pathology of tube, ovarian and peritoneal cancer.
Let me oh. My microphone is okay. Sorry. [00:03:00] If at any point you like, stop hearing me, let me know in the comments. 'cause I don't wanna go on mute for too long. I'm gonna be losing you. Again, whenever you're joining, let me know where you're tuning in from and if anything technical.
Technique-wise or like setup-wise doesn't work. Let me know. And we're back to the tubal ovarian peritoneal cancer. This, it has a lot of work to be done to diagnose it. And what do I mean by that? We have a bunch of other tissue than just the tumor tissue to evaluate and. The, there is a large amount of non-primary tumor related tissue requiring assessment of the presence of metastatic disease, and we're talking about lymph nodes, omentum, and including and these are resection cases so not just biopsies and also anything else here.
Yeah, volume. Volume. And I have [00:04:00] a little smiley here. Why am I happy about this volume being so big? Not happy about volume being so big, but I'm happy because this is a fantastic use case for ai and you're gonna see why. Actually, have you ever heard of this challenge to detect metastasis in lymph nodes?
It was called chameleon Challenge. One of the first challenges oh my goodness. Okay. I have to stop because we have the, a first time a person who's joining first time the live stream, but has been following the newsletter for years, and this makes my heart just it's so heartwarming. Thank you so much for joining.
And now Chameleon Challenge. 2016 2017, different image analysis companies were detecting metastasis. In lymph nodes and they did pretty really well done. They detected a lot. And here this is the approach that is being used. Let's check the method that [00:05:00] we used. Attention-based multiple ins.
Learning with a Vision Transformer foundation model to classify whole slide images. And we are classifying as either those who have ovarian cancer containing ovarian carcinoma, metastasis or not. Which is such a fantastic task for ai. Like you don't have to screen it, you get it pre-screened and then presented to you.
So training and validation there, it was conducted. With the five whole slide image of surgical resection specimens of 404 patients at this lead teaching hospital, those trailblazers in digital pathology and what happened here, like the results were very good. Or the area under receiver operating curve of oh 0.998.
And this was in some assembled classification from [00:06:00] holdout testing and the balance accuracy of a hundred percent. Like really, but I believe it actually, because this task is visually simple. In terms of distinguishing the cancer from the surrounding tissue. So visually, you don't have doubts when you see it, but the problem is that there is so much tissue to go through and so many square micrometers to cover.
And this is the problem for human eye and basically like our cognitive and visual strain. Whereas for AI, it's not a problem at all. So I actually believe these results. What else do we have? In the lymph node set, we have an org of oh 0.963 and a balanced accuracy of 98% in the omentum set.
So the a hundred percent the better, like better. It's all are like really good, almost perfect results where the omentum. We also had fantastic results. Inclusion [00:07:00] is there is great potential in identifi the identification of ovarian carcinoma, no. And on mental metastasis. And this could provide clinical utility through its availability to prescreen whole slide images prior to histopathologist review.
A big heart to this application because this is. Like I say, visually easy task that constitutes a big burden on the pathologist and could totally be done with AI, pre-screened with AI and then shown to a pathology formation. So it's still computer-aided diagnosis. AI-assisted diagnosis. I think we're gonna have this abbreviation in another paper.
A-D-A-D-D. No ai, DD something like that. We'll see. But anyway, so the pathologist is still in the driver's seat of this particular diagnostic process, and I'm struggling here with my mouse. The pathologist is still in the driver's seat. We want driver's seat. We wanted that way, and that's how [00:08:00] it stays, but with the superpower of ai.
So let's go to our next publication. In our next publication, this is something at Scientific Translational Medicine. We have a group from San Francisco in a second, but we're talking about deep beam, a high-performance generalizable. I don't know why my underlining is so crooked here. A deep ensemble.
Or ensemble? Ensemble, I don't know how to pronounce it for bone marrow, morpho and hematologic diagnosis. And I was like, what is this ensemble? And basically it's multiple models. So it's not just one model. And this group is from mostly from the us. We have San Francisco, New York, Boston Memorial Sloan Catering in New York.
North Carolina, chapel Hill, Oakland, California. These are usual subjects in the digital pathology [00:09:00] space. So let's talk about this particular application. Oh, and we have new people joining on LinkedIn. Fantastic. Thank you so much for joining. Let me know if you have any questions. Let me know where you're dialing in from.
Deep. He, deep. What's the problem here with, we have this cytological analysis of bone marrow aspirate. This is a pivotal diagnostic workup for all hematologic disorders. When there is something flagged on your, work, then it triggers making a smear from your blood. And then at some, maybe if they the hematopathologist decide, okay.
More work then that's gonna be the bone marrow aspirant. So bone marrow aspirate is a bunch of different level of development blood cells, right? Both white blood cells and red blood cells from the very beginning to when they are mature. So a bunch of very [00:10:00] similarly looking little cells. And why am I saying that?
I'm saying that to pre-frame this. That this skill of this pro, highly complex and time consuming.
And I can say yes to that and anybody who has seen a bone marrow smear can also confirm that. Okay. Oh, and we have even more people joining on Saturday. You guys are amazing. I love this. This is fantastic. Let me know where you're dialing in from. Going back to the paper, blah, bone marrow aspirate evaluation difficult.
So let's have the AI help us. Deep learning model based sorry. Deep learning based models for the automatic classification of bone marrow cell morphology demonstrate the potential to improve. Improve diagnostic efficiency and accuracy. Yes. Whenever AI can recognize something well on the image, it can help.
The problem is when it doesn't recognize something very well, and when it's visually [00:11:00] difficult to recognize, often it doesn't. But in this case we have the potential to improve everything. But the existing deep learning approaches are not that fantastic. They. Also they're not that fantastic compared to human evaluator and also not that fantastic in terms of general generalizability beyond the training dataset, right?
Or beyond the, like a single dataset. So what they did, they curated a dataset from the University of California, San Francisco, and it included a training set of. 30,394 images from 40 patients with morphologically, normal marrows, so normal bone marrow. And then they had a test set, eight and 507 images from 10 different patients and.
All derived from 400 equivalent whole slide images. So we [00:12:00] have images and whole slide images. So we have like probably snapshot screenshots of images and that are derived from four whole slide images. What is this? 400 x This is the CX objective, which actually surprises me a little bit because I was expecting bone marrow evaluation to be done on.
At higher probably the visual publish evaluation is, but here they trained on 40 x, which is fantastic because if it works on 40 x. With ai, then the scanning is faster. And then we developed deep heme, a snapshot and ensemble, deep learning, classifier and snapshots of a classifier are very basically different versions of this classifier.
So like different levels of training of the same. Classifiers so you don't have to retrain from scratch. And then you deploy them all on this particular data [00:13:00] set. And this outperformed previous models in accuracy, which is fantastic. And this while expanding the total number of.
Differentiate the ones that you can differentiate cell classes. So, before it was a limited number of cell classes that the model could differentiate. And now it's a lot more and I see more people joining. So whenever you join, just say hi in the comments. And let me know where you're dialing in from and also what time it is for you.
So we have this snapshot ensemble of models that recognizes more classes and then they externally validate. So how cool is that? It's just good science, right? But because I was exposed to bad science before I'm, every time I have a test set, training set, external validation, I'm ecstatic that they actually did the work that they were supposed to do.
'Cause in the beginning of the digital pathology days you would go to conferences and you would hear like amazing performance. [00:14:00] On the training set. And that's not how you do this. Computational pathology. You do need a training set, validation set, and all these other different data sets.
Otherwise, you can just throw away your model or just run it on your training set in. Endless loop. Okay and then this externally external validation was on the dataset from Memorial Sloan Catering and spoke super active in the space of digital pathology. They even organize digital pathology workshops.
So if you ever get the chance to go there to learn digital pathology, please do they included. 2,694 images from 10 morphologically normal patients, and 11,076 images from 6,655 patients with normal or disease matter.
And it demonstrability and also at the level of individual [00:15:00] ification they systematically compared deep heme to three medical experts from different academic hospitals. And the accuracy was comparable or exceeded the one of human experts. So I'm so happy that we now have something or, these are not really commercializable tools, but this is already research done proofing a concept that this stuff can be done, which is amazing. Our next. Paper Abstract slash abstracts is gonna be AI for head and neck. Before that I'm gonna give you an update to which is the next conference I'm going to, actually on Monday I'm going to STP.
So this is gonna be relevant for my fellow Toxicologic pathologists. It's the annual meeting of Society of Toxicologic Pathology and I am honored to speak there about digital pathology and AI in neuropathology. Yeah, the, these presentations are invite [00:16:00] only, so I feel pretty honored and that I'm gonna be speaking about that.
But, that's just like my ego and feeling good. Oh, somebody invited me to speak. But the more like the more relevant thing is that, there is interest in AI and digital pathology and the conference attendance and the organizers wanna know, okay, what is out there? Yeah, I cannot say what I'm gonna be talking about, but hopefully I can record my practice talk and then maybe share it with you in one way or another or not.
We'll see. But STP is my next conference. Let me know where is which conference you are going to next time. Either when you're watching live or as a recording. Sorry. Yeah, recording. Let me know where you're going. Next from the digital pathology conferences or pathology conferences because maybe we can meet each other there.
And now let's move on to AI for head and neck. [00:17:00] Squamous cell carcinoma from diagnosis to treatment. So this is a review that is gonna tell us about, okay, what's going on for head and neck and when it comes to digital pathology, AI, and all these things. And. And this head and neck is a globally prevalent malignancy with high morbidity and mortality.
And I think that all the cancer papers start like this type of cancer has high mortality and high morbidity. But that's, these are the rules of writing papers, right? You have to do an introduction and specifically in the abstract where people from outside your, your area are gonna do it.
And after this one I'm gonna address some comments 'cause I'm getting some comments. Thank you so much for chatting. We have high morbidity, high mortality, and despite the therapeutic advancements, patient outcomes are hindered by tumor heterogeneity, treatment related [00:18:00] toxicity, and also limitations of traditional prognostic tools.
So obviously we wanna leverage AI to improve personalized. Had the next squamous cell carcinoma management by integrating. And this is the keyword hour in decorating. This is our keyword for this one. Radiologic, pathologic, and molecular data into actionable information and insights. And this is a review of recent developments in AI application across head and neck cancer.
AI has shown promise in enhancing diagnostic accuracy through automated tumor burden assessment, extra nale extension prediction and endoscopic image analysis. Deep learning applied to radiology and digital pathology enables extraction of prognostic features that may inform risk stratification and treatment [00:19:00] deescalation and the, again, let me do this.
Put a different color. Can I do different color? This one, multimodel. Multimodel is the integrating and multi. Model are our keywords here. Because multimodal a models they fuse imaging histopathology and electronic health records have demonstrated superior performance in outcomes compar uni model.
So we have multimodel
versus uni model.
And these are better and these are not that great. So this is basically what they're saying here and. Additional applications may include early toxicity detection during radiotherapy, adaptive treatment [00:20:00] planning surgical complication forecasting predicting immunotherapy response identifying imaging and histology, correlates of tumor immunogenicity.
So there is potential as we know, and but there are also barriers to clinical translation. Because we don't have explainable models because the implementation is difficult because there is a continued development necessary. Yeah. And adoption is lagging. So all the classical problems with the digital path adapting actually any.
Technology like adoption. Adoption, but I would want the digital pathology adoption to be more than what it is. But you know what, let's make it happen together. And what I wanted to say is that on Wednesday, XStream Histology livestream, and it's it's funny because this livestream those publications, all comfortable, so confident with how to [00:21:00] present them with, how to go through these abstracts.
But this is after 24 times and the histology one, even though like I, I'm fluent and I'm like. I need to know histology. I know histology and I know that I know histology. I was so much less confident on this livestream. So if you go to my YouTube channel and wanna check out a less confident presenter, who is the same person who's presenting to you right now?
Pretty confidently I would say. You can check this other livestream and I. When I finished, I was like that wasn't that great. Did those people didn't like it? Did they learn enough? Was I not rambling too much? I probably was, but then I thought, hey, my first Digi Path Digest sucked as well. So I just gave myself permission to suck at the beginning of the process and I'm gonna be improving it.
And we have a participant here from, sorry. From [00:22:00] Nigeria, and we have a long comment who developed a counting and classifying blood cell blood cells from multiple blood smears image as a personal project. And how can you improve this tool for bare accuracy and performance and other functions?
So this is a perfect comment for our previous previous paper, right? Heme, sorry. Deep heme. Where is our deep heme? This one. Get this one deep heme a high performance generalizable deep ensemble of bone marrow, morpho and hematologic diagnosis. And check what they did and check what you can replicate in your project and let me know how that went.
Let me know which methods you went for and. How you're gonna be using it? Are you gonna be using it in practice? Is it for research or is it for anything? I'm just curious. I'm always curious about these digital pathology projects, [00:23:00] especially in the under-resourced areas, because I think this is where digital pathology has the biggest potential, but this is also.
This is also the place, or the place. It's not one place. It's different places where we have to look at alternative low, like super lean methods to deploy it. So yeah, I'm curious. Let me know and. What are we, Ooh, we are talking about something super cool because here's Tiffany, because this paper called Enhancing Organ Alions allocation Efficiency, a Pilot Study evaluating AI Assisted Assessment of Donor kidney Pathology was done by tech site.
And tech side is a digital pathology place sponsors. So big shout out to tech side. I met the team at US [00:24:00] Cap in Boston a month or two ago. When were we there? Probably in March or April. Anyway, I met their team. I have recorded a podcast with Tiffany and Ben, the CEO of tech side. We also have a demo of the platform that was used.
In this very paper. It's funny when I show it digitally, it's on this side, but actually it's on this side for me. Anyway, so I saw the platform from firsthand. I know their vision behind the platform and this publication is an example of what they did with it. So let's dive into it. Shout out to Tiffany for helping with this publication.
The purpose of the study was to evaluate effectiveness of ai, sorry, effectiveness of AI assisted review. So this is a r AI assisted review system for in improving diagnostic accuracy, efficiency, and [00:25:00] concordance with expert assessment during the evaluation of donor kidney viability. I like these papers this week because they had like very practical applications.
And like the ovarian cancer screening for metastasis and this one, this has to be done fast, accurately, and AI could help. So 60 H and E stained frozen section kidney biopsies. So this is important. Frozen. Why is it not okay? Frozen sections. Most of this stuff is being developed on FFP Forline, fixed paren embedded material.
And for some stuff you need frozen sections, right? For intraoperative, for fast stuff, you don't do the full forline processing. You do frozen section. Or you can also do direct to digital at some point when the method is developed. Like another digital pathology place sponsor does muse microscopy, but here we have frozen sections.
And they were frozen sections from exponent. Kidneys [00:26:00] obtain the for on organ donation. They, these were evaluated and a board certified renal pathologist established ground truth through manual digital evaluation on the tech side fusion platform. So there's another thing that I liked in this paper.
Other than that, I know the people and that I just talked to them at the conference and they have a pretty good product. I'm gonna tell you about it in a second. The slides were independently. Independently reviewed by an AI algorithm, a board certified pathologist, reviewer two, and a board certified transplant surgeon reviewer one.
After a washout period, AI assisted reads were performed. The performance of this ar. AI assisted review and the manual digital review. So this is where I have my heart. They were not comparing this stuff to glass. This we already passed. That glass is okay and digital is okay. And now we can [00:27:00] do on digital and then compare digital review with AI assisted digital review.
So we are like advancing. That's why I have a little heart here. Manual digital review. We're not doing glass comparison anymore. So this was compared to the ground truth for total and sclerotic glomeruli. So this is all counts. So this is also important. You have a specific object that you can detect with ai and you can count it like perfect use case for ai.
Beautiful use case for ai. So as well as concordance with kidney viability T thresholds. And this threshold was 20% scra sclerotic glomeruli cutoff rate and secondary outcomes included also comparison of revenue times and concordance rates for the AI assisted and manual digital. And AI analysis alone with the ground truth.
So what were our results? Across parameters. [00:28:00] We had concordance. This is amazing. We have concordance of AI assisted versus ground truth. And then there were different like numbers for different reviewers. But basically agreement at the 20% sclerotic glomeruli threshold for kidney via viability was 98.33% for both the AI assisted and manual.
So it's very good. And also AI assisted they re, it reduced meantime review times, minutes by 54.83% half of the time when you use AI versus non-AI. Per slide review. Times decreasing from 17 minutes to eight minutes, 35 seconds. That's a lot of time for reviewing this freaking biopsy. 17 minutes for a kidney frozen section biopsy because you have to count this glomeruli, and [00:29:00] now you could reduce by more than 50%. How cool is that? This all different, correlations, Pearson's correlation coefficient concordance correlating correlation coefficient for AI assisted review were generally higher than manual review, particularly for the percentage of sclerotic.
Glomeruli does not surprise me at all because we have a computer counting and a person counting. So we're not that great at counting. I'm gonna tell you a little story. When I was little, my dad I dunno how old I was, like maybe 11. My counting skills maybe weren't that great, but like I was focusing, I was like even had my abacus or whatever and he gave me something to calculate and then of course he corrected and I was wrong so many times.
And that was when I was 11 and I realized people are not that good at counting no matter how hard you try. So I learned that lesson early in life [00:30:00] and okay. But basically what our tech side colleagues are saying here. And that ai assisted review is Im, has improved alignment with ground truth, and there they analyzed for it and there was no systematic bias.
And AI assisted review aligning more closely with ground truth compared to manual review for both reviewers. And the conclusion is that the tech site algorithm reduces review time while maintaining accuracy and concordance with experts promoting AI adoption to improve workflow efficiency and expedite transplantation decisions.
So that's something that has to be done like super fast transplanting stuff. So if you can leverage ai, fantastic. And also this platform that, fusion platform. It basically fuses your visual manual digital workflow with AI assistance. You can have this algorithm probably when the it's ma mature enough [00:31:00] and plugged in into the Fusion platform and use it.
Whenever you need to use it. And I'm so happy that this is also developed on frozen sections. So I think we have one more. So guys don't leave yet. Stay one more and then we can go about our weekend and we have. Agent, AI agent. Let's talk about AI agent. This is a relatively new concept. Stay with me and if you're just joining or watching the recording, let me know where you're tuning in from and what time it is for you.
And then at the end, I'm gonna tell you a bit more about these earrings. Actually, if you wanna learn more about these earrings, I'm gonna put something on the screen you can add. Just it's called DPP store. You can see them there if you're already thinking of tuning out, but don't because we have this agent paper to discuss.
So let's discuss it. In the [00:32:00] meantime, check out the earrings. And we are gonna be talking about development and validation of autonomous artificial intelligence agent for clinical decision making in oncology. The first time I heard about agents was. Just before I recorded a podcast with a radiology expert Dr.
Nina Koler, and she was explaining it nicely in the in the podcast, so you can have a look at this particular podcast, but here, this is a group from Heidelberg and they are freaking cutting edge. I've heard of them. And the person in charge there is Jacob. Nicolas Catter, you're gonna see his name on many digital pathology publications.
I heard about them when I started in the digital pathology space around 2016. So we're like a decade into this stuff, and they were already cutting edge there. What they are saying is that clinical decision making in oncology is complex. We agree with that [00:33:00] and it requires integration of multi-modal data and multi-domain expertise.
We also agree with that and we already saw in this other paper multi-model is a little than uni model. And they developed and evaluated an autonomous clinical artificial intelligence agent leveraging G PT four. So we're talking about the generalized pre-trained transformer, our Chad GPT friend with.
Multi-model precision oncology tools to support personalized clinical decision making. And the system incorporates vision transformers for detecting microsatellite instability and KRAS and BAF mutations from histopathology slides. So we have a vision model, we have a text model, and I don't know what else.
They have a search rag or retrieval augmented. Generation. They also have Med Sam. This is a radiology model for radiological image segmentation. I'm gonna tell you something about these mutations and other the [00:34:00] microsatellite instability because there was a podcast with a person from from their team with Oliver Salani.
Yes, and it was a good one. Check the one with Oliver Saldana. And what else did they do? They did web-based search tools such as onco kb, PubMed, and Google, and evaluated 20 realist, realistic, multimodal patient cases. And the AI agent autonomously used appropriate tools with 87.5% accuracy, reached correct clinical conclusions in 91% of cases, and accurately cited relevant oncology guidelines in 75.5% of the time.
And you may think, oh, 75% of the time, that's not so much. But when you compare it to GPT-4 A loan, this integrated agent improved decision making accuracy from 30%, which is [00:35:00] don't even approach me with that number to 87.2%, which is oh, let me work a little bit more on this and it's gonna be.
Clinic ready. This finding demonstrates that integrating language model with precision oncology gene and search tools substantially enhance clinical accuracy. And they establish a robust foundation for deploying AI driven personalized oncology support systems. And that were all the papers that we were supposed to discuss together today.
What I want to show you, it's always. Like not showing me. I wanna show you the earrings that I was bragging about and I'm gonna keep bragging about them 'cause I'm so proud of them. We have three different different designs. We have multinucleated giant cells. We have cartilage and we have Ian Blue special stain of colonic crypt, and [00:36:00] they are such an amazing conversation starter.
So if this is something that you like for yourself or for somebody who you know I take them to conferences and people started recognizing them. So I thought, Hey, let me make some for digital pathology trailblazers. I don't have too many of them. Oh, and I see somebody scanned this QR code. Thank you so much.
I don't have too many of them. I only ordered 25 of each design and actually I probably have less 'cause I have to send out some. But I didn't count how many. So if this is something you're interested in, check the QR code. And obviously if you are here the first time or if you don't have the book yet, and by the book digital pathology one-on-one, all you need to know to start and continue your digital pathology journey.
This is available for free through this other QR code that I'm showing on the screen right now as a PDF. And if you wanna buy it on Amazon, then you can as well. And, but what I would. Rather recommend at this very [00:37:00] moment. You can do both. It's not expensive. It's I don't know, I put the like lowest price and I could do so that it's accessible.
But the one from QR code is free. It's A PDF. And if you have the PDF, you're gonna be on my mailing list. And when the new version comes out, because this one was in 2023, this was like baby chat, GPT, and now we have like multi-model stuff, foundation models, all that stuff. So I need to update the AI chapter, but if you have the previous versions.
You automatically will get the next version. So don't worry for the digital, don't wait for the next version. Just grab the one right now. And when I have everything updated, you're gonna get the updated version and also give somebody these earrings or get them for yourself. Thank you so much for joining.
I always appreciate it. It was Saturday morning, so you are amazing. Thank you. My digital pathology trailblazers, and I talk to you in the next [00:38:00] episode.