Operationalizing the Patient Voice for Long-Term Protocol Planning and Execution

execution of the clinical trial. So I'm going to start by providing a little bit of context for you. Many of you have seen this data before, but I want to sort of set the stage by just showing you how complex our protocol designs have actually become and this is something we measure routinely. And then I'll move into specifically how we are able to systematically introduce and integrate patient input based on the characteristics of the protocol itself.

It's scientific characteristics as well as the executional elements of the design. And then I'm going to turn it over to Ben, who's going to really talk about how you apply this and you determine ways to integrate patient input in to really drive sustainable value for the organization overall, and how to use those insights to inform cross functional

decision making. First, the context on the left hand side, you see some data that was published a few years ago, just to really give you a flavor for some of the most common elements in a protocol, scientific and operational or executional. And I'm showing you really just two time periods here. We've been measuring growth in the prevalence of

these variables for quite a long time. If you look at the total number of endpoints, for example, this is on average, looking across multiple therapeutic areas, we've seen nearly a doubling. Eligibility criteria has remained relatively flat, but that's in part a function of the way we count inclusion and exclusion criteria. Some companies group multiple criteria and counted

as a single as a single inclusion criterion. For example, total procedures to support those endpoints has grown dramatically, And you can look across any element of the design, the number of sites, number of countries, virtually, they have all grown dramatically. Look at that bottom data point. Total data collected by the protocol has risen far more dramatically than any other area, nearly four times the level we observed

in twenty ten on average. On the right hand side, I'm showing you just another measure of customization and complexity, and that is the number of intermediaries involved in supporting any clinical trial, both the design, but more so the execution of the study and the data collection. About four or five years ago, total spending by pharma on contract service providers technology services, study conduct services, cro services surpassed the total amount that is spent each year just supporting

internal infrastructure. And the key takeaway with all of this is complexity is associated with poor with poorer performance. It's correlated inversely with poor performance. So the higher complexity rises, the more performance suffers. Our recruitment rates are lower, our retention rates are worse, our timelines are longer. The number of protocol amendments that we observe actually increases the more

complex our designs. On the right hand side, I'm showing you just one measure of site burden to administer our protocols, and we have countless measures that really show how difficult it is for sites to manage and administer our protocols today. Here's just one measure showing you site enrollment achievement, the percentage of sites that were activated that ultimately achieve target enrollment, and that number has been declining steadily every year for

more than a decade. Here are just two other men measures that now bring it down to the patient burden level, and we have many others that I could share with you. The percentage of patients who drop out prematurely due to their own choice, not the choice of the site personnel, or due to a response to the study drug has skyrocketed. Nearly two thirds of patients in recent studies are dropping out out of their own choice first, so they're pre empting the site or they're choosing to drop out because

of the burden of participation. On the right hand side, these are now exit interviews that were conducted with patients who completed their participation and we asked what did you like least about being in the study, and four of the five areas all relate to the burden itself, the location of the research center. The study visits were so time consuming, there were too many procedures, they were cumbersome,

and I love that one compensation was not enough. You could not pay me enough for the kind of requirements that this protocol post. We have been looking to simplify protocol design for a long time. You know, I've been measuring as an outsider looking in as an academic for nearly thirty years. We've been looking at protocol design behaviors. Early on, one of the most common approaches was to

use economic input into protocol design. The idea that if you cut out a procedure, or you cut out a certain number of times that the procedure is performed, would save the study budget some expense. That was a really highly prevalent approach in the early nineties, a period where we started to see a lot of companies look for tools that would help them manage the economics and the

financial requirements of a lot of their activity. You might remember, for example, the PIKS database, which was a tool that gave pharma companies the ability to understand the typical pricing

for an investigative site to perform a particular protocol. When we shifted in the two thousand twenty ten period, this was one of the early phases of sponsors looking to establish a more meaningful and more effective relationship with their sites, and so in that twenty twenty ten period we saw height and focus on executional feasibility, and you may recall that a lot of the feasibility assessments that were introduced started to get larger and larger and larger as more

data was collected by the site. We then shifted our focus and we're still in this environment where we're looking to reduce the number of endpoints. We're looking to prioritize our designs so that we can minimize avoidable amendments, one of the most disruptive experiences that we have in any clinical trial, and that's still a very very important way

that we look to simplify protocol design. And the patient engagement movement has introduced and elevated our sensitivity, our awareness, and our interest in integrating patient input into design and using the patient's own perceived a sense of burden, the relevance of the trial their own concerns about the convenience of participation to help identify areas that we can modify. So these are essentially the four optimization areas. We know

if many organizations that have integrated these approaches. Patient engagement and the measurement of patient burden is perhaps the newest area, and we mostly see companies using advisory boards to gather

that input. These are like focus groups, and I know many organizations are doing them, but they typically appear when the protocol is nearly finalized, so they're actually coming into the planning and the design phase slightly late to the game, after many internal champions and thought leaders and others have

provided input into the design of the protocol. The idea for the burden assessment is to move patient input into the earliest stages so that before we move to solicit input from thought leaders or at the same time that we're soliciting input from multiple parties or already incorporating patient input. So there are a number of approaches a number of

organizations that are offering methodologies. What the Tough Center has done is created essentially an approach that organizations can ultimately build in house and a take in house so that the can do this internally. We are not specifically designing a product that can be licensed. We're developing essentially an approach that any organization can adopt and use. We've published

extensively and I'm listing three of the publications here. What we did is we based on our work in protocol design and the benchmark activity that is now coming up on thirty years of experience, we were able to narrow down the sixty most common procedures that we see in every protocol, and we presented those procedures and other elements of the participation experience, distance to travel, whether a procedure might be performed on site or at a different location,

the length of the visit, for example. All of that was presented to patients globally and they would rate the relative perceived bird of every one of those procedures or of those experiences related to essentially a reference point, and that reference was a routine physical exam, something that most patients are familiar with, even just by interacting with their

own primary care physician practice. And we gathered ultimately responses from thirty six hundred global patients, so we have some granularity by disease conditioned by demographic characteristics, by socioeconomic status, and many others, and we're now able to use that input and its variation by all of these different subgroups to map to every protocol design. So we're looking at

a number of different areas. We're looking at the procedures that are performed, we're looking at burden to adherents, we're looking at lifestyle restrictions, we're looking at other convenience factors, and all of that can be mapped based on the input that we've received from patients. To date, fourteen companies have been applying this approach and bringing it in house. And I'll just share a little bit of the high

level aggregate data with you. The chart on the left here is not intuitive, and it's because we've created a burden's score, which does not have meaning in and of itself. You have to sort of compare that to benchmark data. For companies, they can compare it to their own internal

burden score to see how it's changing over time. I'm showing you aggregate data here, and what's most important is it tracts so well to changes that we've seen in protocol design over time anyway, So it's sort of a way that we have helped to validate a lot of our measures. They map very well to change in design decisions. On the right hand side, I'm showing you another measure of burden. This is now the distribution of visits by

their average duration. Where in twenty eleven to twenty fourteen, nearly sixty percent of all patient visits averaged about an hour or less. That has dropped down now to below half. And you can see the proportion that has grown the most are now those visits that are lasting more than two hours. This may include travel, right, we've sort of consolidated that to summarize the statistic here, But that's just

another measure of burden that we can assess. And again the punchline and all this is you can start to relate burden to performance outcomes and that helps you isolate those areas of that may be most contributing to burden that might be predictive of different outcomes in the study. Here, I'm pulling together data from all of the fourteen companies that have participated so far, and what I wanted you

to see is just a pattern that is emerging. Essentially, when burden is low, we typically see faster relative cycle times, fewer protocol amendments, fewer protocol deviations, and even lower dropout rates and drop out for many of you, you'll know, it's a very complicated measure because there are so many factors that play a part, so we were surprised to see a slight decline in the overall rate here in

this assessment. With that, I'm going to turn it over to Ben, who's going to really show how his organization has approached trying to integrate and leverage this insight into their own protocol design activity. So Ben, I'm turning it over to you. Thanks so much, and.

We have in my company, like the theme you see across many of the sponsored companies right now, what we consider to be a productivity issue in our clinical trials, and that is something that has some very serious legs

behind it to try and actually uplift that. But also we have a growing need and understanding that we need to make sure that the trial designs themselves are much more patient centric, and we have the opportunity therefore to essentially bring together a mutually beneficial relationship where we can make the trials much more patient inclusive, much better for patients, but also actually for us cheaper to run, faster to run,

and get to market more quickly. When we started down this path several years ago, we decided that we didn't want to reinvent the wheel for obvious reasons, so we reached out to Ken in the Tought organization to try and actually start leveraging the standard framework and methodology that

they can and tough to actually brought. And what that enabled us to do very quickly was to actually have a point of comparison of where do our trials actually land against other large industry bodies, because we wanted a point of comparison that's actually sort of we run extremely

complex global trials. We have a reputation for being amongst i think it's fair to say, the most complex and the most expensive and also the slowest trials amongst most of the large organizations which is not something we're terribly proud of, so we needed a framework that would actually help us actually bring this together and to directly address

R and D productivity. What we identified quite quickly through this work was not just the KPI is underneath this in terms of how do we actually measure the burden of our designs and linking that to for example, the investigator burdens and the site level burdens and different feedback from patient bodies and collecting evidence around for example, the value of patient engagement for our patient engagement organizations, but

actually to contextualize those scores against key drivers and determinants of burden, which allows us to then actually influence decision making and designs within our very large and complex organization

across different functions that we have there. Of course, to do this you can see through the different talks earlier today, there are a large variety of different complexities involved across For example, the people that actually develop the protocols and study designs need to essentially make sure that the protocol is for purpose and actually determining the key end points of evidence it needs to actually work for the science,

but then actually in operationalizing design and running the studies. Effectively, we also have a second series of stakeholders internally, and they have very different priorities and incentive structures. And then, of course we have the people that actually are participating in our trials who have a whole different range of issues and complications that we need to make sure that

we're actually taking sufficiently into account. When we started presenting the results of these initial analyses and insights, we very quickly gained across pharma reaction by a cross farmer, I mean internally across our early research organizations and late research organizations and a medical affair organizations. Suddenly we started gaining this critical mass of different people across the different functions who were very motivated to actually participate in this effort

and to bring together those insights. So I want to obviously, I can't talk too many specifics around the data that we're actually seeing. A lot of this is in flight. I'm going to talk about it sort of in an aggregate way, and I'm happy to answer questions offline and off recording at a later date if you would like and be interested in sort of how we're approaching this

and some of the value we're seeing. But I think the suffice to say, in the initial efforts that are actually coming through with study designs that are actually changing and are in flight with patients right now, we are seeing substantial improvements in different productivity scores that can just displad improvements in our cycle times in terms of patient recruitment activities, reductions and patients dropouts and reduction reductions in

protocol avoidable protocol amendments and different drivers there a value and the feedback from the patient organizations has also been positive. So we are starting to see the shift that we are looking for through this effort. And say, this effort, but it's simple an amalgamation of many, many different efforts coming together with a single lens. I'm talking a little

bit generic here. We don't actually call this the triad in company, but it's more sort of how do we think about this and how are we framing it as a construct, Because what we need to do is actually make the data that Ken was talking about actionable across

different functions that have competing priorities. And not only do we need to understand what performance is and what's driving it, we need to actually be able to get to the root causes of that, what specifically is driving the increase in burden and protocol complexity and how do we nip that in the bud where we can and how do we recognize what is necessary burden and what is necessary complexity that actually achieves a beneficial outcome for patients, and

where can we actually start cutting back to try and make sure that we are optimizing and minimalizing all of those efforts. So I'm going to talk through, in fairly vague detail, a little bit of what we mean in

each one of these different elements of the triad. In the performance sector, which is largely my part of the organization portfolio strategy, we want to understand the mutual benefit through simulations and measurements of what do we actually expect the trial to perform across key milestones and how is this affecting the budget. So essentially, how do we do the trade offs in the trial performance factors that we are looking for and how does that influence financial decision

making investment decisions across our portfolio. On the other hand, we want to make sure that we are actively reducing patient burden, reducing, minimizing and managing our site patient burdened our complexity, saying before that we know that some of this is necessary will always be there. There will never be a trial that doesn't have any burden. It would be nice if we could get there, but I don't

think it's going to happen. And then we want to make sure, on the other hand, that we are actually using directly patient feedback from the get go, not simply as Ken said, like after the protocol is largely designed and then we tweak it if we can, or too often. I think we know that the trial design is often sort of said and doesn't change terribly much after we

get that engagement. How do we make sure that it's there from the start and actually that we have an active learning loop so that the next trial design actively incorporates what we've already understood from different patient communities, rather than sort of going back and starting a new advisory

boarder or asking the same questions again. So again, that's both easier and cheaper for us to do, and it's better for the patient community because we have the feedback that they get a little bit sick of us asking the same questions every single time. It doesn't have a terribly professional appeal to it. Some of the different measures here and this is not comprehensive, and I'm mixing across some key performance indicator type data points and some drivers

of what is actually what is underneath that. But here are some of the things that are actively looking at. Many of them are measurable through the Tuft's work that Ken is presented. So we're in terms of the trial design measures, the additive patient and site burden scores. What I mean by additive is contextualized, it's against standard of care.

So when you looked at what Ken was presenting an aggregate in terms of the total burden score, what we've done to try and actually make it more actionable by our clinical study teams is actually put against the standard of care. So what is the additional burden above that

standard of care? And then against our folio? How does that compare to other studies within the same disease area or the same therapeutic area within our portfolio, and then how does that compare against other similar studies from other sponsors within that externally, so that we can really understand is this trial adding more burden than it's worth? Do we have a sufficient understanding of the disease and the potential benefit of this that it's actually worth the additive benefit.

So the additive burden, on the other hand, we have, for example, we know that we need to collect the actual patient insights around their support needs, the additive burden from their perspective, not what we have measured it as using our internal expertise and our scientific russianale, but actually what do they think it is, because that's quite different and we need to make sure that that's taken sufficiently into account. So I'm happy to answer questions around how

we are using these this. There's obviously a lot of different calculations in the background about how these relate to each so for us to say that it is relatively complex, but we have started to make this work for us in terms of influencing decisions across these different data points and this informs our learning loop. Now this is in

company fairly early maturity. It's not fully embedded yet, it's in process, and I can't talk too many specifics, but essentially, as I said, we are making sure that the key insights from patients and healthcare providers and sites is actually informed from the start before you start designing the protocol, and that sort of goes into a study design tool technology against our portfolio and pharma strategy to actually allow us to simulate the likely performance of this trial design

as we tweak it, so you can then live make changes and see the likely results of how does that perform or how would that likely perform? And we can adjust that for value, we can adjust that for speed, and we can make different trade off decisions if we need to make this happen faster, what would it cost in addition or what would the patient trade off be? Are we satisfied with that? If we take that back

to patients, how would that actually look? How would they react if we talk about regulator interactions amongst all of this as well. So ipsyn colleagues talked about how does that all fit in and how do we make the right level of informed decisions? That is actually the third point as well, so I've to covered both. Then we follow the loop, so any data point that actually goes into actual we feedback into the system so we can understand did the scenario play out like we expected? Why?

Why not? And then we make sure that this is all sort of self reinforcing across the text, and we add that to the data set to make this actually work, and you can appreciate maybe the complexity of this Organizationally, we have an elaborate partnership across the organization of people that are actively involved in getting this across. I've highlighted our patient partnerships function and a strategic insights function, but it's far more comprehensive than that. It's across our patient

centered data groups. It's across our clinical operations groups, because our clinical science groups in early and late commercial everything like that. The most important insights for actually getting this off the ground that we've noted that I wanted to highlight first of all, is transparency. So this doesn't really work if it's only a senior leadership view or if

it's only a clinical science view. The way that we have actually really gained ground here is making sure that everyone can understand the implications of what choices they are making.

If you change the study design in these ways and it compromises something else, if you're making a trade off decision, that is visible across functions, and we found, we have found so far that this is instrumental and actually driving sustainable change because now everyone is much more accountable to actually making this work and happen, and the other one is incentives to make sure that, for example, we can embed this in corporate goals and our sort of bonus

multipliers for every employee. This is starting to happen, and that means that it's public for everyone in the company and everyone is actively incentivized to work towards this and make sure that we are acting as an organization in this route. And this is I think the most important part of actually making this sustainable is it's a lot of effort to actually help make sure that we can

do all of this. There's obviously a lot of extra data collection that we need to do internally in data processing. The technology investment is not insubstantial, and then we have to make sure that we can validate all of this, that we can actually make sure that this is driving the right behaviors, that it's driving the right performance that we actually want to see, that it's not compromising anything

in terms of quality for example. So the only way to then actually pay this out is to tie it back then to these different areas and make sure that we are understanding the value against any one of those factors in the triad. So far that seems to be going in the right direction for us. We are seeing sort of a trial design costs coming down, a performance

going up against that effort. However, we still have a long worad ahead of us to actually making this fully embedded across the organization and fully maturing this, and yeah, I hope to actually see that through with Ken in the near future. Near future meaning the next year or so. Okay, so it didn't even get dinged, I think. So we have thirty seconds per tect