[00:00:00] intro: Welcome to the Research Culture Uncovered podcast, where in every episode we explore what is research culture and what should it be. You'll hear thoughts and opinions from a range of contributors to help you change research culture into what you wanted to be.
[00:00:24] Nick: Hi, it's Nick, and for those who don't know me yet, I'm open research advisor based in the library here at the University of Leeds. You're joining us in season three of the Research Culture Uncovered Podcast where I'll be speaking to colleagues from both the University of Leeds and from other universities and organizations about open research, what it is, how it's practiced in different disciplines, and how it relates to research culture.
If you haven't already, you can catch up with season one, which was an introduction to the podcast and to my co-hosts and season two with my colleague Tony Bromley, who is in conversation with a number of presenters from the REDS Conference of 2022. That's the Researcher Education and Development Scholarship international conference held hear in Leeds.
But now I'd like to introduce my guest for today, Dr. Madeline Pownall, a lecturer here at Leeds in the School of Psychology. Dr. Pownall completed her PhD this year in 2022 in Social Psychology. She's a Leeds Institute for Teaching Excellence Fellow, that's LITE for short and has contributed extensively to the Framework for Open and Reproducible Research Training, or FORRT for short.
So welcome to the podcast, Maddy. Is it okay to call you Maddy?
[00:01:28] Maddi: Yep. Yeah, please do.
[00:01:30] Nick: And, and thank you for taking the time to talk to us.
[00:01:32] Maddi: Thank you. I'm pleased to be here.
[00:01:34] Nick: Um, so perhaps we'll get onto your work with LITE and, and the FORRT communities. Uh, but firstly, congratulations on being HE Psychology teacher of the year 2022 which I spotted on on Twitter.
[00:01:45] Maddi: Yeah, that was recently. Um, yeah. And part of that was actually about some of the open research teaching bits that I do. So yeah. Thank you.
[00:01:54] Nick: No, yeah. Congratulations. And I suppose, um, that is related to teaching, which we'll come on to...we want to talk to you about teaching and, you know, open research skills in the curriculum and that kind of thing. But perhaps to start with perhaps a little bit on, on your academic background, as I say, um, you are a lecturer here in the School of Psychology, but you only very quite recently finished your PhD?
[00:02:13] Maddi: Yeah, so, um, I joined the School of Psychology in 2018. I had like a split, um, postgraduate researcher with teaching contract.
So it meant that when I first joined I did a lot of undergraduate teaching, mainly research skills, statistics, things like that. And my PhD was in, um, kind of like hardline social psychology, so sort of experimental social psychology, and then gradually more qualitative stuff. Um, but the, one of the main theories that I was using in my PhD was called Stereotype Threat Theory, which, um, kind of unbeknownst to me, or I quickly learned very quickly became the kind of like poster child for psychology's replication crisis.
So I started this PhD in 2018. I didn't know a huge amount about like open research, open science, and then really quickly had to learn, um, 'cause I kind of found myself being embroiled in conversations about like reproducibility and replication and all of that kind of stuff. Um, so it was mainly from then that I then developed interest in open research, open science, rigorous research, all of this stuff, and now that's kind of what I do. So I finished my PhD, uh, in April of this year, and now I'm a full-time teaching scholarship lecturer and, yeah, all of my kind of research scholarship, things like that now centers quite a lot on open research and particularly how we can embed it in pedagogical context.
[00:03:39] Nick: Yeah, well certainly we've been working on that, uh, you and I a little bit and I'm really interested to talk about that in a bit more detail, but I suppose first to sort of address the, the whole psychology thing, I, I spoke to our colleague, uh, Kelly Lloyd recently, who also has a background in psychology, as I said to her, um, you know, it's come from psychology has it open research in, in some ways or a big part of it because of that replication issue?
[00:04:02] Maddi: Yeah, I think the replication issue was really having a bit of a moment in psychology, because it was essentially, um, so there was this 2015 paper that I talk about all the time, uh, by, uh, a group of authors called the Open Science Collaboration, where basically they took a hundred of psychology's, most, I think they were like the most influential studies, and tried to, um, replicate them and found that in the, in the original studies or something like 97 or something, percent of the studies had significant, uh, results, so they found what they expected to find. But then when they tried to replicate them, it was something like 36%. So there was this whole kind of conversation, movement, moment, um, after that paper where I think psychology, particularly social psychology, started really seriously asking itself, well, what does it mean if we can't replicate some of our most influential findings? Is it because there's something wrong with the original? Is it because times are achanging and just, you know, social psychology moves on? Or is it something more kind of sinister? And I think that that whole conversation around what does it mean if we're humans studying other humans and we can't replicate each other's work or reproduce each other's work, what does that actually mean, um, has sparked a lot of these big conversations now about what does it mean to have robust evidence and all this kind of thing. So I think, yeah, it sounds, or my impression is that a lot of the open research conversation has come from this kind of panic about, what does it mean that we can't replicate our findings in social psychology? And now it's kind of there's lessons learned across disciplines. I guess...
[00:05:42] Nick: I was just gonna say, I mean, I don't wanna put you, you know, on the spot for other disciplines perhaps, you know, ask you to talk to those disciplines, but is there perhaps a danger, do you think of people saying, you know, this is a, is a psychology problem, whereas actually there's evidence emerging that it does affect a much wider range of disciplines than, than just psychology?
[00:06:00] Maddi: Yeah, well I think so. One of the things that really interests me about open research is the kind of questions that it allows us to ask. So I think that one of the most promising things that have come out of the, I don't really like the term replication crisis cause it feels a bit dramatic, but has come out of this kind of replication conversation in psychology, is that it allow...or it's provided a language to articulate questions like what does it mean to have robust evidence? When are we sure that a finding is robust and that we can trust it? What does it mean if our research is credible? And I think all of those kind of questions, whether or not there's evidence for a replication crisis or not, all of those questions are applicable and interesting to any discipline right? So I think that one of the, kind of parts of the open research conversation that really interests me are the really like, kind of meaty, almost like epistemological questions about how do we know that our findings are meaningful? Um, and what are the kind of standards that we judge how meaningful or how trustworthy or important our findings are.
And I think that kind of set of questions you can ask for any discipline or any subdiscipline. So I've given like open research talks in like applied linguistics conferences and like physics conferences. And when I talk about things like what does robust research mean, that question kind of chimes with everyone, I think.
[00:07:27] Nick: Yeah, well, I mean, just as an aside, because um, I've learned an awful lot, in my role, and again, I, I'm always, you know, talking about, I'm essentially a layperson, you know, I'm not a psychologist or I don't have a discipline. Um, and as a central service, it's a challenge for that, for supporting different disciplines. Um, so I learned, for example, about pre-registration from, from Kelly, um, and the ReproducibiliTea journal club, et cetera, and then in the course of my work, I mentioned that to, uh, an earth scientist, I think, who, didn't know what it was and hadn't used pre-registration. But when I explained, understood how that could be of value for his discipline. So I suppose it's trying to bridge the gap between disciplines and bring different practices into different disciplines where they're using statistics, I suppose, so it might be relevant. Um...
[00:08:13] Maddi: Exactly.
[00:08:15] Nick: Um, so I mean that all that side of things and, you know, the reproducibility is, is really interesting, but I was also interested looking, um, at your sort of, um, feminist perspectives, and I know you wrote, um, a paper with, again, with Kelly, you know, um, Kelly's was the last podcast that you would've heard before this, if you haven't, you can go and check it...you can go and listen to that. So you wrote a paper, I think, with Kelly and, uh, other colleagues around navigating open science as an early career feminist researcher. And I was really interested in that, how you started pulling out, not just the, the, you know, the, the, the issues around reproducibility, but inclusivity as well? Perhaps you, I'm sure you can talk about much more articulately than I can if you could maybe...
[00:08:57] Maddi: Yeah. So that, that whole paper kind of came out from...so all the authors on that paper are kind of self-defined early career feminist researchers. And what I was noticing is I was having a lot of coffees with a lot of my kind of PhD friends who all were really sympathetic to the values and kind of mission of open research or open science. They're like, yes, research should be transparent, it should be robust, it should be all of these things. We get it, we're on board. But what was...and there was kind of this kind of...what I was noticing is the sort of party line, if you like, of, oh, we think research should be robust and transparent that loads of people were subscribing to and endorsing. But then what was happening more kind of on the ground, particularly for early career people, and particularly for early career researchers who occupy research bases that are quite kind of contentious or marginalized, is that people were saying, yes, it's all well and good that this is good for research, but what does this mean for me as a researcher and I started having these conversations. Then we kind of brought together this dream team of, I think there's 10 of us on the paper, um, who were all kind of having these individual conversations around really trying to stress the importance that...of kind of bridging together the stuff that's good for research and what that actually means for the people doing the research.
Because if you have, um, all these efforts to, for example, like, make data more transparent and make things more open, which means that by definition it's more easy to find mistakes and it's more easy to kind of, um, question claims, then that's all well and good, but then you need to make sure that you've got a culture that can respond to things like questioning claims and finding mistakes in a way that's compassionate and in a way that's fair and kind of collegiate and what I was kind of noticing, from the more people I spoke to is that actually there wasn't, or there was a perception particularly in certain disciplines within, for example, like certain parts of social psychology or, um, psychology more broadly that people were saying, you know, I really, I really believe that data should be made open, but what then happens if I'm kind of out, what happens if I find a mistake? And there were a couple of notable examples of early career feminist researchers who, um, particularly on things like platforms like Twitter, were really experiencing like proper hostility, um, and aggression in the name of like open science. So we kind of wrote that paper to try and centre those experiences of basically that you're just trying to kind of remind the field, um, that when we're talking about research, it's not done in a vacuum like it's done by humans and the need to kind of humanise and make compassionate our, um, research culture above and beyond caring about the, the actual integrity of the research itself.
[00:11:56] Nick: Yeah, that, so it became termed didn't it bropen science? So we touched on that also with Kelly, and that was an article wasn't it by Kirsty Whitaker and Olivia Guest. That's, um, bropen sience and, uh, but Kelly made the point as I think it does in your paper aswell, that's not necessarily gendered, although strictly, I mean, because of the power imbalances, it tends to be...
[00:12:19] Maddi: yeah.
[00:12:20] Nick: White men in positions of power perhaps?
[00:12:23] Maddi: Yeah. I think it's really important though that it's not a gender thing. This isn't a kind of like men versus women thing. Like I've, I'm, and I guess that my whole kind of feminist approach, um, makes that quite clear that I think that there's way more that kind of a feminist lens can offer above and beyond looking specifically at like gendered experiences.
I think for me it's about power, and it's about voice and about inequalities. Um, and Kirsty Whittaker and Olivia Guest make it really clear in their...um, where they first coined the term bropen science in a, um, article and The Psychologist that they're like, not all bros are men and not all men are bros or something. Um, and I think that that feels quite important because it is a little bit kind of tongue in cheek colloquial, this is the thing, but I think what has been really powerful is it has given, I guess, people like me and people like my early career colleagues and not early career colleagues, um, a kind of set of, or I guess a language to articulate that kind of feeling of, I don't really know if this culture is safe for me to be really...
[00:13:28] Nick: being exposed
[00:13:29] Maddi: Of being exposed and being...so whenever I talk about like bropen science, I always have this slide that kind of has a scale and on, um, one axis is like researcher vulnerability and on the other there's research transparency and I think they are almost a direct correlation with, yes, it's great to be as transparent as we can be, but at least we need to have a bit of a conversation about what that means in terms of vulnerability of researchers, particularly researchers who might not have, kind of, good support that they can rely on or who might, um, occupy researcher topics that are, um...or that lend themselves more to kind of hostility and politics. You know?
[00:14:12] Nick: Yeah, yeah. No, no. Yeah. It's, uh, I mean, as I say, I'm just be really interested in the context of research culture. How, how this, how this sort of, you know, um, relates directly to open research in a way that might not be immediately obvious to people coming to it from the outside. Um, so you sort of explained how you became interested in open research through your PhD, et cetera, and then I think you've become involved with quite a lot of communities. Um, I mean, there's a lot happening in the open research space isn't there at the moment or the open science space.? So, um, one of the ones we mentioned was FORRT, um, perhaps tell us a little bit about FORRT or maybe some of the other organizations you're part of?
[00:14:50] Maddi: Yeah, so, so I guess I kind of went through my PhD and was interested in open research....I was kind of forced to be interested in open research because, um, nothing was replicating and then kind of turned my attention to...okay, well all of the stuff that I've been learning about open research has kind of come like on the grapevine and has been, you know, oh, I happened to be at events and happened to learn about this stuff. And now one of the things that I just think is really, really important that open research and these kind of conversations are integrated into training and like kinda undergraduate grassroots training. So FORRT is the Framework for Open and Reproduce Research Training. Um, and it's basically a community was started by, um, PhD students and now it's kind of grown and grown and grown.
It's basically a community of people all around the world who I guess believe that open and reproducible research should be integrated into, into training. And that includes everything from kind of, um, well everything actually from like kind of high school science all the way up to, um, sort of PhD training and beyond. Um, so we do things like...uh, there's different projects that are happening, so I led a project in FORRT that was all about articulating the impact of open research on student outcomes. Um, there's a whole load of work being done in FORRT around like EDI issues and neurodiversity, that we run systematic reviews, we write papers, we do studies, like it's a really kind of, um, productive community as well as being a space where people can chat. Um, so I think getting involved with FORRT was one of the best things that I ever did, um, because it is, it is a proper kind of big team science initiative, if you like. So we actually produce things as a, as a collective, which I think is quite powerful.
[00:16:45] Nick: Yeah. And you did, of course you did a talk for us which I'll...cos we, I'll put show notes on this when it, when it goes out. So I'll link to the talk you did for us on how does open research impact student outcomes, and you talked in alot more detail in that talk didn't you about that particular paper, about the approach and the methodology, um, to that kind of thing. I think you were...you touched on three aspects. I mean, it might be worth just revisiting now, if, if that's okay? I mean, scientific literacies, student engagement and attitudes towards science were the three things you were, were interested in. Um, could you, do you wanna tell us a little bit about...
[00:17:18] Maddi: Yeah, sure. So that whole project, so this came from...so most of the projects that happen with FORRT come from some kind of conversation on our Slack Channel and then someone says, right, let's do it, this will be our next thing. And a few people were saying in the community, that they really get the kind of moral case for why we should integrate open research into the curriculum or into training or whatever.
Um, so like, oh, it's, it's, you know, it's good for science and we need science to be transparent. All of this kind of stuff. Um, but what they were, or what some people were struggling with, is how to articulate the, kind of, impact on students. Um, to, to things like their, for example, university department. So what we wanted to create is basically a kind of synthesis of the evidence that looks at whether integrating open research actually does anything in terms of, um, student impact. So does it actually impact student outcomes in any way? Or is it more of a kind of, well, it's good for research, so it should be integrated into the curriculum. So we ran, um, a really big, I think there were 75 of us on the eventual paper, a really big systematic review that looked across those three different domains. So attitudes towards science. So does it impact how students trust science? Whether they think it's credible, whether they wanna have a career in science, does it impact their engagement? So do they like their studies more? Are they more motivated, are they more interested, um, and does it impact, um, statistical literacy? So are they able to do stats better if it's taught in an open research way? Um, and one of the big things from that review that we highlighted is the lack of good evidence. So now we are kind of working more on, I guess trying to like fill that gap in the evidence, um, and to really advocate for like good, empirical, robust investigations about what does integrating open research actually do for student outcomes?
[00:19:15] Nick: Yeah, and I, I mean, as you know, we're really interested in that work from our perspective in the Library and the, the broader picture across the university is we try to promote open research and open science principles, because in my experience, it can be quite difficult to articulate the benefits, you know, not the...it can be a bit abstract, and that's to do perhaps with the incentive structures that aren't yet perhaps in place do you think? I mean, what, what are the other barriers to open research? Systemic issues with...?
[00:19:49] Maddi: What, for researchers or for students?
[00:19:51] Nick: Well, I, I suppose that's, you know, I'm, I'm sort of thinking out loud but, are they related? I mean, they are aren't they, I suppose because there's people like you who are obviously, um, passionate about open science and are in a position to be able to teach your students, but do we perhaps lack the skills across the university to, to teach them? I dunno. As I say, I just sort of, it's interesting to think about how we can embed this on the practical side I suppose, you know, we talked about your paper and need more evidence, et cetera, but what can we...
[00:20:26] Maddi: yeah, well I guess that...cause I have a, a very complicated thoughts and feelings about things like incentives, because, so I've, I've kind of over the past few years have made the switch from doing like kind of experimental, quantitative, statistical type psychology to doing now, like the 99% of the stuff that I do is, uh, qualitative or participatory and that kind of stuff. And actually what that's meant is that there's loads of open research practices that all of a sudden are just not compatible with the kind of research that I do.
And not in a way that, oh, it's because I need to think about it more, or because I need to frontload the work or need more time or more training, just purely because it's just not compatible. It's not, doesn't, it doesn't make sense to, for example, pre-register some of my qualitative work because it's really exploratory and it doesn't add anything. And so I've kind of come from, um, a place a few years ago where I was really advocating for like, there needs to be the kind of really formal top-down incentives for like engagement and open research to now thinking, Ooh, well actually there's some really legitimate reasons why people can kind of happily opt out of some open research practices, and I think the thing that we, the, like, the royal we, haven't yet figured out is how to differentiate people who are thinking about open research and have kind of made the, I dunno, epistological, methodological, ontological, whatever, decision that that open research practice isn't appropriate for them versus the people who don't engage with open research because they either kind of need more training or cause they haven't heard of it or because they don't want to, so, um, yeah, I think that that's...that feels like where we, the collective we, are kind of at with incentives. Um, cos it is like, there's a, there's a lot...and I've, I've written about this quite a lot recently, but there are a lot of open research practices that just don't make sense for some qualitative research. And some of my colleagues say this and they're like, oh yeah, but it, it kind of does make sense because it's flexible, but it doesn't, for me, most open research practices should be aiming to, um, kind of reduce questionable research practices, and if the questionable research practice isn't relevant for your method, then what is the point of engaging with that tool? Does that make sense?
[00:22:51] Nick: Yeah. I mean, that nuance, um, I think is a really important conversation. I mean, it's ...we've had similar conversations, I think, in the library, because I keep sort of saying, well, we keep talking about open and immediately that switches, people off, because well, I can't be open. Um, so we talk about FAIR data, for example, which isn't necessarily open. Um, but the language of open can perhaps feel exclusionary. And, um, had a conversation with Kelly again, actually, just to mention Kelly again. So a paper that she did and qualitative interviews, um, and she, you know, the ethics around anonymizing those and making sure that that was all done ethically.... I mean, it's a major issue for us in this team actually trying to manage that, kind of, manage that kind of thing. And you've touched on it in your paper as well, I think, its sort of suggesting that actually early career, feminist researchers might be actually doing that more qualitative research as well. So there's a cultural aspect to the type of research that different demographics are doing as well perhaps?
[00:23:54] Maddi: Exactly. And I think that there's a danger in the conversation around open research and qualitative research or creative research or participatory research that is, kind of, seems to be centering a bit around, oh, well these are all the ways that you can participate if you just think about it enough or if you build it into your ethics or things like that. And actually, what I would really welcome is a bit more of a kind nuanced, as you say, um, appreciation that yes, there's, there's all of these open research practices in a kind of buffet and actually not all of them will be appropriate and not all of them will be relevant. And that kind of goes beyond, oh, it's because you need time and training and more because you can make an informed decision about actually that that tool or practice isn't appropriate or isn't relevant.
Um, and yeah, so I think that that kind of move to...more like legitimising, opting out, which sounds counterintuitive., but I think it feels important in order to be, um, really inclusive, you know?
[00:25:02] Nick: Yeah. No, I, I, I totally agree, and I, I think, you know, as you say that the whole conversation around incentivising is, is problematic or measuring, you know, how do we measure these things? Um, but at the same time, there are existing incentives that are perhaps, against the open agenda as well? You know, like, um...
[00:25:23] Maddi: yeah, that's true.
[00:25:23] Nick: Publish or perish and just...so again, it's a cultural aspect that...but trying to unpick those I think is, is very difficult.
[00:25:30] Maddi: Yeah, and it's like, we talk in the paper about a term, I, I don't even, we actually use this term, but about "slowpen" science, which I really, really like. And I think that this whole move to slow down research, um, feels really important. Like that feels like a really good progress. And I guess that kind of also by definition, um, like truly participatory research or citizen science or qualitative research is inherently slower. Like it's kind of, it's inherently more thoughtful and slower and your data's richer and it takes longer to analyse , and this kind of thing. So I do think that there's potential for, um, the kind of open research conversations and some of the existing, I guess, kind of like feminist research principles that they can be allies to one another. Um, and I think that this whole move to just really just slow, slow down, slow down, really think about kind of, what is your research question? How can this be done as open as possible, as closes as necessary, in a way that kind of is as close to the values of transparency and rigour in a way that makes sense for your approach to your research? And I, and I think that that to me is, is the real hallmark of really good research, when it's just slowed down a little bit.
[00:26:47] Nick: Yeah. Uh, no, no. Yeah. The, the concept of slow, slow....I haven't heard the term Slowpen before, but yeah, slow slowing, slowing, slowing science down and you know, and again, that's to do with the culture isn't it, and the salami slicing and making and publishing rapidly and maybe, you know, getting the best REF results and all that kind of stuff. Part, part of the culture. Um, so I'm aware I sort of took that back towards staff, perhaps away from students, but I suppose that's an interesting part of the conversation as well for me because the conversations that, that we've had already around, you know, in my role and our team as a central service in the library, we support PGRs and staff.
You know, we, we, we are not employed to support students, which will apply...so again, how do we sort of collaborate, I suppose, in, in terms of encouraging, um, better practice with students as well, as well as staff. Um, but perhaps just try and talk about that a little bit in terms of how we can perhaps embed it a bit in the curriculum? There's a program at Leeds called Curriculum Redefined isn't there, that we're hoping to maybe leverage a little bit in this direction?
[00:27:56] Maddi: Yeah, so part of my role, um, is I'm a Curriculum Redefined lecturer. So what that means is that part of my job is to contribute to the kind of redefined... redesign of the, I guess, of our like psychology programs. So what we're doing with that is trying to really explicitly, integrate open research into the curriculum. Um, it is really hard! It's really hard because, um, I guess because there's so many, like in psychology, there's so many students and there's three years. And then in terms of like the major project at the end of third year or in third year, um, there's so many different supervisors. So trying to kind of have, a kind of cohesive, like these are our principles and this is what matters, and this is what we think research should be in a way that's inclusive to all research methods and in a way that it doesn't make for a kind of confusing or disjointed student experience, is really difficult because it's a bit like...one of the lessons I do currently, I think is in like week two of students when they first join on the BSC in psychology, is I talk to them about replication and reproducibility and open research and it's, it's a bit of a kind of delicate game because for some of them they really get it and it's like, oh great! You know it...I'm hoping that it can feel a little bit empowering cos it's like, oh, you can contribute to knowledge because we're kind of having this like reappraisal of psychology at the moment. But I'm also very aware that, I'm sure that there's some students who are sort of, it feels all, all of that kind of lack of, not necessarily credibility, but lack of, um, concrete answers...
[00:29:38] Nick: unsettling?
[00:29:39] Maddi: Can feel a bit unsettling yeah. Um, so I definitely think that with Curriculum Redefined we've got our work cut out for us to try and think instead of just trying to kind of sprinkle it across the curriculum that already exists, what would this actually look like if we, um, yeah, if we kind of integrate the principles of open research from day one in terms of how we structure the curriculum, what we teach, how we teach. Um, so yeah, so we're, we're only kind of a few months into that process. Um, but I think it'd be really interesting to see what that looks like.
[00:30:12] Nick: And I guess you're just talking from a psychology perspective. Cause again, immediately I'm, I'm, I'm naturally trying to think broader than that, you know, and how can we bring that into other disciplines? I guess there's no easy answer to that is there? It's just, uh, we need to take a sort of longitudinal view and take it one step at a time, and hopefully the things that you learn from, from doing it in psychology, we can try to extend to other disciplines.
[00:30:39] Maddi: And I think as well, it's really important to me that open research, or that engagement in open research doesn't become about kind of ticking boxes of like practices. You know, like open research doesn't mean you pre-register or that you make your data open. It means that you are kind of attentive to principles like transparency and openness and FAIR data, and those kind of more like abstract principles about how we do research rather than you have engaged in X, Y, and Z behavior.
So I think I would really kind of encourage, particularly Curriculum Redefined colleagues, also everyone, to think about, well it might mean that, you know, preregistration doesn't make sense for your discipline or you don't work with data so open data doesn't make sense, but those kind of fundamental principles....um, and I think things like inclusivity and compassion and that kind of thing, I think I see as being open research principles as well. So how can we, you know, design a curriculum that, um, champions those values?
[00:31:39] Nick: Yeah. And, and again, I don't wanna throw too much of a curve ball, but again, I'm just thinking out loud, to what extent is this relevant, do you think, outside the academy? Or is it? You know, I mean, what I'm thinking of, I suppose is again, public trust in science. You know, it just made me think when you were talking about your students and they might be unsettling that, you know, you know, all the lack of perhaps scientific literacy and that, well, science...all scientists think something different, so how can we believe what they say or trust what they say, you know, through the Covid pandemic, or whatever it may be, I mean, it's a bigger societal question. I mean that's, I mean, that's a big question.
[00:32:18] Maddi: That's, I think that's above my pay grade. But the , that was a big question. Yeah. Cause I think things like, like all the, the things we looked about in the systematic review around like, um, engagement...attitudes towards science, scientific literacies, that really, that beyond kind of what does that look like for undergraduate students, that those kind of things, um, you'd hope that there's some kind of initiative to combat those just generally in the public. Um, and that's why I think that, cause I always try and talk about open research and psychology, not in terms of like, oh, we're having a crisis and nothing will replicate and everything's miserable. But more that this is such an exciting time to be, even just kind of thinking about science and knowledge and life because we're having this, like we're really taking seriously, kind of, evidence and methodology and how we communicate our science. And I think because there's things like, you know, um, it's all well and good...you could do a really great, really robust research study and it's all preregistered and the data is open and it's really robust and lovely. But then if the media reports it in a way that then kind of overstates the claims, for example, and then that's what gets communicated. And then there's this whole issue of like, credibility. Um, yeah. Isn't necessarily in the academy, but still is really important in terms of, um, how we communicate our research in a way that has kind of meaningful impact. So I think that yeah, the whole open research conversation is for everyone.
[00:33:49] Nick: Yeah, and I'm just thinking as well, I mean, a specific practice perhaps that we haven't touched on, which I think you do, you preprint I think as ..?
[00:33:56] Maddi: Yeah, I preprint all the time.
[00:33:58] Nick: ...as a rule. Uh, but again, there's been some nuance around that discussion, hasn't there? Again, through Covid and the fact that they're picked up by the media and maybe communicated in a way that isn't necessarily...
[00:34:09] Maddi: yeah, I think, I think preprints is, is really complicated. Cause I preprint everything kind of just by default, um, because there's, um, like there's some papers that are being cited all the time now as preprints, that are still stuck under review somewhere because,I don't know, couldn't find reviewers or anything. So that's really useful. And I think when I was kind of on the job market, it was really useful because I could show, oh, here's everything I've been working on, and it's all under review, but trust me, it exists. But I also think that, you know, peer review is there for a reason. Um, and trying to, kind of promote my work, but also caveat it gently with it hasn't yet been peer reviewed. But that is quite a nuanced thing to say and I get that like other kind of scientists and researchers may understand the nuances of that, but what does that look like, as you say, when it comes to, um, comms and media and kind of more public, uh, consumption of that science? So I think it is, it is complicated, but generally I'm a big fan of preprints. Cause I think that, It is one way of kind of, I guess like democratizing access to research about...
[00:35:20] Nick: well, the whole open access thing is, you know, that's another, another conversation. But I suppose in that...just interested in your experience or opinion of so-called open peer review? You know. Does, does that work in your discipline, you know where, because it's in the public domain so your peers can look at it and scrutinize it and comment on if they wish, but do they or..?
[00:35:41] Maddi: Yeah, I've done, um...so it was a general rule whenever I peer review, I sign all of my reviews. So I think, in the spirit transparency, I sign my reviews, um, which feels a lot easier when you really like the , when you really like the paper. It, it felt very different, um, when, for example, you're a reviewer and you're like recommending rejection or when there's like, You know, things wrong with it. Um, I think what it does, um, or the kind of practice of open peer review is...it's, it's slightly kind of two sides to it, but I think it has the potential to combat some of those issues around compassion and hostility and that kind of thing, because more often than not, that stuff is done when there's a kind of veil of anonymity. Um, and I think kind of, having a culture where yes, we can like criticize, or not criticize, we can critique each other's work because that's important. But if you have to put your name to it, I think it could potentially be one mechanism to um, yeah, to kind of create more of like a compassionate culture.
But then we also see examples where people are just kind of actively hostile on Twitter, where their name's attached to the comments. So it doesn't work all the time, but I'm hoping that it, it should in theory be better for things like accountability. Um, or that's kind of my own experience that I feel it's a bit like, um...someone said to me when I was like, marking my first piece of student work, they were like, don't put anything on that piece of student work that you wouldn't be happy to sit across having a coffee with someone and say to them. And I think that having more of that kind of vibe with peer review, um, should help some of those things we talk about in the paper in terms of like hostility and unkindness and separating the research from the researcher and things like that.
[00:37:28] Nick: Yeah. No, that's great. That's a really interesting conversation. Thank you. I suppose you've already referred to some of the, you know, the need for more evidence, et cetera, but I suppose so I suppose the question is, you know, what's next for you? Have you got other projects? And I know that there's one project that's, uh, yet to be, um, secured around trying to embed in the curriculum, isn't there? So you and I have been, have been looking at. Um, anything else in the pipeline for you into, in this sort of arena?
[00:37:54] Maddi: Um, so one of my big things is...so I'm trying to, um, I'm trying to design something that's, that's gonna be, um, useful to try and look at student outcomes more nationally, particularly in psychology. So it's very like early stages at the moment, cause I need to apply for funding, which...that whole process just scares me a bit. But I'm trying to design some kind of like randomized control trial thing that I'm sketching out at the moment with some colleagues in psychology that aims to look at...um, so for the institutions that really explicitly integrate open research into their curricula, do they show any difference in student outcomes compared to the institutions that don't? To try...everything that I'm doing at the moment is trying to kind of fill this gap in the literature of needing really good empirical evidence, to, I guess, kind of evidence or show any impact that open research has on student outcomes. Um, so that's, that's the biggie that I'm working on at the moment.
[00:38:54] Nick: Well good luck with that. Um, so no, that's been really interesting. As I say, lot, lot of. I think really interesting themes of how it sort of, you know, how open research really is part of research culture that, you know, hopefully explore with you in um, more detail, um, in the future. So, uh, but thank you for that.
Thanks very much for your time and, um, I'll let you get on. Thank you.
[00:39:16] Maddi: Okay, thanks for having me.
[00:39:20] Outro: Thanks for listening to the Research Culture Uncovered podcast. Please subscribe so you never miss out on our brand new episodes. And if you're enjoying the discussions, give us some love by dropping a five star rating and written review as it helps other research cultures. Find us and please share with a friend and show them how to subscribe.
Email us@academicdevleads.ac.uk. Thanks for listening, and here's to you on your research culture.