(S3 E7) Octopus: a new way to open and transparent research with Dr Alexandra Freeman - podcast episode cover

(S3 E7) Octopus: a new way to open and transparent research with Dr Alexandra Freeman

Apr 19, 202342 minSeason 3Ep. 7
--:--
--:--
Download Metacast podcast app
Listen to this episode in Metacast mobile app
Don't just listen to podcasts. Learn from them with transcripts, summaries, and chapters for every episode. Skim, search, and bookmark insights. Learn more

Episode description

In our weekly Research Culture Uncovered conversations we are asking what is Research Culture and why does it matter? This episode is part of Season 3, hosted by Nick Sheppard, who is speaking to colleagues from both the University of Leeds and from other universities and organizations about open research, what it is, how it's practised in different disciplines, and how it relates to research culture. In this episode Nick is joined by Dr Alexandra Freeman, the creator of Octopus, an innovative new open science platform designed to replace journals and papers as the place to establish priority and record your work in full detail.

Dr Freeman is Executive Director for the Winton Centre for Risk & Evidence Communication at the University Cambridge. Previously she had a 16 year career at the BBC, working on series such as Walking with Beasts, Life in the Undergrowth, Bang Goes the Theory, Climate Change by Numbers and as series producer of Trust Me, I’m a Doctor. Her work won a number of awards, from a BAFTA to a AAAS Kavli gold award for science journalism.

You can connect with Alex via Twitter.

In this episode we talk about:

  • Alex's professional background and her long standing interest in science communication
  • Her work at the Winton Centre to help patients make evidence based decisions about healthcare
  • How academic journals aim to both disseminate useful findings to practitioners and to act as the primary research record, which can pull them in different directions
  • Problems with how research is incentivised, rewarded and assessed which can lead to 'questionable research practices' and contribute to the 'reproducibility crisis'
  • How she came up with the idea for Octopus and how it aims to address these fundamental issues
  • The Octopus model which disaggregates the journal article into its component parts in a structure that more closely matches the actual process of research: problem, hypothesis, protocol, data, analysis, interpretation, real world use, review
  • The sheer ambition of her vision for Octopus and the challenge to fundamentally change the research ecosystem, to encourage and incentivise intrinsic research quality
  • How Octopus is currently STEM focussed, reflecting Alex's own background, but that she has begun to explore the model for the social sciences and humanities
  • How you can start using Octopus right now by visiting https://www.octopus.ac/ and publishing your first research problem

Be sure to check out the other episodes in this season!

Links:


Transcript

[00:00:00] Intro: Welcome to the Research Culture Uncovered podcast, where in every episode we explore what is research culture and what should it be. You'll hear thoughts and opinions from a range of contributors to help you change research culture into what you wanted to be.

[00:00:25] Nick Sheppard: Hi, it's Nick, and for those who don't know me yet, I'm open research advisor based in the library here at the University of Leeds. You're joining us in season three of the Research Culture Uncovered podcast where we'll be speaking to colleagues from both the University of Leeds and from other universities and organizations about open research, what it is, how it's practiced in different disciplines, and how it relates to research culture.

If you haven't already, you can catch it with season one, which is an introduction to the podcast and to my co-hosts in season two with my colleague Tony Bromley, who is in conversation with a number of presenters from the Reds Conference of 2022. That's the Researcher Education and Development Scholarship International Conference held in Leeds.

But now I'd like to introduce my guest for today, Dr. Alexandra Freeman. The creator of Octopus, a new open research platform and radical new approach to academic publishing designed to replace journals and papers as the place to establish priority and record your work in full detail. Open by design, the platform is being developed in partnership with Jisc and the UK Reproducibility Network with a grant from Research England.

Dr. Freeman is executive director for the Winton Center for Risk and Evidence Communication at the University of Cambridge. Previously, she had a 16 year career at the BBC working on series such as Walking With Beasts. Life in the Undergrowth, Bang Goes the Theory, Climate Change by Numbers and as series producer of Trust Me, I'm a Doctor, and, uh, you've won a number of awards, I believe Alex, including a BAFTA. What was the, what was the BAFTA for?

[00:01:51] Alex Freeman: Oh, the BAFTA, the BAFTA was when, uh, my very first job at the BBC was on walking with Beasts and Walking with Dinosaurs. And, uh, it was for walking with Beasts Interactive, which was an idea that I came up with to help people explore more about how we know what we were sharing on screen and how we did the animation. So you could switch between the kind of the main program and go off and explore more kind of behind the scenes. So, yeah, it was good fun.

[00:02:22] Nick Sheppard: I think I remember that. I remember the program, I'm not sure about the interactive side. Is that still out there? I might have to have a look at that with my kids myself. Is it still there?

[00:02:29] Alex Freeman: Oh, it might, it might be. It's probably, everything's available somewhere on the internet, isn't it?

[00:02:34] Nick Sheppard: And uh, and I was listening to another podcast recently, that you did with Jisc and you were, you worked with David Attenborough. Is that right?

[00:02:41] Alex Freeman: Yes, yes. I worked on Life in the Undergrowth. I was one of the researchers on that, so spent several years going around the world with David. Um, yeah, I mean, it all sounds very glamorous, but it's, uh, a lot less glamorous when there's two of you and 26 cases of kit that you've gotta get up a mountain somehow.

[00:02:59] Nick Sheppard: And do you miss the BBC? Did you? Um.

[00:03:03] Alex Freeman: Well, I don't miss, uh, trying to drag equipment up mountains at five o'clock in the morning, but I mean, yeah, there are, there are certainly things that I do miss. Um, but things that I don't, so yeah, it's all change every few years in my life.

[00:03:19] Nick Sheppard: But it...so it does sound that, you know, with that sort of background, you were interested in communicating science, I guess?

[00:03:26] Alex Freeman: Yeah, yeah. I've always been interested in communicating science. I mean, right back from when I was a little child, I used to watch all the science programs and natural history programs and I loved them and that was what I wanted to do. And then I studied, uh, communication in animals. I studied animal behavior and communication between butterflies was my, uh, thesis, and then I went into what I'd always dreamed of doing, making natural history programs and writing. Um, and it was a surprising change really to go back to academia, but it was to study communication, uh, between humans. Um, so we worked on, or work on, uh, communicating evidence, which is quite a different thing from normal science communication. Excuse me. It's quite a different thing from normal science communication because it's about communicating research evidence to those who might want to use it to make important decisions in their lives.

Um, and having started doing that, uh, I changed to doing that in 2016, and I found myself leading a research group with people, you know, doing postdocs in research. And that was when I really realized how much the sort of research assessment and the research culture relied on the way that researchers communicate their work. And that was when I started thinking, hmm, I think something needs to change here cos I'm not sure that the current journal system is really encouraging people to communicate in the way that would be best for science and research in general.

[00:05:10] Nick Sheppard: That's really interesting. Just, I mean, I dunno a great deal about, so...what you're referring to now is the Winton Centre for Risk and Evidence Communication as, as you say, so...I mean, it immediately puts me in mind of, I mean, we'll come on to maybe discussions around reproducibility, and whether or not research and scientific findings are actually robust. I mean, is that a problem that you address in your work now?

[00:05:39] Alex Freeman: It is, I mean, it's not...so the research that we do in the Winton Centre is about how to communicate numbers and quantitative evidence and the quality of the underlying evidence to people. So those people might be, uh, patients trying to make decisions about healthcare, they might be a judge or a jury in court. They might be policy makers trying to develop policies. So all of these people are trying to make decisions based on evidence, and we work on how to communicate that evidence in such a way as to support that decision making. So how to help people weigh up the pros and cons and know about the quality of the underlying evidence. And so that quality of the underlying evidence can sometimes, you know, overlap with things like the reproducibility crisis. But it's not directly related to it.

[00:06:36] Nick Sheppard: Right, right, right. And...but this then your sort of work and coming back to academia that led you to the idea of Octopus? So how did that

[00:06:47] Alex Freeman: it did, yeah.

[00:06:49] Nick Sheppard: ...sort of come about? Was it, was it just, uh, as you said, there was...you'd identify problems with scholarly communication. Perhaps you can talk about what you think those are and how Octopus sort of addresses them?

[00:06:59] Alex Freeman: Yeah, I mean, how long have you got? I think, uh, yeah, I mean, I knew about things like the reproducibility crisis before I came back to academia. I mean, you know, we probably all read about that kind of thing in the popular press, but really what struck me more than that, and more than the push for open access, were sort of broader issues with the research culture and the kind of publish or perish culture and also, working on the kind of communications that we did in the Winton Centre, it really struck me how much people needed to...well, there are kind of different ways of communicating. So when I was communicating in the media, it was quite right that my sort of incentives were built around how many people watched the programs and what people's reaction to the programs were. Did they find them engaging? Did they feel they learned something new? These were the sort of metrics that we got back, um, either immediately the next morning or in a week's time, and we were really judged on those.

It's quite, you know, we get quite a lot of feedback and we respond to that feedback constantly in the media and that's right, you know, we're basically an entertainment medium and when I started working on evidence communication, I really realized how different that needs to be, because your incentives now...you are not just trying to engage people with information, you are really trying to help people understand that information at a much deeper level and engaging emotions, and all of the kind of thing that, um, sort of normal science communication does is actually potentially a detriment for that. Um, you know, you're not just trying to sell a story and tell a good story and, you know, get people feeling kind of positive or angry or all of these emotions that you can kind of play with and engage in a narrative. You are trying to get people to really think about the numbers and make serious decisions and weigh things up and really have their critical faculties at their maximum.

And it really occurred to me that when we're writing for journal articles, if you look at the author guidelines, they kind of make sense from a media perspective. They're about, you know, keeping the word counts short, telling a good story, you know, putting the findings upfront and thinking about the impact.

But they don't make sense if your core objective is to help people understand the real detail. Uh, if you want people to be able to really assess the quality of that work. You know, all the sort of stuff that ends up being put in supplementary information is actually the really important stuff if you're trying to assess that work really for its intrinsic qualities and if you're trying to build on it.

So I started thinking, well, there's these two kinds of writing. There's writing to disseminate findings. If you're writing, to try and, uh, engage a policymaker or a practitioner and make them think, oh, this is something, you know, these are findings that I might put into practice. That's one thing, and that's all about the findings and the impact.

But if you're trying to communicate all the real depth of what's been done by whom and what happened, um, and all of the working, then that's a really different way of writing. Uh, it needs to be much fuller. It's probably not a linear narrative. It might use multiple different kinds of media. It's not just a kind of print thing. It might involve code and data repositories and videos, so, really the kind of thinking at the Winton Center that I was doing in the day-to-day research did sort of inform my thinking about the way that we communicate as scholars and seeing the research culture as well as somebody now working within a research group made me think, well, this is having a real impact on people's lives, people's careers, but also how they approach their work.

You know, I went to conferences where experienced, um, you know, really sort of, uh, high, high ranking professors were giving advice to early career researchers, and sometimes their advice was really, um, I felt was really bad advice because they were, you know, it was good advice in the environment that we're in, they were saying things like, if you want to get headlines, think about the headlines and then what research you need to do to get those headlines. And that really put my teeth on it, I thought, you know, that is not how we should be approaching research. It's not about the headlines, it's about the quality of the underlying work. So, you know, that may be how to succeed in the current environment, but if we want to help people do good research and incentivize good research, we need to change the current environment.

So that's why I thought we really need something quite radical that can help address things like the reproducibility crisis, open access issues...but also these bigger issues about what's incentivized and how research is assessed and what we think good means when it comes to research.

[00:12:53] Nick Sheppard: So you set yourself quite a challenge by the sounds of it?

[00:12:57] Alex Freeman: Well yes. And actually I was kind of also set...I mean, all of this was going on in my head, but it hadn't crystallized, it wasn't part of my day job. And then at the University of Cambridge, we have this organization that helps people come from outside academia and come and ask questions of academics. And so we occasionally get these emails saying, um, here's somebody who wants to have a chat. They're interested in X. Would you be able to spend an hour with them? Um, and somebody came and their question was, if you could start again with science, what would you do? And now that's a really big and challenging question, but it made me really think, well, the thing that I think would most change science for the better would be addressing this problem with the way that we communicate research, and hence the way that we assess research and hence the way that we approach research. And so it, it was the day before I was meeting him and I thought, gosh, I'd really better write down my thoughts so that I've got something cohesive to say.

So I wrote down all of this and emailed it to him to say, this is what I think we could chat about tomorrow. And he turned up to that meeting and said, yeah, that sounds great. Why aren't you doing it? So the ball was firmly in my court and I thought, well, fair point, I better do something.

[00:14:24] Nick Sheppard: So then you conceived Octopus. So perhaps you can just tell us a little bit about how Octopus...it launched in June, is that right?

[00:14:33] Alex Freeman: Yes, and they've been working on it....it's had a very long heritage. So it was back in 2017 that I had this sort of night where I wrote everything down on about sort of, excuse me... so it was about 2017 when I wrote it all down and had this sort of night of writing about 10 pages of how I thought this new structure could work and since then I've gone out and talked about it with people. I then managed to raise some money to start building a really simple prototype so that I could show people how it might work. And then a more complex prototype, and then eventually UKRI, the UK's Government research funder, funded a full scale, uh, production of Octopus, and helped me go into collaboration with Jisc, who, um, really have the resources and technical capabilities to build something that can take the amount of traffic that Octopus is designed to take.

And so, yes, we launched that very early product in, in June, 2022 and over the Summer we've been really adding to it and debugging it, and a lot of conversations happening sort of behind the scenes to build all the, um...everything around Octopus that we need to build. So talking to funders and talking to institutions and talking to other service providers that we want to integrate. And now we are at the point where we are about to say to the world, Octopus is there and ready to take your research. So come and publish with Octopus. And so I'm hoping that in 2023 the floodgates will be open and Octopus can really start being a place that people feel that they want to come and put their research to kind of record things to get their priority established. To get peer review, to get their institu...you know, so that their institutions can know what they're doing and the quality of what they're doing. And then they can use that sort of resource of where all their research is as a basis for dissemination work. So publishing, uh, journal articles based on that primary research that's freely available in Octopus.

[00:17:07] Nick Sheppard: And Octopus...it, it refers to, well, eight legs I guess. Is it? There, there's eight , eight components of, of research that you've identified, I think. Is that right?

[00:17:17] Alex Freeman: That's right. So Octopus, the name Octopus has, has sort of multiple origins. I mean, firstly I wanted a, a nice friendly word that wasn't sort of too academic sounding. Um, but also I wanted one that was easy to say, whatever your native language, because Octopus is a global platform. Uh, but yes, you're right that there are eight types of publication that you can publish in Octopus. Uh, so it doesn't...it's not a place to publish papers, it's a place to publish the underlying research that you might then write a paper based on.

So there are research problems, hypotheses, methods or protocols, uh, data or results, analyses, interpretations,and real world sort of implementations and then there are peer reviews. And so they form these kind of branching chains because everything in Octopus has to be linked to an existing publication because if you just, you know, allow people to put material into this kind of, like a repository, a soup of stuff, then it becomes...you know, people can publish data without publishing the method that it was, um, uh, collected according to, uh, or they can, you know, where do you put an analysis of a data set? You want all these things to be linked together. So in Octopus, if you publish an analysis, it has to be linked to the data. That data also has to be available, so you get these branching chains. Uh, each publication can have multiple peer reviews attached to it, and that forms this kind of, um, the, the skeleton, ironically, behind the invertebrate, Octopus structure. Um, and what we've done to start people off, because every publication needs to be attached to an existing publication, is that we've created, we've pre-seeded the data with existing research problems that we've extracted automatically using natural language processing from existing papers. So there are already something like 7,500 research problems or topics within octopus. And that means that there's a kind of...it's, it's almost like seeding a coral reef. There's already a structure there.

So if you come along with, uh, some research that you want to share. You should hopefully, fingers crossed, be able to find somewhere to attach your first research question to, and then you can publish your research question and then you can publish your hypothesis or theory behind that, uh, how you're going to approach it, and all of those publications can then attach to each other and, and flow from that.

[00:20:07] Nick Sheppard: And, you know, it's, it's really striking listening to talk, you know, as I have heard you speak about Octopus before you did the talk for us, um, last...earlier this year, I think about Octopus as well, but just the sheer ambition of what you're trying to achieve. You know, you've talked about, about, um, the radical nature of it and, uh, the, the, the impact on potentially research culture. Um, I suppose I'm interested in, since you've gone live and you've been engaging with, with the sector a bit more and integrating with other services, I mean, you know, perhaps a, a slightly um, loaded question, but are, are you still confident in the vision? You know, is it, is it, is it working? Is it going as as you'd hoped? Um, are there challenges that have arisen that you didn't anticipate?

[00:20:58] Alex Freeman: Not yet so far. I mean, maybe I just always anticipate a lot of challenges anyway, but I...there's nothing that I haven't anticipated and the issues that people raise, you know, there are things that people worry about. Um, one of them, for instance, is the fact that peer reviews are post publication and they're not anonymous, so once a publication's out there in Octopus, then anybody can peer review it. But a peer review is treated like a public...any other publication type, uh, it is a publication in its own right.

So people worry about that because, uh, peer reviews in the current process, people are used to them being anonymous. And it's a slightly different role from how I see peer reviews going into the future. Um, and so people worry about, you know, can I feel like I can publish a critique of a more senior, in quotes, author's work? Um, there are concerns about...people worry that I wouldn't publish my ideas in Octopus because then I might get scooped. So these sorts of concerns, which I hope in the longer term the research culture will change because octopus is designed to change the way that we approach the way that we work. So you know, you can't get scooped if you are getting credit for the idea.

The idea is a piece of work in itself. You get it out there as quickly as possible. You can only be scooped until it's out there. Once it's out there. Octopus is like a patent office. It's out there with your name on, with the data on. There you go. Your idea, you get credit for that as an idea. You don't need to collect data for that idea.

So there are different...it's a different way of thinking, which I think will take, uh, quite a while to establish itself within the community. But everything about Octopus is designed with the principle of really helping people do the best quality research, uh, and really recognizing that and incentivizing it, and rewarding what we all as a community think is good practice.

So it is very, it is, it is ambitious in that it's trying to tackle pretty much every problem that I can see at the moment in one fell swoop. And some people kind of, you know, have a sharp intake of breath when they think, you know, well, can you really tackle everything at once? Well, I think we can, uh, because I think, you know, these are fundamental problems and if you tackle them at the root, and redesign the whole system, then yes, I think we can tackle them all at once.

[00:23:41] Nick Sheppard: I mean, that, that's, you know, that...my sort of interest and colleagues, you know, working in the sector is that sort of ambition is considerable, but you know, in terms of changing research culture across the piece...so I was listening to um, a colleague of mine who'll be speaking on the podcast aswell, Dr Emma Spary, and she was talking about the challenges of changing research culture and the fact that, you know, you can get to stasis can't you? Because, well, I can't change that, um, over there because we need to work in collaboration with the funders and the institutions and the universities and the...you know, there's so many different moving parts. So you've already indicated people sort of baulk at that ambition, but, you think, well, we we're gonna get there?

[00:24:36] Alex Freeman: Yes, I do. I do. Because everywhere I talk about Octopus, you generally get. An incredibly positive response to it. People understand what it's trying to do and why it's trying to do it, and that it is a model that can work. I mean, obviously in the dark hours of the early morning, sometimes I wake up with a slight cold sweat thinking, what if I'm wrong? But you know, it's been several years now of talking about this, and nobody has come up with a really fundamental flaw as to why this model isn't a much better model. You know, it's not perfect, but it's certainly a lot better than the current model.

Uh, so that's my first thing that gives me sort of positive vibes. But the other thing is that we're talking to all the different parts of the ecosystem. Som of course researchers feel that they can't change the world themselves just by the way that they publish. Institutions can't change the world because they also are contingent on what researchers do and what funders do, and so all of these different parts, the researchers, the institutions, the funders, the journals, we all have to kind of join hands and take a leap together. And actually I've talked to all of those four and everybody is happy to start taking that leap together. So I'm really hoping that in the next year we will all move together. You know, the funders are thinking about research assessment of the future, what the next REF's going to look like. Um, and, you know, other independent funders are all interested in, you know, making sure that they fund the best research that achieves the aims that they want.

They want high quality research, so they're really interested in research quality and research assessment and how it's done Institutions. They want to be able to see at a glance what all their different researchers are doing and how their research is being, um, uh, recognized by peers, um, and by funders. And octopus is designed to plug into, uh, institutional systems and provide them with the information that they want and that they need. and for researchers it's designed to make the publication process much easier for them. So, you know, I know what a pain it is to write a paper and get it through the review process and get it to publication. And actually so much of the time I've got a lot of research that it would be really nice to be able to get out there quickly, um, and get out there easily. And Octopus is designed to be able to do that and allow me to get credit for that. And journals who I initially wasn't sure how journals and octopus would work hand in hand, but actually had really positive conversations with several journals now about the fact that it...journals can still publish articles and in fact, increasingly journals are moving towards, especially the larger, more general journals, they're moving towards shorter articles, articles that are more to the point as far as their readers are concerned, and a lot of...the sort of the research work is therefore ending up in supplementary information, which can be a bit scattered and difficult to get hold of.

Well, Octopus can essentially form the basis for a research article. It can include all of the data and all of the analysis and all of the other work that goes on in a research project in a structured way. And then the journal article can kind of sit on top of that, and journals really appreciate that.

And in the future, you know, peer review is a difficult and expensive process for journals to organize. If there are peer reviews within Octopus....they're thinking about how they can use those to streamline their own systems and choosing what then gets published in their journals. So they're really positive about it as well. So I really think that, you know, we can all take this leap of faith with our, you know, holding hands in a row and stepping out into the, into the new future together.

[00:29:07] Nick Sheppard: No, I mean, as you know, I'm sold. But I suppose the, um, just to, to challenge that a little bit, I suppose one of the conversations, or some of the conversations I've had with academic colleagues, and in my role as well as sort of research support and, you know, open research and the sort of stuff that I do within the university, you know, we've seen so many different platforms proliferate and I suppose I worry that the infrastructural space for research dissemination is quite crowded, you know? It's, it's quite confusing. You know, we've got preprint servers. Increasingly there's things like pre-registration that people may have heard of through the Open Science Framework, there's commercial services like ResearchGate, then there's ORCiD and people don't quite know how that interrelates with different things...there's Science Open, you know, there's institutional repositories like we manage here, the White Rose Research Online Repository and our data repository. So with that confusing landscape, I mean, is that a challenge? Yeah, well, I I think it is a challenge.

[00:30:04] Alex Freeman: Well, it is a challenge, yeah. I mean, I find it a challenge in my researcher job, uh, and I use several of those outlets. Um, but Octopus is designed to really bring all of those together. So if you are pre-registering already somewhere, if you are using protocols.io, if you are using Open Science Framework, any of these repositories or services, then Octopus can tie all of those together so you don't have to repeat work that you're already doing. If you're already using somewhere, then you can just use the URLs or the DOIs that you've created in these other services and put them in Octopus and kind of get them stamped in Octopus that, you know, this is how this fits in with all my other research. So if I've published a pre-registered hypothesis. In Octopus, I just need to say I have pre-registered the hypothesis, here it is, here's the DOI. And that then allows that hypothesis to be within my research chain to be research assessed. So I hope that by working with, um, you know, UKRI and people and institutions, we can really help researchers navigate the complex structure and bring it all into one place so that it's easier. It, it means that you don't have to give up doing what you, you are already doing. It allows you to tie together what you're already doing, and if you're not already using these services, then you can either purely use Octopus or you can use whichever service works best for you and tie it into your Octopus stuff, and Octopus will bring it all together for your institution.

[00:31:45] Nick Sheppard: It's already linked up with ORCID, right? You log in with ORCID?

[00:31:47] Alex Freeman: Yes. Yeah, you log in with your ORCID, um, and it pulls in all the information that's attached to your ORCID so it already knows about all of your other publications. It will automatically pull those in from ORCID and present them to your institution. So when your institution wants to look at an author's page, they can see everything that that author's done, all the peer reviews that they've written, all of the publications outside of Octopus, all of their Octopus work. And increasingly we'll have little graphics to show, you know, how that...what sort of researcher that person is. You know, are they doing lots of collaborative work? Are they coming up with lots of ideas or are they a data collector? Are they an analyst? So it'll provide institutions with little miniature profiles of each researcher.

[00:32:36] Nick Sheppard: Well that, um, I'm conscious of time, so we'll let you go in a moment, but just, you know, how, how...you touched on this a little bit, but I suppose, how does Octopus sort of address the question of research quality? You've mentioned reviews and, you know, there's this issue of hierarchies perhaps in the fact that, you know, an early career researcher might be unwilling or, you know, reluctant to criticize the work of a more senior colleague, that kind of thing.

[00:33:05] Alex Freeman: Yeah. So Octopus is all about encouraging and incentivizing intrinsic research quality. So what I mean by that is that so much journal articles end up being about the impact of the findings. But findings, although they're important and they might be our ultimate aim of a lot of research, they...they're not robust unless they're based on really high quality methods and ideas and hypotheses and data collection and all the other kinds of work. And I think that at the moment that means that we're driving people far too much towards it all being about the results and the findings and that we're putting less emphasis on the fact that we need specialists who are all about analysis, say, or data collection or hypothesis generation, and that these are important specializations and the quality of their work...it needs to be judged in its own right. Um, so that's why in Octopus we break up the unit of publication into these smaller pieces. because, you know, I think that each of those pieces represents a genuine sort of body of work that a person can produce.

And so that can increase, um, specialization because now if you are specialist in one part of the research process, you can be publishing just that part of the research process and collaborating sort of worldwide, you know. Um, but it also means that we can assess each of those units individually. So, as I said, peer review...you'd be peer reviewing, not a whole journal article, which might be, you know, seven or eight or more people's work, all kind of integrated together. Instead, you'd be peer reviewing a data set, or a hypothesis, or a protocol or an analysis. Um, so it means that the peer review is much more focussed, and we also have a rating system. And that now scares some people because they think, oh, you know, we don't wanna put a, a number on the quality of somebody's work. But actually it allows us to set criteria, predefined criteria, which define what we as a community think good means. So what does a good data set look like? What does a good analysis look like? And if we can come up with predefined criteria, and that will help people structure the way they're thinking about it, um, and rate those, which then once you've got a numerical rating, that does help a lot with the sort of digital world that we live in with sort of ordering search results and, you know, graphing results. But it will be also combined with the qualitative peer reviews. So I think together all of that helps us really, uh, refine research assessment from being something that's about quite a broad diffuse thing, like a journal article, and instead being much more focused on the real integrity and quality of a much smaller piece of work. And I think that's what we need to do.

[00:36:23] Nick Sheppard: Yeah. No, brilliant. That's great. Thank you very much. So I suppose my final question would just be...I mean, you need people to use this platform, right? That's the challenge, you know, it is easy perhaps, or easier, to build the infrastructure, but you know, how can we at the University of Leads, for example, or across the sector, you know, support Octopus? I guess we just need to encourage our...to explain to them what it is, get them to, to, to, to start using it, right?

[00:36:50] Alex Freeman: Yeah, absolutely. So I think, um, first of all, go to Octopus - octopus.ac - and explore what the platform looks like. Have a read about how it works. But then what I would really encourage people to do is think about what your research questions are and publish your first research problem, which is quite a short, you know, it's about a paragraph or so, uh, publication that describes the research question that you are working on. And that then gives you a flavour of how easy it is to publish on Octopus. You shouldn't be worried...hopefully people won't be worried that their research questions are going to be scooped, so that's a kind of gateway entrance to it. And then have a think about. What, um, you might not be able to publish in the journal article at the moment.

Have you got small data sets or ideas that you are really, genuinely never going to carry on and work on further? Why not publish those in Octopus? Have a go see how it feels. You can then get credit for publishing those. So those will now appear on your record for your institution to see. Maybe somebody will peer review them for you and rate them.

So have a think about the things that you know are low barriers to entry for you and start that way. And then, as I said, next year, 2023, we're working in association with some journals who are going to be writing their policies about how Octopus might integrate with journal articles. So hopefully next year there'll be, you know...the way that journals, for instance, integrate with some, um, preprint platforms where you can publish a preprint and then automatically send it to a journal afterwards. There'll be ways of integrating your Octopus work directly with your journal articles. So that's going to be coming in the next few months or so.

So, you know, I think really think ahead now about work that you are doing. For instance, I've got work in my team where we've got quite...multiple analyses and complex work that's gone on. When we publish it as a paper, it's only going to be a small aspect of that work. So we've talked to the journal about publishing all the stuff that we've done on Octopus, and then relating that to a sort of narrative thread that we're going to submit to the journal, and they're very happy with that concept. So have a think about whether you've got work like that that might fit the Octopus structure. So yes, have a look, have a think, and then I...

[00:39:25] Nick Sheppard: I would just be remiss though, as well, I know this is something else we've touched on before in conversation, um, this is, it's quite...we're still quite STEM focused, right? Um, have you, I know you've had a few conversations with colleagues in the humanities or other types of disciplines. Do you think it...have you had much feedback that it does or doesn't lend itself to different types of research or..?

[00:39:46] Alex Freeman: Yeah, I mean, it was originally a STEM proposition because that's my background, but actually we've had really good conversations with people in arts and humanities and social sciences, uh, and many of those feel that this way of working will actually suit their disciplines as well. So out of the, um, 7,500 or so problems in there already, uh, there are some topics that cover all of hopefully, uh, humanities, um, history. We've got lots of that in there. So there are places for arts and humanities to start branching their problems out as well, I hope. But we're really keen over the next year to work closely with people in areas where they don't feel that Octopus currently necessarily sounds like it's suiting them because I think a lot of it is about we need to learn what language people use in different disciplines, and make sure that that helps people then see how, for instance, you know, a protocol or a method might be better described to fit the way that people work in other disciplines that we're less familiar with.

[00:41:03] Nick Sheppard: And there might even be, I mean, I'm just thinking out loud, but, you know, interdisciplinary work, you know, bringing humanists and scientists together, that kind of thing.

[00:41:10] Alex Freeman: Absolutely

[00:41:11] Nick Sheppard: That's an, an interesting area, I think, for open research more generally.

[00:41:14] Alex Freeman: Yeah, absolutely.

[00:41:16] Nick Sheppard: Great. Well, thank you very much. Um, I'll let you go. We've talked for quite a while. So thank you very much for your time and for

[00:41:22] Alex Freeman: Oh it's been a pleasure. Thank you. I can always talk about Octopus for a long time.

[00:41:26] Nick Sheppard: Well, we may, we may well have you back again after, you know, you've made a bit more progress perhaps next year, and, uh, obviously I'll put links to Octopus in the show notes, et cetera.

[00:41:35] Alex Freeman: Yeah. And do drop us a line...dpo drop us line if anyone wants to chat.

[00:41:41] Nick Sheppard: Yeah. Brilliant Great. Thanks very much.

[00:41:43] Alex Freeman: Perfect. Thank you.

[00:41:44] Nick Sheppard: Alex, and, uh, we'll speak to you again.

[00:41:50] Outro: Thanks for listening to the Research Culture Uncovered podcast. Please subscribe so you never miss out on our brand new episodes. And if you are enjoying the discussions, give us some love by dropping a five star rating and written review as it helps other research culturists find us. Please share with a friend and show them how to subscribe.

Email us at academicdev@leeds.uk. Thanks for listening, and here's to you and your research culture.

Transcript source: Provided by creator in RSS feed: download file
For the best experience, listen in Metacast app for iOS or Android