Welcome to Talking Teaching. I'm your host, Dr Sophie Special, and today we're exploring a topic at the heart of education, assessment. Traditional metrics like ATAR have shaped how we evaluate students. Yet as educators, we know that these measures don't tell the whole story. So how can we better capture the richness of what students achieve and who they become? That's where Melbourne Metrics comes in. It is a framework for assessing and credentialing capabilities beyond academic achievement.
Developed under the leadership of Professor Sandra Milligan at the University of Melbourne, welcome to Talking Teaching, Sandra. Hi Sophie, thanks. It's great to be here. Melbourne Metrics has been shaped with deep partnerships and collaborations with schools and one key partner and early adopter of Melbourne Metrics is Kerry Baptist Grammar. And today I would like to welcome Principal of Kerry, Jonathan Walter. Welcome to Talking Teaching, Jonathan. Thanks very much. Great to be here. So let's get into it. Sandra.
I'll start with you. You've spent years researching assessment. So why do you think rethinking assessment is so critical right now?
Sophie, I've formed this opinion about the last 15 years and we kept getting feedback, mainly from schools, saying, look, the assessment system isn't working for us anymore. More and more students were opting out of things like the ATAR and opting into a broader range of programs that are now available, particularly in senior secondary. Kids themselves were walking with... their feet they were saying no this is not what we want we've been trying to get all kids to stay at school until the end of
secondary, but we're completely stuck at about 80 to 84%. We're losing up to 20% of kids even before they get to the end of secondary. Then teachers were telling us, look, there's more to teaching than just cramming them full of curriculum knowledge. They need that, maybe not crammed, but they need the curriculum knowledge, but they need a range of other things as well, things that will help them thrive, like capacity to be social beings.
and all of those general learning capabilities, as they're generally called. And schools were telling us these things are important, but they're not reflected in the current assessment system. So what can we do to the assessment systems that will enable equal recognition of those kinds of skills? That's where we're coming from. Jonathan, how does that resonate for you as Principal of Caring? Well, we're absolutely...
committed to trying to find a way to celebrate the breadth of a student and none of this diminishes the relevance of having great academic measures through ATO. That gives us a slice of where a student's capabilities are in a certain but limited set of domains.
What this project does is open up then a whole lot of other dimensions of the students so that we can provide a much more fulsome description of who they are, their skills, their strengths, their capabilities as they head out into the world beyond school. And that is incredibly empowering for our students because not every student can show
their full range of abilities captured through a single number at the end of the journey if we're able to then enter into a much deeper dialogue with them about what is it that you bring to the world where are your great strengths where are your passions and how do we capture those and then tell that story then you've got students exiting school feeling much more capable and confident in themselves and indeed ready to make a positive contribution
Sandra, what do you think about that? Yeah, look, I totally agree with what Jonathan said and how I sort of frame it is that having a measure of how well students do in the traditional metrics of learning, like examinations and so forth, for many students, particularly those who are going into high demand courses like medicine and so forth, it is necessary for them to have those. However...
it is insufficient for them because they need to be good at other things as well and every student needs to be good at those other things and those other things that they need to be good at, the assessment system has just always ignored and even though schools may have been teaching those things, there's been no recognition for it. One of the reasons I think our program is interesting to people is...
For any student, any person, you can get a reference and they say, oh, good at this, good at that. Or you can have a checkbox, you know, interested in music or et cetera. That's not enough anymore. We want something that is a solid and truthful representation of what students know and can do, that can be trusted.
by parents, by the students themselves. So, you know, it's real, this is me, that can be trusted by universities. So we want something that is equally trusted as the results in an examination system, but focusing on those other absolutely must-haves that kids need these days. So can you just tell us a little bit about what...
Melbourne Metrics is and how it was developed and where you're at at the moment? Well, it's very simple. We've been working in close partnership with schools in the new metrics program. We've developed assessments that are robust and common based on a common assessment architecture. I know that sounds too technical, but it's really important for the Trust. We've developed assessments that can be used to represent students in a lot of
social interpersonal critical thinking skills so picking up all those extra skills that students need that cannot be assessed in examinations so we have developed some tools the schools have been stress testing this for years I'm just so grateful for the effort that they've put in I'm wondering sometimes Jonathan why are you still standing because it's been a very challenging research project we now have some assessments
They're now being used beyond the new metric schools. They're of great interest to universities now because the universities are saying, well, our graduates need these skills as well, so maybe we should take that into account when we're selecting. So the whole idea was to get some assessments that everyone can trust that's based on a common currency, sort of like a Rosetta Stone.
that can work in any school regardless of the curriculum that they're using, you know, whether it's VCE or HSC or IB. And that's what we've been trying to do. And I think we're, you know, we're five years into it. I think we're nearly there. And Jonathan, from your perspective, what does the Melbourne metrics look like at your school and how has it met?
perhaps the limitations of these traditional methods. I might just take us back a step in that education's shifted so significantly in the last 20 or 30 years. So we're now all really embracing this sense that we want to be holistic educators. We recognise that this is more than just having a body of knowledge absorbed by students and then regurgitated in an exam. Schools do more than that. We uplift kids. We set them up.
really to be great humans as they go out into the world. So what this piece does is start to have the assessment landscape respond to that reality of what we're delivering every day in our schools. And so the fact that we can now start to capture the rigour and the warranted nature of this through University of Melbourne gives enormous comfort to our families and our students around, well, I am actually a good collaborator.
or I'm a good communicator in these different settings, I bring a great sense of agency to my learning and I'm able to be a good citizen. And being able to put some sort of tangible measures around that piece really does then value the individual and their profile and what they bring. And that piece around us then being able to say, well...
How can we help you make that step beyond school and develop a pathway for you, be it into university or indeed straight into work? When we're able to help you describe more fully where it is that your strengths lie and have a trusted credential that sits alongside that, that brings great confidence to our students and our families and indeed as we're hearing from universities and workplaces.
to them as well to be able to say, well, this means something now, that we're not just going off a one-page reference that might say lovely things about a person, but what does it actually mean when we're balancing this up against others? And none of this is easy. I mean, the work that goes into these assessments is really rigorous. So we have teachers and students wrestling with what this means as to how do I describe my ability to collaborate in this context and what's the evidence that's there to provide.
proof to that and they're great rich discussions around who our young people are and just where they're wanting to head.
So you think that the language that's associated with it and the understanding of self has been enriched by doing this? Completely, yes. And it is that recognition that we are doing so much more than just teaching five or six subjects at the end of the day. There's all of this life learning that is so rich in our schools that we are just doing a disservice to our students by not capturing all of that. So this is a really huge step in that direction.
I mean, ultimately, this is about sending them out feeling capable and confident in themselves. And so being really clear about their skill base and their strengths. So it is very much a strengths mindset that we bring to this that says, how can we help you understand what it is that you can do? And then letting them head off with that greater sense of confidence into the world.
Sandra, did you want to add to that? One thing I'd say is we've focused a bit on Year 12, but this is not just for Year 12. It's a K-12 program, and we have people credentialing Year 6 kids who are just about to head off into the adventure of high school. We've got schools that are credentialing at Years 9 and 10, where students are heading off into their next phase, sometimes employment and sometimes the scary upper school. So this is...
not something that you just do in the last semester of year 12. It reflects, as Jonathan said, the way schools have changed and the way almost every school in the country is trying to develop a broader base, a good learner, a good person, a good citizen and that starts at kindy or before and goes right through. So our assessments cover that full range.
And what do the students say? Oh gosh, Sophie, the main thing, whenever we run workshops or conferences, what I always want is the kids up there to begin with, because they don't have the educational jargon and they don't actually know much about assessment, but they do know about their lives. And when they talk about...
their lives they talk about on the one hand sometimes the damage that only being represented by a number does to themselves and also the sense that I've now got a language that I as Jonathan said that I can talk about myself with I mean it's interesting I just might make one point about the method of assessment because people always say, are you sure you can measure these things? And the answer is, we looked to the approaches to assessment that
organisations take, when it's really high stakes, for instance, medical education. If you've got a young surgeon just about to go out and slice someone up, you want to be absolutely confident that they know what they're doing. So it's competence. So we've adopted the same approach that organisations like medical schools take to assess the competence of doctors.
And we've applied it to the schooling context where we're trying to work out to what degree are kids competent social animals, competent citizens, competent communicators, competent learners. And we've adapted those methods to the schooling situation. That's why it is quite, I'm sure Jonathan would say this, it's quite a challenge to change the thinking about how assessment should be.
work and that's the essence of the difference. So it's a very different approach to assessment. I don't think we've got it completely nailed yet. This is a continuing research project but we're certainly at the point where schools can deliver some assessments in these kinds of competencies to students and that they can be trusted. Sandra, most senior secondary schools will use examinations.
such as VCE, IB, for people to believe that this is an accurate representation of where a student sits or their assessment. So why would people trust or believe that this new way is an accurate representation?
That's a really good question, Sophie, because examinations cannot test the competence of people in these complex competencies that we're talking about. Now, we're not the only ones who've got this problem. You know, med schools, dentistry schools, bricklayer schools, they've all got the problem at the base. How do we know how competent this person is? to it is not as simple as
do an exam at the end of Year 12. It's got a range of things in it. There's an external audit associated with our certificates which Melbourne University conducts with each school so that we know that the school is taking this seriously, they have the learning design that supports the assessment, they have the professional learning for their teachers that they require. We use calibrated, empirically validated
assessment instruments that the teachers use so we know how they operate. We're increasingly providing feedback loops through the 12 years to the teachers who do the assessment so that they become sharper about what to look for. So we've got a range of things in place that enable the university in conjunction with each school to say, hand on heart, we believe that this certificate
is an accurate and true representation of who a student is and what they can do. Now, that's the fundamental aim of the research and we're moving along that very nicely, I think. Jonathan, do you think that these are fairly accurate measures of your students? Well, yes. Our first impression is that actually they're really tough.
you look at a student who is doing everything in our school highly able and they're not able to get just easily to the top of the the scale at all so it's actually creating a more thoughtful dialogue around actually what are the skills that matter in these areas and
I think what's going to happen is that it's going to take us to a much deeper level of sophistication around not only our language to be able to describe this, but actually pulling apart really what does collaboration look like in this context. And then when you've got a really sophisticated assessment tool there that will help lead you on to the next stage of sophistication there, we are getting a better outcome for the students because we're now moving to the next stage.
And so it's all upside for us. We just see a huge potential for this to really deepen the education. And as we move from the old model to this new model, I don't think we've fully articulated or even thought where it might head, but it's super exciting. I agree. Super exciting. Jonathan, you're fortunate enough to be in a school that is ELC to 12.
You're able to speak to children of all ages from three to 18. What does it sound like in these different parts of the school? I've just come this week from the kitchen garden program in grade three for our students. So they're out growing vegetables, taking them into the kitchen, turning it into a meal that all their parents then come and celebrate.
that great achievement and that cycle that's been going for several months over the course of the year and culminates in this restaurant experience that they do at Grade 3. The skills that they are showing through that journey around understanding the natural cycle of the garden growth and the connection to the elements and the way in which we need a relationship with our natural environment in order to cultivate good food and then
putting that all together and then presenting it and talking about the menu and all of those things. That is so much more and such a richer experience for those students than just simply doing a textbook driven exercise or doing something in the traditional classroom setup, which we would just give a grade, standard grade to.
The dialogue that then can flow out of this when you're starting to look at, well, what have I learnt now about my ability to communicate in different settings? What have I learnt about my need to collaborate with my peers to make sure that we've...
being able to pull all of these learning experiences off and then show that proudly to our parents. And so we have our parents come in and see all of that firsthand, of course, but then now we're able to capture that and say, there are these other dimensions that we're assessing on now. And for teachers, we're all learning the new language or the new grammar, as Sandra says, around education.
Because it is a new set of skills that we're talking about now. And so we're running trials at different year levels across the school. All our year sixes have been engaging in it, all our year nines. And as the years go on, we will build this greater capacity within our staff and our student body around what's the new things that we're looking at valuing here. And then ultimately, every student will... hopefully be able to graduate with a profile that speaks to
more to who they are. I'm wondering in thinking about these new ways that were sat within the capabilities or sat within the curriculum but were not intentional so now you're making them far more intentional. Yeah I mean we've always done some of this stuff in our programs but this is more shining a light on it and saying now that there is a measure that's warranted by the University of Melbourne and we can look at and say well there's some
When we apply this level, there's been a whole lot of research that's gone across multiple different, I think we're up to 700,000 assessments now through the project. So it's really building this sense of confidence and rigour so that when you reach this level, you can say confidently, yep, we'll...
We feel good about that. And relative to others who have been through this similar journey, this is where you sit. So as principal, you're able to really share with your entire community that there is research and rigour behind this. What does your community say about it? What do the parents say? Well, they're watching with interest because it is this evolving space that we're working in and we're five years in. I was just delighted to hear that there are...
100 schools next year in the project. So that's grown and grown and grown. So it's built real momentum here. And we continue to learn more and share more and the dialogue's rich for us. And so our families are really watching with great interest to see just what surfaces and how this can help describe the pathway for our students. One of the most exciting bits for us, I think, is this sense now that
It's opened up a dialogue with universities around matching students to a pathway or a career grouping rather than just ranking them. Yeah, that's really exciting, isn't it? That means that we're getting kids heading in the right direction.
I don't know what space in the caring industry I want to go to. I've got a much clearer sense that that might be where my happy space is going to be. Or similarly, I'm not sure what sort of engineer I want to be, but I love solving problems and I've got that mindset. And this helps us determine more clearly for students just which job set or category they would fall into and should be looking towards as they exit school. It's really exciting and it's definitely a new way of thinking.
I think that the children being able to actually feel it themselves, so no matter what their cognitive ability or their ranking or whatever is happening in school, that they're able to really understand themselves more and actually use a different way of being assessed. Is that something that is a big focus around the research and what you're doing, Sandra?
Well, I think the learning sciences, as they develop particularly neuroscience and social psychology and so forth, they're all saying now that sitting down in front of a book and being coached on predetermined methods and content is not the way to get breadth and depth of learning or to get students to love learning or to get them to thrive. So we feel very supported.
by the learning sciences that are underpinning this, which we're very happy about. I know one of the things that parents are always concerned about is, is there a trade-off here? Like if you do more of this, do you do less of the other? So they're worried about the NAPLAN scores or the ATAR scores or whatever. And we're building a database to answer that question, is there a trade-off? But the early indications are...
that there is not a trade-off at all. In fact, it's a, what's the opposite, a virtuous circle.
and that students who have these strong learning skills, strong collaboration skills, know how to learn, know how to collaborate in their learning and so forth, they do a lot better. And in fact, they go well beyond the standard curriculum in terms of depth and breadth, which is why some of the universities are very interested in them. So we're all learning about this, but we're feeling more and more cheered, I think.
So you take feedback on from the schools, from the community, from parents, from students themselves to allow the program or the ideas to evolve? This is not a research project. It's a research and development partnership. So the whole idea is we're out there for feedback. That's our number one search here because this is really new. We call this Next Generation Assessment.
next generation recognition. So we're all on a learning curve and that's the power of the partnership, I think. So looking ahead, what's your personal vision?
for the future of assessment in schools? And how do you see this approach evolving, say, over the next five to ten years? Well, paradoxically, I would say I think there needs to be less assessment all over, which sounds sort of strange given that we've spent the last five years developing assessment materials. So I think kids have got to be given plenty of time to grow. But what we've built into our assessments is that there are opportunities for reflection, opportunities for formative assessment.
opportunities for kids to learn how to be self assessing as well as the sort of high stakes end so my maybe Pollyannaish view of assessment is that it should be a really
powerful tool that students and teachers can use to continue their own growth in the areas and directions that are going to help them thrive in their life. Now that's very broad, but it's not there for ranking. It's not there for saying you did it or you failed. It's there to support every individual to grow. That's my vision of assessment.
Fantastic and Jonathan what excites you about the future and what would you like to see for your own community and for the broader education system that you're so passionate about? Well I think it's measures that help us.
have students come out with a really strong sense of knowing how to think and how to learn and having some excitement around that sense that I'm going to be a lifelong learner. I am going to continue this throughout my life. And so with that, what gets them out of bed every day? So help them find their passions and understand what it is that gives them the ability to contribute positively to the world. And this deeper dive into...
The factors that determine the whole person help us then send them on with a greater sense of confidence about those things around, I'm a good thinker, I've got this commitment to continuing to be inquiring about.
what's coming across my pathway. And I've got a really strong sense of who I am, what motivates me, where it is I'm wanting to invest my energy in the world, and the things that drive me from an ethical stance as well. And all of those things can start to be really articulated much more strongly through these measures.
And for us, I mean, we're really conscious of this is a transitional phase too. So we're very much describing this as ATAR+. So ATAR sits there. It's a really important measure for us right now.
But there's so much more we can say about our students and send them off with this greater sense of identity through understanding that and having this deep dive, which has got rigour behind it now to be able to justify what it is that we're measuring. It's such an inspiring example of what's possible when we think differently. And even moving into next year, what are your final thoughts that you'd love to share with our audience? Sandra, I might start with you. What's next for Melbourne Metrics?
Well, firstly, I'd like to continue the partnerships that we've got because there's plenty more to learn and we're keen to continue that. One of the exciting things, though, is that we've opened up active dialogues with universities around the country and I have to tell you around the globe about how they might use these sorts of assessments along with, or in some cases instead of, some universities don't use the ATAR and so forth.
So either together with or independently of those measures to help universities do a better job to guarantee success for the people they choose. So that's number one exciting.
Number two is we've just agreed with the Council of International Schools, which is 1,500 schools around the globe who have students who are sort of international in their orientation. And we're doing a sort of a version of New Metrics International with them to join with our New Metrics schools in Australia, which I think is so exciting. Congratulations, Sandra. It's such brilliant work.
Jonathan, for your school next year, 2025 and beyond, what's next? Well, we're all in once again next year, really just to go to the next level and just bring more staff, more students in on this learning journey with us as we start to build our knowledge of these new...
measures and I guess start to share the story more broadly out so we can build a bit more momentum around the sector. Clearly there's something here that we're excited about. Can we get other schools on board? I hope then provide a little bit more encouragement for universities to think carefully about what are the entry methods that they're willing to look at as they're...
bringing our young people on. If we could give them more information, how might they use that to make some better selections so that kids are going into the right courses and staying there? And for employers, what information do they need? And we're in that dialogue with our parent community and then employers beyond to say, well, how else can we help you?
make the right selections and ultimately just get a pathway for every student as they go through. That's it. If we send every kid out feeling confident and capable in themselves, having found a passion, job's done really. Thank you both for joining me today on Talking Teaching and sharing your wisdom and expertise and feedback about how we can think about assessment differently. Thanks for joining me. Thank you. My pleasure.
As we close out the 2024 year, I would like to extend my deepest gratitude to those who make this podcast possible. Genevieve Siggins, our wonderful podcast producer, to Podcast Services Australia, and Chris, our audio engineer, and Jessie and Eva, our videographers. A huge thank you to the Faculty of Education here at the University of Melbourne. We're looking forward to welcoming many new guests and sharing fresh perspectives and continuing to explore the transformative power of education together.
Until then, have a wonderful summer break and see you all in 2025. This podcast was recorded at the University of Melbourne in November 2024. Talking Teaching would like to acknowledge the Kulin Nations, true owners, caretakers and custodians of the land from which we produce and broadcast.
We pay respect to Elders past and present of the Kulin Nation and extend that respect to other Indigenous Australians listening. We also acknowledge the place of Indigenous knowledges at the University and in education.