S1: From the newsrooms of the Sydney Morning Herald and The Age.
S1: This is inside politics. I'm Jacqueline Maley. It's Friday, April 25th.
S1: This week we're doing a special podcast about our complicated
S1: relationship with political poles. As journalists, we like them because maybe,
S1: just maybe, they can tell us something about what voters
S1: are really thinking. But we are a bit wary of them, too,
S1: especially after the federal election in 2019 where the polls
S1: were wrong. That caused a massive rethink in how polling
S1: is done and how we in the media rely on it.
S1: So let's get into it. How are polls actually conducted?
S1: How are participants recruited? How are the questions formulated? And
S1: are they right this time around? Today, as always, we
S1: are joined by our chief political correspondent David Crowe. And
S1: today our special guest is Jim Read, who conducts the
S1: Resolve Political Monitor poll for our papers. Welcome, both.
S2: Thanks. Great to be back on.
S3: And it's great to be here. And it's also really
S3: good to be face to face in in the studio,
S3: isn't it?
S1: I know it's so old school, I love it. And
S1: we're being filmed, so everybody's on their best behaviour. Jim,
S1: I want to start with you. First. We're going to
S1: do a deep dive into polling and how it works,
S1: what it's looking like for this election. But can you
S1: just talk to us a little bit about how the
S1: sausage is made? What exactly is a poll? How do
S1: you find the people and how do you make sure
S1: that the sample is representative?
S2: Sure, there's the the how the poll is done, but
S2: also the why. I guess the why we do polling
S2: is which dictates the how is that we really want
S2: to inform readers and listeners about what's going on and
S2: hopefully inform the commentary. So it's not just one person's opinion,
S2: you know, there's a sort of safety in numbers, if
S2: you like, by by assessing public opinion. So it's important
S2: that we get it right. And the way you do
S2: that in research is by asking the right questions of
S2: the right people in the right way. And that's the
S2: same with any market research, regardless of whether it's political polling,
S2: you know, so the the first thing we do is
S2: make sure we're asking the right people. And increasingly we're
S2: doing that with online research panels rather than calling people up.
S2: And that gives us a pretty cost effective way of
S2: getting to a representative bunch of people nationally. And we
S2: set minimum quotas by state, by area, by age, by gender.
S2: So we make sure we've got a good bunch of
S2: people and we can, you know, fiddle about with the
S2: back end a little bit with what we call weighting factors.
S2: So if we're slightly down on females, for example, we
S2: may upweight them a little bit in our sample. So
S2: it's representative. And then we asked them the right questions
S2: in the right way. So you know very obviously that's vote.
S2: But also we go a bit deeper into some of
S2: those diagnostic measures like what issues are important. How committed
S2: are you in your vote. How likely are you to change?
S2: What do you think of the leaders? What do you
S2: think of this policy, that policy, etc.? And I think
S2: that's one of the great strengths of our approach is
S2: we we go deeper and talk about policy and issues
S2: and events, and we've got very particular ways, for example,
S2: of asking vote. Um, after the 2019 polling episode, we
S2: came in and we thought, you know, how do we
S2: change this? And one of the major things, I guess,
S2: that we did is to take out an undecided option.
S2: So most polls give an undecided option. We took that
S2: out right. Simply because it doesn't appear on the ballot paper.
S2: So we thought let's you know, let's emulate the real
S2: decision as it is in the in the election box.
S1: That certainly makes sense. David, how do you use polls?
S1: I mean, you report the polls that we publish the
S1: Resolve Political Monitor polls, but how do you use them
S1: more broadly in your work.
S3: Really as a way to find out the mood of
S3: the electorate across a whole range of fronts. And so
S3: we ask the personal questions because we're interested in how
S3: people view the performance of the Prime minister and the
S3: opposition leader. Sometimes we've asked questions about the likeability of
S3: different leaders. We're always conscious of asking a question as
S3: pointed as possible about an issue that's running in the community,
S3: whether it's Donald Trump or whether it's health policy or
S3: education policy and so forth. So we're looking for ways
S3: to really explain to our readers what the mood of
S3: the electorate is, because each of us can can be
S3: in our own bubble in politics in a way, because
S3: sometimes there's a self-reinforcing thing that happens in communities where
S3: you you're more inclined to read the stories that reinforce
S3: your own opinion. So sometimes we well, we we can
S3: use the poll to reflect a better sense of what
S3: the wider community feels. And sometimes this leads to polling
S3: stories that people disagree with. You know, if if Peter
S3: Dutton is making inroads into Labour support by by seizing
S3: on complaints about the cost of living. Our polling reflects that.
S3: And that's an important sort of statement about where the
S3: community is at. And it counters some of the spin
S3: that might be coming out of Canberra. Actually, it's it
S3: serves quite a useful function in that way.
S1: Yeah. It's a corrective to political spin. Jim, how do
S1: you find the people that you poll and do you
S1: pay them?
S2: We do. Usually when they're collected by online polls, they're
S2: usually offered an incentive for their time. We sometimes also
S2: mix a bit of telephone polling in with that, simply
S2: because we find that if you want to, you know,
S2: be as inclusive as possible. There are certain people, you know,
S2: for example, the, you know, people aged over 80, um,
S2: you know, younger people with better things to do or
S2: more affluent groups who simply don't go on online panels,
S2: or at least not in the numbers that you'd like.
S2: So targeting them with mobile phone calls and other things,
S2: you know, helps to to balance that. But certainly with
S2: online panels, it's pretty normal that you provide an incentive for,
S2: for their time. Um, the great thing about online panels
S2: is they're very easy to reach, but they're online panels
S2: and online panels. So the most important thing is that
S2: they're recruited properly, because if they're just a sort of
S2: opt in consumer panel, I think you tend to get
S2: skews and biases in there. And I think that was
S2: part of the 2019 problem with the polls that were
S2: around then. The best polls are the ones where you
S2: recruit them randomly off the back of telephone surveys and
S2: other things like that, and you get a random sample
S2: of people.
S1: And you make sure that they're sort of serious people
S1: who are going to take the exercise seriously. I mean,
S1: how do you do that? And just, you know, you
S1: don't have some young kid, you know, yeah, a donkey
S1: vote or.
S2: You know, it's one of the things as a pollster
S2: sort of keeps you awake at night. I guess so. We,
S2: you know, the best polls. And I'm sure there are
S2: lots of polls, not just mine do this, but you
S2: have very rigorous quality control and security checks. So you
S2: make sure things aren't bots or AI by doing little,
S2: you know, those little checks that we're all used to
S2: and you do internal consistency checks. So you might ask
S2: the same question twice, right? You know, throughout the survey.
S2: And if they answer differently between those two times you
S2: start to get a bit suspicious.
S3: Yeah, right. The voice referendum was an interesting example of,
S3: I think, the use of polling and the rigorous way
S3: in which it was done. Jim's work showed by the
S3: middle of 2023 that the majority were against the indigenous voice.
S3: And this was a very uncomfortable finding for people in
S3: the indigenous community and in labor. About. And I often
S3: think back to that moment and think because the polling
S3: became very consistent all the way through to the actual referendum.
S3: And I think back on that and wonder why people
S3: didn't act on the signal would have been a hard
S3: thing for people to do to say, okay, this is
S3: clearly headed for defeat. So we're going to have to
S3: come up with a different proposal to take to the referendum.
S3: For instance, take out. Putting the voice into the Constitution
S3: and maybe just focusing on recognition instead. But I often
S3: think back to the polling at that time and wonder
S3: about whether a different pathway could have been found, given
S3: what the poll was showing.
S1: I mean, that was one example where the polling was
S1: spot on. Previously, we've had examples where the polling has
S1: been very misleading. So I'm talking about 2019 election, which
S1: obviously showed that Labour under Bill shorten was going to
S1: win government, and they most certainly did not to the
S1: extent where Scott Morrison called it, you know, a miracle.
S4: I have always believed in miracles.
S1: So what happened? Why didn't the polling pick up the miracle?
S2: Well, I think there's really no excuse for it, because
S2: we had polls that were published on the Friday and
S2: even the Saturday. You know, we were saying the opposite result.
S2: The results were a few percent out. You know, it
S2: just it became very important because they called the the
S2: result the outcome incorrectly, I think. But it's not unusual for,
S2: for for polls to be a few percent. The most
S2: important thing is that it's within your margin of error,
S2: which is usually a couple of percent. And therefore we
S2: usually get it right. I think the polling industry did
S2: a lot of navel gazing and a lot of soul
S2: searching after that resolve was brought in, partly because of that,
S2: you know, after that 2019 result and we've, you know,
S2: had a pretty good record since then. We've done, I think,
S2: four elections in a referendum for the age and the SM. So,
S2: you know, which is proven to be accurate within our
S2: margins of error. To be fair to other pollsters, we
S2: didn't have those legacy issues of, you know, using a
S2: certain panel, using a certain question format and having those trends.
S2: So we we were able to start afresh. But I
S2: think the other thing we made a conscious effort to
S2: do was to look afresh at the vote question. And
S2: one of the things, because the polls were saying a
S2: certain thing right up until Election day, one of the
S2: uncertainties was where the 5 or 10% undecided actually went
S2: on the day. So they usually excluded from polls in
S2: their in their voting result. And so we've decided to
S2: get rid of that. And we forced people, much like
S2: when they go into the voting booth and they're forced
S2: to tick an option or, you know, scribble a message
S2: on the vote paper and spoil it, or just do
S2: a donkey vote, we allow them to do that same
S2: thing in our in our vote choice. And regardless of,
S2: you know, us usually doing a poll about a week
S2: out from from Election day, it's still proven to be
S2: pretty accurate. I think really because of that.
S1: Um, well, I mean, we're going to we're going to
S1: find out in nine days, aren't we? Um, Jim, in
S1: the interest of transparency, can you tell us more about
S1: the relationship between your company resolve strategic polling and whoever
S1: commissions the polling, Like our newspaper, newspapers. I mean, what
S1: are the expectations of you from The Age and the
S1: Sydney Morning Herald? And how do you make sure that
S1: the sort of final set of questions doesn't accidentally or
S1: inadvertently reflect somebody's bias along the way?
S2: Well, again, I think there's safety in numbers. We're interviewing
S2: usually 1600 people. Sometimes it's more than that. Our last
S2: poll will be 2000 people. So as long as you
S2: pick that sample and analyse that sample carefully, I don't
S2: think it shouldn't be biased. I think also there's a
S2: how should I put it, an expectation of fidelity. So yes,
S2: we want interesting results. But they've also got to be
S2: balanced and fair and true. And I think one of
S2: the problems that that polling has in terms of, you know,
S2: fidelity and balance, or at least the perception of it,
S2: is that, you know, you might be any polling company
S2: is usually working for other entities as well, you know,
S2: industry bodies, companies, etc.. And there's nothing wrong with that.
S2: There's nothing illegal in that, but I think transparency is
S2: a big thing as well. So if you're a pollster
S2: and you have an interest in working for a company
S2: or an industry, as long as you declare that it
S2: allows people to, you know, decide what they make of
S2: the poll.
S3: Jim's being very polite about the way it's all done,
S3: I think. Let's, um, we're very happy with using resolve.
S3: We did learn from the experience of 2019. Back then,
S3: the consensus, and there was a report done about this
S3: by some independent agency. After the 2019 election. The consensus
S3: across the polls seemed to be labor ahead, 5149. The
S3: actual result was the mirror image of that. For a
S3: lot of the polls, the difference was within the margin
S3: of error. But a lot of the focus on the
S3: two party preferred number boiling poles down to two numbers
S3: only created this impression of a Labour certainty, misleading impression.
S3: And one of the lessons for us out of that
S3: was let's not emphasize only those two numbers. So we've
S3: broadened the polling and we've tried to emphasize things like
S3: the primary vote for for the major parties, which is
S3: so important, especially in this fragmented electorate where about a
S3: third of the voters are not choosing Labour or Liberal,
S3: they're choosing all these other options. And so Labour and
S3: Liberal rely on the preferences. So this was really important.
S3: Jim is not working for any of the political parties.
S3: That's important to us as well. If there's any change
S3: to that, we would declare it so that everybody knows
S3: where they stand. And people have to bear in mind
S3: the margin of error for us, for Jim is 2.4%.
S3: So if it's 51, 49, remember the margin of error.
S3: It's real. But we're very happy with the polling because
S3: every month Jim is polling 1600 people. A lot of
S3: others are doing 1100 people or 1200 people, and their
S3: margin of error is greater because of that. You know,
S3: these are just the technical things where we've got to
S3: be as good as we can, but we acknowledge that
S3: it is not as precise as perfectionists might like, and
S3: it's not a predictor. It's a poll of a particular
S3: view across the electorate at a point in time, like
S3: last weekend, not May 3rd.
S1: Yeah. How do we make sure that the polls serve
S1: a democratic purpose or a journalistic purpose? And they're not
S1: just a horse race, you know, almost for their entertainment value.
S3: It's really hard because I get feedback from people saying,
S3: why aren't you writing about more about policy? But when
S3: I write about polls, there's huge engagement. And just to
S3: be really frank about this, when I write about polls,
S3: I get more feedback from politicians than I do when
S3: I write about policy.
S1: The same politicians who say that they never look at
S1: the polls, they don't read the polls. The only poll
S1: that counts is the one on Election day.
S3: Breaking news.
S1: Yep.
S3: I mean, this is the thing. They're deeply engaged in
S3: this stuff because they're professional politicians and they've they've worked
S3: with numbers all their professional lives. And it's all about
S3: the numbers. So they're deeply invested. When we say something
S3: one side doesn't like, they let us know. They think
S3: the poll's wrong. But over time, you know, we're creating
S3: a or providing a consistent picture of where people are at.
S3: And the most important thing is the trend. We found
S3: the trend against the government when the cost of living
S3: was biting. And certainly this year we've found a trend
S3: toward the government as Election Day gets closer. This is
S3: this is the trend to watch.
S1: Yeah.
S2: And we've I think, you know, just to pick up
S2: on your point there about the two party preferred vote,
S2: we have tried to avoid that that horse race. And
S2: we have a particular way, I think, of looking at
S2: vote where we concentrate on primary vote, because there are
S2: a lot of minor party and independent candidates that, you know,
S2: make up around a third of the primary vote. Now,
S2: so the way we think about vote is that it's
S2: a measure, and it's a measure that doesn't move around
S2: a lot. So we we like to concentrate on policy and,
S2: and other things. But we do report it and we
S2: have to report it. Unless there's a leadership change it
S2: doesn't move around too much. But it is a very
S2: handy measure over time of the health of the parties,
S2: including the Greens and Independents and One Nation and other things.
S2: Two party preferred the binary measure kind of masks that.
S2: So we've we've treated though as an opinion measure for
S2: the vast majority of the term. When we get closer
S2: to the election, we start to talk more about two
S2: party preferred to to say, you know, this is where
S2: the the position is because otherwise you, you have people
S2: working out what seats would change hands from a two
S2: party preferred result, you know, two years out from an election.
S1: Yeah. Too far, which is. Yeah.
S2: Which is, which is silly.
S1: Yeah.
S3: There's a there's a great jazz standard. It don't mean
S3: a thing. It ain't got that swing. And I always
S3: think of that tune when I write in my stories.
S3: There's no such thing as a uniform swing. Um, I
S3: kind of hum it to myself because you cannot apply
S3: the national 52, 48 or 51, 49 to the seat
S3: of Chisholm in Melbourne, or the seat of Bennelong in Sydney.
S3: It does not work that way.
S1: I know, and I think sometimes when we're reporting federal politics,
S1: particularly from Canberra, we we forget about the local politics
S1: is everything. And, you know, seat to seat is is
S1: the only game in town. I just want to ask
S1: you both about how polling can affect the democratic process
S1: itself or can affect opinion. So, you know, it's supposed
S1: to be an objective measure. But sometimes, um, polls, if
S1: they go a certain way, will actually affect the way
S1: that people vote. And sometimes the parties will take pride
S1: in their underdog status or claim their underdog status in
S1: order to get ahead. I mean, what do we sort
S1: of do with that? Is that is that a risk?
S3: Um, isn't there a scientific principle where the where the
S3: process of the experiment can affect the outcome?
S1: Yeah, that's what I'm talking about. Exactly.
S2: The quantum observer.
S3: Yeah. But okay, so I was talking to somebody in
S3: the union movement recently who made the point that labor
S3: is doing better, not just based on what we're finding
S3: with resolve, but also based on the union movement's own polling. Yeah.
S3: Does that then lead some people to think, oh, Labor's
S3: going to be in, I'll vote green. Yeah. Yes. That's
S3: a factor for some people. So there is there is
S3: an impact. But the alternative is to not ask, what
S3: is that? Don't ask, don't tell. You know, like, let's
S3: not ask people because it may influence people's judgment. Well,
S3: influencing people's view of politics is kind of what journalism
S3: is often about.
S1: We can't help it sometimes.
S3: Just reporting.
S1: What happens. Yeah, it is an interesting one, I think.
S1: What do you think on that point, Jim? That sort
S1: of idea of a feedback loop, because it must also
S1: affect the parties psychologically, even though they don't want to
S1: claim the win, if they know that the polls are
S1: going their way and their internal polling is going their way,
S1: they get a pep in their step. And you can
S1: see that happening in the Labour campaign.
S2: Yeah, certainly. Um, Anthony Albanese has, um, you know, seemed
S2: a lot more confident. I think, you know, partly that's,
S2: you know, his position in the polls, I guess, but
S2: but also just running a, you know, a very decent campaign,
S2: a disciplined campaign. So they're on the ascendancy, if you like.
S2: But I think, you know, just reiterating really what David said,
S2: an informed electorate Is an electorate that's ready to vote,
S2: and we're just part of the informing process. Reading through
S2: some of the reader comments of of David's stories is,
S2: you know, which I occasionally do the most wonderful comments
S2: or I didn't know people thought this way, oh, I'll
S2: have to go and research this, etc.. So, you know,
S2: informing people about what's going on is important and, you know,
S2: you can't get around that. There's there's feedback loops everywhere.
S2: So you speak to your family, your friends about what's
S2: going on in the election. That's exactly the same thing
S2: as a poll. So I have no issue with that.
S2: The issue is, if polling were to get it wrong,
S2: there's only one thing worse than having no polling as
S2: a political party. And that's having the wrong polling. That is,
S2: it's getting it wrong. For some reason you're getting a
S2: wrong steer rather than no steer. So I think as
S2: long as we maintain our accuracy and our fairness and
S2: our balance in the way we present things, then it's
S2: it adds to the democratic process, not detracts.
S3: We're not going to put a poll on our front
S3: page on Election Day. You know, I don't see it
S3: as my role to try and predict the outcome on
S3: the day of the election. I'll comment on the outcome
S3: on election night. But in the final days of the
S3: election campaign, I want to report to readers what the
S3: leaders are saying on either side, what the policies are
S3: saying on either side. If there are any developments that
S3: are newsworthy in the actual campaign, rather than do polling
S3: on the Friday or Saturday, that says we expect the
S3: result to be this way or that way. Yeah. Because
S3: I think at that point, voters, you know, might want
S3: us to tell them where things are at in the
S3: competing propositions and they'll make up their own mind.
S1: We do, however, have a new poll coming out next week,
S1: don't we? Which is going to be the last one
S1: before the final day. I just want to ask you
S1: both very quickly what the polls are showing now, like
S1: what our last poll has showed.
S2: It's essentially very close to where we we were in
S2: 2022 after lots of ups and downs. You know, Albanese
S2: started with a very, you know, strong honeymoon, which lasted
S2: for about a year. Then the long term trend was
S2: Labour went down, down, down, down, down. Those rate rises
S2: started to bite. The cost of living was biting more generally,
S2: and the feedback we had in focus groups and respondent
S2: comments was that Labour looked, you know, distracted by the
S2: voice and other things and not acting on cost of living.
S2: So they really suffered from that. They lost a great
S2: deal of their vote. And for, you know, the start
S2: of this year, it looked like the coalition were in
S2: with a chance of actually, you know, forcing Labour to
S2: be a first term government. But we've had this huge
S2: bounce back. Labour have gone from a two party preferred
S2: deficit to I think in our last poll it was 53.5%
S2: lead over the coalition, a very slight swing to them,
S2: which would seem to indicate, you know, a majority Labour
S2: government at that snapshot. But, you know, within our margin
S2: of error, as David said, we should always look at
S2: the margin of error. There's also the chance of a
S2: minority Labour government in there. So the polls are basically
S2: we're essentially middle of the pack at the moment, and
S2: the polls are basically telling us that that's the state
S2: of play, majority or minority Labour government at the moment.
S2: But as we've seen in in recent times, a week
S2: is a very long time in politics. So let's see,
S2: we go where we go over the next week and
S2: a half.
S3: I talked to a Labour MP the other day who said,
S3: I'm worried that we're getting complacent and the polls are
S3: telling us we're ahead. And you know, we shouldn't take
S3: that for granted because he was seeing people going into
S3: the early voting booths still grumpy about the cost of living.
S3: And I think that's worth mentioning. We don't just ask
S3: about leaders and, you know, primary vote when we ask
S3: who's better to keep the cost of living low? Labour
S3: were way behind on that 2 or 3 months ago. Now, 30%
S3: say Labour and Albanese are best to keep the cost
S3: of living low and 30%, say Coalition and Peter Dutton.
S3: So it's now neck and neck on that key question.
S3: But MPs are still on the government side, are getting
S3: this blowback on the cost of living still. So I
S3: think we've got to emphasise it's still close.
S1: It's still anybody's game. Yeah. We will have to check
S1: the price of the sausages on. The sausage sizzles on
S1: the day and see what's happening with inflation of the
S1: democracy sausage. Guys, thanks so much for joining us. That
S1: was really interesting. Thank you for coming in, Jim.
S2: Thanks for having.
S1: Me. And good to see you in person.
S3: David, wasn't it good? Thank you. Cheers.
S1: Today's episode was produced by Julia Katzel with technical assistance
S1: from Josh Towers and Taylor Dent. Our executive producer is
S1: Tami Mills, and Tom McKendrick is our head of audio.
S1: To listen to our episodes as soon as they drop,
S1: follow Inside Politics on Apple, Spotify or anywhere you listen
S1: to your podcasts. And to stay up to date with
S1: all our election coverage and exclusives, visit The Age and
S1: The Sydney Morning Herald websites to support our journalism. Subscribe
S1: to us by visiting The Age or SMH. I'm Jacqueline Maley,
S1: thank you for listening.