The value of a poll question depends heavily on how you ask it. Take this “finding” from Forum Research, as reported by the Toronto Star:
“Ottawa’s tougher rules for…mortgages are a good idea, according to 63% of Canadians.”
That’s an interesting conclusion given the question that Forum Research asked:
“The Liberal government has tightened mortgage-lending regulations, and cancelled the primary residence tax exemption for foreign real estate buyers. Do you approve or disapprove of this decision?”
As written, the question combines two distinct rule changes: the new insured mortgage rules and the banning of the primary residence tax exemption for foreign real estate buyers. Combining the two has likely impacted the responses because a majority of people (rightly or wrongly) strongly support curbing foreign buyers. Even if some folks were not necessarily supportive of “tightened mortgage-lending regulations,” the grouped-in foreigner question could lead many to say they “approve.”
What’s more, the surveyor does not explain how “tightened mortgage-lending regulations” will reduce affordability, a big pain point for young buyers. Nor does Forum Research explain how such restrictions might jeopardize an existing homeowner’s equity. Nor does it touch on the potential economic ramifications of the rules, or even the future potential benefits for housing stability.
“We generally do not provide ‘context’ for fear of biasing the question and for fearing that we are providing information to respondents that the general population does not have,” says Forum Research President Lorne Bozinoff. But that cuts both ways. Had more people understood all of these points, and had the questions been asked separately, the “approval” numbers (for the mortgage changes specifically) might have looked rather different.
Knowing Canadians’ true feelings on one of the most impactful mortgage rules of all time would have been valuable. It’s too bad these pollsters dropped the ball.
Reading your article, I concurred that conflating the two issues in one question may have been a poor choice.
However, after looking at the survey results, I think your argument is moot. Higher education correlates strongly to more agreement with the policy change. Higher income correlates to more agreement with the policy change. In the face of that, it’s hard to argue that Canadians only agree with these policy changes because they’re poorly informed, don’t understand the possible consequences, or were unable to disentangle the two factors in the question. Per the foreigner angle, respondents who listed their ethnic background as “Canadian” agreed/disagreed/were unsure in the same proportions as all those who listed other backgrounds combined, and the HIGHEST “agree” numbers came from those who named their background as “Asian” (i.e. East Asian, as South Asian was a separate choice).
“Higher income correlates to more agreement with the policy change.”
Which policy change? That’s the very question Ralph.
Recall that an overwhelming 90% of Vancouverites supported the 15% foreign buyers tax: http://www.cbc.ca/news/canada/british-columbia/vancouverites-support-tax-poll-1.3700421
The fact that a few specific demographics leaned one way on this commingled question implies little about the majority’s agreement with the insurance rules specifically. Given so many mortgage professionals are unsure about the potential outcomes, it’s hard to argue that most laypeople — no matter how rich/educated — have sufficient insight into how they’ll be affected.
Wow – that is pretty condescending.
On the contrary RS, it’s likely quite factual. And note the word “most.”
Virtually every demographic had a majority agree with the conjoined question, and “don’t know” was an option. The two exceptions were ethnic Black and income < $20k. The demographics I cited above had particularly high levels of agreement. You think that if the question was split, more would agree with the measures against foreign buyers and fewer with the other changes? You're probably right. But if you think it would be less than a majority, I'd say that's highly unlikely, given the numbers on the combined question. To say otherwise is to suggest that answers to both questions would have low correlation, or that Canadians' minds are so focused on the foreign buyer issue that it dominates their thinking about home prices in general. I don't think there's much basis for that.
Looks like we’re going in circles. :) The takeaway was simple. A strong anti-foreign buyer sentiment has biased the responses to the second question. Despite our respective opinions, there is no way of knowing: A) the degree of bias (i.e., whether it would have been a minority or majority in agreement with all of the mortgage rule changes), and B) how the responses would differ if respondents were given more context on the changes.
Well, we’re both trying to glean as much insight as we can from what we agree was a flawed survey.
I try never to comment on the “Ralph” posts. They are always the same boring one trick pony. This response was extra silly though. Anyone who does not understand the innate bias created by the precise wording of a survey question knows nothing about polling and surveys at all.
Distinct concepts should always be sampled separately. Some surveyors combine questions simply to save money. Perhaps that was the case here. Each poll question can cost $1500 or more for 1000 responses depending on the polling company.