CMHC’s emili Under Fire

CMHCThe Globe and Mail’s top story on Wednesday suggested that CMHC is overvaluing the homes it uses as mortgage collateral.

It insinuated that the automated valuation model (AVM) built into CMHC’s “emili” underwriting system routinely overestimates property values. The implication is that taxpayers are at risk if mortgage defaults spike and CMHC cannot liquidate properties at their anticipated prices.

The story is portrayed like a scandal, which will likely undermine confidence in our housing market a bit more. Unfortunately, it’s yet another mortgage-related media story that is long on speculation and short on substance.

Before we begin, it’s worth noting that CMHC says emili (which has been around for 16 years) is not technically an AVM. Its main function is to assess overall borrower risk and not to determine a specific property value. We therefore use the term “AVM” loosely when referring to it.

AVMs exist for a reason and they’re used in dozens of countries. Their purpose is to generate objective and accurate valuations with less cost for the consumer, less managing of appraisers by lenders and much faster credit decisions. (CMHC often confirms property values in 7 seconds or less. That compares to 2-3+ days if a traditional appraisal is required.)

CMHC-emiliNow then, here is something that shouldn’t come as a shock: CMHC’s emili system, at times, overvalues properties. We all know that. It also undervalues properties (but that sort of thing doesn’t make for scintillating headlines).

Emili’s job is not to pinpoint home values with 100% accuracy. No model can do that (nor can any appraiser for that matter). As such, over- or undervaluation alone is not the issue. What matters is the variance from true values. In other words, how much and how often is emili deviating from market value? If it’s 1% on average, that’s one thing. If it’s 10%, that’s another.

This data is unfortunately not publicly available. But you can be sure that regulators have it (or will soon). Anecdotally, we’ve heard that the percentage of properties valued more than 5-10% above actual appraised value is quite small, but there is no data to confirm it.

Of course, physical appraisals aren’t perfect either. On purchases, it is “extremely rare” for human appraisals to come in less than the purchase price, according to U.S. research. (There are very few Canadian studies on this topic.)

On refinances, evidence suggests that appraisers may undervalue properties more than AVMs. That can reduce lender/insurer risk but it also has obvious downsides for refinancers (who may be unreasonably declined based on a bad valuation).

The tradeoffs between AVMs and human appraisals have been well-known for years. AVMs cannot easily evaluate factors like upkeep, view, property flaws, sun exposure, finish quality and other things that could add or detract from value. AVMs are most vulnerable in predicting values for remote, new or unique properties. That’s why insurers and lenders send out appraisers for properties that are more difficult to assess.

appraiserAVMs are also arguably less effective than appraisers for fraud prevention (assuming the appraiser is not involved in the fraud). The research differs on this point, however.

On the other hand, AVMs benefit from being emotionless machines. According to a U.S. National Appraisal Survey in 2007, over 90% of appraisers admit they’ve felt pressure to return a specific property value. By contrast, AVMs are completely uninfluenced by bias or pressures that afflict human appraisers.

While AVMs sometimes misjudge individual home values, there is ample evidence that they effectively value properties on a portfolio (i.e., overall) basis.

Emili has evaluated millions of properties since its inception. CMHC says the system logic is based on:

“physical characteristics of the property, the municipal property tax assessment, historical and current sales activity, and prior sales activity of the property being assessed, when available.”

(emili) does not use property value averages, but uses the specific characteristics of the property being assessed. The database and models are continually updated and independently reviewed by a third party.”

 CMHC calls its database “the most comprehensive…in Canada.” It includes property information on approximately eight million homes.

*******

CMHCWith its reputation and profitability hanging in the balance, and with regulators keeping it under a microscope, we’re quite certain the country’s biggest mortgage insurer is not about to take shortcuts—not with hundreds of billions of dollars in real estate on the line. If anything, it will be overly conservative on values going forward, especially now that prices are softening.

Despite what detractors believe, this isn’t a game for lenders and insurers. Erroneous property valuations are clearly linked to higher default losses. That’s why some lenders have reportedly increased usage of human appraisals to reduce their risk—even on insured applications.

Unfortunately, we have yet to find Canadian data that quantifies the difference between human appraisers and automated valuation systems (AVMs). There is plenty of research from the U.S., however.

Some studies have found that certain AVMs overvalue properties over specific timeframes. The median overvaluation in one study we saw was 4%. In addition, independent rating agencies like Fitch see it fit to deduct 5% from property values when those values are arrived at without full appraisals.

On the other hand, there are U.S. studies like this suggesting AVMs overvalue properties less than human appraisers—even in falling markets.

Either way, you can bet that insurers who rely on these models know the risks and adjust for them to the best of their ability. That may be why the ratio of auto-approvals at CMHC is down as of late.

avm-algorithmDespite all of this, every lender executive we’ve talked with asserts unmitigated confidence in CMHC’s emili system. Indeed, some lenders we spoke with today questioned the very basis for the Globe’s story—which seems to be based on a few comments submitted last spring to OSFI. Those commentators were unnamed and their motivations are unknown.

Many also question the timing of this story since OSFI already addressed AVMs months ago in its B-20 guidelines.

In the coming days, we’ll undoubtedly hear housing finance critics cite the Globe’s article as “proof” that CMHC’s valuation mechanism is reckless. But those charges are completely unsubstantiated based on the available evidence.

If proof ever materializes that CMHC is consistently and materially overvaluing properties, we’ll be among the first to report it. But we highly doubt that to be the case—in part because we routinely see more undervaluation than overvaluation (with our own clients).

We’ll be investigating this story further in coming weeks. But for now, it seems irresponsible to publicly discredit CMHC’s valuation model without clear data to back it up.


Sidebar: Numerous AVMs are in use today. Here’s one made for consumers: Zoocasa


Rob McLister, CMT

  1. EXTRA! EXTRA!
    Unnamed sources say the world will end in two months.
    It must be true because it’s in the newspaper.

  2. Outstanding article, Rob. Once again it’s reassuring, though rare, to read well-researched unbiased articles when it comes to CMHC.
    The Globe piece was nothing short of a hatchet job. Quoting a few dubious and anonymous “opinions” as fact is hardly top-notch journalism.
    Having said that, as a professional appraiser, it was disturbing to read yesterday from other mortgage broker web sites how handing the valuation job “back to appraisers” will cause an uproar in the mortgage business.
    The reason cited was that appraisers evidently come in too low on valuations. Too low in relation to what exactly? The value that the homeowner gave to the lender regarding the estimated market value of their home?
    I perform over 100 appraisals per month, and the homeowner’s EMV on 90% of the orders I receive is invariably too high, sometimes way too high. It is no secret that homeowners routinely over-value their own homes.
    Mortgage brokers have to remember that their client is not my client. I sometimes wonder if it was their money on the line, would mortgage brokers be so willing to approve the deal?

  3. Here’s the B-20 language. It clearly says a green light from EMILI alone isn’t enough. Some form of physical inspection is required. How many lenders are onside with this “new” rule???
    “…FRFIs should take a risk-based approach, and consider a combination of valuation tools and appraisal processes appropriate to the risk being undertaken. The valuation process can include various methods, including on-site inspections, third-party appraisals and/or automated valuation tools.”
    and
    “In general, FRFIs should not rely on any single method for property valuation. FRFIs should undertake a more comprehensive and prudent approach to collateral valuation for higher-risk transactions, such as residential mortgage loans with a relatively high LTV ratio.”

  4. Just what we need is more hysteria! Thank you again Rob for your clear and concise “rational” explanation of how Emili (AVM) works. It could never be an exact valuation system as changes in the markets take awhile to catch up in sales data available. The authors of that article should have talked to you first!!

  5. I agree that the story is pretty flimsy on facts.
    The real problem is why we don’t have the data. Wouldn’t it be trivial for CMHC to release data showing how emili valuations compare to the actual sold price of the property? Then we’ll see very quickly what the average deviation is.

  6. Well said Rob. This was a bogus attack. you hit every nail on the head. The system is not a perfect valuation model: both directions, up and down’ but its pretty close. Human appaisers often have the same valuation variances and there is no question Emili has saved Canadian consumers a ton of money in the last 16 years.
    The professor who is the so-called “source” of this breaking news was on the television last night giving giddy commentary on his “revelation”. He announced that current mortgage default rates were at 1.00% which we all know only misses the correct figure (0.38%) by 200%. Clearly he was an amateur enjoying his Andy Warhol moment.
    There are so many factual issues to address about Canadian property values today we do not need a fake crisis to distract from real concerns.

  7. I think the more important point is that banks routinely sell mortgages for properties transacted at prices slightly above their appraised values, whether those values are by Emili or human. This results in a virtuous cycles of ever-higher prices. However, this game is now over.

  8. with someone like CMHC, it certainly should be about the portfolio results rather than any one appraised value.Yet CMHC format does seem to cycle with ? mood ? attitudes or ?. I would suggest that 2 yrs ago, CMHC was fairly aggressive ( high) on refinance values)and last year generally low on purchase valuations ( meaning declines on someone overpaying for a black bathtub in the bidding wars). And as much as I have a great deal of respect for “good” (human touch)appraisers, the “science” often seems weak in the “guesstimated” values assigned to the differences in comparables used. (with good appraisers those questimates are based upon a wide experience, but not all appraisers are as experienced.) On any one property, I could easily get a wide range of values from several appraisers on a refinance. I hope that the values I get from my handpicked appraisers for my portfolio are consistent ( ie close to market value over time). What I also get from human appraisers is local property trends of the neighbourhood and, on refinance, an extra item- the home care attitude of the owner.

  9. You said, “It is no secret that homeowners routinely over-value their own homes”
    Well, it’s no secret that F.I.’s and their assigned appraisers support UNDER-VALUING homeowner’s homes. As Rob notes, this is especially indicative of re-finances. How is that professional?

  10. The lack of disclosure in Canada is absolutely poor compared to other G7 countries, and without full disclosure, people will draw their own conclusions whether it’s right or wrong.
    Before anyone defends CMHC, they should ask why hasn’t CMHC released hard numbers or a comprehensive report comparing Emili to on-site inspections? Or providing other crucial information that is abundantly available in other developing nations? i.e. foreclosures, second mortgages, foreign investment, HELOCs, etc.
    Identifying and Deflating Asset Bubbles by Hugh C. Beck, a member of the SEC: http://blogs.law.harvard.edu/corpgov/2009/07/11/identifying-and-deflating-asset-bubbles/
    “A bubble inflates as public information about an asset class diverges from private information possessed by the public information’s sources. Accordingly, the key to deflating a bubble is to gather relevant private information, compare it to the corresponding public data, and then publish the comparison for all market participants to see.”
    As I stated in an earlier post, there is already a blueprint for how this crisis will unfold. Nothing is different about our housing bubble compared to any other. You can debate right or wrong facts all you want — it’s irrelevant — it is however, dependent on the disclosure of private and institutional information for the public to conclude it’s views on.

  11. Hi Rob, great story as usual,I agree I have always felt the emili system was being pretty good for average markets and neighborhoods. I have also used the same appraisers for 20 years so I am comfortable with their work too. One question I have is (unless I missed it) what about input? I haven’t read anywhere where input could be flawed. The system is only as good as the person inputting the information. Value can be quite different based on square footage and no garage to double car garage. When branch banking moved to sales with their staff I expect this is where their could be some significant differences in values. Converting high ratio customers to the new and great conventional products branch staff get rewarded on could ques the results. Not pointing fingers just another angle on the story. Any thought on this?

  12. As I said already, under-value based on what? What “value” is being undermined. Where is this magical value that you refer to? How was it determined? Who determined it?
    Help me out here.

  13. So, big surprise that home appraisers think that Emili system valuations are inaccurate!
    In other news, Single family detached home builders report that condo’s are unsafe, lumberjacks suggest cooking and heating homes with firewood and local electricians report that you contact a certified electrician to change burnt-out light bulbs.

  14. Hi,
    I am not a professional, but all this sounds like hearsay, both sides. The Globe story is weak. Also “we trust them” or “they know what they are doing” have no reasoning value.
    The only fact is that there is no full disclosure. Could be national security but could also be lack of competence. It`s just what we choose to believe.

  15. The true valuation of a home or any asset is what a qualified buyer is prepared to pay for it at this point in time.
    No asset valuation model is perfect but cash price is as close to perfect as you can get.

  16. Do you have any idea of the courses and work that appraisers go through to get their designation? Certainly a lot more than real estate agents and mortgage agents do.
    Do you honestly believe that an appraisal will come in “low” because a bank asks them to? Would you risk your professional license and income to do this?
    Homeowners always believe their home is worth more than what market value would indicate. Market values can change in a few short weeks but homeowners always think of the price that their neighbors got last month or last year.
    By the way 2 different appraisers can assign a different value to the same home using the same comparables because they use a different value for the square footage value. No different than 2 highly trained doctors can have slightly different opinions on the treatment for a patient.

  17. Also Rob says “evidence suggests” but offers no data to back up his claim. How professional is it to make a claim with no data to back it up? Maybe the appraiser is more on track than you give him/her credit for.

  18. Doug do you dispute that human appraisers are more conservative than computer appraisals on refinances? If so then I assume you do very little mortgage business.

  19. Thanks for the support Doug, much appreciated.
    At the risk of belabouring this issue and boring some to tears, it is important to note that “emili” is not stictly an AVM, and and AVM is not an appraisal. AVM’s meet almost none of the requirements of the Appraisal Institute of Canada or the Canadian Uniform Standards of Professional Appraisal Practice (CUSPAP,)that all appraisers must adhere to.
    However, it is also important to note that AVM’s are often contructed with considerable input from professional appraisers. They are complicated and require advanced statistical techniques and multiple regression analysis. I know because one of the courses that Doug alluded to that must be completed by appraisers, is to design an AVM. To put it mildly, the course, which is administered through the business school at UBC, is a “b_ _ _ _.”
    AVM’s are most useful, in my humble opinion, for Computer Assisted Mass Appraisal systems (CAMA), which are utilized mainly in the assessment field to aid municipalities for taxation purposes. Appraising every home individually every year or so for tax purposes is not practical.
    AVM’s have leaked into the residential mortgage and refi business by default. They are fast and efficient but they are not designed for individual pin-point accuracy an a given property.

  20. You say: “Do you honestly believe that an appraisal will come in “low” because a bank asks them to? Would you risk your professional license and income to do this?”
    Appraisers don’t lose their license for undervaluing a property. They’re more concerned with overvaluing a property and being held accountable by the lender.
    Appraisal management companies like Solidifi and Centract facilitate this mindset. Appraisers on those systems don’t have to kiss up to anyone for business. They can be conservative on refinances without the backlash.

  21. “Modern studies of the accuracy of home mortgage appraisals in the U.S. began with an article by Man Cho and Isaac Megbolugbe, economists at Fannie Mae’s Office of Housing Research, who studied the 1993 Fannie Mae loan acquisition file, which contained over 600,000 home-purchase mortgages. They found that in this group of prime mortgages, only 5 percent had appraisals that were lower than the transaction price, while over 30 percent had appraisals that were exactly the transaction price. The other 65 percent were above the purchase price. On its face, these data suggest that appraisals may be biased. Too many mortgage appraisals are exactly at the transaction price, and the distribution is highly asymmetric.”
    http://www.philadelphiafed.org/research-and-data/publications/business-review/2010/q1/brq110_home-worth-appraisal-bias.pdf

  22. Actually I have a good mortgage business.
    Calling an appraiser more conservative than Emili is really not a good comparison because Emili is programmed to look at overall borrower risk whereas an appraiser determines what the market value would be to the best of their ability. As I said elsewhere 2 different appraisers can appraise the same house and come up with different final values. This is due to the value they assign for $ per sq ft for a replacement cost.
    Appraisals are used in a lot refis especially when the credit history may not be as strong as desired.
    The article also mentions that most of the stats used are from the US.

  23. Doug,
    As mentioned, there isn’t a ton of academic research on appraisers vs. AVMs, especially on refinance transactions. The full sentence from which you quote is: “On refinances, evidence suggests that appraisers may undervalue properties more than AVMs.” When we believe something to be true, but there isn’t enough conclusive data, we use qualifying words like “suggests” and “may.” We try to cite sources for most numeric claims but sometimes take shortcuts after writing for 5-6 hours. :)
    In this case, the evidence for that statement is both internal (we clearly see a higher percentage of refi’s declined due to value when an appraiser is dispatched — not including instances where an insurer sends out an appraiser) and external (brokers seem to confirm [example] that it’s easier to “hit” a value on a refinance when lenders use AVMs). There are exceptions of course and market or regulatory changes may affect things going forward.
    In addition, there is circumstantial evidence, including research (again from the U.S. – http://bit.ly/R5dHZo) showing that lender-commissioned appraisers tend to provide appraisals slanted in favour of lenders. Lenders have an incentive not to overvalue properties on refinances (especially on high value, uninsured and maximum LTV files) and appraisers have an incentive to stay on lenders’ approved lists.
    There is also another study I recall (but can’t remember the citation) which suggests that appraisers tend to undervalue a home being refinanced if there is obviously an adequate loan to value. There is little motivation for AVMs to do that kind of thing.
    Hopefully this explanation salvages some of our professionalism. ;)

  24. Thank you for an excellent answer.
    I would like to go back to appraiser’s comment “in relation to what” I think that he hits the nail on the head. The homeowner overestimates the value of the home in relation to the true market value (for that point in time) usually 9 out of 10 times. Does this not then shift some of the blame onto mortgage agents who put the application value down as to what the homeowner suggests? Then the appraisal comes in “low” and people say it cannot be correct or the appraiser was wrong.
    When the homeowner is paying for the appraisal I ask them “do you think that appraisal will come in at your number and what are you basing it on?” An agent has to be responsible for the value they put into the application.
    Perhaps some are right here and some appraisals do come in low. Perhaps location has some effect. However the applications that I have done usually have their value pretty close to the appraisers. And as I mentioned elsewhere 2 appraisals can be down on the same property and come in with a $20000 difference simply because one appraiser uses $120 per sq ft for replacement value and another feels $110 per sq ft is an accurate amount.
    There will always be discussions like this when humans are involved and a computer program.

  25. zoopraisal did a good job in less than 7 seconds for me just now.
    I think about it this way: accurate appraisals are a hell of a lot more important in flat markets than upward or downward ones. In that way this focus is timely.
    The fact that we insure 75% of the mortgage market is the problem. If that were 20%, as it should be, the quality of emili valuations wouldn’t be so scandal worthy

  26. In the process of renewing with our current lender right now, in the province of Quebec. Based on MLS listings (12) in the neighbourhood, I estimated our house would “list” for about $343k (if we were selling) based on ratios of listed prices versus municipal evalution and property taxes. The lender, through Emili, came in at pretty much the same number $343k. Zoopraisal low=$254k mid=$290k high=$325k. This has been a very interesting read!

  27. upward trending markets do not bode well for AVMs when they correct. clearly, in a market correction (which is most relevant to CHMC) the estimated values will be significantly overvalued.
    this much should be obvious.

Your email address will not be published. Required fields are marked *

Copy link