If you’ve ever purchased a property, sold a property, or even browsed available properties online, chances are high that you’ve run across an automated valuation model, or AVM. AVMs are a common feature in real estate, including models like the Zillow Zestimate, CoreLogic, and the Xome ® Value Model.
AVMs can be great tools to help you estimate a property’s value, but there are a few important things you need to understand to make sure you’re getting the full benefit of an AVM.
What is an AVM?
An automated valuation model (AVM) is a model that uses mathematical or statistical methods along with a series of data points to estimate the value of a property. According to the International Association of Assessment Officers, “the purpose of an AVM is to efficiently provide an accurate, uniform, equitable estimate of fair market value.”
So, in layman’s terms, an AVM gives an educated, data-driven guess of how much a property is worth on the current market.
Competing AVMs may offer different information, depending on the data source(s) and the particular algorithm they use to work with that data. Each AVM vendor will have their own unique approach and may even use different models and rules for different property types to gain higher accuracy and coverage in their valuations.
Why are AVMs used?
AVMs are typically used as a convenient starting point to determine the value of a property. They make it easier for both consumers and lenders to get a general idea about property value because they are often faster and more affordable than traditional property appraisals.
Formal appraisals are still a necessary step in most lending situations, but they can be a powerful tool in the early stages of a property purchase. Financial institutions also use AVMs when doing things like making loan decisions, sometimes in combination with traditional appraisals to gauge the accuracy of the AVM results.
How do AVMs work?
Most automated valuation models contain five elements that help them come up with a valuation on a property:
- Property sales and related information
- Listed property and related information
- Property data, which can include different property characteristics, features, property condition, and more
- Price trends, such as the Home Price Index
- Machine learning models — a set of mathematical, statistical methods — used to identify the relationship between elements 1-4 with a hypothetical value of a property
AVMs do not usually include subjective data such as curb appeal in their models.
What are AVM confidence scores?
Automated valuation models do have a margin of error given that they are statistical methods — they are only as accurate as the data behind them. A natural question when it comes to AVMs is how trustworthy and accurate they might be. How are you supposed to know which one might be more correct?
That’s where confidence scores come in. Confidence scores help users gauge the accuracy and precision of an AVM estimate based on how similar it is to estimates from other AVMs.
Confidence scores are often derived from a metric called the Forecast Standard Deviation (FSD), which measures the statistical probability of an AVM’s valuation falling within the range of the property’s actual market value. A lower FSD metric means that the AVM estimate is more accurate, whereas a higher FSD would indicate that the metric is less accurate.
For example, if an AVM estimates the value of a target property to be $150,000 with an FSD of 10%, that means there is a 68% statistical certainty that the AVM will fall within 10% of the actual sale price.
(Why 68%? Because 68% represents one standard deviation in a normal statistical distribution according to the empirical rule.)
That means in this example, there is a 68% chance the actual value of the property would be somewhere between $135,000 to $165,000.
The confidence score is then found by subtracting the FSD percentage from 100. So, in this example, the score would calculate like this:
100% – FSD 10% = 90 Confidence Score
Confidence scores can be a helpful way of evaluating the accuracy of an AVM. But there is one important thing to note — the calculation of FSD and Confidence Scores can vary across the industry.
A confidence score of 90 for one AVM might not been the same thing as a confidence score of 85 for another AVM, depending on how they are calculated and what factors they might be based on.
What are the elements of a reliable automated valuation model?
Given that confidence scores can vary, how do you know that the model you’re working with is a good one? There are five factors you can look to, based on the historical performance of that AVM, that can help you determine if the model you are using is the best one for your needs:
1. Higher accuracy
When a property is sold, the transactional data can be used to help attain the percentage sales error. The lower the sales error of the property, the better the accuracy of the model. The sales error is calculated like this:
(AVM Valuation – Selling Price)/(Selling Price) * 100
So, if the AVM valuation was $150,000 like in our previous example, and the selling price was $155,000, then this would be the percentage sales error:
($150,000-155,000)/($155,000) * 100 = -3.2%
A positive percentage sales error means that the AVM overvalued the property. A negative percentage sales error like in our example means that the AVM undervalued the property.
2. Higher Precision
Sometimes, accuracy and precision are used interchangeably across the industry. Precision typically refers to how consistent the set of valuations is.
Two precision metrics that can be utilized are the Median Absolute Percentage Sales Error (MdAPE) and Mean Absolute Percentage Sales Error (MAPE). These are two statistical measures that predict the accuracy of a forecasting method. MdAPE gives the middle data point of sales error when ordered from least to greatest and MAPE gives the average of all sale error data points.
Usually, strong AVMs would have lower MAPEs of 10% or below, whereas MAPEs between 10 to 15% might be considered good for certain scenarios. Another metric used for gauging the precision is the Forecast Standard Deviation (FSD).
3. Higher Coverage
Hit rate is a metric that is used to determine the percentage of properties that were able to be estimated. For example, a 98% hit rate might mean that 98 out of 100 properties have an estimate.
Sometimes, different vendors might classify the hit rate in different ways. For example, vendors might classify an AVM valuation as a valid hit only if it is above a certain threshold of confidence score or below a certain threshold of FSD.
It can be difficult to attain higher coverage and higher accuracy/precision at the same time. The larger the set of data looked at in the hit rate, the more possibility there is for the margin of error to increase.
Also, since trends can differ across geographical regions, states, counties — even down to the neighborhood market level — AVMs might estimate most of the properties with a reasonably good accuracy and precision measure but might fall short of some properties that did not fit the trends observed from the models.
If these properties are excluded from the estimation, the hit rate decreases and the accuracy improves, but if these properties are included in the pool of estimation, then the hit rate increases but the accuracy decreases.
For example, an AVM’s performance might decrease in rural areas where properties are scattered across the county, compared to suburbs where there is a lot more sale activity.
4. Data Quality
One of the most important elements of a good AVM is the underlying data it uses. The machine learning or mathematical/statistical models are entirely dependent upon the quality and accessibility of the data.
Because of that, it is important to have good data sources for things like property characteristics and features (such as bathrooms, bedrooms, fireplaces, etc.), tax assessment records (like land value), neighborhood data (school scores, walkability), price trends, and market trends.
This is why many AVM providers focus on improving the quality and reliability of their data to improve the results of their valuations.
5. Quality of methods and models
Every AVM vendor might have a different set of mathematical and statistical methods [they use in the form] of machine learning models. It is rare to see an AVM provider use only one model to determine a property’s estimated real estate market value.
The most common trend observed is that a variety of models are used to determine the value of a property. Some vendors might use a suite of neural network models, which mimic the brain’s behavior. Other vendors might use a suite of linear regression models, which statistically predict the relationship between two variables, to estimate the price of the property.
An important element of a good AVM isn’t just the type of model used, but also whether the most appropriate models are deployed for different scenarios across various property types, geographical regions, and more.
Automated valuation models are not an exact science, given that they are all based on slightly different models, data sources, property information, and algorithms to calculate property valuations. But even so, AVMs can be particularly useful for investors when it comes to time savings, more informed decision making, and augmenting the traditional appraisal approach.
Want to try out an automated valuation model today? Visit Xome.com and search our properties for auction and for sale.
