consolidation magic?

One of the key assumptions made in the white paper and presented without support from any actual sampling data (guess we don't need no stinkin science!)is that, compared to regional MLSs and single MLSs, consolidation of MLS data at the statewide level produces 1) more accurate data, 2) more timely data, 3) cheaper data and 4) more comprehensive data.

I'm going to go down this list and ask some questions about what attributes might factor into measuring the characteristics of each item that might demonstrate the supposed superiority that consolidation produces at the statewide level.

I'll also consider some means of testing that set of hypotheses (which I assume hasn't been done or a "white paper" would certainly provide the statistics of the sampling).

1. Accuracy of Data
What is accuracy? That may seem too rudimentary for a starting point , but if we're interested in a metric for measuring a component of MLS quality, we should understand the metric. More importantly can the metric be measured in a precise way. If we're pimping for a political agenda that doesn't matter much. We can just assume THE TRUTH. I'm not pimping (at present anyhow), so I'll refer to Wikipedia for a refresher definition.

Ok, without launching into a discussion of accuracy vs precision etc. etc., lets agree that accurate MLS data approaches the TRUTH. In this case the information in the MLS system should be very similar to the TRUE values associated with the house (or lot or whatever) and listing agreement (price, commission, term of listing).

The problem with MLS data is how do you know what the TRUE values are? If you don't know, how can you determine the accuracy of the data? The MLS staff isn't going to run out with a clipboard and review every house. Agents familiar with the house might know what the truth is (but if it's not their listing, they may not care much--they know the truth already). As far as listing agreements--MLS staff can audit the agreements, but that nearly never happens--so in most cases there's no real checking of house data or listing data.

So, how does a statewide MLS "know" it's data is more accurate than the data present in component MLS containers pre-consolidation? What changes magically with consolidation? Does the statewide MLS have an omniscient algorithm that sniffs out inaccurate data and flags it. That is a cool system, if that's what they really have in Connecticut. You'd think the Federal Government would want a piece of that, huh? They could certainly use it!

Ok, so measurement of accuracy is going to be a significant hurdle. We'll come back to that during discussion-- after we cover more of the complexity behind the contention that statewides are more accurate in their data.

IF we could indeed measure accuracy against the TRUTH, what factors might influences accuracy and are those factors likely to exhibit differentials between statewide MLS and those MLS databases covering smaller areas?

Agents input much of the MLS data but in many MLS systems certain fields are pulled from public records. The public records part is as accurate as the public records--usually pretty good for recently reviewed parcels and for permitted or assessed improvements.

So, public records data is easy--it's accurately pulled across into the MLS, but it's pulled across in the same manner whether it's dumped into a statewide container or the container of a regional or a small MLS. No advantage to a statewide there.

The motivation of agents to do a good job of inputting accurate data is another major factor. That involves several sources.

Agents have varying degrees of personal pride in their professional skills and responsibility to act in the best interests of the Sellers they represent as well as the other agents they interact with professionally. Hard to rationalize how inaccurate data is a benefit to a Seller or other agents--at least in the long run.
Are agents likely to do a better job when entering data into a statewide system?
The data may be seen by more people, but listing information is already up on the Internet for the whole world to see. Exposure is not a likely factor.

There is also office culture/protocol/office handbook directives etc. The authority of brokers/managers over agents is all over the place, but I haven't talked to many old timers who think it's as strong as it was 20+ years ago. Back in the day, the broker was THE AUTHORITY. Nowadays, not so much. The rise of state/national associations as a standardized source of education/information probably plays a role in that trend. When I started in the business, the broker or manager or old timers were THE place to get much real estate information. There was First Tuesday as well--we had an extensive collection of back issues. Back then if your broker told you to enter accurate data--you did. If you didn't, the understanding was you might find yourself looking for another office. Sounds tough, but the listings BELONG to the broker. Before computers, Internet and profuse data leakage brokers felt that ownership with intensity.

Some larger offices have a clerical enter the data, but there's a worksheet somewhere that the agent fills out and one would hope the clerical is skilled at data entry. No reason to think there's a differential between statewide and some smaller database in that regard.

Then there are MLS rules and sanctions. Most rules include a statutory fine for confirmed data errors. The fines vary. In regionals the fines are imposed according to the MLS rules of the association the agent committing the infraction belongs to. A deterrent? Somewhat, depending on the fine and the agents concern about having a record in their file. The other factor is that someone who discovers an inaccuracy must file a complaint. Our system allows that to be done right online-click a "report violation" link and fill out the report, then submit. However, the "glass houses" effect kicks in and keeps much of the bad data from being reported. Are the fines higher in a statewide? That might actually decrease reporting of errors. Agents who think the fine is grossly excessive would probably think twice about filing a report. Within a statewide framework, who administers the fines, where do the fine monies go, etc. etc. BUT does any of that indicate a statewide will necessarily compel agents to participate in achieving more accurate data? Nope.

OK, there's the lay of the land for data accuracy. How is statewide data likely to be different (more accurate) than data in a regional or a local MLS? How do we prove that hypothesis? Take random sample of listings and compare the accuracy of their data field entries to the TRUE values as verified by objective observers checking the actual properties--sorta ground truthing. Has anyone actually done that? Not that I know of. It would take a lot of effort and it's possible no one really wants to know.

Here's another angle. There are some self checking features in some MLS vendor applications, but they don't check every field and they don't know the TRUTH. If a statewide application had more of those self checking features you could possibly say that for THOSE fields where self checking was applicable, the accuracy of statewide data was superior to that of a regional or local MLS that had a lesser number of self checking features--but only for that subset of fields. However, that's not the hypothesis we're trying to test. It would depend on the vendor and that a different issue entirely.

Statewide databases are inherently challenged by compromise. In fact all MLS databases are a compromise. They can be very large to accommodate MORE information that agents and buyers might have an interest in. In reality, most of the data field are rarely used. Most searches use 4-5 fields. The database may have several hundred. Some are necessary to identify agents, offices, contact info, etc., but setting those aside, there are still many field never used and most of those aren't required. That raises an argument regarding attention span. At what point does data accuracy tail off because agents loose enthusiasm during the lengthy process of completing the fields?

Statewide databases might tend to have more fields than regional or individual MLSs. Regionals and individual MLS are more organic--i.e the containers include data about a geographic area with a degree of commonality. How would one data set represent the diverse properties of a very heterogeneous state like CA. Mountains, coasts, tracts, resort areas, etc. In a very general way--one size fits all--just not very well. That effect could decrease accuracy in a statewide MLS. Fewer of the data fields would be relevant to a particular listing. The MLS application could flag fields depending on access ID or property zip, but that could be a problem too.

Here's an interesting "what if". What if an agent is confronted with entering data for a listing into a hypothetical MLS system in which ALL the agents using the system, and public viewing the system have intimate knowledge of ALL the houses, including the one in question. Under those circumstances, we might expect the agent to feel a need to carefully enter accurate data-- knowing that any error would be quickly detected. That's taking things in the opposite direction from statewide superiority, but is more in line with the very early MLS systems.

It's similar to giving a seminar to an audience who's not very knowledgeable on the subject, as compared to appearing before acknowledged experts in the field. In which case are you going to be at your very best?

In summary, I see no verifiable reason statewide MLS data should be any more accurate than data from smaller MLS entities. If Connecticut did before and after sampling and analysis, I'd love to see the data and results. At best, accuracy in MLS data isn't easy to measure in the first place. To complicate matters, MLS data accuracy is largely determined by factors outside the direct control of the MLS, be it statewide, regional or individual.

If statewide MLS consolidation has an upside, accuracy of data is not likely a major contributor.

Next up: Timely Data? Does time become warped in proximity to a really large MLS?