April 21st, 2011
By Jonathan Bednarsh, President & COO
Part I: The Boiling Point
I am absolutely awed by how rapidly the lid has blown off the data distribution pot in the last month or so. We’ve all heard the gripes, frustrations, and tussling that’s gone within our industry during the last few years regarding listings data distribution (which I’ll use as a generic term to cover IDX, VOW, Syndication, Indexing, etc). But just this week alone we’ve seen:
Today our industry is at a crossroads and we’ve got to use this opportunity to find a better way. The current data distribution models are colliding and creating unneeded friction without shared benefit. To be clear, I strongly believe these problems can be solved – we’re not talking rocket science, folks. What’s required is taking a fresh look at established norms in our industry (which most agree are unsustainable) and leveraging today’s widely-accepted technology approaches to drive a benefits-driven change for the better.
Here at Onboard Informatics we’ve been working on just such a solution for several years and are ready to introduce our approach to the marketplace. Over the next week or two I’d like to tell you more about our approach to the Data Distribution Dilemma. We’ll cap that discussion with a very exciting announcement I promise you won’t want to miss.
Exploring Challenges
I’d like to begin by commenting on the excellent white paper by Clareity Consulting published this past Monday 4/18, Syndication to Real Estate Portals: Problems and Solutions Whitepaper. If you haven’t read it yet, open the link above in a new browser window and read it now. They have been a leading voice on this topic for some time now, most recently fostering some rather spirited debate at their MLS Workshop back in March. In their latest paper, the Clareity team has done an outstanding job of exposing many challenges with today’s third-party syndication model and they pull no punches. They have also proposed some insightful paths towards possible solutions, including the Syndication Bill of Rights. The paper also addresses some “alternative approaches”, most significantly two related options dubbed “Transient Download” and “Partial Download”. To summarize their conclusion on these alternative approaches, Clareity doesn’t really see either one of these approaches working as a total solution for a variety of reasons.
While I agree that there are challenges to overcome, here at Onboard we look at this a little bit differently. In fact, the concept of real-time data access and tracking is at the heart of our solution. I think the easiest way for me to begin describing our viewpoint is simply to address the points raised in Clareity’s whitepaper directly. I’m going to excerpt portions of their paper below and address each point one by one.
Let’s begin with their concerns regarding “Transient Download”:
“This [Transient Download] does not provide a substantial advantage over markets where publishers have a relationship with the MLS and receive updates via RETS every 15 minutes to an hour. Because there is no way to preclude brokers/franchises from sending their listings direct to the publisher for storage in their database the current issues with duplicates and data accuracy among duplicates are not solved.”
OBI: Many publishers get listings via brokers and/or syndication networks that do not provide updates more frequently than once per day, even if the local MLS does make greater update frequency available via RETS. Typically this is a result of the cost and complexity of managing frequently updated data. As Clareity reveals in its paper, the business models of most publishers really don’t benefit from highly updated data despite the impact on consumers. There is plenty of room for improvement on data currency that can be addressed by moving beyond the current data distribution models.
The paper does make a good point with regard to the problem of “duplicates”. If the MLSs and brokers are not cooperating, Publishers still have to deal with the issues of duplicate data. However if there is cooperation…(more on that below)
“If listings are not stored in the publisher’s database it is unclear how enhanced listings would be made to work.”
OBI: There are a number of best practice approaches that enable users of a remote database to append, enhance, or otherwise segment individual records or groups of records. In fact several of Onboard’s current APIs for public records and listings data support this kind of functionality today.
“There are doubts that this model could provide the system search and display performance and reliability required by publishers, and now expected by consumers.”
OBI: Most non-techie types would be amazed at how much of today’s web experience is built on a foundation of far-flung databases accessed in real time via Application Programming Interfaces (API’s): everything from critical financial data to by-the-minute news updates; from popular real estate applications from companies like Trulia & Zillow to the Microsoft & Google maps on most of your websites. All rely on real-time data access via APIs. There are literally thousands of high-volume, high availability success stories – many of which you interact with every single day. A properly constructed API platform built on fast, scalable architecture and infrastructure can provide the search and performance capabilities demanded by today’s consumers. You can read some more about the positive impact of APIs in our industry here in a RISMedia article written by Onboard’s VP of Product Development Scott Petronis.
“Innovation on website and mobile apps would be limited by the capabilities offered by the source database, effectively reducing diversity and options for brokers.”
OBI: My respect for the Clareity folks notwithstanding, I have a very different point of view on this. Take a close look at today’s real estate companies and publishers: most of these companies are not technology firms. They sell real estate, advertising, etc. The ability of their technology teams (or vendors) to innovate tends to be dramatically limited by resources and other core priorities; not the least of which is listings feed management. Reducing or eliminating the need to manage complex and frequently changing MLS databases (a problem that grows exponentially as you aggregate more MLSs) by leveraging a robust and flexible search API platform will allow these firms to focus their resources on innovation, rather than detracting from it. Further, API platforms enable much easier cross-platform development and faster (lower cost) development cycles. Migration to an API-driven model will be a boon for creativity and application diversity.
Moving on to “Partial Download”:
“Again, unless brokers and franchises stopped sending duplicates, the way they do today, this doesn’t solve the problems.”
OBI: I’ll address MLS/Broker cooperation below.
“And, if publishers are required to link to listing detail pages on other websites, the user experience will surely suffer. Again, issues surrounding performance and flexibility would be significant.”
OBI: I couldn’t agree more with Clareity on this point. Any proposed solution that requires a consumer to be bounced around from website to website is a huge step backwards and will not work. Look at the hoops franchisors have to go through to offer IDX search on their websites as a perfect example (though indexing has helped improve things a bit).
“Clareity believes the chances of MLSs being willing or able to mandate that brokers or entire franchises not send their own listings to the publishers is negligible, and if there is but one large broker that does not let the MLS fully control syndication, being provided an advantage by the publisher for doing so, then an attempt by the MLS to fully control syndication will be fruitless.
Both of these approaches and variations being discussed amount to MLSs pulling back from syndication. Brokers and franchises that had the resources to send listings to publishers directly and efficiently would be advantaged in the immediate term, and other services would, no doubt, spring up to serve broker and agent syndication needs. Syndication would become more fragmented and problematic to manage than ever, and MLSs could entirely lose syndication relevancy and control. [emphasis added]”
OBI: Clareity’s conclusion on this topic is premised on one very important dynamic – that MLSs and Brokers are inevitably on opposite sides of the data distribution dilemma and their interests are in conflict. To be fair, that premise is founded on the many years of experience Gregg, Matt and team have spent working with the industry to address many of its problems, including this one. However I firmly believe that there is some common ground between MLSs & Brokers that has yet to be explored.
As Clareity points out, there is little to no chance that MLSs can “pull back” the reins on syndication; nor are they in a position to dictate where and how brokers distribute their own listings data. However, MLSs are in a terrific position to implement data distribution systems that provide tangible benefits to their members: data accuracy, unprecedented usage visibility and business intelligence. These business benefits inherently encourage cooperation and partnership.
A Path Forward
- What if Brokers & Franchisors (even Agents) derived material benefits from allowing MLSs to take a lead role in distributing data on their behalf to Publishers and Syndication Networks?
- What if Publishers gained greater cooperation from MLSs and access to better, more frequently updated data while reducing the headaches of duplicate data management?
- What if Consumers were able to rely on listings data without having to deal with outdated and inaccurate information?
Onboard has developed an approach that we believe can provide these benefits. While not a silver bullet or magical cure, our solution addresses many of the problems highlighted by Clareity and others. In fact, key elements of our approach have been in production successfully servicing thousands of real estate websites for more than two years now.
Stay tuned to this channel for more details about Onboard’s solution to the Data Distribution Dilemma over the next week or so.
Tags: data distribution, data distribution dilemma, listings distribution, listings syndication, MLS listings issues.
