ARTICLES
The Asset Record Download: Google invests $50 million in Auction.com The Asset Record Weekly Download. February 24, 2014 The Asset Record Weekly Download. February 10, 2014 The Asset Record Weekly Download. February 3, 2014 The Asset Record Weekly Download. January 27, 2014 The Asset Record Weekly Download. January 10, 2014 The Asset Record Weekly Download. January 3, 2014 How to tame Big Data in Real Estate The Asset Record Weekly Download. December 20, 2013 The Asset Record Weekly Download. December 13, 2013 The Asset Record Weekly Download. December 6, 2013 The Asset Record Weekly Download. November 29, 2013 Signal vs. Noise: Seeking Credibility in Real Estate In The News: Minnesota Business Magazine- December 2013 The Asset Record Weekly Download. November 22, 2013 Article Archive

How to tame Big Data in Real Estate

By Jason Baker, Chief Technology Officer at Asset Record Company

Let’s say there are roughly 140 million properties in the United States. Roughly 20% of those properties are commercial. Now, let’s assume that each property has 100 documents worth of information associated with it. A document could be something like a title, mortgage, inspection report, or appraisal. Some properties could have dozens of documents and others may have hundreds or even thousands of documents. How would you design a software application that allows thousands of people across the country to simultaneously access a data store containing potentially billions of documents? That’s the key challenge facing our real estate technology team at Asset Record.

Attempting to build a massively scalable real estate platform would have been out-of-reach for most technology startup companies in the past. Today, we can leverage new technology architectures and software deployment models enabled by cloud computing to realize our vision. It’s almost as if the planets are aligned in the tech world as inexpensive cloud infrastructure, next-generation software application platforms, and mobile computing technologies converge. We use all of these converging technologies at Asset Record to deliver a new type of data collaboration platform to the real estate marketplace.

Building a large mission-critical application used by thousands of real estate professionals is very different than building an application that runs on a laptop or corporate server. The most important thing to understand is the difference between designing an application that scales vertically and an application that scales horizontally.

We’re all familiar with applications that scale vertically. If you have a really slow application running on your laptop what do you do? You might add more memory to the laptop or if the laptop is really old you might decide to buy a new one. The application you are running on your laptop is designed to scale vertically. In other words, to increase the performance of your application you need to buy a faster laptop. We do the same thing with enterprise computing. If our enterprise applications are running slowly we simply go out and buy bigger servers.

The problem with applications that scale vertically is that at some point you can’t buy a laptop or server fast enough to support the user demand for those applications. Google Search can’t run on a single computer because a billion people use it every day. The same is true for a real estate application storing millions of property documents.

So the solution to building a large-scale real estate application is to scale the application horizontally. What does this mean? It means that we design the application to run on multiple computers in parallel with each computer supporting a certain number of users. For example, instead of just running the application on one computer that supports 100 simultaneous users, we run the application on 10 computers to support 1,000 users. Supporting even higher numbers of users simply requires more computing power. Instead of using bigger computers we just use more of them.

Building an application that supports horizontal scaling is easier said than done. However, it’s much easier to do today, using modern web application technologies, than it was a decade ago. We used a very modern application framework at Asset Record built on Node.js and Redis to support horizontal application scaling. We deployed our application on the Platform-as-a-Service, Heroku cloud in order to easily manage the computing infrastructure powering our application. We can literally scale our application to support hundreds or thousands of simultaneous users in minutes.

Okay, building an application that scales horizontally in the cloud is definitely doable today. What about building a database that supports potentially hundreds of millions of documents? Databases have the same scalability challenges that applications do. If you look at any enterprise IT server farm typically the biggest servers are reserved for databases. That’s because traditional databases are designed to scale vertically. Need to store more data or support more users? No problem. Just buy a bigger database server.

Scaling database servers vertically only goes so far when you are talking about a large Internet-based software platform supporting thousands of simultaneous users. Once again, modern database technology comes to the rescue. A new generation of database platforms is making it possible to build real estate platforms like Asset Record.

We used a next-generation database called MongoDB and Amazon’s S3 storage to store the data in our Asset Record platform. MongoDB is a document database rather than a traditional relational database. Instead of storing data in tables and rows, data is stored in “schema-less” documents. MongoDB makes it really easy to partition, or what we call shard, the database information. Instead of storing all of your data on one big beefy database server, you store separate pieces of your data on individual database servers. The individual database servers function together as a group in order to respond to your database requests. All you need to do is add more database servers as your storage needs and user demands grow.

Building a real estate application like Asset record was once a daunting task – accessible to only the largest and most technically sophisticated organizations. While it still requires specific expertise in cloud application architecture and modern web application technologies to develop this type of platform, the barriers to entry are much lower today. The benefit to our customers is that no matter how many properties they work with or how many documents they store on our platform, we can handle it.


Do you have something awesome to say and need a platform to say it? Asset Record is looking for a few good contributors. We want to showcase movers, shakers, leaders and innovators of the real estate industry. Whether you’re an owner, manager, appraiser, lender, broker, investor… we want to hear from you.

We’re all about the collaboration so contact us at info@assetrecord.com and let us know.

Take A Test Drive

TAKE A TEST DRIVE
  • Convenient dashboard for your real estate projects.
  • Store, share and collaborate with your documents and information.
  • Connect with owners and professionals for faster deals.