Real Estate Data Scraping on Real Devices
Real estate data scraping across country-specific property portals, where listings vanish once sold and prices are personalised by geography.
What makes real estate data scraping different
A property listing is a perishable record. It appears on a portal, moves price once or twice, goes under offer, and is taken down. Nothing archives it publicly. That is what separates real estate data collection from most other web data work: a collection window you miss is not delayed data, it is data that no longer exists.
Price history is the asset. Days on market, reduction cadence, withdrawal and relisting are all derived from repeated observation, never from a single fetch. A collector blocked for a week leaves a hole in the series that cannot be backfilled at any price.
Portals are national, and so is the blocking
There is no global property portal. Rightmove and Zoopla in the UK, ImmoScout24 in Germany, SeLoger in France, Idealista in Spain and Italy, Funda in the Netherlands, Zillow and Redfin in the US, realestate.com.au in Australia. Each has its own markup, its own pagination, its own anti-bot vendor, and its own idea of what a request arriving from outside the country means.
Two things follow. Coverage is a per-country engineering problem rather than one scraper with a locale switch. And the traffic has to originate in the market it is reading, because portals personalise by geography: currency, floor-area units, which agency contact is shown, and sometimes which listings are shown at all.
Archonum runs every session on a real consumer smartphone on a local carrier network, across a fleet of 250,000+ devices in 175+ countries. The page a device in Madrid renders is the page a buyer in Madrid renders. There is no proxy split between the browser and the exit IP, so there is no mismatch for a bot detector to act on. The difference between that and a proxy pool is visible at the packet level, not just in the address.
The hard part is reconciliation, not fetching
One flat is listed by three agencies, with three photo sets, two floor areas that disagree, and the address written three ways. Portals rarely publish a stable identifier that crosses portal boundaries, so matching falls back on geocoded position, floor-area tolerance, image hashing, and listing-text similarity. Every one of those signals needs the complete page.
Which is why partial collection is worse than no collection. A response with listings withheld, or with the contact block stripped because the session looked automated, does not raise an error. It quietly degrades your match rate, and you find out weeks later in the numbers. Instrument for blocked-but-looks-successful, not only for HTTP failures.
What a listing collection run has to hold
- Cadence over throughput. A daily sweep of the markets you care about is worth more than a quarterly bulk pull of everything.
- Session continuity. Search, filter, paginate, open detail. Losing the session mid-pagination drops listings without telling you.
- A local vantage per market. One exit country cannot represent twelve.
- Provenance on every row. Source portal, country, and an observed-at timestamp, so a disagreement between two records is resolvable later.
Where the line is
Public listing pages only. Archonum is infrastructure for loading pages any member of the public can open in a browser. It is not a route behind an agent login, a broker back office, or a paywalled valuation product, and we do not build one.
MLS data is a licensing question, not a technical one. Access to MLS records is governed by the agreements you hold with the MLS, and no collection tool changes what those agreements permit. If a data set needs a licence, get the licence.
The rest is ordinary discipline: honour robots.txt as specified in RFC 9309, keep request rates to what a human session would plausibly produce, and treat the personal data that sits inside listings, such as agent names and phone numbers, under whatever regime applies where you operate.
What we contribute is narrower than that, and worth stating precisely: the collection layer looks like ordinary consumer traffic because it is ordinary consumer traffic. The property market intelligence use case covers how teams wire it into an existing Playwright or Puppeteer pipeline, and you can talk to us about a specific set of markets.
FAQ
It depends on the jurisdiction, the portal's terms, and what the collected data contains, so there is no single answer and this is not legal advice. Archonum's own boundary is narrower and easier to state: public listing pages that anyone can open in a browser, nothing behind an agent login, and no MLS data beyond what your own licence permits.
Many portals vary what they serve by the visitor's location, including currency, floor-area units, the agency contact shown, and in some cases the inventory itself. A request arriving from the wrong country can therefore return a page that no local buyer would ever see, which is why sessions should originate in the market being measured.
No. MLS records are distributed under licence agreements between the MLS and its participants, and that is a contractual matter rather than a technical one. Archonum loads public listing pages on real devices; it does not grant access to licensed MLS feeds.
Archonum runs browser sessions on a network of 250,000+ real consumer smartphones across 175+ countries, so a session can originate in the market whose listings you are reading. Coverage of a specific portal still depends on your own parsing work, because portal markup differs by country.
Ready to get started?
Run the same script you're running today against Archonum and compare the block rate yourself.