Experimentation & Third-Party Data Integrations
Overview
We’ll often get asked to supplement experimentation in platforms like Optimizely or Convert (among others) with third-party data from services like 6sense, Clearbit, or Demandbase. Pulling in this data is typically very doable, but frequently comes with some tradeoffs and caveats which we’ll try to explain here.
Timing
This third-party data is typically loaded into the web browser asynchronously via an additional “fetch” api network call. The async nature of this data creates a delay between the base web page loading and this third-party data eventually becoming available. This delay can lead to noticeable “flicker” or “flash of unstyled content” issues.
For example, say you want to change the hero headline copy from “Welcome!” to instead say “Welcome Cro Metrics!” and you intend to swap out the name of the company based on this third party data. What you’ll find is that since the third-party data enters the browser after a short delay the headline copy will suddenly update once the data becomes available, creating a jarring “flicker” between the before and after.
One caveat is that the data is often cached within the browser’s Local Storage after it has initially been requested. This means that if the visitor navigates to another page (or refreshes the current page) that cached data is available immediately with no delay whatsoever since it doesn’t need to wait for the network call to come back, and since there is no network delay anything relying on the data will be displayed without any flicker.
This can lead to confusion because unless you’re clearing your browser’s cache (or opening a new incognito window) between every page load you won’t be seeing the flicker issues. This can obscure the problem until you launch the experiment out to real visitors and their first page load consistently sees this flicker issue that you didn’t realize would happen.
Potential Solutions
Design the experiment display to be asynchronous friendly: Use a modal, butterbar, some sort of delayed animation, etc. Essentially build that async delay into the treatment so that it doesn’t appear jarring or look like a mistake.
Disable the treatment on the first page load, and only show the experience on subsequent page loads. This is especially unfortunate when attempting to target the homepage since it’s often the first page the visitor sees.
Hide the entire page until this third-party data becomes available, make the experiment changes, then display the entire page at once. This way the visitor doesn’t see it suddenly shift. This has the drawback of delaying the visitor experience until this async data shows, although it’s generally only a few milliseconds.
Other solutions may include getting rid of the asynchronous nature of this data. Essentially that would mean either hard-coding the needed data into the webpage via your server-side code, or using a third-party data integration that supports loading the data synchronously (typically via a script tag).
