Nine Principles for Designing Great Data Products

From data collection to analysis, to the usage and communication of results, using data to deliver quality customer experiences takes careful consideration. Here are nine principles for designing engaging data products that your users will love.

1. Collect Data Passively

Collecting user data should never interfere with the quality of the user’s experience. While privacy is certainly an important issue, consumers have growing expectations for how their shared data can be used in exchange for better experiences. This is especially true for millennials—80 percent say they have some or a lot of trust in the companies they do business with to keep their personal information secure. Smartphones present an enormous opportunity for passive data collection: accelerometer data, GPS data and app usage data can all be used to learn about your users and to provide better user experiences. Google Maps uses GPS data from its millions of users to provide the fastest routes and help users avoid traffic.

Passive data collection also unlocks hidden business value. For example, most companies collect clickstream data, but few go beyond the use of standard analytical solutions. At frog, we helped a Fortune 100 client assess one year’s worth of clickstream data from its web application to better understand their users’ behaviors. What we found were ways to redesign the interface to help more customers complete transactions digitally, rather than contact the call center. To do this, we generated User Interaction Flow Diagrams, which visualized the flow of users through the experience and allowed us to find pain points. Then, the data was used to generate user behavior models able to estimate the impact a UI change would have on the overall digital completion rate. The result is a better experience for customers—and call center cost savings of over $600k per year.

2. Don’t Exhaust the User

Until a user interacts with your product, no data exists and personalization is impossible. Active data collection techniques can help your product overcome the cold start problem, but they have to be a natural part of the experience. Successful data products get around this by giving users an easy and engaging onboarding experience capable of collecting the necessary data without being overly burdensome. Apple Music asks new users to Tell us what you’re into and presents a few bubbles containing genres to select. Stitch Fix guides users through a questionnaire that helps ensure their first Fix contains items they’ll love. Netflix asks new users to select three movies they like at sign up; they handle the rest over time.

Onboarding surveys aren’t the only solution. Frog worked with a Fortune 500 financial client to design a web application for helping customers find the perfect car. We built a sophisticated recommender system that utilized the attributes of every car on the market, as well as the preferences of users that interacted with the application. However, the application was unable to provide recommendations to new users. We could have designed an onboarding survey to get around this problem, but it would have been a tedious experience. Instead, we used existing survey data to assign attributes such as Sporty, Family Friendly, Luxury or Unique to each automobile. Then, we presented users with an interface that allowed them to toggle between these attributes and view recommended vehicles with a single click. The aspirational nature of the attributes kept users engaged—and we were easily able to collect large amounts of user data that could be fed into the recommender system.

3. Constantly Validate with Data

Launching your data product is only the beginning. Once users start to engage, it’s important that you are continually validating your data product by tracking important, quantifiable metrics. The world is always changing, and a model that works well today will not work well forever. Additionally, tracking important metrics gives you the ability to perform experiments, or A/B tests, that can help you improve the performance of your data product. Airbnb is constantly running hypothesis driven experiments by iterating on the user experience and product offerings. This includes anything from changes to the appearance of the website to optimizations for their smart pricing algorithm. By leveraging an internal tool used to perform A/B testing, Airbnb can measure the impact changes have on important metrics like click-through-rate or the number of bookings.

While collecting user feedback to improve the user experience is important, the best data products can instantly and automatically incorporate feedback in to the overall experience. For the vehicle recommender application, frog added a button to recommended vehicles that allowed users to place the vehicle in their Garage. This let users view all their favorite vehicles on a single page, but also provided an excellent mechanism for us to collect feedback. This feedback was stored in our database, which we used to calculate the recommendations in real time. By storing user feedback in the same database that was used to make recommendations, the performance of our vehicle recommendation system improved as the number of users grew.

4. Give Users Control

An overly eager machine learning system that makes too many decisions, however accurate, will leave users feeling bewildered and frustrated. Yet, striking that perfect balance between anticipating needs and giving users the right amount of control can be challenging. Designers at Nest Labs learned this principle through experience: Making users fight against temperature schedules they did not select or want caused not only irritation and discomfort, but also thermostat usage that resulted in higher energy usage than before. By nature, people don’t like being told what to do. For the Nest Thermostat, letting users feel in control led to a better experience and to increased energy efficiency. Their initial Auto-Scheduler algorithm was optimized to reduce energy costs, but because they failed to take the end user experience in to account, this algorithm led to higher energy usage. The Nest designers listened to their users and updated the Auto-Schedule algorithm to ensure comfort and respect user inputs.

In a frogVentures collaboration with Heatworks to bring the MODEL 3 connected tankless water heater and app to market, one of the primary goals was energy conservation. Part of this conservation is achieved through increased heating efficiency. However, the majority comes through encouraging users to use less hot water. The straightforward solution to increased energy efficiency would be to put strict limits on the amount of hot water a household could use in a day, but that would lead to frustration and attrition. Instead, the data collected by the MODEL 3 provides the user with historical savings, goals and recommendations that encourages the user to take control of their own water conservation.

5. Meet Unexpressed Needs

Collecting user behavior data, passively or actively, is only part of creating great data products. It’s also crucial to understand how to use that data to anticipate the needs of users and respond accordingly. Tracking clickstream data, purchase data and any other user behavior data gives us the opportunity to create models of customer behavior that can be used to predict future behaviors. It also helps segment users into groups to develop personalized recommendations. Predictive texting on your iPhone, Netflix’s personalized recommendations or Mint’s budgeting advice all rely on massive amounts of user behavior data to provide you with timely information that meets your needs. Despite the wide variety of uses, these predictive applications all rely on the same approach: finding correlations in historical user data that can be used to predict unmet needs.

Connected vehicles represent a new opportunity to collect massive amounts of user behavior data that can be utilized to anticipate the wants and needs of the user. Working with a Fortune 500 insurance client, frog designed and built a mobile application that collected driving behavior data from GPS and accelerometer smartphone data, as well as an Automatic ODB port reader. The mobile app used this data to offer timely, location-relevant promotional deals and financial advice, as well as encourage safe driving behavior.

6. Invoke Discovery and Delight

Recommender systems are one of the most common types of data products. Providing your users with high quality recommendations keeps them engaged by providing personalized content and product recommendations. But what is a quality recommendation? Simply providing the most relevant recommendations can lead to obvious or boring results. To truly capture the attention of users, we need recommendations that invoke discovery and delight—serendipitous content users will enjoy but wouldn’t have thought of on their own.

Out-of-the-box recommender system solutions provide a passable experience, but it will take a truly bespoke solution to create a sense of discovery for your users. At frog, we developed a web application to help users find the perfect college. Behind the interface is a hybrid recommender system that combines content recommender (in this case, a list of schools) and collaborative recommender (schools similar to ones you like) systems. A hybrid approach offers relevant and serendipitous results, and leads to a richer customer experience.

7. Build Trust with Transparency

Even if your data product is working properly, users will be skeptical to engage with your product if they don’t have any understanding of how a decision was made. Providing transparency into the inner workings of your data product can help earn the trust of users. For example, Spotify will make recommendations with the tagline Because you listened to… By providing this information, users will have a better understanding of what to expect and can make a better-informed decision about what to listen to next. Many machine learning algorithms generate a probability or confidence score in addition to a prediction. Sharing these confidence scores with users can help them make informed decisions. This is commonly used in weather forecasting, where users are given a percent chance of rain instead of a binary result of rain or no-rain.

Transparency is especially important in areas such as healthcare and finance, where decisions can have major consequences. Working with one of the largest banks in Mexico, frog created a dashboard to be used by bank employees assisting customers. This bank served primarily low-income customers, who tended to complete transactions in person. With this insight, we created a dashboard for bank employees that displayed relevant customer information. It then presented a set of recommended actions and financially responsible guidance personalized to the customer that a bank employee could then offer. For each recommendation, the system displayed the rationale, citing relevant information such as recent life events or payment history. Including this extra layer of transparency allowed bank employees to have confidence in their recommendations.

8. Visualize the Complex

Making data easy to interpret is essential when designing great data products. Wayfinding apps help commuters easily avoid thick red lines of heavy traffic. Fitness tracker apps show simple charts and trend lines that can be understood at a glance. News sites like FiveThirtyEight use data visualization to help readers understand complex stories and concepts. Data visualization is everywhere, yet visualizing data without becoming overly complicated or busy is a difficult balance to strike. Making careful use of location, shape, color, size, weight, motion and other means of visually encoding information draws attention to important information.

Data visualization becomes increasingly challenging as the size and complexity of the data to be visualized grows. This challenge is especially common in IoT applications, where massive amounts of streaming data need to be visualized in real time. At frog, we worked with a leader in the oil and gas industry that was collecting sensor data from the instrumentation in their drilling equipment to assess the integrity of their wells. By employing a user-centered design approach, frog created a data visualization dashboard that allowed both technical and non-technical employees to make decisions using this complex data.

9. Blend In

Sci-fi movies portray a future where machine learning and AI exists among us as robots, intelligent chat bots and fully autonomous vehicles. While many articles about machine learning focus on these far-fetched realities, the truth is that machine learning is already in our lives today in much more subtle ways. Often, users are unaware of the sophisticated machine learning that powers their favorite products. They simply notice the improvements these algorithms inform. The lesson here is that the best data products will be ones that work with the way people live today by integrating with the products they already use.

At frog, user experience is always top of mind, and we perform extensive research to ensure the products we design fit in to the users’ lives. When designing data products, it’s tempting to create futuristic products that force users to change their behavior. While it’s important to push the boundaries, the best way to keep users engaged is to create intelligent, responsible data products that fit seamlessly in to customers’ lives.