Decreased errors by 43% and increased success rates by 2x with content design changes

Summary

Revised information hierarchy and UI content to decrease 429 errors by 43% and increase validation success rates by 2x.

Background

We released a beta version of a feature for real estate agents. To access the feature, our users have to enter credentials in a validation modal. This modal is critical to feature adoption and we weren't hitting our goals.

During stand up one day, one of our lead engineers mentioned that he was concerned about the volume of failed validation attempts.

To try and diagnose the problem, I jumped into my test account. I came up with a list of issues I saw in relation to this modal:

• Inconsistent and incorrect labels and terminology
• Multiple modal versions
• The help link was too far from the relevant field
• 500 error copy appeared for 429 errors
• No inline error messages or feedback
• The numbers-only ID field accepted random symbols and letters

Next, I went to the lead engineer who reported the problem to get more data about the error volume for this modal. I put together a plan and pitched the changes to our product manager.

She said yes to the changes, the lead engineer said the scope was easy to manage, then we got to work.

Deliverables

First thing’s first: there were three versions of the modal and we consolidated them down to one version.

• The header now focuses on the purpose of the modal and matched each of the three CTA/entry points for this modal
• Terminology updates:
• We now use correct name for the feature (Forms → Form Libraries)
• We now use the term “form source” to prepare for the release that scales to include more form sources — real estate associations across the country manage their own form collections and access to those collections
• We introduce an acronym per our content guidelines (NAR membership → National Association of REALTORS©)
• I confirmed and corrected all NAR terminology by logging into a NAR account
• The description leads with instructions that briefly describe the value of taking this action
• The description also clarifies that you can only get access to forms from associations that we’re working with, to help manage expectations
• Added inline error messaging
• Relocated instructions related to finding the NRDS ID so that they’re right next to the relevant field
• Added a description beneath the “Associations” header that clarifies to folks that they need to be an active member of the association for this to work

• Cleaned up all of the toast error messages for clarity
• “Unable to process the request right now” is copy that relates to 500 error messages — aka server issues that would be our fault
• The copy for 500 errors were also appearing for 429 error messages — aka rate limit issues where the user has tried to authenticate too many times and now they’re in time out
• The 429 errors were the most common for this modal and we were miscommunicating the problem in a way that made it seem like eSignature was broken
• There are more error messages that aren’t featured here, mostly because I don’t have the “before” copy for those ones

Results

These changes alone — particularly the changes we made to error handling — had a measurable impact on product adoption:

• We reduced our highest-volume 429 errors by 43% 🎉 
• We increased the validation success rate 2x — aka we doubled feature adoption because this modal was the only way to activate the new feature

Portfolio

Check out another project.