How Important Is Concept Testing? Ask Amazon.

Published on Mar 08, 2021 by Ed Rodgers

Updating mobile icons is a common practice for businesses.  It keeps a brand recognizable on mobile devices and attracts users’ eyes.  Amazon’s recent mobile app icon update did not go as planned, resulting in a quick and quiet switch to a less controversial icon after some negative reactions.  It is unknown if Amazon employed any concept testing before launching the new icon, but a well-designed concept testing program might have saved them from the recent gaffe.  The snafu serves as a good reminder to consider concept testing for any planned brand changes.

Amazon updated their mobile app icon in January 2021.  The previous icon that had been in place for five years showed a shopping cart and the Amazon logo.  The problematic updated icon resembled an Amazon delivery package using a brown background with a jagged piece of blue packaging tape and the Amazon smile logo and presumably designed to resemble their distinctive packaging.   However, many on social media quickly saw a certain Nazi dictator's mustache above a smile, an image that is hard to unsee after it is pointed out.

As a result of negative feedback, Amazon quickly changed the jagged blue stripe to a different shape that no longer looks like a small mustache.  The quick adjustment by Amazon was an appropriate course of action. Still, it did not seem like the result of deft design or high-level communication we’d expect from Amazon and may indicate that the new icon was not properly vetted through concept testing.

Using concept testing before launching the new icon, Amazon might have learned of the potential issue and could have avoided the unintended attention to their update. Amazon could have considered the following concept testing approaches to ensure the updated icon would deliver the desired reaction.

Concept Testing That Provides Useable Insights

There are two important factors to successful concept testing – the who and the how.  Leveraging internal staff's thoughts and opinions and "trusted" people for opinions exclusively would be a mistake.   Including opinions from customers and others who know the brand will ensure greater objectivity.   You also need to ask the right questions of the audience.  Using the correct testing approaches and well-crafted questions will yield insights that give explicit direction about the best option and the data to support the decision. With the proper audience and concept test design, Amazon may not have launched the offensive mustache icon.  

Have More Than Two Concepts Prepared For Testing

Testing of new concepts, whether a mobile icon, a product design, or new messaging, requires iterations of options and features to help arrive at the best attributes and identify potential detractors.  By starting with several concepts with discernable differences, the tests can narrow the attributes and combination of features that give the concept the best chance for success.

It is unknown how Amazon arrived at the choice of the replacement icon. Was the replacement an icon that was not as well-received when socialized, or did Amazon triage and simply adjust the mustache?  Proper concept testing with alternative concepts would have allowed Amazon to pivot to the updated option and have insights associated with that second choice.

Test Domestically, Internationally, And Culturally

Amazon operates in 13 countries where different colors, images, and visuals can garner unique responses from country to country.  When launching in multiple countries, the importance of translation across language, geography, and social norms are critical in concept testing.  Using concept testing in different countries and languages can prevent a product from falling flat on arrival due to poorly written copy or product builds not tailored to the needs of those in a particular country. There are countless examples of products failing in different places around the globe due to narrow concept testing practices, whereby data from the global audience is not collected or ignored.   

In addition to having global insights for a concept launch, obtaining cultural perceptions can also be integral. Conducting studies for a specific audience segment based on cultural, religious, or lifestyle norms can ensure your concept does not unintentionally offend.  In the case of Amazon, the updated icon's association to the Nazi dictator might have surfaced more quickly among Europeans or older people.  Suppose Amazon had learned of the association with the dictator in an early concept test. In that case, they could have run additional tests to understand how strong the association was and the level of distraction from the desired response.  This information could have potentially informed decisions to launch with low concern or move away to a better, less risky option.

Using Text Science To Identify Associations And Perceptions

Using visuals for concept testing to capture feedback may seem straightforward but gaining a deeper understanding of what influenced a choice is equally essential. Learning the customers’ underlying thinking and motivations can uncover the concept attributes that strengthen or weaken willingness to use or engage.

Using open text answers as part of concept testing captures respondents’ opinions and thoughts on the concept in their own words. This type of feedback can be insightful when pairing it with machine learning and algorithmic approaches to uncover deeper insights associated with answers.  Common themes and influencing attributes can be found across all responses, as can the emotional sentiments related to the responses.  These influencing drivers can pull people towards the concept or push them away from using or buying a product. For Amazon, this “in their own words” approach might have uncovered a strong association to a smile and a mustache instead of packaging tape.

Test Early -- And Often

Concept testing is an important step in marketing and product development. Using concept tests for only large investments can be a short-sighted approach and can distance the company from properly understanding the customer.  Concept testing allows your company to know what is important and relevant to customers.  Adopting a well-crafted concept testing program for highly visible and widely distributed changes can save a company from bad press and having to make fast, expensive, and unplanned changes.

Amazon's recent update to the mobile app could and should have been at the very least an unremarkable or satisfactory event. On the other (much more strategic) hand, proper concept testing could have propelled their branding toward optimal new heights – and who doesn’t want that?