Google search is an incredibly advanced product that we're all accustomed to. When visitors use search on your store, they expect to be able to make typos and spelling mistakes and still get relevant results at near instantaneous speed.
Spree Commerce offers built-in search which does a decent job, but with nearly 4,000 products and 12,000 variants it wasn't optimal for our client, Manhattan Wardrobe Supplies (MWS). Our journey to a better search option for MWS led us to try a new service, Algolia.
When I first picked up Algolia, I got excited. It offers many advanced search features right out of the box, including the ability to customize how ranking actually works.
Rather than displaying results based on how many query terms they contain, Algolia’s ranking algorithm rates each matching record on several criteria. You can even assign custom criteria to consider additional factors that are most relevant to your customers (SKUs, variants, options, etc).
The downside of external services
But what about the problem of response time? A common challenge with outsourcing search is latency. When a user searches for something, the request goes off to an external server, waits for a response and displays the results. The latency introduced can be noticeable and hurts the user experience.
Using external services forces you to trade off this delay with new capabilities. In practice Algolia crushed any worries I had because two key areas have been addressed:
Algolia currently has twelve server locations around the world. They call it a "distributed search network". When setting up, they guide you to store your search index at a location close to your customers. This keeps latency to a minimum.
Now that the connection has been made, we have to wait for the search result response. The search engine itself is implemented as an nginx module which makes their stack unbelievably shallow. This is a brave, creative and frankly a brilliant solution that works like a charm in practice. With over 12,000 records to search, 99% of our response times are below 7ms. Your results will differ based on how many records you have indexed and how you've configured Algolia.
Algolia's response time from 99% of queries made
Algolia is so fast they should really measure their response times in microseconds not milliseconds.
To give you some context, a typical request to a website might take 150ms to respond. Search taking ~7ms to respond is so small it doesn’t matter. In fact it may actually be faster than the built in search we're replacing.
Putting it to the test
Once the implementation was complete, to measure the impact of Algolia I set up an A/B test. 50% of the traffic would see the original search, the other 50% would see Algolia search.
We put it live and the client’s response was: "OMG IN LOVE. ITS SO FAST."
The first A/B test
After waiting several weeks the results were in for the A/B test. The conversion rate declined from 4.31% to 4.28% when introducing Algolia. I wondered how could this be? It’s so much faster.
Clearly I’d overlooked something. Thankfully I had a raft of data in Google Analytics to help figure me it out. After much theorising, I realized that the layout of the search had changed too and I must have made it worse. Not Algolia’s fault at all.
The next question was: What was it about the new design that had this negative impact?
You have to think deeply about end customers: Who they are and what are they trying to achieve?
Manhattan Wardrobe Supply’s best and most loyal customers are wardrobe professionals running productions for theater, film and TV. Ordering supplies is a vital part of the job. They don’t typically buy one or two items, they buy for a whole production.
The old search listed all product options and allowed customers to buy many items directly from the search results. The results page was almost like an order sheet. And there was enough knowledge and trust that customers often didn’t need to visit product detail pages to order.
The newer interface took away the “buy now” option from search results. We also limited the number of results by not listing all product options. This is how you’d usually see search results on an online store, but this obviously didn’t work as well for MWS customers.
Revising the search interface
With these thoughts in mind, I went about changing the Algolia search interface to more closely match the original. Not only that, I could confidently go further in this direction now I had a better understanding of what the customer is trying to achieve.
I added the ability to buy products from the search results page. I also listed out variations of a product in search results, though it did make the results really tall. Finally, I decided to make every product buyable from search results.
I was now using advanced Algolia techniques to store custom fields about stock levels and variant information. I also added some neat features like being able to drill down your results by category and highlighting words on the page that exactly match what you’re searching for. This makes the results page very scannable.
The final A/B test: Improved UI combined with Algolia
We observed a 8.4% improvement on conversion rate with this new test.
Conversion Rate by A/B Segment:
To further understand this I segmented this result by new vs returning visitors.
Returning Visitors: Conversion Rate by A/B Segment:
I found an 11.2% improvement on the conversion rate for returning visitors and a 6.2% improvement for new visitors. This makes sense to me because the layout of the search has been designed to be more useful for existing customers than new ones.
I encourage every store owner to evaluate your site search or (contact us for a professional evaluation). Does it offer the Google-like quality users expect? Think carefully about your customers and what they’re trying to achieve. Are you losing potential conversions by not providing a superior search experience? You might find, as we did, better search generates a measurable return on investment.
Join The Conversation
More On The Blog
Are Spree sites meeting Google’s page speed standards?
A study of 680 sites built on the Spree OS platform reveals nearly all are mobile friendly, but only a handful achieved a mobile speed score high enough to be considered by Google as performing well.
Barry Harrison — Sep 12, 2016
Machine Learning and eCommerce Talk at SolidusConf
I recently spoke at SolidusConf about Machine Learning and eCommerce
David Jones — Jun 14, 2016
Top 7 Product Recommendation Options
Top 7 Product Recommendation Options Ranked by Revenue Generation
Barry Harrison — May 31, 2016