Blending Art & Science: Quality of Search Testing

Out in the field I’m seeing that a good number of merchants and marketers using our merchandising tool are stretched a little thin and barely have time to do any more than look at their “failed” or “null” searches. Well, if that’s you, you’re not alone.

Our Merchandising Consultants have been preaching to you that you need to pay attention to not only failed searches, but also your Top 100 successful keyword searches. What does this exactly mean? What should I be looking for in my reports? How should I approach analyzing my data when it comes to “search”? What defines a “successful” search?

The primary purpose of the Quality of Search (QoS) testing that our consultants teach you is to ensure that the results we’re returning are as relevant as possible – indeed a very important part of the optimization process, but that’s just the beginning. Basic QoS testing directs you to look at your Top 100 Searches sorted by ”Instances” or the number of times that a particular term has been entered into the search box on your web site.

Now if you really want to optimize site search you need to sort and analyze these reports in a variety of ways – slicing and dicing the data. You may be surprised by what you find when you start looking at your internal search reports from various angles, not just instances. Consider looking at your top searches by conversion, orders, AOV, visits and total revenue. You’ll be surprised by what you learn.

Next you need to run through the customer experience. What is the customer seeing when they search for these terms? Are they coming into the site from a Google search and have landed on an internal Search Results page? Is the messaging clear and the refinement path intuitive for that customer so he or she can narrow the search accordingly. Where are shoppers falling out after they have executed a search?

The data is there, but what you do with the data determines whether your site search is optimized or only randomly relevant. The process is not completely scientific or mathematical – rather, it’s a blend of art and science. This requires that your merchandisers and analysts engage in equal amounts of right brain/left brain activity – balancing intuition and creativity in terms of what is a great shopping experience, with analysis and metrics-based merchandising.

• Michael Klein
• 1 response
•  Online Merchandising, Site Search

• Share on   Digg Reddit Facebook MySpace StumbleUpon Propeller Slashdot Newsvine ma.gnolia
• Bookmark at   del.icio.us Technorati Google Bookmarks Yahoo Bookmarks Yahoo! My Web Windows Live Furl