How ConsumerAffairs ranks the Top Picks in each category
The editorial team at ConsumerAffairs uses an algorithm based on a number of category-specific factors. That algorithm produces a set of Top Picks.
What are Top Picks?
Top Picks are the companies listed with “best” designations at the top of every ConsumerAffairs buyer’s guide, in categories from home warranties to moving companies.
You’d assume the company with the highest star rating is number one. But when we recorded the video above, the extended car warranty guide ranked a 3.8-star company above a 4.4-star company. How can that be? Your star rating is an important indication of quality, but it’s not what sets your Top Pick ranking.
That’s great news for you, because it means you can climb in the rankings over time by paying attention to customer feedback and acting on it. Think of Top Picks like a ranking algorithm you’d find in Google or Instagram.
How ConsumerAffairs determines Top Picks
Each category has different sets of review aspects and features, scored on the same three principles.
Review substance over volume
A brand has to have met a review count threshold in the past 24 months. After that, you don’t want a lot of short “This company is great” reviews. You want customers talking about things such as coverage, customer service and price.
The product and offerings, not just reviews
Company features count too: the facts about what a company actually offers customers.
Relative to other brands in the same category
Each brand is scored relative to the leader in its category. Reviews are part of the algorithm, but not star rating specifically.
Inside the algorithm: extended auto warranties
Here are the review aspects and company features for extended auto warranties.
Review aspects
For each aspect, ConsumerAffairs looks at reviewer sentiment and determines, for instance, what percentage of reviews mentioning customer service are positive versus negative.
Coverage
Customer service
Price
Claims handling
Auto warranty looks at reviews over the most recent one-year period, with quarterly updates. Inside that window, ConsumerAffairs also looks at the number of reviews people are leaving and the company’s response rate to reviews.
Company features
Unlike reviews, ConsumerAffairs gathers these features directly from companies.
Number of levels of coverage
Maximum vehicle age and mileage for eligibility
Additional benefits a company offers
Whether any maintenance benefits are included
To improve their score there, a company would have to, for example, cover vehicles with more miles on the odometer.
Other Top Picks have other category-specific aspects and features. Medical alert systems, for example, have a punctuality aspect. Each component has a specific weight in the algorithm, and ConsumerAffairs doesn’t list the weights publicly. So the general recommendation is to do things that create happy customers. What ConsumerAffairs is trying to measure with Top Picks is: are you a company with a quality offering that delivers on what it says it will, based on customer reviews?
You’re scored against the category leader
For a given aspect or feature, the top-performing company earns a 10. Everyone else is scored relative to that leader.
Coverage satisfaction, fictional example
| Company | Positive reviews | Score |
|---|---|---|
| John’s Extended Warranty | 77% | 10 |
| Chris’s Auto Care | 70% | 9.1 |
70/77 is standardized to 9.1/10
Chris’s Auto Care improves to 76%
| Scenario | Leader | New score |
|---|---|---|
| 1. Leader stays at 77% | 77% | 9.9 |
| 2. Leader goes up to 99% | 99% | 7.7 |
The same improvement can raise or lower your score, depending on the leader.
You aren’t being scored against a static standard. Instead, you’re being scored against whoever in your category is currently the best at that one specific thing. You can perform better, but if your competitors are performing even better than you, your score won’t improve.
How to increase your chances of becoming a Top Pick
Create happy customers by doing what you say you’re going to do.
Because reviews make up a good chunk of the algorithm, getting reviews that talk positively about the components of the score can improve your rankings. With that in mind:
-
Re-read the methodology on the guide you’re interested in. Find the three or four factors where you’re being scored against a leader and try to be better at those.
-
Get reviews from all your customers about the factors mentioned. You want the customer who’s been with you a year and has filed a claim and had it handled to talk about customer service. You want a wide range of real experiences from customers.
-
Improve your offering and deliver on what you promise. ConsumerAffairs is asking you to be good at the things your customers were already going to talk about.
Frequently asked questions about Top Picks
How can a brand make sure it is considered for Top Picks?
To be considered for Top Picks, a brand has to have met a review count threshold in the past 24 months. You do need to get reviews to be considered.
Why can a brand with no recent reviews rank above a brand that consistently collects reviews?
As long as a brand has a sufficient number of reviews in the past 24 months, it will qualify for initial consideration. But if a brand stops getting new reviews, eventually it will not be eligible to be a Top Pick.
How can a brand with fewer reviews or a lower star rating rank higher?
Star rating isn’t actually considered directly. ConsumerAffairs looks at the reviews behind the rating, specifically around the components listed in the methodology. Other factors like price, specific offerings and category-specific features are also important. In medical alert systems, for example, if the maximum range of the in-home unit is significantly higher than systems from other companies, it could help that company compensate for slightly worse reviews.
Can the Aspect Marketplace team change my Top Picks ranking?
No. The Aspect Marketplace team does not influence editorial outcomes, and Editorial and Marketplace are kept separate for that reason. The ranking isn’t the editorial team’s taste either, and it isn’t a committee in a room arguing about who deserves it. The algorithm produces the order.
What’s one thing I can do this week to improve my chances of being a Top Pick?
John, head of editorial at ConsumerAffairs, recommends making a list of all customers from the last two years and asking them to write a review, from those who have just signed up to those who have been a customer for two years. All of them.
Talk to an Aspect expert
ConsumerAffairs built Top Picks to help someone when they need it most, by matching them with a brand they can trust. Work with Aspect to learn from customers and improve your product offerings.