To Review or Not Review?

Imagine running a very successful idea campaign, which had 80% participation and generated 1,368 ideas – WOW! Your initial response would probably be euphoria, but that would quickly be replaced by the daunting task of reviewing the massive amount of ideas. If you follow some standard review best practices, you have a review team of about seven people (an odd number to break ties). Everyone would have the appropriate implementation expertise to know which ideas are actually actionable considering the constraints of the sponsor. Further, they also have the personality traits to be able to make quick decisions.

You are likely using a phased or multi-round review process, where each round reduces the number of ideas to be considered for the subsequent rounds. So far you're doing everything right. The review team members even played the role of moderators and actively participated in the campaign while it was running pre-screening content and asking contributors for clarification and/or supplemental information if necessary. Your team is ready for the task, as arduous as it may be.

On day two of the review process, the sponsor calls you and says Reviewer #1 has found a couple of ideas they can use. She sees no need to continue the review and thanks you for your support. How do you respond? Do you inform the sponsor that there may be even better ideas and that you recommend the process continue, or do you agree and conclude the process?

There really is no right or wrong answer here. There are merits to continuing and merits to stopping. My personality is such that I would have a strong curiosity to review and debate all the ideas, but my experience has shown that it's best to stop. Here are a few reasons:

The internal cost of the review (7 reviewers x $60/hour x ((1,368 * 5 min) / 60) ~~ $50K-$70K (use your #s);
The opportunity cost associated with the loss of 114+ hours of work;
There may not be any more good or better ideas; and
Even if there are more good ideas, organizational constraints typically restrict implementations to one or two ideas.

Do not construe my advice to stop as a disregard for the other ideas. I presume that a proper idea management system is in place and that an idea bank or warehouse is present to store all contributions for future consideration. I'm sure, however, someone reading this, could up with an equally compelling list as to why to continue and I welcome debate. But in either case, I strongly recommend always asking “To Review or Not Review?”