Improving the Candidate Evaluation & Selection Process

Summative Offers Venture Executive Benchmarks for Profiles International’s ProfileXT

By Peggy Thompson and Jenny Steelman, Summative Executive Search


Executives are More Important than the Product and the Market

Carpenters build three-legged stools knowing that each leg is essential to the stool’s success.  If each of the three legs isn’t equally strong, the stool will crumble under load-bearing weight.

Most investors and CEOs believe that the success of a private venture technology company is also built on three equal, load-bearing legs:  1) viable, clearly differentiated products 2) large, well-timed markets and 3) strong, smart executive leadership.   They believe that if all three legs aren’t equally strong, the company could fall under the weight of unanticipated challenges (e.g., the product doesn’t scale; the targeted market’s adoption rate is slower than planned; an essential VP is lured away to step into her first CEO role).

Summative’s experience and observations suggest otherwise.  We believe it’s the executive leadership that is the most important factor to the success of a private venture technology company.

Over the years, we’ve seen clients launch complex technologies that initially failed.   We’ve seen others work to define new markets that never materialized.  But, instead of crashing under the weight of these failures, these clients went on to become hugely successful.

Their secret?   Great executive leadership.

One Case in Point:  OpenPages’ Executives Overcame Disintegration of Market

When Summative recruited Michael J. Duffy to lead OpenPages in 2000, the peak of the dot-com bubble, the company was in the then hot content management space.   Within months of Mike joining the company, the bubble burst and most of OpenPages’ targeted market evaporated.  At that point, many companies would have failed.   Many companies did fail.

Not OpenPages.

Instead, Mike worked closely with the Board and executive team to identify a new market opportunity for OpenPages’ existing IP.   Beginning in 2002, they re-launched

OpenPages in the nascent Sarbanes-Oxley compliance market and then went on to become the top vendor in the much broader GRC (governance, risk, and compliance) market.

Thanks to his leadership, Mike, playing the role of strong war-time general, successfully led the company from a disintegrated market to a much bigger market that they defined and dominated.  OpenPages’ exited via a great acquisition by IBM in 2010.

Products fail.  Markets evaporate.  But, exceptional CEOs like Mike can compensate for market and technology weaknesses by bearing a greater weight of the company’s ultimate success.

Widening the Funnel to Include Non-Traditional Executive Candidates

As discussed in Summative’s white paper, Rules of the Game Changing in Search for Private Venture Executive Talent, the pool of “easy decision” candidates – executives who have previously grown and successfully exited a private venture technology company – is shrinking.  We are again facing the phenomenon experienced in the dot com bubble: the demand for executive talent to lead private venture technology companies is greater than the supply.   But, the factors driving today’s private venture executive talent shortage are different and, in some ways, more complex than those in the late 1990s.  They include:

  • Private venture companies are experiencing longer timelines to exits, so the rate of recently-exited executives entering the candidate market has slowed.
  • Executives rolling out of successful private venture company exits aren’t as motivated to commit seven to ten years (or longer) to another early-stage company.
  • More private venture software companies are being funded than are exiting.
  • Private venture executives are growing risk weary and increasingly seek the greater career stability and larger cash compensation plans that big companies offer.

In response, the top of the candidate funnel must be widened (much like it was in 2000) to once again include candidate profiles not typically contemplated in recent years.   They include, but are not limited to:

  • Bright up and comers who may have limited private venture experience but possess other valuable traits, like deep domain/market knowledge.
  • Executives with strong private venture experience, but limited experience with the domain/market.
  • Bright up and comers who have never held a VP or CEO/GM title.
  • Executives with some private venture experience, but no high-profile exits.

Given the disproportionate role a strong executive team can play in the success of a private venture technology company, the shrinking volume of “easy decision” candidates, and the greater number of executives with non-traditional profiles entering the top of the recruiting funnel, it’s more important than ever that we accurately assess candidates and put ourselves in the strongest possible position to confidently select the right executive.   To understand how to do that, let’s start by looking at how the candidate evaluation and selection game has traditionally been played.

Traditional Evaluation and Selection Process:  The Pot Holes and Their Costs

Despite the disproportionate role that the executive team plays in the success of a private venture technology company, evaluating leadership remains an imperfect science. “Finding the right CEO and judging people in general is the toughest part of our job,” according to Deepak Kamra, General Partner at Canaan Partners, in the NVCA’s paper, Emerging Best Practices for Building the Next Generation of Venture-Backed Leadership.

The imperfect science of evaluating private venture executives traditionally involves mapping candidates’ past experience to the job description while running them through a gauntlet of interviews.  Candidate selection also relies heavily on informal (or “back door”) reference checks.   While this process may be mostly effective when evaluating “easy decision” candidates, there are innate pot holes that can trip up even the most experienced Boards and CEOs, particularly when candidates with non-traditional profiles enter the funnel.

The interview gauntlet demands the largest investment of time.  A study by the Harvard Business Review found that 90% of hiring decisions are based on the personal interview.  Yet, other studies show that interviews are only 10% effective, at best, in choosing the best hire.  Why?  Directors and CEOs, particularly in the private venture community, receive very little training on how to conduct effective interviews and sometimes overestimate their ability to assess talent during interviews.  Instead, they often fall into one of two extreme pot holes by either placing too high a premium on executive candidates who have been a part of a great company exit or by relying too heavily on subjective judgment when assessing candidates who have not.     Either pot hole creates “false positive” hiring decisions.

Executive candidates who have been a part of a great exit may have simply been the beneficiaries of good luck or good timing.   Or, they may no longer have the motivation, skillset, or cultural DNA to lead the hiring company to its own great exit.  Feeling a false sense of security from a candidate’s prior participation in a great exit, Boards and CEOs make a “false positive” decision, jump into the hire, and land in the the first pot hole.

Lacking a great exit, some hiring Boards and CEOs turn to subjective judgment – gut feeling – to make a decision about a candidate.  However, studies show that gut feelings too often rely on how easily and quickly the interviewer relates and connects to the candidate during the first few minutes of an interview (a particularly common phenomenon when interviewing VP Sales candidates).   Instead of accurately assessing objective factors that can reveal the candidate’s true future potential, hiring Boards and CEOs are seduced into a “false positive” decision, jumping into the second hiring pot hole.

We in the private venture community also rely heavily on “back door” references.   “Back door” references are other investors and CEOs whom we know and trust, not necessarily provided as a formal reference by the candidate, that have direct experience with the candidate.   But, what happens when “back door” references for an otherwise strong candidate aren’t available?  Summative has seen some investors pass on candidates who lack “back door” references – making a potentially “false negative” hiring decision in the process – rather than assume the risk of hiring an executive unknown by a trusted associate in their network.   Finding “back door” references will become even a greater challenge as we widen the candidate funnel in response to the shrinking pool of “easy decision” candidates.

Hiring mistakes are common and costly across all companies and all industries.  Peter Drucker suggests, in his frequently cited quote, “Chances are good that 66% of your company’s hiring decisions will prove to be mistakes in the first 12 months.”  Bradford Smart, in his book, Topgrading, estimates that an executive hiring mistake can cost up to 24 times their first year salary.   Drucker’s and Smart’s stats are based on big companies across all industries.  Imagine how much greater the frequency and the cost of executive hiring mistakes are to private venture technology companies.  (Unfortunately, Summative has found no central aggregator of the stats specific to private venture technology companies.)

Given the frequency and costs of bad hiring decisions, and given private venture’s susceptibility to hiring candidates based on “false positive” decisions while passing on potentially exceptional talent based on “false negatives”, it’s time to improve our candidate evaluation and selection process.   To do so, we turn to an unlikely source for inspiration:  big companies.

Big Companies Leverage the Value of Objective Online Assessments

For years, big companies have improved the effectiveness of executive candidate evaluation and selection by incorporating into the process objective online assessments, scientific tools created by third parties that accurately gauge a candidate’s thinking style, behavioral style, and personal interests.  The best assessment tools then map the candidate’s results to the profiles of other executives who previously held the same position in the hiring company or across other large companies.   Now considered a best practice by big companies, the veracity and benefits of assessments are well documented and have eliminated at least some of the guess work.

Organizations Using Assessments Realize Benefits (Compared to Those that Don’t)

75% greater year-over-year improvement in hiring manager satisfaction

75% year-over-year decrease in hiring   costs

2.5 times greater year-over-year   increase in profit per full time equivalent.


Source: Aberdeen Group, Talent Assessment Strategies,   March 2010


Other   Benefits to Using Assessments

  •   Standardization of the selection process
  •   Ability to predict performance
  •   Objective overview of candidates
  • Reduction in time to fill
  • Improved retention rates
  • Improved accuracy, quality of hires

Source: Bersin &   Associates, Pre-Hire Assessments Research Bulletin, 2009

Given the results that big companies are getting from assessments, why aren’t private venture technology companies using assessments?

Private Venture Has Been Slow to Leverage Assessments

Two primary factors have contributed to private venture’s slow adoption of objective, online candidate assessments.   First, unlike big companies, startups typically have limited HR and administrative resources.  They have neither the time nor the money to administer assessments.  Second, and more importantly, the best assessments build customized performance models (benchmarks) based on the results from having prior successful executives in similar roles (in the company or across other big companies) take the assessment.  The assessment then measures future candidates against the benchmarks, highlighting key variances and suggesting additional interviewing questions around those variances.

In contrast, smaller private venture companies have a limited – or no – history of prior successful executives in the role.  Often, the role is simultaneously being defined and filled for the first time.  No history.   No benchmark.  Given the vast differences in the thinking styles, behavioral styles, and personal interests of executives leading big companies versus those leading private venture technology companies, without a private venture benchmark it didn’t make sense to incorporate assessments into the executive candidate evaluation and selection process.

Introducing Profiles International’s ProfileXT

Now, for the first time, Boards and CEOs can see how executive candidates compare and contrast to some of private venture’s most successful operating executives.  With >40,000 clients in 122 countries, Profiles International is one of the leading global providers of online assessments Using Profiles International’s ProfileXT, hiring boards and CEOs can receive objective answers to these questions:

  • Cognitive abilities (including verbal skills, learning index, numeric reasoning):  Can they do the job?
  • Behavioral traits (including energy level, assertiveness, independence, objective judgment):  How will they do the job?
  • Interests:  Will they enjoy the job?

Additionally, Summative has enlisted some of private venture’s top executives to complete the ProfileXT.  Their assessment results serve as the performance benchmarks for four key executive roles.

Private Venture Benchmarks



VP Sales

VP Marketing

Summative’s private venture retained search clients can now objectively assess how a candidate’s cognitive abilities, behavioral traits, and interests compare and contrast to top performing executives in the industry.  For example, CEO candidates can now be evaluated against some of the most successful CEOs in private venture, including Bill Conroy (previously CEO, Initiate Systems, which experienced a great exit via acquisition by IBM in 2010) and Doug Erwin (previously CEO The Planet, merged with SoftLayer which was acquired by IBM for $2 billion in 2013; prior to that CEO, PentaSafe, sold to NetIQ in what was the largest software transaction in 2002).

ProfileXT also provides custom interview question suggestions based on the each candidate’s unique results, putting the hiring team in a stronger position to maximize subsequent interviews.

Summative and ProfileXT is an Easy Decision

While Summative may be the first recruiting firm to have built ProfileXT benchmarks for tech venture executives, we are certainly not the first in the industry to recognize the value of online assessments in the candidate selection process.   At least one of our clients – one that understood very early on the disproportionate weight that executives bear in the success of a private venture company – deployed assessment tools as well as the services of a psychologist to complete in-person, deep dive assessments on finalist executive candidates.   We also know of at least one national private equity firm that requires an executive to take an assessment before their first meeting with the candidate.  These private venture early adopters have leveraged the experience of big companies by recognizing the potential value of objective online assessment tools.

As the pool of “easy decision” candidates shrinks and the top of the funnel opens again, it’s time for private venture to incorporate online assessment tools into the executive candidate evaluation and selection process.  As was the case with OpenPages, there are exceptionally talented executive candidates, like Mike Duffy, who may lack, for example, prior noteworthy private venture exits.   Nonetheless, beyond the traditional pool of “easy decision” candidates there is a world of executives that have the aptitudes, behaviors, and interests, like Mike Duffy, to lead private venture technology companies to great exits.  Unlike OpenPages in 2000, however, we now have the benefit of online assessments to supplement our evaluation and selection of those candidates.

While assessments will never replace interviews and “back door” reference checks, they will give the hiring team greater and more objective insight into each candidate.  With unique private venture benchmarks for CEOs, CFOs, VPs Sales, and VPs Marketing, hiring teams can now objectively evaluate if an executive candidate possesses some of the same thinking styles, behavioral styles, and interests of top performing private venture executives.

Vinod Khosla, Sun Microsystems’ co-founder and one of the leading venture capitalists in the country, once said, “It is not the what, but the who.”    If the “who” (the executive team) has to be right in order to make the “what” (the product and the market opportunity) successful, why wouldn’t we leverage a tool proven to improve the candidate evaluation and selection process?   Given the high frequency and cost of hiring mistakes to private venture technology companies, and the low cost of incorporating Profiles International’s ProfileXT into a Summative retained search, this is one decision that is easily made.



Almost or recently public?

Recruiting diversified, independent directors is critical to investors, analysts, and performance. Summative's independent director practice can help. Read blog post.  

Independent Director shutterstock_95175814


Are you a tough CEO? Are you working with a tough CEO?

A challenging CEO can be a positive leading indicator for success. Read blog post.

Mad Exec shutterstock_100444810