How Much Can Demonstrations Teach You About a Solution? (Part 2)
In the first installment of this series, I provided a few observations -- and referenced some useful observations from Gartner's Debbie Wilson -- on the ideal place for fitting product demonstrations into procurement/sourcing technology selection. In this second post, I'll share handful of lessons I've learned over the years to get the most out of demonstrations in a real-world situation (i.e., when you're the prospect contemplating the purchase of a solution rather than how technology vendors typically brief or educate analysts and channels about their products, in a far more generic manner, except when based on an actual prospect/customer evaluation).
To begin, I think the key to getting the most from demonstrations is to require vendors to demonstrate a range of product responses to actual business scenarios. In other words, take the concept a step further beyond what Debbie recommends in her previously referenced post (which is nearly right on the money). My colleague and friend Brian Sommer of firm Vital Analysis and ZD Net fame, suggests that the only way to learn about provider solutions are to request a series of demonstrations based on specifically prescribed scenarios.
Brian suggests in one such guide, "Business case scenarios are a technique used to identify particularly difficult, unusual or complex business problems and relate them to solutions providers. When used in software selections, scenarios help focus software demonstrations away from mundane, non-differentiated function and feature demonstrations. Instead, the value of scenarios comes from the focus they provide on the most important business problems of the customer. When all providers are required to respond to the same scenarios, it prevents the providers from showing the functions and features they want to demonstrate and makes them respond to the scenarios that matter to you."
Brian also suggests that, "scenarios work best when they are developed across many dimensions. Each scenario should originate from a given role or perspective within the company. Scenarios should consider a number of environmental, process, technology and strategy issues. The more complete a scenario is the more valuable it becomes. Great scenarios clearly communicate a key business problem. The more specificity a scenario contains, the better a provider can respond to it."
Further, "Scenarios effectively highlight how different providers' solutions will (or will not) solve material business issues of your firm. The chief alternative to scenarios is the function/feature checklist. While these can be scored and weighted, this technique does little to illustrate the lack of elegance present in some solutions. More to the point, function and feature checklists do little to illuminate how a process works but scenarios do. If you want to see how a prospective provider handles some of your more vexing project-based processes, document them in a scenario. A checklist provides a glimpse into a process' components provided you knew to list them all."
Stay tuned for the next installment in this mini-series when we look at how to write a scenario, using the sourcing area as an example. And in the meantime, if you're either a practitioner or a vendor/consultant looking to hire an adviser to help you (or your client) create the right set of scenarios to think through complex issues, I can't recommend anyone more than Brian Sommer. You can reach him directly: brian (at) vitalanalysis (dot) com.
Tweet
TweetBacks
The "business scenario" approach assumes that the ability to address "business scenarios" is a useful measure of anything. Typically "business scenarios" are crafted from melted-down aggregations of vendor marketing messages and feature lists, combined with some business problem that the vendor has already got a pat answer for unless he's a complete idiot. If he's on the ball at all, you won't discover anything interesting; or if you do, it will be something on the order of Debbie Wilson's mouse clicks. "OMG this vendor takes seven mouse clicks and that one takes twelve1!!!1!!"
You have to go deeper than chin-stroking about a suite of standard reports and baked in transaction processing flows for "business scenarios." What's behind the curtain? What if you want to write your own report, or modify one of theirs? What if you want to change the way the transaction processing works, fundamentally? If you find a solution that's extensible in those dimensions, then it's tons better than any canned solution, by definition. Because you can make it do whatever you want it to do.
There are many more such questions to be asked, but I won't belabor the point. Ask them, with the aid of a technical resource. Or don't ask them, and... buckle up.
I think it is generally assumed that software sourcing will follow a flow similar to this: OA, Industry & requirements analysis, RFP, Q&A, Demos, References, Shortlist, Detailed Demos, (potential Pilot), contract negotiation and implementation. As with sourcing many commodities this process acts as a funnel where the organization’s requirements are defined up front and potential vendors are pared down until the buying organization arrives at the vendor that most closely matches their requirements.
I’ve seen a wide range of organizations purchase software with varying degrees of requirement definition. My hypothesis is that those organizations who are asking for demonstrations based upon business scenarios naturally do so because they have a better understanding of their own requirements. Those organizations that run functional checklist style demos are relying on industry research more so than introspective requirements analysis. After following this string everyone seemed to naturally arrive at the answer that the business scenario demo is a better way for the buying organization to go and I agree.
The important lesson here is a reemphasis on the classic sourcing lesson of defining your requirements and critical business scenarios up front. If you do not, the natural tendency will be to use what is available in the open market as a basis for your evaluation which will leave you with a vendor who was selected based upon their fit to industry requirements and not your own organizational requirements.
How many times have you been in a software procurement meeting where people spent most of the time asking detailed questions about the quantities and columns on a report? All of those questions are pointless, if (a) the data are accessible to you, (b) you can write your own reports easily, and (c) you can modify the standard reports trivially.
There's really no excuse for vendors delivering software that can't be adapted to your needs, or that can't be modified and extended by you, at will. If you look for solutions that can adapt, and can be modified and extended, that's how you avoid the shelf-ware trap. It's all too common in our space for someone (usually some hands-off CPO) to charge off and buy software with a "classical" software procurement process, only to have the software under-utilized (or not utilized at all) 12 months down the road.
Expert advice is cheap. It really is, compared to what you'd pay to some analyst firm, or waste on some solution that compares well on an analyst checklist, but is actually hopelessly inflexible.
I wish analysts would ask these questions. But, with (very) rare exceptions, they don't, probably because most of them don't have the necessary background to understand the answers.