Stage-Gate Process: 2020 & Beyond, ft Dr. Robert Cooper

In this episode of Innovation Capital

We look at the stage-gate process invented by world renowned researcher and writer, Dr. Robert Cooper, and examine whether it still holds up in today’s climate of innovation and what processes and frameworks will shape the future of innovation.

Listen now

Episode transcript

Ray Chohan: This is the first episode of Innovation Capital presented by PatSnap.

Welcome to the inaugural episode of Innovation Capital, where we welcome Dr. Robert Cooper, the original creator of the world famous stage-gate process, Dr. Cooper was named the top innovation management scholar by the prestigious US Journal of Product Innovation Management, and he’s also a fellow of the Product Development and Management Association, the PDMA. Dr. Cooper’s process is now deployed by thousands of R&D teams worldwide, and the majority of the Fortune 500.

So welcome, Bob Cooper to Innovation Capital, I just want to say on a broader note, and massive thank you, Bob, our team here have consumed so much of your content over the years, your book has literally been handed out to everyone in our organization. I can’t tell you how many times your slides or content has been put into our content internally. And your work has been an absolute inspiration to our wider team here at PatSnap. So Bob, what we’d love to do is, I’d love to start from the beginning, your original genesis story, and just your background and how you ended up in this wonderful world of R&D and new product development, Bob. So I’d love to kick off with your background and the context around that Bob.

Dr. Robert Cooper: It’s sort of an interesting story of how we ended up with stage-gate and all these best practices for product development. A bit of a circuitous route, I was a graduate engineer, had an MBA, I was a young guy working in a small company a number of years ago, and the owner of the company who had hired me, fresh out of university, was a brilliant innovator and had built the company on the success of a new product. And I was really in awe and amazement that he was so successful doing this. He was manufacturing process equipment. And so we decided to launch our second new product, expecting to double the size of the business yet again. And lo and behold, it failed and it almost bankrupted us. And it was devastating. And at the tender age of about 25, I recall discussing this with my former thesis advisor at the university. And he said, you know, Bob, there’s got to be a better way, maybe that would be a good PhD subject, why don’t you come back and do a PhD in this field of innovation management and new products? And so I did. And that got me on the way into studying companies, what they were doing properly, what they were doing badly, and how there, there could in fact, be a better way. And that was sort of the beginning of the story and ultimately led to the best practices on stage-gate.

Ray: Okay, thank you. On a broader note. Obviously, your process that you’ve codified has been adopted by thousands of innovation practitioners globally now. But for a wider audience joining today, what is stage-gate in a nutshell?

Dr. Cooper: Well stage-gate was based on research that looked at what successful entrepreneurs – or, more correctly, intrapreneurs – people working for companies did in order to take an idea and drive it all the way through the various steps and stages to launch a successful new product. We studied dozens of these entrepreneurs within companies and it was a bit like interviewing winning football teams; if you interview enough of them after the championship is won, you begin to get a picture of how to play the game. And that’s exactly what stage-gate was, it was a model that replicated what these folks did. And it broke the innovation process into a series of stages. Most of these guys thought of taking the project a step at a time or stage or phase at a time, such as coming up with the idea, and then building a business case, and another stage was developing the product and then validating the development and then launching the product. So, multiple stages, but between the stages were a very key component that many models miss. And those are the gates, and the gates are sort of the go/kill decision points, or senior management buys in and continues to support and resource the project team as they move forward. So, that’s where the name “stage gate” came from: stages preceded by gates that give you the go ahead and provide the resources to do the next phase of the project. So, that’s basically what stage-gate is.

Ray: And so, Bob, unpacking that further, when you’re on that journey of adding that gate process throughout stage-gate, what was the route to that pattern recognition? Was it kind of meeting X number of innovators and spotting that pattern? What was the background of actually building out that gated methodology, how was it actually born?

Dr. Cooper: Well, research, as I said, Ray was conducted on a large number of entrepreneurs within companies, and I call them entrepreneurs. Because these were not your typical, fairly conservative, project leaders. These are very dynamic, charismatic guys that drove fantastic products to market, often against all odds, and often against the bureaucracy of a larger company. And I recall, some years ago, after we’d done a study, in about a dozen of these guys, I was asked by the dean of my business school where I was a professor, by that point, if I wouldn’t mind giving a talk to the Friends of the business school, a group of business leaders, and I’m up there on stage, putting up different diagrams of showing how company X did it, and company Y didn’t, company Z did it, and finally, some guy from the audience puts up his hand and says, “Cooper, enough, we’ve heard all these neat stories of Company A, B, and C, and X, Y, and Z – how about you just put all that stuff together, package it up, and put it out as a playbook for how to do a new product project?” And that’s exactly what happened, Ray. We set up a package, all the things, these guys did all the best practices. And it wasn’t just gates array and stages, it was, it was about mitigating risk, it was about doing voice-of-customer, it was about building a robust business case. There were a number, at least a dozen best practices that these folks, these very clever project leaders and teams, did to make their project successful and that got built into the game plan. So, Stage Gate is more than just stages and gates, there’s a whole bunch of very, very precise prescriptions for how to do different things in the process.

Ray: I’m fascinated to see the rise of this methodology because it blows our mind. Now here, when we speak to our customers every day, they always typically reference your methodology. But if we go back into a bit of a time machine, do you recall how quick it scaled? Was it like a viral launch, which just went global really quick? Or did it take time? How did it spread so fast?

Dr. Cooper: Good question. Right. And I wish I could say it was brilliant marketing and planning and the usual things that you use to launch a new approach. Not so, actually. I, as a professor, I happened to be on a sabbatical at a US university, I happened to pick Orlando, Florida. My kids were pretty young in those days. And, anyway, I’m sitting around the swimming pool, wondering what to do. So, I decided to take the advice of this guy that put up his hand at that conference and said, “why don’t you pull it all together and put together a cookbook,” so I did. And I remember I fired it off to two friends of mine in two of the companies that I’d investigated. One of them was a company called Exxon Mobil Chemical. They are a Toronto operation. And the other one was DuPont of Canada. I was living in Canada at the time. And both guys wrote back to me immediately. In fact, one guy phoned me immediate and he said, “this is perfect, this is exactly what we should be doing – let’s implement it.” And so, both DuPont and Exxon Mobil implemented it. And they implemented it first north of the US Canadian border in Canada. And the next thing you knew is the Americans were saying, What is this new system? This looks fantastic. Let’s do it. And when those two big companies roll it up, before you know it, a number of other companies in similar process industries did, and then it rolled over very quickly across the US and then Europe. Scandinavia was the first area in Europe to pick it up, then Denmark, followed very quickly by Sweden. The Scandinavians always seem to be fast out of the gate, when new things come along. And then before you know it in your country, Ray, and Germany and other continental European countries. So, it just started with two big companies largely because of friends, and grew from there.

Ray: Brilliant. Looking at now, we sit here in 2020 in unprecedented times, to say the least. Looking back at recent times, and most importantly, looking forward, what’s next for stage-gate? Where’s it evolving to? Because I’m guessing now over years, people, the community have contributed to the methodology and giving you some really interesting feedback. So, where do you see it going in 2021 and beyond?

Dr. Cooper: Yes, you’re so right, Ray, the community has been an enormous laboratory to improve the process. In fact, we’re into our fifth generation version of the process now. So if a company is using the cloud stage-gate, you know, from 20 years ago, it’s been superseded many times and you owe it to yourself to look into the newer versions, which are faster, better, cheaper, and more effective. One of the biggest changes, of course, that drove the new generations was the fact that the world was changing so quickly, markets changing so quickly. And also information was very fluid, things that were true one day were not true the next day. And so the newer model, and the newer versions of stage-gate, had to deal with the fact that things were changing. And also things had to be much more adaptive and flexible. And as a result, we are borrowing concepts from the software world, namely agile development methodologies, which deal with fluid information and unstable and uncertain situations, and started building them into more traditional stage-gate to come up with what we now call agile-stage-gate, which is really the fifth generation process. And that’s been one of the most significant changes, Ray, in the last, say, 10, 15 years. A handful of companies started doing it in Europe, a handful of companies started doing this in the US, and before you know it, this new emergent hybrid agile and stage-gate combined, started to become increasingly popular, and with with darn good results, too, I might add. So, that’s been a key. And I guess the other big key is the fact stage-gate by the way, was originally developed for and by companies that made physical products. I mentioned two companies already both were happened to be in the in the polymers and chemicals business, but also large companies such as Procter and Gamble, and GE and others adopted stage-gate quite quickly. They’re typically physical product companies or traditional companies, which still account for 75% of R&D spending in the United States, you know. Physical product companies that is, are now as you know, building software into their classic products, to make them smart products. So whether you’re in the packaging business, or the food business, or the medical device business, lots and lots of software is being built in. And of course, the question then becomes how do you handle a new product project where half the project team are traditional, mechanical, electrical, or chemical engineers, and the other half of the team are all software code writers? And of course, some kind of an agile – or a process like agile-stage-gate – seems to work best for both these different types of teams, and can unify them and bring them together in a single process and single project. So, that’s been another major change smart products.

Ray: Yeah, so this is interesting. This links to one of my favourite investors, Bob, Marc Andreessen, he coined the term in 2011, saying software’s eating the world. And that phrase, when I first read that piece, it stuck in my mind since 2011. And sitting here in 2020 now, it could not be more true the way software’s at so many different industries, so many different processes. It’s unprecedented. And then this links to pre-COVID, when we could fly, we are attending a customer gathering and there was some interesting debate and conjecture in the room talking about can stage-gate really work well in the rise of AI application. So, so many of our clients now are deploying AI across all of their business units, because it’s completely transforming where they are heading, and also in response to customer demand. So, what are your thoughts on Stage Gate being applied to AI applications? What are your thoughts on how Stage Gate can evolve to really address AI applications?

Dr. Cooper: Well, AI, of course, is largely from the IT and software world, which is where the agile methodology was developed, largely in the US, Palo Alto, California, Silicon Valley back in the 1990s. In those days, the software folks were not doing very well at product development and it’s a good thing. It was such a growing industry and was a very forgiving one. But, every time you’d meet these guys at our conferences, invariably when they talked about their projects, they were behind schedule, over budget, and very often the “product one” launch didn’t work anyway. So, they had the relative launch release to 2.1, 2.2 to get all the fixes in and they came up with agile and it was a heck of a good method for the software world. Now fast forward 10 or 15 years after that, and so that’s why artificial intelligence being part of that software or IT world would fit very nicely with agile – fast forward to this century. And a handful of companies in Europe for example, one of the first I saw was Lego education in Billund in Denmark, started to combine their software, their digital development groups with their hardware groups to come up with new educational products software and hardware base using an agile-stage-gate approach. In fact, the term “agile-stage-gate” was coined by Danish researchers in about 2015. So, what they were doing was taking the best from agile (and agile has very good features), and combining it with the best from stage-gate. And when and when they did that, they had to throw away some of the traditional components of stage-gate. This classic stage-gate relied on classic project management techniques, such as Gantt charts and critical path plans and milestones and timelines, all that classic project management. They threw that out, and they put in Agile Project Management into the stages of Stage Gate. So they got that aspect. But what agile lacks, Ray, and this is a very important thing. And this has been pointed out to me by a number of industry experts. Agile lacks gates, and gates perform a very, very important function in any new product project, be it AI or medical device or developing a new vaccine. The gates are the point where management sits down with the project team and says, okay, guys, what have you done today? How are things looking? What’s the progress? And is this still a good project? It’s where management intervenes and either supports or kills the project. That’s where management provides the resources that the project team needs in order to move on to the next phase very, very critical. And the other thing that gates do is they kill bad projects. Most companies development portfolios are full of a lot of mediocre projects that probably should have been killed months ago. But nobody does because there’s no gates, they just go on. And that’s one of the faults of agile, it’s a very efficient process for developing software, but maybe so they may get it done efficiently. But it may be the wrong project, and not a very good project in terms of financial rewards, etc. So, that’s a very important part that. That’s what agile stage-gate brings. It basically brings the best of both worlds together. And a number of software companies have adopted agile-stage-gate. It’s a little bit easier for hardware companies who are familiar with stage-gate to adopt it, because they’re just replacing one form of project management with another with putting in or throwing out classic project management, putting in agile project management, but they’re still keeping stage-gate. So for them, it’s a bit easier, but I think it has equal applicability to the software world and that would include artificial intelligence, Ray.

Ray: So, Bob you raise a wonderful point here and we’re seeing this a lot in the last three years, we were speaking to one of one of our key clients who leads R&D in a world-famous consumer packaged goods company, and he was talking about how he’s completely reimagined his recruitment philosophy, Bob, for R&D. So, on a broader level, is that a trend you’re seeing across the board? Folks who are technically gifted, that’s table stakes and research and development, innovation, but also they’ve got marketing chops, they’ve got sales shops, they can engage the rest of the team. Is that is that a broader trend do you think is going to further part of growing industry?

Dr. Cooper: I’d be hesitant to say that I’m seeing it take place, but I sure would say that I’d like to see it take place, because I think that client of yours is right on. Far too many technical people with their Bachelor’s and Master’s and PhD degrees in engineering or science, have very, very limited marketing and communication skills. And even less willingness to engage and interface with customers. You know, I can remember, Ray, just to tell a little story on the side, I was giving a seminar in the UK a few years back and there was this woman, a very, very sharp lady who sat in the front row with her PhD in chemistry. And I was talking about how important it was for the project leader and her team to get out there and do face to face interviews with prospective commercial customers in our case, and how it was critical for the whole team to get out there, not just the sales and marketing department folks, but the R&D folks from the lab to touch real customers. Understand their points of pain, understand what was keeping them awake at night, and commit to solving their problems and coming up with a great new product. She looked at me and said, Bob, that’s fine and well, but I have a PhD in chemistry, I wasn’t trained to do that, that’s not my job and I’m not going to do it. And of course, you look at her and say, “Well, you know, do you want to be successful in product development? If you do, forget the fact that you just got a PhD and can’t get out there and talk to customers. It’s critical. That’s the difference between winning and losing.” So, I would like to see a lot more of that built in. Now, part of it, Ray, of course, is personality. Some people are just outgoing and gregarious, etc. Part of it is training, I think some of that can be trained. And we try to in the programs I’m working with, in Canada, for example, where I’m located in Toronto. The Canadian government finally sponsors us and others to work with smaller business, to help them develop new products. They’re all technically very clever people typically but they lack this market orientation and so a good part of our training is on voice of customer how to do it, how to talk to customers properly. And it’s tough, getting people who are software writers or electrical engineers, thinking that way. So, I’d like to see it, I think that packaged-goods firm company that you talked about is right on but it’s got to be right across the board, not just in that industry.

Ray: One thought leader we track a lot here at PatSnap (and again, we probably talk about him a lot internally and he’s on a lot of our PowerPoint presentations) is Ray Kurzweil. He’s published a number of fascinating pieces and he really talks about, we are living in special times, Bob, an era of unprecedented exponential innovation, driven by a number of force factors and macro forces. So, based on that kind of underpinning, what are some of the pitfalls you see ahead? Between now and 2025 2030? What are some of the emerging challenges that you’re seeing in the current innovation processes out there? Do they need to evolve?

Dr. Cooper: Well, that’s a that’s a tough question, Ray, because don’t you wish you could crystal-ball gaze the future, especially coming out of this pandemic, where many people are predicting, this is going to be a whole new world. It’s easy to talk about the positive side of innovation, because based on the new the newer technologies that are coming to fruition, whether they’re artificial intelligence, or quantum mechanics, and quantum computing, electric vehicles, you name it, there’s so many technologies that are reaching the springboard point of just taking off, and so the world is going to be a very different place in the next 10 or 20 years. I guess the big dangers are that we don’t get so mesmerized by this possibility, that we start thinking that everything is good. I can recall as a kid reading a Popular Mechanics, which is a very popular book that often young boys read back in those days, about how we are going to have flying cars by the year, I don’t know 1970, etc, you know, and as kids were just mesmerized by Sputnik and the prospect of flying cars and space trips, and what have you, well, you know, here 1970 or 1980s come and gone and we’re still not having what we may have one of these days, but we’re still not flying to the moon for vacations or holiday. And we’re still not seeing the flying cars and so on. So, you got to be a little careful that you don’t become so mesmerized that you throw caution to the wind. I mentioned before, Ray, that a significant number of projects that get initiated and invested in come to nothing. I think that according to the US data, and this is American data from Product Development and Management Association (the PDMA), of every four projects that are approved for development – that means heavy spending – only one out of four becomes a commercial success, which suggests to me that there’s an awful lot of people saying yes, the projects that probably will come to nothing. I think we’ve got to be careful about that. It’s easy to say yes, yes, yes and be a little bit dazzled by the possibilities of the future, but one also has to be recognized one is investing scarce resources and shareholders of course expect returns and if you want to keep your job you got to be successful. So, that’s one of the concerns I have is that we still need thoughtful investment about where we want to place our resources as businesses. And by all means new product development is critical. And our new business model development, especially coming out of this recession. The last great recession, from 2010, taught us that the companies that did well, in the 10 years that followed between 2010 and 2020, they were the ones that continue to invest in R&D, and product development and focus on innovation. The ones that cut back, they didn’t do very well in that 10 year period that follow. So, it’s important to keep the focus on innovation and new products and not cut back like so many companies have recently during the pandemic. But it’s also critical to recognize you need a good strategy and also you have to take some care about where you say yes, and where you say no, where you put your R&D dollars, and perhaps more importantly, the projects you say no to. So, portfolio management, and a good gating process are two tools that will help you there.

Ray: Do you see any interesting examples of how data and analytics is being overlaid with stage-gate? Do you see some form of convergence potentially, in the future? This is where the system of record becomes the system of intelligence, where you have workflow and analytics partner together. Is that anything that you’ve seen in terms of some of the audience in the community that you work with?

Dr. Cooper: Absolutely, Ray. An excellent topic for discussion. A colleague in Germany and I wrote a paper recently that appeared in a journal or magazine called Innovation Management, and it was called the digital transformation and its impact on product development. And that was one of the themes, how artificial intelligence and data analytics, and those tools generally in that category, are impacting on product development. For example, today, many companies including the British company, AstraZeneca, are focusing on developing a new vaccine. Hopefully, they’re successful. They’re using artificial intelligence data analytics to basically select candidate molecules and solutions for the problem, and really accelerating the process of developing a vaccine. That’s not happening just in the area of pharmaceuticals, by the way, that’s happening in a number of areas, from things that you might even consider a little trivial. I was chatting with one company that was in the area of custom dress, design and manufacturer for ladies for social events. And they were using data analytics to analyze Facebook pages, and to see if they were going to special events. Or if some woman was talking about going to her son’s graduation or something, I don’t have a thing to where they are analyzing all that information, looking at some pictures of her, what kinds of dresses she looks good in, what kinds of colors seem to suit her, and they would custom design, using artificial intelligence, a dress based on an analysis of that data, and send her a picture and say, Well, this dress suit your son’s graduate suit you for your son’s graduation. And of course, you sort of say, Well, you know, an interesting application. But I guess the world’s coming to that. So I’m seeing a lot of use of these types of tools, more and more so to not only develop the product, or assist in the speedy development of the product as they are in the pharmaceutical industry right now. But also even in the conception, and coming up with the ideas in the first place as this dress manufacturer was doing. We’re seeing enormous use of these new digital technologies to accelerate and enhance product development, not only in the areas I’ve just described, but also in terms of coming up with rapid prototypes and virtual prototypes and augmented reality prototypes. FedEx, the large US company that’s in the delivery business, uses virtual reality to test new drop off products and containers, before they even appear in people’s homes. They bring people into their lab, put virtual reality goggles on and have them imagining they’re already using the product without even having it so we’re seeing a lot more of that type of use. I know that’s a little beyond your question. But digital technologies are playing an increasingly big role in product development, to help companies conceive, develop and even launch the products. So yeah, key area in the future.

Ray: Now Bob, this is absolutely brilliant. Our audience loves your broad perspective and your depth of experience. And I’m guessing you’ve got so many stories of fascinating organizations you speak to, and have worked with over the years. So, it’s interesting now going into 2021, especially after this year, it’s been an interesting year, to say the least, and a challenging year for so many people in so many different ways. Looking at the bright side, in terms of innovation R&D, we are speaking with and hearing in the market that we are potentially entering a special era, Bob. In the next eight, nine years, potentially by the end of this decade, the innovation process potentially could be fully digital. I know that’s a long shot. But just for your professional opinion, do you see that as a reality in the next kind of 7, 8, 9, 10 years, where even when it comes to ideation, it’s actually executed by a piece of AI? Do you see a future like that in eight, nine years?

Dr. Cooper: Ray, we’re already sort of moving in that direction now. There are some software producers who produce software in support of product development. There are several American companies, I shouldn’t mention names here (this is not a commercial for them). There’s also several European companies that have superb software packages that support R&D departments, that support new product project teams, and their management. And they do everything from embedding stage-gate through to portfolio management to handling ideas. I was at a conference recently of one of these companies in the US, and was quite amazed at how they are now thinking for the future of building in a lot of these analytical techniques, data analytics, into their artificial intelligence and into their software, not only for enhancing it for coming up with the ideas, but also accelerating development. And also even for project selection. Can you imagine software that has a this project button, artificial intelligence that says do this project, but not that project. There’s actually a venture capital company, I believe it’s in Hong Kong, that has put an artificial intelligence robot on its investment board. And you know, after studying a number of past investments, it learned – and this robot is now making better calls than the rest of the members of the investment board. This has been happening for a couple of years now. So, I can see that happening also within the stage-gate process that the management gatekeepers may have an extra gatekeeper called an artificial intelligence robot sitting at that meeting, and helping them make decisions. So, this is already starting to happen, Ray. If you hear of anybody that wants to pursue this, I’d be happy to talk to them with your help and involvement. In pursuing this, there have been several requests to me that companies are starting to think about doing this already.

Ray: We see so much talk and conjecture around this topic. On another broader note, we feel quietly bullish the next five, six years is going to be fascinating, Bob, in this space, but it’s going to be a community effort. I think no one organization or group of people will make this happen. It’s going to be a community effort and there’s so many ways to slice and dice it but it was great to get your overview on that perspective on data and analytics and how it plays into that innovation workflow. So, just to wrap up Bob, today’s been wonderful. It’s been great getting your global perspective and all the stories and all the experience you have in the space. On a broader note here at PatSnap, there’s a number of of our team are who are huge fans of Elon Musk, he’s the rise of what I call the R&D facing CEO. What I mean by that is, if you said to Elon, if you could do any other job, what would it be? He’d probably say, I’d be with the R&D team. I’m an engineer and we’re also seeing that with Microsoft, Satya Nadella. He was originally from the R&D organization and he’s doing a stunning job with Microsoft since Steve Ballmer. Do you see that as a trend moving forward where boards of large companies prefer having a leader from that deep technical background who was formerly a leader in R&D, in a conventional organization, is that a macro paradigm that you see just constantly scaling in the future?

Dr. Cooper: I hope so. You know, for a number of years there everyone looked to Steve Jobs before his untimely death, as the that kind of a leader too because as you recall, he and Wosniak started in his garage and built Apple from nothing. And you know, Hewlett and Packard, you know, way back in the beginning, they too, had a similar background, both engineers, I think they were at a Southern California Technical University. So, we’ve had a long story of highly charismatic, and highly innovative technical people, be the people that have led and built these great companies. So, this is not a new phenomenon with Elon Musk, this has been around for 100 years that I suspect and hope it does continue. Because look what they’ve managed to do, you know, reading through the history of R&D, and great inventions, and great innovations and more modern times, you know, this century and the last century, it’s been the story of these men and women. I suspect it’ll continue on. I would like to see most – you know, I think, one of the most unfortunate things, and I apologize to my friends in the financial community – and I think one of the most unfortunate things that happen in the UK, and in the US, is the rise of the financial person in these large companies whose basic focus was on the next quarter’s results. Very short term orientation, and maximizing value to the shareholder, which invariably meant a very short-term orientation. And under those folks, R&D and innovation often was discouraged, I think, to see those folks, at least in many companies, replaced with the kind of people we’re describing now, the entrepreneurial innovation, technically trained person leading the charge, I think it would be a very good strong breath of fresh air versus the financial driven mess that we’ve seen for the last couple of decades.

Ray: Yeah, we’ve definitely been champions of that school of thought as well, we absolutely love and adore these first principles and thinkers who lead these amazing organizations because they inspire us everyday here at PatSnap as well. But, Bob, thank you so much for today, getting your perspective on stage-gate, the history, where you see things moving forward has been absolutely fascinating today, and I’m sure our audience can’t wait to hear your perspective. So, really appreciate your time today and thank you so much for spending time with us and you have a have an awesome week.

Dr. Cooper: Thank you, then good luck and be safe.

Ray: Until next time, continue to embrace your childlike wonder. Stay curious.