The needle was never in the haystack

Have you ever actually tried to find a needle in a haystack? Of course you haven’t, because nobody has. It is, after all, a metaphor. But it is worth pausing on, because the next time someone tells you the answer is in there somewhere, you might want to ask them how their sinuses are. You will not find the needle. Instead, you will find itchy eyes, a streaming nose, and a hand full of straw. That is not insight. That is hayfever.

 

What is strategic discovery?

Strategic discovery is the practice of returning to the original customer signal, which a structured voice of customer programme turns into decisions that built a business, before scale and accumulated structure obscured it. Drawing on Rita McGrath’s Discovery-Driven Planning, it is the diagnostic move most scaling businesses skip when they most need it. The argument here is simple: the answer is not in the haystack of dashboards and decks. It is back in the field.

This is the position scaling businesses keep finding themselves in. The business is large now, the team is busy, and the dashboards are full. While marketing runs campaigns and sales closes deals and operations ships, an unease has settled in somewhere underneath all the activity. The things being done are no longer quite the right things. Strategy meetings end without clear conclusions, and the pitches that used to land are landing less. So you suspect the answer is in there somewhere, and you ask the team to search harder.

You are searching the haystack. What you actually need is strategic discovery, which is something quite different.

 

What the haystack actually is

The haystack is your business as it currently stands: scaled, layered, structured, and possibly impressive from the fielder’s-eye view. Over the years, it has been built outward from the original idea. Each season has added more bales: more product, more process, more people, more dashboards, more decks. As a result, there is now a lot of hay. It is a nice problem to have, until you need to find something specific and discover the needle is somewhere near the bottom of bale 47.

The needle, meanwhile, is what you originally went into the field for. It is the human truth at the bottom of the business: the customer reality, the signal that told you, back in year one, that what you were building was worth building. That signal has not vanished. However, it has been bundled up, stacked, and sat on by the rest of the business. So it is still in there. But it is no longer where you are looking.

 

Why scaling businesses keep searching the haystack

Because the haystack is what you can see. The dashboards, the surveys, the quarterly results, the customer service reports. The strategy decks from the offsite. The PowerPoint with the four pillars. All of it is hay. Useful hay, sometimes. But hay.

The structural problem behind this was named recently by Bain & Company, who put it crisply. In most organisations, there are now too many degrees of separation between the executive and the customer. As a result, the people closest to the original signal are buried inside functional teams. Their observations get diluted as they move up through the layers. By the time anything reaches the leadership room, it has been summarised, slide-bulleted, and stripped of the texture that made it useful in the first place. In other words, you are reading a transcription of a transcription.

CB Insights, meanwhile, puts a number on the consequence. Forty-two percent of business failures come down to no market need. A further fourteen percent come from ignoring customer feedback. Most of those businesses did not lack data. Instead, they lacked a working relationship with the original signal.

 

A small aside

I am sometimes asked whether anyone has ever, literally, searched a haystack for a needle. The answer, as far as I can tell, is mostly no. Because nobody is mad enough. The phrase exists because it works as a metaphor for a hopeless task. Which makes it odd that so much of strategy practice has been built around variants of it. Search the data harder. Commission another deck. Run another survey. Find the needle. Then, six months later, present back. Still no needle.

 

Strategic discovery starts in the field, not the haystack

The reframe, once you see it, is simpler than it sounds. The needle was never in the haystack. Instead, it was on the ground, in the long grass, long before the hay was made. So strategic discovery is the practice of going back to the field. Walking it. Looking down. Recognising what was always in the soil before the business got built on top of it.

This, by the way, is not a new idea. Rita McGrath at Columbia Business School has been arguing a version of it since 1995. That year, her Discovery-Driven Planning paper in Harvard Business Review reframed how scaling businesses should think about uncertainty. Her argument was that conventional planning assumes you already know the answer and just need to execute the plan. However, discovery-driven planning starts with what you do not know. It then treats every plan as a set of testable assumptions, to be checked against the actual ground.

Strategic discovery is the same move, applied to the question that matters most. Not “is this plan working” but “are we still working on the right thing.” Because of that, you stop assuming the answer is hidden in the haystack you have built. Instead, you walk back to the field and look at what is actually there.

 

What walking the field actually looks like

Walking the field is not another survey, and it is not a workshop with sticky notes. Instead, it is the disciplined practice of going back to the people who experience your business directly. You listen with fresh ears for what was always there. So we mean customers, frontline team members, and the people who answer the phone. The ones who can describe what they are seeing without needing to translate it into a dashboard. Sometimes, the most useful sentence in a year of strategy work is something a delivery driver said in passing.

Then you listen for the qualitative signal. Not just the metrics, although the metrics are useful. You listen for the texture. The things people say when nobody is asking them to fill in a Likert scale. The pattern of language. The half-finished sentence. The thing they keep almost saying but never quite landing.

 

After that, you check the lens. Every business is looking through a lens that got set sometime in its first three years. Most of the time, it has not been deliberately re-examined since. The lens is the unspoken assumption about who the customer is, what they want, and what the business is for. So strategic discovery is partly the practice of finding that lens and asking whether it still fits. Often, it has drifted, sometimes substantially, from what is actually happening in the field. Our work on the invisible ceiling goes deeper on this drift.

 

Every harvest changes the field

Here is the part most strategy frameworks miss. The field is not a fixed thing. Every year, you grow a crop, you cut hay, and you change the soil. New customers arrive while old ones leave. The competitive landscape moves around you. As your team grows and ages, the story the business tells about itself shifts. So by the time you have done one full cycle, the field has changed.

That is why strategic discovery is not something you do once at the start and then file. It is something you go back to. The Good CX DISTIL methodology builds this in deliberately. Six stages, one flywheel: Discovery, Insight, Strategy, Tactics, Iteration, Learning. Because the Learning stage feeds straight back into the next Discovery, every harvest is read again. The haystack has grown. Whatever was buried last year is buried under a different layer now.

 

The CX research work we publish consistently shows the same pattern. The businesses that stay sharp are the ones that walk the field on a rhythm. While the ones that drift are the ones who decided, somewhere around year four, that they already knew what was in the soil.

What this asks of senior leaders specifically, the people standing in the gap between scaled functions, is a related practice. We look at it through Richie in The Bear, Ronald Burt’s structural holes, and Cal Newport’s Slow Productivity in scaling leadership lessons from Richie.

And once that leader has made the bold strategic commitment, the work shifts again. To keeping their head once the Rubicon is behind them. We look at the discipline of sustained leadership through SAS: Rogue Heroes, Marcus Aurelius and polyvagal theory in lessons from SAS Rogue Heroes: cross the Rubicon, keep your head.

So if your strategy feels like it is running on assumptions you can no longer fully name, it is probably time to walk the field again. Not search the haystack harder. Not commission another deck. Instead, walk back to the ground, look down, and see what is actually there.

 

Ready to walk the field?

This is the part where most thought pieces try to sell you something. We will not. Soz. Instead, the honest invitation is smaller than that. So if any of this is landing, the first move is just a conversation. Not a six-month proposal. Not a pitch deck. Just one conversation. What have you been searching for? Where have you been looking? And is there a chance the field would tell you something the haystack will not? If it is, we will tell you so. If it is not, we will tell you that, too.

The needle was never in the haystack. You already half-knew that. But the good news is the field is right where you left it.

Let’s talk