The Pattern is Not the Path
Why the Patterns we Study Cannot Tell us the Path Forward
Listen on the NonTrivial Podcast👇
CASTING FORWARD
Life is very much about looking for patterns. Those parts of a situation or phenomenon that speak to something fundamental. To do so is to understand something more profoundly, expand human knowledge. But what is a pattern exactly? A pattern is a set of unmoving parts that stay still amidst the immense complexity and swirling of everyday life, and of phenomena in the natural world. Imagine seeing water slosh around in a bathtub. While most of what we see is random and chaotic, there are parts of that movement that repeat. The crests and troughs of the water as it moves up and down occur again-and-again. They take on a pattern.
Imagine looking at a mountain range, the jutting rocks and crevices are randomly dispersed, but there are also aspects shared between all mountains. Regular, intelligible forms that keep popping up, that make a mountain a mountain. Characteristics that define the thing.
To seek out those defining characteristics of life and the natural world is the story of humanity itself. But we don’t do this just to learn, we also want to cast those patterns forward; use the discovered patterns to improve our lives, to make bigger and better things. Casting patterns forward is the narrative behind progress. Throughout history historians, writers, philosophers, engineers and scientists have taken what they’ve learned and used that knowledge to create new works of art, new philosophies, new inventions, new theories. Like ripples in a bathtub or growth markings on a mountain, our brightest minds have picked-up on the parts of reality that seem to reoccur, and that speak to something profound. Human progress is a story of noticing patterns and casting them forward.
Or is it? Are the patterns we notice in life and natural systems things we can cast forward? Can we take patterns we see and use them to make a better version of the thing studied? This is what I want to talk about in this article. In our personal and professional lives we are told that the more we understand about our environment the better we’ll be able to create things that improve the quality of our lives. But this rests on a fundamental assumption: that the patterns we see around us can show us the path forward.
That the patterns we notice contain the information needed to take the next step; the “how” behind the pattern. This is very much the academic narrative, where schooling supposedly teaches us things that can be used in the real world for innovation. It’s very much the political narrative that suggests government can take the knowledge and recommendations from experts and cast those forward into policies that improve lives. And it’s the narrative of our everyday lives, where the patterns we notice often form the basis of not only our world views but also our actions.
But what if this narrative isn’t correct? What if a closer look at how information works reveals that such casting forward isn’t possible. That there is something fundamentally wrong with the story that human progress owes its march forward to the learnings that occur at each step. What if there were aspects of nature and information that preclude the notion we can make something better by learning more about it? That regardless of what you know about a situation, understanding it cannot serve as a foundation to create new things. What happens when the pattern is not the path?
Let’s go see.
MULTIPLE REALIZABILITY
I will argue that the reason patterns cannot be cast forward is due to a fundamental aspect of complex things; something called multiple realizability. If something is multiply realizable it means there are multiple ways to achieve it. If you take the temperature of a block of metal, there is not *one* way to achieve that temperature. The same temperature can be achieved by friction, radiation, absorption from the surrounding environment, etc.
Temperature is multiply realizable. Same goes for wetness. There is no way to trace the path from individual water molecules to wetness. We can talk all day about intermolecular interactions and forces, but to move our story from those local interaction rules to the behavior we see at the larger scale isn’t possible. Sure we can take averages, do tricks with course graining, and renormalization but this does not expose the logical chain of causality from inputs to outputs. Wetness, as a property, is multiply realizable.
To go beyond materials into real world situations is of course just a matter of increased dimensionality; increased complexity. Everyday situations have attributes to them, and those attributes bear out via patterns, since a pattern is just a regularity that appears. Only through regularity can we define things amidst the vagaries of our environment.
This is quite intuitive. Take any nontrivial task you’ve worked on, whether it’s writing an article or book, finding a solution to negotiation, job searching, etc. These situations all have properties to them. Books have chapters, and paragraphs, negotiations have interests, options and commitments, jobs have expectations and deliverables. To see these patterns is to define the thing itself. This is simply human pattern recognition. We group and categorize based on the what we observe, and this allows us to communicate situations to others.
To suggest there is one way to arrive at good writing would be ridiculous. There are countless styles and approaches people use to bring their thoughts to paper. The same goes for any real world situation. Negotiations are organic, messy, involve listening, instructing and compromise. Jobs benefit not from some robotic adherence to process and culture but from diverse experience, serendipitous discoveries and creative contribution.
Arriving at the thing you deliver, the thing you create, does not move by a path from beginning to end, rather it precipitates out from highly varied interacting pieces. To be good at any of these activities is not about knowing *how* it’s about having the constitution to jump in and pay attention to signals; something I’ll talk more about later.
The patterns that define a thing or situation are multiply realizable. There is no *one* way to achieve them. The higher the complexity the more multiply realizable the pattern. While metal blocks and wet water already show us the notion of a path is ridiculous, life itself has near infinite dimensionality and are thus “infinitely” multiply realizable. To achieve things in life is not, cannot, be about finding paths, despite what every business book, classroom lecture, or so-called “causal analysis” might tell us.
It is multiple realizability that precludes the chance there is a path to achieve the pattern. The corollary is that this means patterns cannot be cast forward. They cannot act as instructions on how to take the next step, since it wasn’t a “step” that produced the pattern in the first place. To be clear, it would be statistically impossible to reproduce the interactions that led to the pattern we are noticing. They would never happens the same way twice.
The pattern emerges from a statistical ensemble of impossibly intricate interactions. Patterns don’t have a history that can be replayed, rather their stories are lost forever; a transient spark that gave life to the pattern but is no longer part of its makeup. To notice a pattern is to see something that has no evidence of how it got there.
Of course this doesn’t stop scientists from adding narratives to *how* something supposedly arose. But the how that is given is not some end-to-end connected chain of events, rather it’s, at best, a litany of other properties that work alongside the pattern of interest. How those other properties work alongside is not fact, it’s speculation, and if we’re being intellectually honest, barely an informed one at that.
Multiple realizability is what makes information travel in one direction in any system of appreciable complexity. We can see the end result of what we want. We can see birds fly and calm individuals speak wisdom. That information is available. But going the other way, where we reverse-engineer those patterns to see how it came about is not available in anything but the simplest of systems. The *how* is lost forever. When we make a discovery we are seeing something that has already emerged. To believe one can cast a pattern forward is to believe one has access to information they do not.
The pattern is not the path.
FOR INSTANCE
Airplanes
Say we are observing bird flight. We notice patterns in the wing of the bird. The shape and length of the wing. The angle it makes with the air. The more we study the patterns inherent to the bird the more we learn about things that fly. But is this the same as saying we learn *how* flight is achieved? Does observing the patterns inherent in bird flight instruct us how to create an airplane?
I argue it does not. Now this is hard to wrap one’s head around, since it seems obvious that observing bird flight would help someone build an airplane. After all, it’s well known that the Wright Brothers spent a good deal of time observing birds, and applied “wing warping” designs that look very much like the shape of a bird’s wing.
But this narrative that says the inspiration was also the path is always going to be told. Ask anyone who achieved anything and they will create a narrative around *how* it came to be. I talked about this in the episode called We Can’t Create the Way we Consume.
To create is ad hoc and messy and doesn’t make for a good story. In fact the process is downright nonsensical because of the way we explore the possibility space; also discussed in the last NonTrivial episode. Scientific articles are a perfect example. The so-called “scientific process” is largely fiction, but there’s no other way to communicate how things get figured out. We cannot create the way we consume.
So we are going to hear about the study of the thing leading to the thing, regardless. It’s how we tell stories. But this doesn’t mean it’s how it happened. I argue that studying a bird cannot tell you how to achieve flight, and it’s not how human flight was achieved. Birds undoubtedly inspired us to fly, but the close inspection of their bodies and behavior did not. Only through massive tinkering and exploration did something capable of flying precipitate.
After all, flight is far more than a set of wings shaped like an airfoil. A bird’s body has a certain weight distribution and hollow bones. And the hollow bones aren’t even for making the bird lighter, they are to allow the bird’s lungs to extend into the bones. Bird bones are pneumatized, allowing the bird to take in oxygen while both inhaling and exhaling. Birds have a mix of feather types, some of which support flight. As a bird flaps its wings down certain feathers move together to help with lift, and when a bird moves its wings up certain feathers move apart to allow air to pass through.
We can study more and more about the attributes of birds but none of this can tell us the confluence of interacting factors that lead to flight. Flight, like anything of real world value, is achieved by synthesis and emergence. We cannot know the course it took to get there.
In fact, whatever bastardized version of a bird we create as a plane is not some set of bird discoveries pieced together to engineer flight, rather it was largely happenstance that arrived at a design that worked. All the “hows” that get told in our stories of invention come AFTER the thing is discovered. But of course, as the educational and scientific narrative goes, we tend to believe the details gathered through study came before the creation of the thing itself. Rest assured, they did not.
The next step taken is granted to us because we’re willing to embrace variation and iteration. From a biological standpoint massive variation is the huge amount of genetic diversity that makes countless versions of bird; and iteration is the millions of years worth of evolution that selectively filters for the designs that work best. In short, throw enough crappy designs off the cliff and the only bird’s who pass on their genes are the ones who don’t smash into rocks. Obviously an oversimplification but the point stands. There was no instruction set for flight, there was trial-and-error, massive sampling of the possibility space, until a solution emerged. Things don’t get structured forward, they can only show us their properties after it works.
Nutrition
Let’s look at another example. In nutrition the narrative is always the same. Someone did a study that compared healthy to unhealthy individuals and noticed something that might suggest what makes one healthy. Maybe it’s the preponderance of a mineral in their bloodstream. To take note of the common pattern (healthy people have lots of mineral X) is the same as any other pattern recognition; this time using statistics to try and confirm the existence of such a pattern. If the difference is statistically significant, then the preponderance of the mineral will will be “confirmed.”
But nutrition, like any other study, doesn’t stop at noticing the difference. The interpretation is that consuming this mineral is a healthy thing to do. This mineral will appear in certain foods, and to include such foods in one’s diet will be considered a smart decision. But this is casting the pattern forward. This is assuming that the preponderance of a mineral in healthy people means adding that mineral to your diet will make you healthy. It’s the same narrative of thinking we can notice a pattern and use it to achieve or even supercharge an effect for ourselves, like bird flight. The entire healthcare industry, whether it’s medicine or nutrition, is based on this premise; notice what occurs in the outcome and assume it must also be part of the input.
This doesn’t mean there isn’t a lot of work behind the assumption. Statistical and meta analyses will attempt to tease out the causal connection, but this isn’t quite what most people think it is. The correlation discovered (highly speculative in its own right) is telling us that if we increase the consumption of the mineral we will get the outcome. But what is the outcome we want? Is it the abundance of the mineral? No, it’s health. But what is health? How was it defined to begin with? Say it was long life. We have no idea why that mineral was present, and to assume it played a role in the longevity is exceedingly naive. How do we know that mineral isn’t merely a byproduct of a highly complex process that leads to health?
To be clear, it might be true that the presence of the mineral is indeed a marker of health. Even if it were some byproduct, it would still be a signal. But the subsequent intervention of choosing to ingest the mineral to improve one’s health is where things fall apart. While the mineral may play a role, we have no idea WHAT role it plays. And, it might play no role at all, even though it’s always present in healthy individuals. Most assume statistical analyses can answer these kinds of questions. Again, other than for the simplest of systems this is not the case.
Statistical analysis can show the difference between populations, and will attempt to show the affect of an intervention, like adding more of mineral X to one’s diet. But the so-called “causality” that is reported is simplistic correlations that operate in far few dimensions than the phenomenon being studies. It is a story told with statistics, not some fact laid bare. The replication crisis in science is hardly surprising, but that’s a topic for another article.
Like any other pattern, the story of what is needed for health comes after the discovery. Scurvy wasn’t prevented because people studied fruit and determined it contained something that healthy individuals had an abundance of. Scurvy was prevented because those who ate certain fruits didn’t suffer the symptoms. It’s tempting to think that now that we know about vitamin C we can cast health forward. But this isn’t how it works. Wisdom only travels in one direction.
Philosophy
The same thing applies to philosophy, specifically in the application of a given ideology or outlook to our lives. Take stoicism as an example. Stoicism has a lot of great wisdom baked into it. Read about what people were thinking over 2000 years ago and you’ll find they understood many of the same patterns we face today, and understood them profoundly. The patterns stoicism picked up on were arrived at through various people trying to survive the trials of life.
But like many philosophies a core component is the practical application of the ideas. Rather than just understanding something about life, why not apply it to your own life? Think about the Stoic take on anger. Stoics recognize that much of our consternation in life comes from our preoccupation with things we cannot control. We can all relate to this. How often does our worry seem to amount to nothing? What is the point of being paranoid, or stressed about situations we don’t have control over anyway? Hindsight tells us that our preoccupation about things outside our control was very much a waste of time.
Or does it? The problem with this idea is that it assumes that what see in hindsight is all that was needed to produce the outcome. Just because we cannot see the role paranoia, worry and stress play doesn’t mean they didn’t play a role in producing the desired outcome. The emotions we feel exist for a reason. Emotions are high-level signals our mind uses to guide our responses to situations. They are signals used to solve complex problems in our environment. Emotions are not unfortunate byproducts of an otherwise calm process, they are critical ingredients to arriving at outcomes that matter. Evolution would not have equipped us with emotions if they didn’t assist in producing outcomes we need.
And so philosophy is interesting in that it notices fundamental truths about life, and I am big proponent of paying attention to those patterns. I also believe that paying attention to those patterns can dramatically improve one’s life, but ONLY if we are thinking about the directionality of information correctly. Noting the calm after the storm does not preclude the need for the storm. If you want to execute on your life’s philosophy you need to understand that patterns are not to be used as recipes, but rather signals as to when you are on the right track. What we see in hindsight is not the instruction set for getting there.
Government
Another example is government, which has perhaps the most dramatic impact on our everyday lives. If we think beyond our personal and professional lives we enter into the realm of social responsibility. We don’t just want what’s best for ourselves, we’d like to think we can improve the world at large in some way.
After all, I am very fortunate to have the opportunity to run a business, write my own thoughts freely, create things like podcasts and embark on life as I see fit. Most importantly I am healthy. I want this kind of opportunity for everyone. If I see a homeless person on the street, or masses of people getting evicted, or some injustice happening I want to speak up and do something. We all have different degrees to which this is true, but I think everyone naturally wants the world to be a better place; even though we may not all agree on what “better” means.
And so beyond our own abilities to affect change we hope there are systems in place that help people generally. Social programs that assist in alleviating homelessness, a healthcare system that makes medicine and therapy widely available, and outreach programs that get people back on their feet. As an outcome I think we can all agree these are good things for a society to have.
But if we start asking what the government should do to help produce these outcomes we can easily end up falling into the trap of mistaking the pattern for the path. For example, if we want to alleviate homelessness it seems obvious we should have funding directed towards setting up shelters. But is this correct? How does the pattern of not being homeless arise? Perhaps the shelter is a starting point, but perhaps it’s also a crutch that is too easy to lean on. Just like bird flight, to not be homeless is to have a number of things converging. There is self-reliance, your own ability to create value, add to the economy, and improve your station in life. There will undoubtedly be cases where shelters do help, especially for those in immediate need. But the notion that a shelter is the opposite of homelessness is far too simplistic.
It might seem like a somewhat contrived example, since obviously there is more the government does than just direct funding at shelters. But here’s the point. There is nothing in the pattern of homelessness, or its opposite, that can tell you where funding should be directed. The successful outcomes of individuals is not a result of some well-defined path, which is what you would need to direct funding. Recall the property of multiple realizability. What matters is not how someone got there. In fact it can’t matter, because we cannot know the path, narratives and “life stories” notwithstanding. All we can know is that the outcome itself, having a roof over one’s head, is a signal that indicates a better station in life. It is a good outcome.
This has massive implications for the role government plays. In order to pass bills and direct funding there must be some narrative attached to where those funds are going and why. But most of that narrative will be a story that suggests the pattern is the path. One must assume they understand *how* outcomes are realized in order to create programs and systems that make things better. This is a problem.
Another example is government injecting money in an attempt to stimulate the economy. The government noticed a pattern that more money in the economy means a healthier system overall, with more jobs, better inflation rates, etc. And it’s true, these are all markers of a healthy economy. So it seems obvious that if we want to achieve a healthier economy we should just add more of the signal. Inject money into the system.
But more money in the economy is supposed to be a result of a genuine cycle of supply and demand. Companies have something to offer, and a significant portion of the population wants it. If demand rises naturally, companies bring more goods and services to meet the demand, and prices adjust. All these signals propagate throughout the market and investors act accordingly. The problem with injecting money artificially is the same issue we saw with the nutrition example. It confuses the pattern for the path. Injecting money doesn’t consider how a surplus of cash is generated in the first place. Injection of cash sends false signals into the system making it appear as though there is more demand than there is. There is great risk in confusing the pattern for the path.
Education
Perhaps the most blatant infraction on confusing the pattern for the path is the academic narrative, specifically the one attached to advanced education. University is there to provide a foundation to enter the real world. A vetting program for businesses to use to filter applicants.
But the entire academic premise is based on the idea that the pattern is the path. Everything we read about in textbooks are patterns that have been noticed by historians, writers, philosophers, engineers and scientists. Everything taught is a set of concepts or principles that have withstood the test of time. The American Experiment as a model of a republic, France’s fight for liberation in the French Revolution, the Abolition of slavery in Britain and the US, the Opium War in China, the first women voters in New Zealand, Hamlet by Shakespeare, “Für Elise” by Beethoven, Stoicism, Nihilism, Marxism, Rationalism, the invention of Motion Pictures, the invention of the airplane, the compass, the printing press, the telephone, the lightbulb, Einstein’s General Relativity, Darwin’s Origin of Species.
All these topics have deep work attached to them telling us a story of their core properties; what makes them important points in time. But school doesn’t stop there. School suggests that by studying these works we will not only know more about them, we will have what it takes to extend them. To cast their patterns forward.
But every work of writing, art, philosophy, invention and scientific theory did not have a “path.” It was realized despite massive epistemic uncertainty. The outcomes of all these examples were and are multiply realizable. There is nothing in them that can be cast forward. We can improve on existing constructs, but we cannot make the next discovery or leap forward based on the narrative attached to “how” these things came to be.
Business
As a final example, businesses make this mistake all the time. In their quest to become efficient machines that generate revenue they look at what did and didn’t work. They run A/B tests and market trend analyses to see what actions supposedly led to what outcomes. But these patterns, while telling of what markets are up to, do not tell us how to construct the solution going forward. They cannot tell us how to operate within the market. To see the patterns in the market is not to understand the journey that produced those patterns. Market patterns are multiply realizable. The pattern is not the path.
SIGNALS INSTEAD OF REASONS
So if we cannot cast forward the patterns we discover what does this mean? Should we stop studying airplanes? Should we give up investigating nutrition and medicine? Should philosophy be cast aside? What about government? Should we strip society of politics and bureaucracy? Should we eliminate advanced education? Should businesses stop looking at market trends and building their models?
Perhaps. But remember, humans have been making progress all along. Despite our narratives we move forward. We improve airplanes, we come out with new medicines, we write about new philosophies, we get some people off the streets, we invent quantum computers and we enter new markets. But the argument here isn’t to stop looking at the pattern. As I stated previously, we should be looking at patterns and learning about the properties of things we deem important. The argument is about how these patterns should be used. Instead of intervening under a naive premise that we know how to fix or improve things because we can see the outcome we want, we should instead use patterns as high-level guides to signal when we are on the right track.
In engineering this means embracing trial-and-error instead of “best practices” and rigid rules. Good engineers don’t think using strict logic and nerdy adherence to engineering principles, despite the image promoted by schools, hiring managers and Hollywood. Good engineers are messy, experiment frequently, build naively, and let great solutions emerge. They study things to know their attributes but they don’t assume what exists in industry now is what should be cast forward as the next step. What’s more risky, locking things down with rigid known practices so you cannot adapt or embracing the unknown so that the next discovery can be revealed?
In nutrition or medicine, let’s stop cherry-picking studies to support some health narrative. Statistics are capable of telling any story. The data can be narrowed, simplistic methods used, and interpretations layered-on to suggest the causal “how” has been figured out. In reality, very little about the path from input to output can be uncovered. Really we should say “recovered” given multiple realizability.
The paths that gave us the patterns we see in healthy individuals are not there for us to study. Suggesting we should ingest the patterns we see in healthy individuals to better our health is speculative at best. There is nothing in the scientific or statistical arsenal that can change this. The opacity is absolute. What we CAN do is look at what has survived over millions of years. One of the most scientifically rigorous things you can do for your health is adhere to what your ancestors did. The reason this is far more scientific than some statistical study is because true science works by surviving years of refutation. This is why Occam’s Razor should be followed. It’s not about providing the simplest theory, it’s about using the simplest theory that survives, and simple things can be tested.
In philosophy, rather than tying our lives to the properties seen in wise individuals we should instead see their attributes as goals to achieve. They are signals of what is possible; beacons that tell us when we’re on the right track. The character attributes of a sage are not there to dictate the path to becoming a sage, only to allow our multiply realizable lives to converge on something positive, as long as we’re paying attention. And that is key; paying attention.
In government we should only keep intervention at the highest level. This isn’t a political opinion, it’s a scientific and philosophical one. Having any governing body reach too deeply into the process meant to converge on solutions will stop that system’s convergence. The intervention will retard the ability of paths to be attempted at a more local level. The incorrect assumption that the patterns noticed in society are themselves the paths forward is deeply flawed. It doesn’t align with what we know about how solutions emerge out of complexity. A given level of governance can do wonders, but cross that line of intervention and catastrophe is inevitable. Government should use patterns as signals for when things are going well, not as recipes for making things “better.”
In education this means dropping the false narrative that what we learn can tell us how to achieve things. The notion that graduates have been prepared to take-on the challenges of the real world, that their knowledge serves as a foundation for making society a better place, is borderline ridiculous. It’s a nice sounding story, that makes universities a lot of money. But it doesn’t hold up to how information flows and how new things get created.
Instead we should understand any education as a time to learn about the fundamental properties of the things discovered by historians, writers, philosophers, engineers and scientists (keeping in mind, There Were No Giants, Only Shoulders). Learning about those properties is not to learn *how* but to learn about the signals the environment makes available. Instead of pretending causality is something that can be teased out of complex tasks, let us acknowledge that we can never such information. Let us instead focus on high-level processes that converge to good solution by paying attention to signals. This is what education should be about. Perhaps project-based learning is a close proxy to this.
And in business, organizations should learn to redefine their risk in terms of adaptation. Thinking the patterns that have worked to date can be expended into the future is problematic. It fragilizes companies into rigid processes and policies that cannot-course correct where and when needed. We need to keep teams multiply realizable, allowing them to find solutions we never could have seen coming. Because if your company doesn’t, it’s only a matter of time your competitor does.
