Historical Perspective
Michael Polanyi, a Hungarian-British polymath (1891–1976), introduced what later came to be known as Polanyi’s Paradox in 1966. Polanyi started his career as a distinguished physical chemist before turning to philosophy, bringing a scientist’s insight into how knowledge actually works in practice. In his book The Tacit Dimension, he famously wrote “we can know more than we can tell”, referring to the tacit knowledge that underlies many human skills. For example, Polanyi pointed out that we recognize familiar faces effortlessly without being able to list the exact features or rules that distinguish each face; a nod to Gestalt perception and intuition.
He illustrated the gap between knowledge and articulation with everyday instances: “The skill of a driver cannot be replaced by a thorough schooling in the theory of the motorcar,” he noted, just as one’s intimate know-how of using one’s body is entirely different from a textbook understanding of physiology. In short, there is always a residue of know-how that we carry in our minds and muscles that we cannot fully put into words.
In a time when many thinkers believed all knowledge could be neatly defined and broken into rules, Michael Polanyi pushed back with a simple but powerful idea: we know more than we can tell. He argued that intuition, experience, and personal judgment are essential parts of how we understand the world. For Polanyi, knowing isn’t just about facts and logic; it’s also about the unspoken, hard-to-explain feel we develop through doing and observing.
Relevance to Software Engineering
Although formulated 60 years ago, Polanyi’s Paradox is alive and well in modern software engineering. Despite the field’s basis in logic and code, much of what good software developers and teams do resides in tacit knowledge rather than explicit rules. In fact, studies suggest that only the “tip of the iceberg” of knowledge in a software organization is explicit and documented, perhaps ~20–30%, while the vast majority (70–80%) is unwritten, experience-based tacit knowledge.
In software teams, explicit knowledge (documents, code repositories, formal processes) is just the visible tip. The bulk of “what we know”, such as the skills, experience, intuition, team culture lies beneath the surface, undocumented but crucial. This tacit layer includes things like unwritten best practices, intuitive design sense, and the many gotchas one learns only through hard-earned experience.
This means that a software team’s most critical understanding often lives in people’s heads and habits rather than in manuals or comment blocks. Seasoned engineers accumulate a deep reservoir of intuition about architecture, code quality, and problem-solving approaches that is hard to articulate formally. They just "know" certain things which is a direct reflection of Polanyi’s Paradox in the tech world.
Polanyi’s Paradox in Software Engineering
In day-to-day software development, Polanyi’s Paradox shows up in numerous ways. For example:
- User Interface Design: Good UI/UX designers often rely on an instinctive feel for what is intuitive to users. There is no complete rulebook for “easy to use.” Much of it comes from empathizing with users and leveraging subtle design sensibilities developed over time. A designer might not be able to fully explain why one layout “just works” better than another, because they are drawing on tacit knowledge of human behavior and aesthetics learned through many projects and feedback cycles. This know-how stands in contrast to explicit guidelines (like style guides or usability heuristics). Those help, but the "artistry" goes beyond what’s written down.
- Debugging: Tracking down a complex bug is as much an art as a science. A veteran developer can often zero in on the likely cause of a problem quickly, guided by a “gut feeling” from past debugging experiences. This skill of knowing where to look in thousands of lines of code or which log hint to pursue is typically acquired through experience and not easily codified or taught. Two different engineers might solve the same bug via very different thought processes, neither of which is fully documented anywhere. Debugging knowledge lives in their heads as pattern recognition: “Ah, I’ve seen something like this before and it might be due to X.” Such intuitions are hard to write into a step-by-step troubleshooting guide.
- Knowledge Transfer in Teams: When a new developer joins a software team, they often go through an onboarding period that involves shadowing others, pair programming, and code review. These practices recognize that a lot of the team’s knowledge is “tribal knowledge”, accumulated wisdom about the codebase and conventions that isn’t in the official docs. From knowing the historical reasons why module Y was built a certain way, to understanding which team member to ask about database quirks, these are things newcomers learn through social interaction.
Much of what senior engineers pass on to juniors is tacit: it’s storytelling, mentorship, and shared experience. If a key developer suddenly leaves, they take a trove of tacit knowledge with them, and it’s often non-trivial to fill that gap. Attempts to capture everything in exhaustive documentation often fall short, because you can’t foresee or codify every relevant detail. This is in spite of the best efforts of Program Managers and tools like Jira, Confluence, etc. Indeed, trying to convert all tacit knowledge into explicit form is “fundamentally flawed due to the very nature of tacit knowledge, which is inherently personal, context-dependent, and difficult to articulate.”
These examples show that software engineering is not just about formal specifications and algorithms; it’s a human craft. The most effective development teams leverage tacit understanding in architecture decisions, code readability, and anticipating user needs. And when this tacit element is missing, say for example, a team only follows rigid checklists without any personal intuition then the results are often mediocre. Polanyi’s Paradox explains why certain programming expertise can’t simply be “transferred” by reading a book.
As a consequence, strategies like code reviews, pair programming, and apprenticeship-style learning are crucial in tech because they help share the unspoken wisdom. Conversely, over-reliance on documentation has limits: no matter how many pages of design docs you write, there will always be nuance that new engineers must pick up by working with others. In short, a great deal of “what developers do” cannot be fully captured in code comments or process manuals. They know more than they can tell.
The Future of Software Engineeing
So all of this should make software engineers feel pretty good about their current job security, because their job is more than just code generation and anyone who has used AI tools knows that even though they can speed up their producitivity tremendously, these tools fall woefully short in many areas primarily because of this tacit knowledge engineers have built up. This is despite some CEOs making announcements that they are replacing engineers with AI. These annoucements are generally made by people who have never built and maintained a large code product or if they have it's been decades since they did so. Furthermore, announcements where they say AI is writing X% of their code are highly suspect because:
- It's unclear what they are counting in the X%. Is it agents writing big chunks of code based on prompts? As opposed to how much of the X% is just code completion. Because, for example, as you type something into Google Docs and it suggests the next word or phrase, do you really think that AI wrote X% of that doc because it's doing autocomplete?
- They fail to recognized the tacit knowledge that goes into code architecture.
However, this is the current state for software engineers. Thinking that this will be the future and won't change is just engaging in "copium", which I see a lot of on LinkedIn. For example, take a look at the replies to the CEO of Zapier on LinkedIn here where he states that AI won't replace jobs. There's a lot of wishful thinking in the comments that things aren't going to change that much. But this is the current state and people are not internalizing the exponential change that is happening in AI.
Additionally, the Polanyi's Paradox of tacit knowledge that is part of software engineering isn't a formidable problem to solve for AI agents and doesn't even require a huge technological breakthrough. So how do you solve the fact that an AI agent in a program like Cursor or Claude Desktop doesn't have all of an engineer's tacit knowledge? Well you give access to all that knowledge to the AI. So access to Slack conversations, entire organization repos, Microsoft Teams meetings, project management notes on customers, stand up meeting recordings, Jira, Confluence, whatever the organization is using to capture that history. If that is done, then you have an AI agent or agents that has more organization and project knowledge than any engineer that works there.
The requirements to do that are larger context windows as extended information, agents that can operate effectively over that context memory such that agents can remember over sessions, and access all of that data. Infinite memory agents are within reach now. There are many possible solutions to this now including MemGPT from Letta along with the frontier AI companies extending their context windows. The biggest obstacle will be helping organizations streamline all of their data and communications to be made available to AI agents. However, organizations are going to be very motivated to do this, because those that do make their data and commuications available are going to have a huge competitive advantage because of their accelerated pace of software delivery and better organizational decision making.
So this is not a big leap to make agents that have this extended memory ability and being able to maintain intent over these tasks. The biggest impediment will be organizations being able to adapt their data and internal processes. Does this mean that all software engineering jobs will be eliminated? No. Engineers who can orchestrate, manage, secure, and generally work with AI agents will be in demand. So those who are using AI as part of their current process and learning as much as they can about taking advantage of AI and looking to innovate around AI Engineering will be fine.
Unfortunately, there is the mindset of many engineers who don't bring anything more to the table other than their ability to pull a story off of a Jira backlog that was written by a project manager and complete that story and who don't think in terms of innovation or design at all. They work in a bureaucratic mindset of just task completion and generating just the code for that task and not in a mindset of generating ideas and innovating. This type of engineer who doesn't innovate and specifically doesn't innovate using AI will be gone and will not be able to find a job like that again in engineering. In other words:
If you are an engineer who is used to thinking like a robot, you will be replaced with a robot.
But I don't think that most engineers can be blamed for thinking of their job in terms of being automatons writing code. Many engineers ever since getting out of school have been conditioned through endless cycles of sprints, planning meetings, story points, and retrospectives to think this way. Also, in many organizations, there are layers of project managers and customer success people between them and the end users, which reinforces the idea that their end goal is code generation and not building a product for a user. In addition, in larger organizations, they may only work on a small functionality of the end product. But if they are going to survive, they are going to need to bring their human-ness to the job, their uniqueness based on their personal experiences and innovate and be creative.
Relevance to Building Better AI
So given the Polanyi's Paradox on engineering, what have been the implications for AI?
Polanyi’s Paradox has long been recognized as a core challenge in artificial intelligence. Let's go back to the beginning. In the early decades of AI (mid-20th century through 1980s), programmers found that many tasks humans find trivial were extremely difficult to specify to a computer, precisely because of tacit knowledge. As Polanyi himself noted, tasks like driving a car in traffic or recognizing a face rely on a vast array of subtle cues and intuitions that we execute without conscious thought. We can perform them, but we can’t easily enumerate how we do it. This posed a problem: how do you write a program to do something you can’t fully explain to yourself? Early AI could handle problems that had clear rules (like solving equations or playing chess to some extent), but it struggled with open-ended, real-world tasks. Polanyi’s paradox was identified as a major obstacle for AI and automation. Unless we have a complete, explicit recipe for a task, getting a machine to do it is extremely difficult.
A classic example is commonsense reasoning: even a toddler understands that a wobbling stack of blocks might fall, or that a snowman in the road isn’t a threat, but such “common sense” eluded AI programs because nobody could fully encode all the needed if-then rules. As one economist summarized, the tasks easiest to automate are those that “follow explicit, codifiable procedures,” whereas the ones requiring “flexibility, judgment and common sense, skills that we understand only tacitly, have proved most vexing to automate.” Indeed, for decades computers remained “less sophisticated than preschool-age children” at tasks like understanding natural language or navigating an unpredictable environment. This is Polanyi’s Paradox writ large in technology: machines hit a wall when facing the tacit dimension of human know-how.
The past decade, however, has seen much progress in AI, largely by finding ways around Polanyi’s Paradox rather than directly solving it. Since we can’t easily tell the machine what we know implicitly, we let the machine learn from experience just as humans do. In other words, “contemporary AI seeks to overcome Polanyi’s paradox by building machines that learn from human examples, thus inferring the rules that we tacitly apply but do not explicitly understand.” Instead of programmers hand-coding every rule, we feed AI systems with massive amounts of data (images, recordings, text, demonstrations) and let them figure out the patterns.
This is the approach of machine learning, and especially deep learning. For instance, rather than attempting to enumerate all the visual features that distinguish a pedestrian from a shadow on the road, engineers train a self-driving car’s vision system on millions of driving images and let it learn the concept of “pedestrian” by example. The AI essentially internalizes a web of correlations and features, a form of tacit knowledge, from the data.
AI can now capture tacit patterns by finding regularities in unstructured inputs. A striking success of this approach was in the game of Go: expert Go players often say they rely on intuition to judge a good move (they “know” a move is strong without being able to verbalize why). Early AI couldn’t crack Go by brute-force logic alone. But a deep learning system (AlphaGo) was able to absorb millions of human moves and even play games against itself, eventually acquiring an almost intuitive style of play that surprised human experts. In effect, AlphaGo learned the tacit principles of Go that even top players couldn’t articulate, and it went on to defeat the world champion.
Similar stories abound: large language models (LLMs) were trained on billions of sentences of human text and as a result, they learned the hard-to-pin-down rules of language, style, and meaning without anyone programming those rules in directly. These models can answer questions, write essays, or carry a conversation in a very human-like way, not because they were explicitly told how to do it, but because they statistically absorbed how humans communicate. In other words, they captured a slice of our tacit linguistic knowledge.
Interestingly, many of today’s AI successes still depend on how well we can communicate our goals to the system; the unfortunately named idea that we are all familiar with of prompt engineering. Because machines lack the shared human context we often take for granted, we’re required to make our implicit understanding more explicit than we typically would with another person. When working with a coworker, you might simply say, “You know what I mean,” and count on shared experience to fill in the blanks. With AI, that luxury doesn’t exist. Instead, we must carefully spell out our expectations, constraints, and context. Getting the AI to produce a useful output often involves trial and error, tweaking inputs, and drawing on our own unspoken know-how about what might steer it in the right direction. This interaction process is, in itself, a reflection of our tacit knowledge; how we choose to phrase a task, what examples we give, and which assumptions we clarify all stem from our intuition about how the system behaves. In this way, human users must surface their internalized expertise just to bridge the gap between what we understand intuitively and what the machine needs to be told explicitly.
The human is leveraging personal know-how about the AI’s behavior to coax the desired result. For example, when trying to generate a specialized algorithm or solve a domain-specific problem, a developer might instinctively shape their request to match what they know the AI handles well; perhaps by framing the problem in terms that are common in training data or avoiding ambiguous phrasing. They may not consciously think about it, but they draw on a kind of internal playbook developed through past interactions: what tends to work, what confuses the model, and how to steer it toward quality output. This back-and-forth of adjusting inputs, interpreting results, and refining expectations—reflects a nuanced, intuitive understanding that the developer might not even fully articulate. It’s a collaborative process that takes some experience by the engineer with working with the AI where the machine brings breadth of data and pattern recognition, while the human brings situational judgment and contextual insight. In this way, navigating AI systems becomes another expression of Polanyi’s Paradox: we rely on what we know but can’t entirely explain to get useful results from a machine that requires us to make that knowledge as explicit as possible.
Polanyi’s Paradox as a Cautionary Tale
Finally, Polanyi’s Paradox and the quest to encode (or exploit) tacit knowledge in AI bring to mind several cautionary tales and analogies. One is the legend of Frankenstein. In Mary Shelley’s novel, a scientist creates a sentient being but cannot control it or imbue it with the values and understanding needed to integrate into society. The creature, “detached from human values, brought chaos rather than order,” as one commentary puts it. This resonates with modern fears that AI systems, if created without sufficient foresight, might behave in ways misaligned with human intentions; not out of malice, but because we never explicitly told them the full story of right, wrong, and context. Polanyi’s Paradox suggests we might not even know how to fully tell them these things, since so much of our common sense and ethics is tacit.
Another example if you have watched enough Disney then you have seen Mickey Mouse as The Sorcerer’s Apprentice in Fantasia as a young apprentice who animates a broom to do his cleaning for him. Lacking the master’s wisdom, he cannot stop the broom as it relentlessly carries out his literal instructions, eventually flooding the place. This is a perfect illustration of the failure of specification. The apprentice knew just enough magic to issue a command, but not enough to specify boundaries or handle contingencies. In AI terms, he had no “off-switch” or nuance in his instructions. The story underscores the risk of viewing powerful AI tools as magical solutions without understanding their limits. If we deploy AI with the mindset of that apprentice and expect it to perfectly do our bidding when we ourselves haven’t nailed down exactly what we want. We shouldn’t be surprised when we get unintended results.
Already, we’ve seen real examples of this: an AI instructed to maximize user engagement on a platform might learn that spreading sensationalist content achieves the goal, to society’s detriment; or a chatbot given free rein to converse might start generating troubling outputs if the designers didn’t anticipate certain prompts. As one observer quipped, if you’re “not careful, like a dreaming Mickey, you too will suffer the consequences of unintended hallucinations” from AI. In plainer terms, the irony of Polanyi’s Paradox in AI is that we are building machines far more literal-minded than ourselves to handle tasks that we ourselves only understand implicitly.
Conclusion
Polanyi’s Paradox remains a guiding concept in understanding the limits of formal knowledge in both human and machine domains. Historically, it reminds us that even at the height of scientific rationalism, that "knowing" something is often inexpressible. In software engineering, it encourages the recognition that programming is not purely formal and that expertise can’t be entirely captured in textbooks and that engineers must adapt. And in AI, Polanyi’s Paradox is both a challenge and a driving force. A challenge because AI must grapple with tasks that we as humans struggle to fully describe, and a driving force behind the rise of machine learning to let computers learn the unspeakable parts of human skill.
As we integrate AI more into our lives, Polanyi’s insight also serves as a warning: we should be mindful of what we haven’t been able to tell our machines. Balancing explicit knowledge with tacit understanding, and combining human intuition with machine computation, will be an important element of buiding stronger AI. After all, paraphrasing Polanyi, we (and our AI) know more than we can tell.
No comments:
Post a Comment