Fun fact, when it comes to software product development, statistically you more likely to fail at building digital products than you are at marriage, unless it’s like your third marriage, which has a divorce rate of around 75% to 77%. In this article, we shall explore why software product development is so hard.
(As an aside, and it could totally just be me, but if you are walking down the aisle a third time, I think you have bigger issues to deal with than grasping the nuances of software product development, but I digress.)
Software Product Development: The Basics
Ideally, software product development should be a repeatable process that produces software with a set of features tailored to meet the needs of a particular market. Sounds simple enough right?
Except for that bit about “a set of features tailored to meet the needs of a particular market”; if software product development was indeed an easily repeatable process, the field of product management wouldn’t be a “thing” and the rate of software product failure wouldn’t be north of 50%.
To be clear, software product development is hard. In fact, as far a professional discipline goes, I’d argue it is one of the hardest out there. This isn’t to diminish endeavors like medicine, structural or automotive engineering in the least, but the advantage those fields have is that there is a combination of laws (of both physics and criminal), regulatory structures, and standardized education that ensure a basic minimum standard of competency and public safety.
Why Is Software Product Development So Hard
Software product development is hard for three main reason:
- As we alluded to above, there are no standardized education or regulatory processes that governs software development, so you have no guarantees of minimal competencies.
The open nature of software is a double edged sword, so while anyone can build software, there are no assurances that anyone can build software competently. While great advances have been made in software, the following piece of internet lore lays bare the challenges of software product development compared to other mass produced consumer offerings.
If automotive technology had kept pace with Silicon Valley, motorists could buy a V-32 engine that goes 10,000 m.p.h. or a 30-pound car that gets 1,000 miles to the gallon — either one at a sticker price of less than $50.
Detroit’s response: “OK. But who would want a car that crashes twice a day?”
Symptomatic of the difficult and unbounded nature of software development are the myriad of development methodologies and books touting the ideal way to build software and that you are “doing it wrong”, but the truth is that you cannot simplify software development by imposing simple models on it.
- Problem identification and solving is fraught with risk. Accurately identifying a profitable problem to solve and then figuring out the set of features a product needs to have in order to solve the particular problem is attempting to hit a moving target.
- Which feeds in to the next reason, and that is because software product development is governed by the laws of chaos and by definition, is fractal in nature. we cannot simplify software development by imposing simple models on it So incompetence in one phase of product development impacts other phases. We shall explore this further.
Software Product Development And Chaos Theory
Software developers and product management professionals throw around the term “iterative” in reference to various agile software methodologies to describe what you could simply call recursive, but to stop there sells the complexity of software product short.
To truly understand the challenge, do yourself a favor and read The Chaos Model and the Chaos Life Cycle as the ideas that follow come from this paper. It eloquently encapsulates the challenges of software product development in ways I could never.
- In order to understand software development, you must understand the flow of an entire project; how to write each line of code, AND how one line of code relates to the whole project
- Software development combines a simple problem-solving loop with fractals to describe the structures within a project (or perhaps, more aptly described as the project within the project), the complexity of software development as well as many of the issues encountered during software development,
- The linear problem-solving loop:
- Stage 1 – Problem Definition: Identify a problem, outline requirements/constraints. Need to make sure you you are solving the right problem
- Stage 2 – Solution Development: Devises a solution to the problem. Does not the solution solves the problem if the problem definition is not correct
- Stage 3 – Solution Integration: Involves the deployment and dissemination of the solution/work product depending on the phase of development
- Stage 4 – Status Quo: The new, improved status quo based on the integration of the solution.
- This is where the fractal nature of development becomes apparent as the linear problem solving loop applies all the stages of a project (as well as at the corporate operating level).
- Fractals imply a very specific scaling in a recursive relationship as the same expansion applies to each level. Each level of software development is composed of all of the levels below it and so each level repeats the structure of the next level down.
- The top levels deal with a few large issues,
- The middle levels deal with more mid-size issues
- The bottom levels deal with many small issues.
- Fractals imply a very specific scaling in a recursive relationship as the same expansion applies to each level. Each level of software development is composed of all of the levels below it and so each level repeats the structure of the next level down.
Where Software Product Development Can Go Wrong
Software development is a combination of technical and non technical development (aka – human factors); non-technical issues affect software development all levels of a project as things can go wrong in the problem-definition and solution integration stages of problem-solving loop on any level, from the operating level of a company to the “single line of code” level.
All the tools, frameworks and methodologies cannot mitigate problems that humans introduce into product development. In fact, I would argue, once you have a basic mechanism to filter out technically incompetent employees, managing the human interaction associated with software product development is a greater challenge than the technical aspects of most projects
Opportunities for product development disasters include:
- Solving the wrong problem as developers can:
- Ignore the problem definition and solve a different problem.
- Over-engineer or under-engineer a solution which either
wastes resources or doesn’t solve the problem. - Not agree on product goals or the scope of the solution.
- miscommunicate with users, preventing them from understanding what a program should accomplish.
- Fail to articulate the benefit of a particular technical solutions or
- Assuming that developers don’t make technical mistakes.
- Organizational politics can result in :
- The rejection good technical solutions.
- The elevation of the “best” developers into management positions with disappointing results as software developer are not typically trained for management positions and many don’t really want the responsibility.
- Assuming that conflicts within a project can be resolved by technical authority or leadership or some new technology.
