My perception of complex problems is that given the examples of any one particular moment in time, the situation can be easily dissected, analyzed, documented, and understood. Take a sample from the next moment in time and the same is true. Try to combine that collection of understandings into a generalization that can be applied to other past and future points in time and the problem suddenly becomes complex.
One of the observations we get from agile development is that solutions will be more correct, given the flexibility to adjust to customer needs versus following a previously defined historical specification. Agility is the ability to adjust to change.
Therefore, complex problems should be solvable by solutions that are simple and agile. Solutions do not have to be complex.
We run into challenges designing agile solutions, though. Many traditional solution design tools often call for static process flow diagrams, swim lane control charts, concrete data models, class diagrams, etc. I think that some of the behavioral object oriented patterns give us some clues on how to introduce agility into solutions, but understanding when to apply those (and how to apply them within some technologies) takes creativity and experience.
I think that introducing that same kind of solution agility into human processes is also very challenging. Often, we want clearly defined instructions and flow charts to instruct individuals on exactly what to do. The only variation being a Madlib style fill-in-the-blank. Perhaps we need more "and then a miracle occurs" steps in our complex processes. And perhaps that's both acceptable and desirable in some processes.
Paul,
ReplyDeleteGood post.
Clearly, from your post here, you have an intuitive sense of something missing in how SW design/development is practiced.
Your observation that "Complexity = Agile Simplicity" is very insightful and is in fact a mathematically proven concept.
As an Electrical Engineer with career focus in control systems engineering and complex adaptive systems, one of the core principles I was trained on at university was, "manageable complexity" is achieved only via the intelligent construction of systems that leverage complexity by building small, manageable "controllable" building blocks.
These blocks then, when interacting together as designed, produce what is called "Emergent Complexity". "Emergently complex" systems are the only kind of complex systems that can be managed/controlled effectively as it grows in scale. Systems designed in this fashion allow the engineer to mathematically model, in a deterministic fashion, the behavior of those systems. If you can model it mathematically you can control it.
If the above sounds very much like what Object Oriented Analysis and Design techniques were/are trying to accomplish in the SW arena it's because OOAD came out of the failures of prior SW design techniques to produce manageable, controllable code bases.
The thing OOAD missed though, and still is missing, is the mathematical underpinnings for why OO concepts like encapsulation of data with function and polymorphism are effective.
Those mathematical underpinnings are well understood and reside in the field of Linear Systems Theory mathematics and the related "Control Systems" theory and methods. These allow a designer to precisely and in deterministic fashion create controllable physical systems and controllable software systems. (See http://en.wikipedia.org/wiki/LTI_system_theory and http://en.wikipedia.org/wiki/State_space_(controls))
These are the techniques that allow engineers to design controllable systems that, for example, make unstable airframes for an advanced aircraft stable and controllable in flight.
The problem that still exists in the software industry is the majority of SW practitioners have no formal training in how to design software systems using the knowledge that has now existed in control theory mathematics and engineering for over 100 years.
Agile development and OO principles represent all the tools needed to create stable, manageable and flexible software systems that can grow and change over time.
But without the rigor of the mathematical techniques, and expertise in applying those techniques, SW practitioners are stuck using the "craftsman", "Then a miracle happens" approach to building quality SW systems. OO and Agile make the "miracles" more likely, but they still are not sufficient to deterministically predict, and then guarantee, quality outcomes in SW.
Most who write software today, even if they embrace Agile and OO, are still "craftsman" versus "engineers". Most degrees ever granted in software would be more accurately titled as a "Bachelor of Arts" versus a "Bachelor of Science".
Until the software industry embraces math/engineering-style rigor, and moves from being a "Then a miracle happens" craft to a truly engineering-style discipline, software will not progress to the next stage, "state in space", of it's evolution.
Cheers,
Roger
Also if you really want to check out an excellent series on complexity go here.
ReplyDeletehttp://nirmukta.com/complexity-explained-the-complete-series-by-dr-vinod-wadhawan/
Roger, thanks for the great comment! That was a very articulate way of describing something that I've always found challenging in software quality. I think the fact that software doesn't waste raw materials when it's done wrong (as would a traditional field involving physical products) and the low barriers to entry into the field (we teach gradeschoolers to program) are also major drivers of low quality software solutions. You're right that the culture has to shift in order for the engineering principles to further develop.
ReplyDeleteIn my post, I was also thinking about business operations -- another place where I personally think engineering principles could play a much stronger role. :D
Thanks again for the great commentary!
nice one! The first agile hacker of modern history to stress simplicity was Henry David Thoreau. He invented the phrase: "simplify. simplify. simplify." and then later changed it to "simplify!" ... he also had a neckbeard! :-)
ReplyDelete(http://thequickword.wordpress.com/2014/01/23/on-the-origin-of-the-modern-neckbeard-hacker/)
:-)