09 September 2011

Agile, Architecture and other things

There are questions about software development I'm really concerned about. These questions raised from my experience being a software engineer, a software architect, requirements engineer and team leader.

One expression of these question is the following problem:
  • How can software engineering become exactly this, engineering ?
This question is not trivial. But first we have to ask, does an answer exist at all ? If there is an answer, it should be possible to compare software engineering with other domains of engineering, say mechanical or architecture domains. Often I try to make mappings from properties of the mechanical engineering to some of software engineering. But the special way of the existence of software - to be not bound to physical resources and to have multiple instances without additional matter or energy - may have impact on the metaphysics of software.

Now, one of this special properties of software is their potentially unlimited effect. Where mechanical devices are limited by matter and energy, software is completely free, not bound, only limited by its runtime, the computer. We learned how fast computer extend their capabilities. In other words, the effects caused by a running software can be very very complex.

This observation raises this question:

  • Is it possible to construct technical systems of any complexity level ? (The Constructible Hypothesis) Or - as contra statement:
  • In general, can technical systems only be grown applying evolutionary principles ? (The Grown Universe Hypothesis)

If the first question has an positive answer, the question above (existence of engineering for software) also has. These both questions also touch another aspect of build things by human: the aspect of control. To construct things means, that the creator has control over every static and dynamic element in the cause chain. The cause chains are arranged by logic reasoning, using basic laws. Here would be the place for an engineering technique for software. But exactly here begins the area of fog, because we know that not every thing can be calculated which means that a full control of any complex system of any complexity level may not possible. If calculability is equal to controllability is an open question.

But if the Constructible Hypothesis would be not true, the only other way of building systems would be to let them grow and arrange solutions by them itself. The evolutionary principle means, to provide just environment conditions (or requirements), and let the system grow. Not necessarily by the system itself like Genetic Algorithms, enhancing a software revision by revision but by human teams is also some kind of growth and evolution. BTW, this would be a place where I see agile and lean methods useful.

If we assume that the Constructible Hypothesis is right, one may ask how we could handle this. One way may be the onion model of control: a few basic laws provide the foundation to build up simple entities and their properties (like atoms from the 4 basic forces and quantum theory, for example). These entities form own rules on their level, from which bigger entities are build with der own rules (molecules, like proteins) and so on (cells, body...). Every level has its own rules, but based on the preceding level, and narrowing the space of possible behaviors. DSLs are operating on the same idea.

Now look at a cellular automata. There are also very basic rules, which generate a special behavior of the automat over time. In the game of life for example, there are patterns coming out which itself show behavior. The "slider" is such a pattern, moving over the 2D grid of the cellular automat. But there is one big difference to the onion model of control: the slider has not its own rules, its behavior is fully and only determined by the basic cellular automata rules ! That means, to control the slider and higher level patterns is only possible by the basic cellular automata rules and the initial conditions. In the onion model of control, you would have constructive possibilities on every level (you could construct different proteins to create a special effect on this level, for example).

From this, it is clear that if the cellular automata model (or let it call after John Neumann the Calculating Universe Model) is true, only the Grown Universe Hypothesis can be true. Otherwise, if the onion model is correct, than both, the Constructible Hypothesis and the Grown Universe Hypothesis have the chance to be true.

So these questions arise:

  • For control behavior of human built systems, is the Onion Model of Control true ? Or
  • For control behavior of human built systems, is the Calculating Universe Model true ?

For me, these are important questions, even if they are seem not that pragmatic :) But it is an important question to investigate what we really can do and achieve in principle in the domain of software engineering. Of course this is no exact analysis, but it should illustrate what I'm thinking about.

If some has a hint about already available work or text from philosophers, computer scientists and so on I would be glad about a pointer :)