Steam Gestalt

At this time, I am productive in drawing. This is cool, because in the life of Software Engineer or even Software Architect, the creative part of my person has no possibility, or at least not enough, to come out. If my information is true told from many people, then I should compliment Google on let their Software People work for some fix part of the whole time on their own ideas. I am shure that gives much value back to the company, even if this benefit maybe not measurable and described by any numbers.

But back to my work. One of my friends told me about a game embedded in a fictive world where is no electricity, but steam instead. I am very excited about this idea, so that I had to draw a gestalt at once (originally I want to ink it, but unfortunatly there is no time in the moment).

So It seems, that I got an big interest on designing gestalts, their equiment, and gesture. Of course I am far from perfect, I need much more experience. But I have a sketch block now, and where ever I got some idea, I will take the pen.

UML in the Functional Programming world

UML is a well known standard notation in the context of objectoriented design and programming. It allows to define classes and their relationsships. It makes it possible to display Use Cases and the Activities performed for them. They way how objects interact together can be visualized in more than one way.

The problem is: it's all about objects. But we are in the age of a new rising star: the functional programming. Here we deal only with functions. There is an input, there is an output, no state. Just a rule to map the input to the output. No behavior. Just a chain of input - output mappings. It seems that UML doesn't fit very well. Some languages interpret functions as objects, but in the end that doesn't help. Functions are a mathematical construct, they are best defined in the way we learned in our mathematics education.

Let's go a step back and ask: what is the goal if we use UML ? It is used during the design phase, so one goal might be to communicate architectures. An other goal might be to write down design elements of the software to build, on a level which allows to use abstractions and to suppress unnecessary details. Of course, abstractions and communication is not only about data structurs, but also about what is done with this data, and by which entities. And in the end this is the nature of using UML: finding out and document what is needed, what happens, and what is get.

Ups. Thats reminds us of functions. There must be something wrong. So, again, one step back: how should we document software ? Actually, the main purpose of any software (which is not used for controlling some device, because their nature is just to be an extension of mechnics) is to provide information. Information aggregated from other informations, as well as calculated, filtered, selected and transformed information. The nature of information is to be data, which have an intention: a question to answer, an goal to achive. If we describe the data and their intention, i.e. the information, and their origins and lifecycle, we have in fact described what a system is. It doesn't matter if the technical implementation is done by functions, or by interacting objects. Thats a matter of the detailed technical description, which is important only for a small set of people (and it has to be decided for every project if the code is the documentation or any detailed UML diagrams or equations are needed). Or in other words: it is in matter of level of detail.

Let us note two additional observations:
1) The data comming from or going into the real world can be modeled better by objects than functions.
2) The context as part of the system environment is the source of intention (what will I achieve, why I do something).

Bring this all together, we can see that UML helps to document the data and their intention and their lifecycle via UseCases (maybe annotated with activity diagrams, for context and intention), Collaboration diagrams (for lifecycles), Class diagrams (for the pure data). In this sense, UML should help document software independently of the programming paradigma used with a sufficient level of detail for the most cases. And if more detail is needed, UML is appropriate for the OOP and mathematical equations for FP (and PI calculus for heavy concurrency systems).

Requirements: The Picture of Software

As we all now, all software development starts with the wish or the need of a customer. The customer has a problem, that he (or she) wants to solve. Then it might be, that the customer has also the idea of a possible solution in mind. "If I had .... I could..." or "if I get ... then I could" and other signs of such ideas is well known.

In most cases, this idea is embedded in the domain of the customer, it adresses data which are key parts of its domain. Or it is related to actions performed in this domain. In either way, it has no technical nature. (It may be that the customers domain contains technical things, computers, machines, but then they are part of the domain as all other and therefore are natural to him.)

Then there is the other case: the customer has no idea, just a problem. But then, he has at least a goal, which means there is an idea of a situation which he (or she) wants to achieve.

Now it is our job as software professionals, to draw a picture of a system, which, beeing embedded in the customers world, would solve his problem and let him reach his goal. If we have this picture and it its approved by the customer, we could build the system and the customer will be happy.

That sound's easy but it is not. The customer has a sound picture of his domain in mind, but a weak imagination of the new system (well, he is no technician...). And this new element must fit in. The software professional on the other side has a strong imagination of "his" new system (what a wonder...), which must be embedded in the - from his point of view - weak picture of the customer domain. It is easy to understand, that the weak parts are in danger of beeing forced to fit and therefore getting distorted. Now the problem is obvious: all parties have different parts of the picture that are weak. In consequence, the resulting overall picture of the customer and the software developer will differ necessarily.

We know, that an iterative process for software development is the best solution. By creating prototypes and mock-ups, things that can be "touched", the picture comes out of the head of all parties and can be experienced directly by each other in early project stages. And because it is not feasibile to come through this in one step, many steps are done, where ever news prototypes and mock things display and stress different aspects. But in the sum, over all iterations, the picture should become more and more equal for all parties. The picture of software becomes clear and sharp, we've got consensus. (BTW, this would be the planing guide for iterative processes: touch all constitutive aspects at least one).

Sounds easy. But software is developed in teams, and not all members of the team can talk with the customer directly. And more worse it is not always possible to talk with one customer, there may be many stakeholders. So the picture in mind must be communicated via documents, but how ?

To find an answer, think about a patchwork image, and the task is to fill a empty place. So the first question is: the shape. Which shape I need to fill ? Related to our software project, this would be the interfaces, logical, functional and physical. Then, one might look which colors would match, then which things the part should show to continue shapes or patterns of the whole picture, and so on. That means: for communication via a documents, one should take certain point of views to the domain and the new system, explain them and why the part fits in the hole. One chapter one point of view, with the goal, to come close to the picture of software.

If all important aspects are described like this, then there is a chance that many people which read the documents become a more ore less comparable picture of software (and its environment) in their minds and the right software at the end.

BTW: There are people, which can design such missing parts of a picture handling all aspects at once - we call them artists.

Started my work on Philosophers Erlang SVG Server

It is time now to start an project which is in my head a long time: the Philosophers Erlang SVG Server. A good friend helped be to get through the kick-off and to realy do it, so now I have set it up as an Google Code project. The beginning is always the hardest part, and it is done.

Details I am writing in the wiki of the Google Code. For now, there is not much to read there, but I hope it will grow steadily. Also, there is not much code yet, but Eclipse is set up with Erlang plug-in Erlide and direct contact to the SVN of Google Code, so, it can really really start.

Loving SysML

Since some weeks I am working with SysML, the special UML profile for Systems Engineering. It uses the most important diagrams of the UML, like Use Case, Activity, Class Diagram, and introduces some additional, like Block diagram and Requirement diagram. The more I work with it, the more I like it because for system engineers it has some advantages over standard UML 2.0:

• The concept of <<blocks>> is more flexible to use. It has class semantics, so it can have attributes like subblocks and parts, and the aggregation and composition semantics. On the other side, a block can be a broad set of things: a feature, a software or hardware component, an functional element. A block itself can be bind on parameters, equations, constraints and other. Therefore, it is a little bit more flexible than the <<component>> of standard UML. And this is, to what a systems engineer is often faced to: very differenct functional elements, which can be a physical thing, a device or a logical set of harware and software, and which are not easily match the classic categories of pure software engineering.
  
• In SysML you can model requirements. You can relate them to each other, for example with <<refine>> You can bind them on Use Cases, on Activities and Blocks and make your solution simple traceable. Using the right tool, you can exchange requirements with Telelogic DOORS (R) and other requirement management systems. 
  
• By using the <<allocate>>, it is possible to relate elements of a conceptual level to elements of the implementation level in a simple way. So you can describe which element in the system is introduced ("allocated") because of which feature described by the concept or requirement. The consequence: bidirectional tracability is easy to maintain.
  
• These simple but powerful constructs make it easy to model product lines: because a block can be many things, it also usable for setting up feature diagrams. Applying some additional stereotypes like <<optional>> or <<variationpoints>> to blocks and using the <<allocation>> and <<satisfy>> mechanism, a variant configurations can be documented and can be mapped to the implementation level. And that's it !

One word about tools: UML or any profile from it will be used by using a special software. Of course, sometimes you write on whiteboards or paper. But for real-world complete models, a software is needed. A good software ! If this has some useful features and good support of the Metamodell of UML, working with UML as a tool comes close to what CAD is to mechanics engineering: the model will not be just a collecton of pictures, of static documentation, it will be a complex database of knowledge, which can be modified, worked with, to which queries can be addressed.

For example, I have set up a empty UseCase diagram, and if I want to add assozations, or I want to find out which model element is connected to which other (a "query"), I drag the elements in question into the diagram. Now I have all functions needed in the context menu of the elements, and can perform my task. After this, I make the diagram empty, which does not delete the model elements, though. The empty diagram is like the texteditor, or the query interface for a database.

What I want to express is, that the way how one works with UML is influenced hard by the comfort and strengh of the software in use. Using UML as a language, as a knowledge constructor and maintainer, not as just a notation to document things, is possible, if the software is good, and only then UML let access the engineer the real power of itself.

Requirement landscape with Croquet

Requirement Engineering is something which I'm looking at for many years now. And I always was and be interested in this. But one thing I am still waiting for: the visualisation of requirements.
So one might ask: visualisation ? Requirements are texutal, semantic-rich objects, nothing which would come close to that what is needed to making charts or some interesting 3D pictures. Well, this may be right. But requirements have structure: they are related to other requirements, they build groups. In addition, they have an inner structure, if they are built using methods available for professional Requirement Engineering. Visualizing requirements related to such structure aspects would be the same as visualize the network or graph of requirements. Clusters or too sparse regions of the requirement network should be visible at once.
But I think, there could be done more. The set of requirements should describe as complete as possible the system to build or to engineer. This set as a whole should be able to induce the picture of software. By reading the requirements, one after one, this picture becomes existent in the mind of the reader, and its shape and details become more and more complete with every requirement read.

But: wouldn't it be nice to visualize exactly that process ?

In the result, this visualization would show which requirements covers to which extent which aspect of a system. For example, the system would be a landscape, or a city street plan, and the size and position of the requirements show there contribution to the whole (system).
Since a while, I asked myself if Croquet would be a optimal tool for such visualizations. The space metapher and even more the collaboration possibilities of Croquet seems promising for such visualization experiments. Unfortunatly, I have no time, so I really hope that some other will do some experiments in this direction.
This is I would do:

• defining a meta-model of requirements, to adress the aspect of their inner structure, just like EMOF for conventional models. I mean not a semantic model, only a structural model (Conditions, quantors, action, object, subject, test criteria ...)
  
• defining a meta-model for describing always needed aspects (=questions) for a software project, let's call it system meta model. You could imagine this as a checklist of questions need to be answered in most projects.
  
• defining a semantic meta model, to describe the relations between semantic in general, requirement parts, and the system meta model.
  

That are only raw ideas. Too bad, I had no time.....

Decisions of a Software Architect

In the past, it was not so clear for me what are the things which are constitutive for the work of a Software Architect. This is especially true if the question comes up how to distinguish a Software Architect from a Software Engineer. At the moment, my work for the company is made up of UML, pure architecture design and requirement engineering. From this, I remarked some things which may be surprisingly important aspects of the work of a Software Architect. I will call them "surprising", not because I never thought about them, but because their real effect and importance for the practice hit me strong in these days.

If a new system has to be made, it all starts with requirements. Everyone know that. Some people regard requirement engineering as collecting a bunch of text or a big set of items in a database in order to describe some properties of the system to build. For me, requirement engineering is much more: it is the search for and identification of the model describing the customer universe at best. Such a model must describe the processes and informations which constitute this universe and in which the new system will be embedded. We will see why this is important.

From theory I know about the value of Use Cases and UML Activity Diagrams in the context of requirement engineering, but I was surprised how useful they are to perform that requirement engineering as desired above. By bounding requirements to Activity Diagrams and Use Cases, it is possible to get an output of a requirement engineering phase which - beside the description of the properties of the new system - give an answer to the question, which actions are performed by who, which are the needed informations, which are the resulting informations and how the system to build will fit in this universe.

This all is the input for the Software Architect, and he must use it to find answers to this question:

Which are the parts of the system to build, and how must they interact ?

Well, this seems trivial, and it is even more easy if processes and information flow in the customer univers is known. But now comes a surprising aspect:

Which of this parts of the system are described at best by the data they use and compute, and which parts are described at best by the processes they perform ?

The answer to this depends not only on the properties of the future system, it depends strongly on the nature of the processes and information flow in the customer universe. The outcome of the architectural decision has very long-range consequences because it determines the complete design- and implementation process: a data-driven architecture has other characteristics than a process-driven architecture. Shure, data and processes are not independend. But for the technical design process it is different if one think first in terms of data or if one think in terms of a sequence of actions.

In the data driven world, the next decisions are related to questions like who, how often and in which way the data access will happen. In the process related world, the Software Architect has to decide about events, sequences and has to work with the time.

Beeing at this stage, the next suprising aspect, the next decision point has to be regarded, which as hughe impact to the design:

What are the points of possible change ?

Here, the Software Architect must invest phantasy and experience to identifiy the "weak" parts in the required system. He must look for the things which may be not so fix as the requirements suggest, and as a result, he must decide what things of the system (roles, data, actions) must be designed in a way that a change would be easy. This is only possible if the requirements describe more than the system itself. I think that the best sources to find such points of possible changes are the relations of the system to the universe of the customer.

And here, we are at the heart of the work of an Software Architect, as I belive it: find a design, which make change in the forseeable way easy. This gives the material to make further design decisions, it helps to reduce the set of possible solutions for a design task, to make the system that what is every time told to us: make it simple, but not simpler. Design for change, but only for that, which will be needed. The power of design for change is that what me really surpised ....

Now the detail work can start. But one thing remain, which I often discussed with colleagues: what about technical constraints ? Must an Software Architect know about the possibilities and properties of certain technical solutions ? Well, at this stage, my assumption is that he must not. In the opposite: the Software Architect must find a design, which can adapt certain techniqal constraints. So the decision point would be:

How can the design made robust against the technical constraints and implementation ?

If writing an required amount of data on hard disk is not possible by simply writing to one single file, an architecture is needed which handles multiple files. The Software Architect can not know all detailed constraints which comes from the many-years experience of a software developer, or from a data base specialist, a web specialist etc. In addition, such constraints are change points "from the bottom". Yes, the architect must have a rough knowledge of what is possible with today techniques, but only in order to be not too unrealistic. Therefore, a Software Architect must keep contact with specialists . By the way, here I also see the border to the role Software Engineer: this one must have detailed technical knowledge, and therefore may be responsible for the "lower" or detailed parts of an architecture. He has to bother with detailed technical details.

So, as a summery this decisions help to come to architectural decisions:

- What are the roles, and the parts of an system, and how interact they ?
- Which parts are data driven, which are process driven ?
- What are the points of possible change in the customers universe?
- What are the points of possible change from the technical implementations ?


Of course, it will not be all of the Software Architect, but it really helps ;-)