Ideas for Processing II

In the last post, I sketched the idea of a graphics subunit, which just takes render tasks describing *what* to draw, an which has the full responsibility and specialisation for *how* to draw. An graphics subunit independent in this way would have many implications, and the technology necessary to implement it goes beyond pure geometrical or mathematical techniques. Knowledge and learning of machines will also play a important role for its implementation.

Now, the questions is how we could come closer to this goal. One interesting path may be to flip the processing chain of vision: instead of recording images by an eye (camera) and process them like the humen vision system does, let us drawing pictures by using remembered pieces of images. In the wide area of computer science research

there exists already a field called image-based rendering, which tries to create renderings out from images recorded by cameras. But as extension to this more more less geometrical way of thinking, I would like to add the approach of the flipped human vision chain to the term image-based rendering (because it is simple the best term).

Where comes Processing on the scene ? Well, it is a big step to model the human vision system. One powerful and promising approach is the Hierarchical Temporal Memory Method introduced by Jeff Hawkins and his company Numenta, Inc. But for the first step, it would be too complicated, even if the underlying principles are not. I would prefer to start with small experiments, done in Processing.

One of the first experiments will be to create an Processing application, which draws some graphics with its own dynamics. It is not the objective that this graphics shows anything special. Then, I will send this application, lets call it the "Drawer", signals in form of data packages, generated by another application (Haskell or Erlang). The Drawer should then react, change its drawing dynamics according to the signal received. I want to investigate how this system can be set-up in a way that wanted graphics can be drawn by providing selected signals. BTW, the signal path is planned to be in XMPP.

The next step would be to add memory, where the course will be taken to the flipped vision processing chain. Processing is a very good tool for this, especially since it is possible to use Scala. So let's start !

Requirements for a Knowledge Management Tool

We are in the information age, aren't we ? Well, we collect information, put it in big data bases, or distribute it accross the web. But in fact, we collect no information, but plain data, numbers, texts, images videos etc. All our computer based technology is designed for measurement and collection, this is their strong part. But I want to access information, and - in the best case - only that one which belongs to my problem.

So, the question is, how can we transform collected data to useful information ? Many philosophers had discussed and investigated, what knowledge, insight and information is. I love this point of view:

The humans are able to act in a free manner, but actions are conditioned by situations. In the course of the lifetime, a human experiences many situations (described by the data provided from the senses), and the actions performed in that situations induce new situations. Therefore, one thing is clear: data must be associated with situations, otherwise, they are meaningless, without any value for the existence of humans.

Another thing emerges from the necessarity of decisions: we all know, life faces us with problems, which force us to decide to go the one way or the other, to choose between options. The decision is the predecessor of the action, and the problem is the predecessor of the decision. Because we remember the association between situation, action (as consequence of a decision as a consequence of a problem) and the result, we can decide better in future, in more or less similar situations.

But background knowledge or experience for itself is not sufficient to decide well. A goal is needed, or let's say more generally, an intention. The experience (the situation-action-result tuples) tells us which action in the given situation may have the biggest chance to lead to success in respect to what we want (the goal, the intention).

Summary: facts (data) - situation (context) - problems(questions) - goals(intention) are the things which distinguish data from information: in combination, they help to determine the best action to take. For short, this is knowledge, in some sense it is the equation of facts - situation - problems - goals - actions solved for actions.

And from here on, I can tell what my requirements to a Knowledge Management Tools are (some things I can not explain more detail here):

• allow to collect any kind of facts, any type of data
• allow to collect context descriptions
• allow to collect problems
• allow to collect intentions
• connect all these elements, by n:m relations
• allow to weight (again) these connections at any time
• allow to add or discard any connections at any time

Unfortunately, we have no advanced Artificial Intelligence today which would allow to code and access any kind of gray and color of the color set of life. We have to categorize and quantize what we experience every day. And we have technological limits. In respect to this, I would add this requirements:

• visualise the connections
• allow to classify context, problems and intentions
• find connections automatically by statistical analysis of words or other means
• allow to create context, problems and intentions recursively out of other contexts, problems, intentions, data.

The last requirement is inspired by the observation, that life is a steady flow:

• allow to create contexts out of sequences of conditions or event descriptions.

Can't wait to play with a system which provides this features. So I hope, someone will implement this *few* requirements soon :-)