Self-Aware Human-Computer Interfaces

How are Human-Computer Interface components (GUIs) similar to virtual life-forms?

Software seems to evolve symbiotically with the human mind – it competes for, and consumes human “attention”, in return producing “utility”.  So consider the development tools (IDE's, text-editors, web-browsers, operating systems) that are used to develop Artificial-life software.

It seems to me that these tools are actually self-contained life-forms or virtual worlds themselves, already (though relatively inactive and behaviorally rigid). Then a-life developers, whether realizing it or not, interact with a virtual world of evolving widgets and logic in order to develop another.

If “living components” or “living widgets” can facilitate their development processes themselves, then it is like a purposeful feedback loop.



GUI design is concerned with the ergonomics of human-computer interaction. In a 2D/3D open space virtual world consisting of vector, fractal widgets, there are many possibilities for maximizing its ergonomics toward specific “functional” or “behavioral” goals:

  • geometry generation and arrangement: position, size, orientation

  • animated appearance

  • interpreting hardware input devices / sensors into control events

  • interpreting control events into widget behavioral reactions

  • etc...



A general-purpose evolved virtual-world operating system needs to involve interaction with and navigation through files, data, links, source code, executing code, etc..

 



Possibilities beyond rectangular windows and deterministic event-driven desktop application or web frameworks, include:

  • 2D / 3D scalable vector widget shapes

  • layout, reaction / behavior – controlled by logic, procedures, or neural networks

  • continuous, reversible “homoiconic process soup” that links abstract data to its possible spatiotemporal embodiments

  • snapshot w/ alternate branches that can be explored and reverted

  • physics (ex: solid collisions preventing overlapping windows)

  • fractal “metaballs” (scalar polygonization), convex hulls, blobs

  • meshes that unfold, or deform when touched

  • physically reactive buttons and sliders

  • noises and tones

  • pulsing, oscillating scalars and vectors → motion, size, rotation, color, lights