Home » Blogs » seh's blog

P2P NeuroSymbolic Interface

25Dec2009

P2P NeuroSymbolic Human-Computer Interface

Experimental language-independent component-based unification of "artificial intelligence", "cognition", "reasoning", "semantic", "social network", and "virtual reality" software systems

Note: This document is a draft and is in development.
  1. P2P Neurosymbolic Web
  2. Design Principles
    1. Paths to AGI
    2. Language Independence
    3. Componentized
    4. Ambient Omni-directional Embodiment: Mind <-> SpaceTime
    5. Possibility
  3. Sub-Components
    1. OpenCogPrime
    2. NARS Non-Axiomatic Reasoning System
    3. Stanford Parser - multi-language grammar dependency parser
    4. RelEx English-language dependency relationship extractor
    5. NLGen2 English natural-language generator
    6. TexAI
    7. OpenRDF (Sesame & Elmo)
    8. WholeBrainCatalog and BlueBrain
    9. Neuroph Neural Network Library
    10. jACT-R
    11. JUNG - Graph (net) data and visualization
    12. HyperGraphDB
    13. JScheme
    14. Groovy Language
    15. JReality 3D Video / Audio
    16. JBullet Physics Engine
    17. Scenario 2D Scenegraph
  4. Functional Components
    1. Graph
    2. MemGraph
    3. UnionGraph (op)
    4. QueryGraph (op)
    5. ScalarMaskedGraph (op)
    6. ScalarMap
    7. NeuroMap
    8. ChangeGraph
    9. Build (Process, noun)
    10. Java Build (Process, noun)
  5. UI/IDE Components
    1. Control Windows
      1. Network Logins
      2. Clear / Cleanup
      3. Set Background
      4. New Message
      5. Edit Message
    2. NetBox Composition Model
      1. Data Properties
      2. Presentation Qualities
      3. NetBox General Properties
    3. Monitors and Control Panels
    4. MetaUI: Reified HCI Component Model
    5. Document / Text Editing
    6. Context Builder
    7. I/O Stream (Console, Shell)
    8. Layered Memory
    9. Dataflow
    10. 2D Pixel Space Simulator
    11. 2D Geometric Space Simulator
    12. 3D Voxel Space Simulator
    13. 3D Geometric Space Simulator
    14. 3D Geometric Space Simulator
    15. Human Face Model
    16. Manipulator Model
    17. Animal / Human Body Model
    18. Object Browser
    19. Audio
  6. Interfaces
    1. Settings
    2. Text Files
      1. Line
      2. CSV
    3. Chat
    4. Development Network
    5. Twitter
    6. Email
    7. Flickr
    8. Virtual Network Computing (VNC)
    9. XML
    10. JSON
    11. HTML
    12. RSS
    13. Blog API
    14. Virtual File System
    15. RDF
    16. SQL Database
    17. REST / SOAP / XML-RPC
    18. FaceBook
    19. MediaWiki
    20. Advertisement
  7. Packagings
  8. Applications
    1. dANN
    2. OpenNARS (2.0)
    3. OpenCog Prime (C++ Implementation) ( http://opencog.org )
  9. References



Design Principles

Paths to AGI

[AGI as Automatic General Intelligence]
[increasingly bootstrappable, increasing potential for self-modification, as the development environment for further improvements and interactions]
[an AI "psychology" tool]
[...]

Language Independence

[discussion re: degree to which this possible?]

Componentized

[Discussion about IoC & contextual instantiation, unit tests]

Ambient Omni-directional Embodiment: Mind <-> SpaceTime

  • Mind in SpaceTime = Embodiment
  • SpaceTime in Mind = Simulation / Imagination

Possibility

Meta-Ego Liquid Ego

Askiom

 

Modes:

  • Get (Completion) Possibilities “uh...|uh?” ?

  • “what will happen if...” X ?

  • do X!

  • Define / tag : X is Y1, X is Y2, ...

 

Any object (or set of objects) is a potential action descriptor, which may be unambiguous, or ambiguous to some degree

 


Sub-Components

OpenCogPrime

http://www.opencog.org

Adapted from the original C++ implementation. Several key classes have been renamed - the changes (and reasons for them) are documented in comments. Includes interfaces to the C++ implementation (CCog?) so that C++ components can be used.

  • Generates / Interprets Combo (language)
  • Hypergraph-based Memory
  • Natural language processing
  • 3D Embodiment Simulator
  • ...


NARS Non-Axiomatic Reasoning System

http://code.google.com/p/open-nars/

  • Interprets NARSese (language)
  • Discrete concepts
  • Discrete statements
  • Interactive, Steppable
  • Accepts Statements
  • Answers Questions
  • Swing-based GUI
  • ...


Stanford Parser - multi-language grammar dependency parser

[get URL]

 

 


RelEx English-language dependency relationship extractor

(Java)
http://opencog.org/wiki/RelEx


NLGen2 English natural-language generator

Java
https://launchpad.net/nlgen2

TexAI

http://texai.org

  • Natural language processing?
  • Generates / invokes Java procedures?
  • RDF support?

OpenRDF (Sesame & Elmo)

http://openrdf.org

RDF/OWL - http://en.wikipedia.org/wiki/Web_Ontology_Language

  • Discrete concepts (RDF)
  • Discrete statements (RDF)
  • RDF input, output
  • RDF net operations
  • RDF/OWL inferencing
  • SERQL query language
  • File / Database memory
  • Java Object Entity I/O
  • ...
 

WholeBrainCatalog and BlueBrain

Physical Neural Modeling
http://wholebraincatalog.org/
 
see also: IBM BlueBrain - a medically/anatomically-accurate physical Neural model that may possibly be open-sourced in 2010
http://bluebrain.epfl.ch/
 

Neuroph Neural Network Library

http://neuroph.sourceforge.net

Adaline, Perceptron, Multi Layer Perceptron with Backpropagation, Hopfield network, Bidirectional Associative Memory, Kohonen network, Hebbian network, Maxnet, Competitive network, Instar, Outstar, RBF network, Neuro Fuzzy Reasoner

jACT-R

http://jactr.org

  • Interprets ACT-R (language)
  • Eclipse-based GUI
  • Sensory (aural, visual, etc...) modules
  • Motor modules
  • 3D Simulator
  • ...


JUNG - Graph (net) data and visualization

http://jung.sourceforge.net

HyperGraphDB

http://code.google.com/p/hypergraphdb/


JScheme

Scheme implemented in Java, and interfaces with it

http://jscheme.sourceforge.net/jscheme/main.html

Groovy Language

JVM scripting language, with dynamic, OOP, and functional programming abilities

http://groovy.codehaus.org/

JReality 3D Video / Audio

http://www3.math.tu-berlin.de/jreality/
Adapted and componentized from original code for consistency with JCog.

JBullet Physics Engine

http://jbullet.advel.cz/ wrapper for http://www.bulletphysics.com/

Scenario 2D Scenegraph

Swing-based API
https://scenegraph.dev.java.net

 
 

 

Functional Components

Graph

directed directed/hypergraph graph index

suitable for RDF among other applications / representations


MemGraph

in-memory transient hypernet implementation

UnionGraph (op)


QueryGraph (op)


ScalarMaskedGraph (op)

filters a net according to FloatMap values


ScalarMap

scalar (floating point) mapping to arbitrary objects
[list of dynamics]

NeuroMap

map neural network (ex: Neuroph) to atoms
bidirectional adapter I/O
Synchronous, Asynchronous


ChangeGraph

path of changes or transformations that can be applied amongst a set of objects.  how do each node/link relate to Build (process) below?

Build (Process, noun)

  • Build is a process/context that ultimately constructs something
  • It has multiple possible destination that can be filterd / refiltered
  • At a given moment, knows present and possibly-present objects, and possibly-actionable actions
  • Functions as a "configurator"


Java Build (Process, noun)

  • Constructs arbitrary java objects
  • Emulates Picocontainer/Spring framework functionality, w/ indefinite / potentialities
  • Interfaces
    • Command line
    • Swing
    • ...
  • instantiates "architecture descriptions" - builder syntax



UI/IDE Components

 

Control Windows

  • Network Logins

  • Clear / Cleanup

  • Set Background

    • color, image, gradient

  • New Message

    • (and to which networks)

  • Edit Message

    • (if possible, ex: a blog post)
  • ...
 

NetBox Composition Model

Data Properties

  • Qualia
    • What: type
    • Who: Contact, source, presence
    • GeoWhere / GeoLocation
    • NetWhere: Internet presence / home
    • When: time
      • Now: in progress
    • In: Intensional Links
    • Out: Extensional Links
    • Related / Nearby: sameAs, similarTo, assocations
    • In: Incoming, InBox
    • Recent
    • Frequent
    • Favorite / Labels (Tags) / Emotion / Quality
    • Trash
    • Channel / Visibility  / Security (Public, Shared w/)
  • node or edge type
    • agent
      • name
      • # of messages
      • avatar image
    • network
      • name
    • channel
      • name
      • topic
      • agents
      • # of agents
      • # of messages
    • message
      • keywords
      • context
      • size
  • source
    • agent author
    • channel / protocol
  • time
  • ...
 

Presentation Qualities

  • visibility
  • size
  • proximity
    • attraction(link(a, b))
    • repulsion(a, b) | repulsion(link(a,b))
  • color
  • attached widgets
    • hide
    • goto next
    • goto previous
    • increase / decrease { size, detail }
    • enable / disable extra details
      • related images
      • related context
    • transformations
      • explode to words
  • lifespan (time to live)
  • ...

 

NetBox General Properties

  • time scale
  • natural length factor
  • ambient force field dynamics
    • ambient gravity
    • whirlpools
    • etc...
  • remove on OOB
  • particle box models?
  • repulsion / attraction balance (+/-)
 

Monitors and Control Panels

  • Video settings

  • FPS meter and CPU throttle

 

MetaUI: Reified HCI Component Model

 (properties and invokable controls) as introspectable (read/write) concepts:

  • Windows
  • Panels
  • Editable Content
  • etc...

Document / Text Editing

  • Syntax Highlighting
  • Contextual hyperlinks popup menu (ex: Eclipse's F3 command) 
  • Contextual auto-complete suggestions
  • Rich-text (HTML) editing?

Context Builder

  • Self-contained instance of a particular AGI core
  • Add New, Remove
  • Instantiation Configuration
    • loaded from file
    • loaded from supplied configuration templates, w/ customization forms
    • produced by another context
  • Context=Core=Engine=Model=System, an instance of a particular AGI entity.  may communicate with other instances, or be isolated
  • Executive Controls:
    • start / re-start (now, or scheduled)
    • stop (now, or scheduled)
    • shutdown (now, or scheduled)
    • send command text
    • send command object (ex: from a form, or wizard)
    • step forward
    • auto-step forward (at arbitrary real-time speed)
    • undo
    • open a view or control panel
    • clone
  • Snapshot Freeze / Thaw
    • if a type of context is completely serializable, its snapshots can be shared via the web and loaded instantly
    • aka "load / save game"
 

I/O Stream (Console, Shell)

  • Channel = Stream = Feed = Console = Prompt = Shell ...
  • Mix content from multiple channels (on/off, including "local echo")
  • Mix channel content in proportional amounts (adjusts control font color and size)
  • Contextual hyperlinks (shared functionality w/ text editors)
  • Auto-complete suggestions for text input box
  • Add files, images, URLs as typed "input" objects?
  • Dialect selector, prior to sending input
  • Input History
 

Layered Memory

  • Plugin-able memory layers that can be activated/deactivated individually
  • Serves as modular knowledgebase
  • Stream Readers/Writers, with optional Buffering
 

Dataflow

  • A MultiContext consists of a set of contexts that are interconnected by
    • Dataflow I/O pipes between individual contexts
    • Topic-based Publish/Subscribe Data Bus "Cloud", read/write
    • Concept "bindings" - that merge readable/writable concepts in disparate cores to the same value
      • if a particular core supports an interface for that particular type - simplest example: strings and numbers
  • Can form diverse "communities" of intelligence engines, possibly exhibiting synergy / symbiosis
  • Can form symbolic networks, where each node is a self-contained AGI core
 

2D Pixel Space Simulator

  • ex: Graphics Software, like Photoshop or GIMP
  • Grayscale or Color per pixel
  • Multiple layers, mixed in varying amounts and blend algorithms
  • View / Edit
  • Can represent visual, or membrane sensory input
  • With timeline, for animation
 

2D Geometric Space Simulator

  • Shapes: Vector, Polygon, Curves
  • Colors, Textures, Strokes
  • ex: SVG Illustration, like Illustrator or Inkscape
  • View / Edit
  • Can represent visual, or membrane sensory input
  • With timeline, for animation
 

3D Voxel Space Simulator 

  • extrusion of 2D Pixel-space into a 3rd Dimension
  • View / Edit
  • With timeline, for animation
 

3D Geometric Space Simulator

  • ex: 3D modeling software, 3D interactive worlds, 3D games, Augmented-reality overlays
  • View / Edit
  • First-person, 3rd-person modes
  • Scenegraph-based (scenegraph nodes may be reifiable "concepts" existing in an AGI memory)
  • Superimposed Enable-able Spatial Layers
  • With timeline, for animation
 

3D Geometric Space Simulator

  • ex: 3D modeling software, 3D interactive worlds, 3D games, Augmented-reality overlays
  • View / Edit
  • First-person, 3rd-person modes
  • Scenegraph-based (scenegraph nodes may be reifiable "concepts" existing in an AGI memory)
  • Superimposed Enable-able Spatial Layers
  • With timeline, for animation
 

Human Face Model

  • at least cartoon "smiley face" (ex: sphere)
  • parametric facial expressions: eyes, pupils, eyebrows, mouth, nose, etc...
  • 3D with ability to look in any direction

Manipulator Model

  • arm, finger, gripper
  • force-fields: suck, blow
 

Animal / Human Body Model


Object Browser

  • List, Tree, Table, Icon, Cloud modes
  • Shows data objects and concepts
  • Wide and square icon representations
  • Add new object / concept
  • Remove concept
  • Edit concept properties and metadata
  • Object <-> Reasoning paths : paths that connect a reason to a concluded/inferred statement/object
  • Object <-> Evidence paths : paths that connect evidence to a concluded/inferred statement/object
    •  

 

Audio

...
 


Interfaces



Settings

 

 

Text Files


Line

CSV

 

Chat

IRC
GTalk (XMPP)
AIM
MSN
...

 

Development Network

Feedback

Demos

Updates

 

Twitter

 

Email

 

Flickr

 

Virtual Network Computing (VNC)

 

http://www.tightvnc.com/doc/java/README.txt
http://ultravnc.svn.sourceforge.net/viewvc/ultravnc/UltraVNC%20Project%20Root/UltraVNC/JavaViewer/

 

 

XML

 

JSON

 

HTML

  • with content filtering rules

  • ex: only images ("media" center mode)

 


RSS

 

Blog API

  • Wordpress, Blogspot, etc..

 


Virtual File System

  • Apache VFS: local, ftp, sftp, http, etc...

 


RDF

 

SQL Database

 

REST / SOAP / XML-RPC

 

FaceBook

 

 

MediaWiki


Advertisement

 

  • AdSense?

 

 

Packagings

TODO Executable package for win, linux, and mac

 

TODO Build procedures

 


Applications


 
dANN

 


OpenNARS (2.0)

...

OpenCog Prime (C++ Implementation) ( http://opencog.org )

Emulate and Extend UbiGraph's API
  • This module displays the atomspace via Ubigraph.

  • Error handling, especially for if the ubigraph command is run (or an

    add/remove atom signal happens) while the ubigraph server is not

    running.

  • A separate command to start the initial display and one to choose which things

    to show.

  • Being able to choose to display only some of the atoms.

  • "Themes" to display TruthValues and AttentionValues visually (e.g. as color or

    size).

  • Add callback for Ubigraph which displays atom info when a node or link is

    clicked. This needs OpenCog XML-RPC or JSON-RPC integration first however.

  • Allow separate visualisation servers, one for visualisation of PLN BIT trees,

    another for NLP processing and dialogue pipeline. Eventually these should all be

    merged into separate views in a generic AtomSpace/OpenCog visualisation tool.

 
 

References

  • Facebook on PS3
  • Mozilla Raindrop