Version 0.5
Copyright © 2004 - 2005 Raphael Langerhorst
Copyright © 2004 Anton Melser
Abstract
The G System Framework is a toolset for building virtual reality style evolution simulation experiments and games.
Table of Contents
When reading the technical documentation of the G System, you will probably find many uncommon and new aspects of software development. This is mostly because of the quite uncommon purpose this project works on. Nevertheless we feel that the G System fills a gap in both social and scientific areas that require some attention. So the purpose of the technical implementation is to model life (or evolution, both are the same), and is thus not a means by itself. This can result in a somewhat uncommon but highly fascinating software design.
Table of Contents
The central library is the G Core System (GCS). It defines the
basic structure of world content. Every such piece of world
content is called an "element", represented by the class
GCS::GElement. This class uses other
classes like GCS::GObject to store
information about the element.
Basically an element consists of three things:
Energy,
Form,
Any number of agents.
Energy gives an element a certain characteristic and
it is fully defined in the core system as
GCS::GEnergy. Energy can take
an important role for agent behaviour as some agents
behave differently for different energies.
Form defines the element's geometry, the basic form attributes are
defined in the core system as GCS::GForm.
Additional form information is usually in the
GCS::GElementData. It is the responsibility
of the application or the agents to create this information there.
The GCS::GObject class holds all data
of an element. This includes the energy, the form, the ID of
the element, the IDs of children and the parent ID as well as
the ID of the element this element is connected to. There is
also a generic data container class, GCS::GElementData
which can hold any desired additional data of the element.
All agents have access to this data and should use it to store ALL
persistent data. This data gets also automatically transported by the
GWE through the network and into the database.
The GCS::GObject class also stores the ID
of the connected element. Every element normally has an element
it is particularly connected with. This is always the element
of interest and towards the actions of this element are directed.
The concept of connected elements is quite fundamental to the G System. Everything in the world is somehow connected with each other. This idea is represented by the connection ID of the element.
Agents perform all of the behaviours of elements, which includes
all laws that the element is affected by. No agents are defined
in the core system itself, the core system only provides the
framework in GCS::GAgent to allow easy
creation of new agents. Have a look at the basic elements
library for some examples.
What the agent exactly does is completely up to the agent designer. Every agent is executed as a separate thread. Additionally the agent has read and write access to all data of its own element and read only access to other elements for which the IDs are known. When working with element data the agent designer has to keep in mind that there can be more active agents in the element which are also working with element data. For proper synchronisation most element data can be locked. It is highly recommended to use this locking mechanism when developing agents.
Please keep in mind that all data that should be persistent must be
stored in the element data object, GCS::GElementData,
that is contained in the element's GCS::GObject.
Agents basically have two responsibilities:
creating influences
responding to influences
The response mechanism is implemented by reimplementing a virtual
receive method of the GCS::GAgent class.
The parameter of this method is the influence itself.
GCS::GAgent inherits QThread
which means that the agent itself is a thread. In its execution
the agent can work with and process element data and can radiate influences
or send influences to specific destinations (often the connected element).
This is where most of the actual logic of the agent is normally defined.
An element can globally be started or parked, which means that all agent threads of that element get started or parked.
See the documentation of the GCS::GAgent class
for more details.
An influence is represented by the GCS::GElementInfluence
class. Agents can send out influences, either by radiation of influence or by
influencing a definite destination element (this is where the ID
of the "connected" element comes in). As soon as the agent has
created such an influence, the agent can send it out and the GWE is
then responsible for bringing the influence to the destination(s).
When an influence is sent out, some other element will receive
an influence. A virtual method of every agent of every receiver element is
called and the influence is the parameter. Using this method the agent can
define the reactions to the influence. The reaction can either be a simple
modification of element attributes - or even just internal agent
attributes - or more complex things that are handled in the agent's
thread (instead of handling the reaction directly in this
influence receiving method).
The GCS::GElementInfluence class is independent of
an GCS::GElement instance, but is used by elements
for interaction. By limiting interaction to one general way, it is possible
to create general rule systems that apply to all kinds of interaction.
The only attribute that is transported with an influence itself is energy. In early versions
of the G System it was common to subclass GCS::GElementInfluence
to provide additional information to the receiving agent. But this brings a close
dependency between sender and receiver of the influence. From version 0.5
agents are able to access data of remote elements (for reading only) and thus
it is not necessary to transport any additional information with the influence itself.
If for example an receiving element needs to react to the sender's position then
the agent would simply fetch the additional information through the
GCS::GWorldData interface.
Influences always carry energy of the source element with them. This means that an element that sends out an influence puts a certain amount of its own energy into that influence, which can be thought of as the strength of the influence. If more than one element receive the influence, the GWE automatically distributes the energy among all receivers. Every receiving element absorbs some of the received energy amount. The energy that is not absorbed is returned to the sender.
The kind of energy that is received with an influence also determines whether the influence is at all recognised. Only if the energies are somewhat alike the influence is received at all. Further improvements on the GWE could result in the GWE making decisions about how much energy is received by what element.
Agent behaviour depends on the energy of the element. If the energy changes then the behaviour of the agent can change as well. This depends on the design and implementation of the agent.
Sending or radiating influences can thus be a means to change the behaviour of other elements. If the rules the other element adhers to are known, then it is possible to control the behaviour of other elements. This is just the way reality works! Everything acts according to its energy, and things can only be modified by influence. Please note that this is a quite radical view of reality and there are many different ways to look at it!! The G System just tries to build a model-able simulation framework of reality and has also more to offer than just this very point of view. In particular you should have a look at the Philocorner. A lot of thoughts are collected there about philosophical aspects of life and evolution.
The element data of IDs proposes a hierarchical world structure, and in fact this is what is used. Every element is in itself a complete unit to the outside and can have a very complex subhierarchy - solar systems are good examples. First, we have the whole solar system that can be seen as a complete unit ("one element") to the rest of the galaxy. This element has some child elements, which are included in itself (in terms of position). These children are the planets, asteroids, large space stations and space ships and so on. Every such element again holds a complex subhierarchy - continents - oceans. Continents hold landscapes and cities. They hold either houses, individual beings or whatever. Individual beings are again a very complex entity - which is probably a topic for our Philosophical Corner (philocorner).
Please note that an element does not have any direct reference (or pointer) to other elements, only their ID. Thus it is not necessary to have the other elements in the same system process - they can even be on a remote machine connected through the network. Large applications probably even should be distributed hierarchically among a network. Reflecting the element hierarchy of the virtual world in the hierarchy of the computer systems used is often a good idea. See the chapter about GWE for further details.
Table of Contents
As we have seen the GCS does nothing out of itself. In order to get functionality one has to create agents, forms (and most likely influences). Some basic classes of this kind are already available and can be found in the GBE (G Basic Elements) library.
The kind of classes you will find in this library mostly has to do with virtual worlds - world creation, element hierarchy management, physics, ... soon this library will also provide you with agents for evolution aspects.
World creation is an interesting topic, some of the required
functionality is already implemented. The class
GBE::GDynamicGeneratorAgent class
provides functionality to generate a randomised
universe based on a deterministic random generator. As this
allows for a huge universe to be created, the generator can
create just as much as the current active elements (whether
they are intelligent computer driven beings or human users)
require. As these active elements start to explore other
parts of the universe the generator brings these parts
into existence - and by the time passed these parts
can determine in what state they have evolved (age of solar
systems or planets,...). Parts of the universe that are not
inhabited by active elements are just put to "sleep" to save
computing power. This is sadly a limitation we have, but it
will have only a minimal (if at all) effect on the overall
simulation if handled correctly.
Now, if the G System is about evolution, how does this get along with a randomised universe? This is rather a philosophical question but as it must occur to some readers we will answer (part of it) it here. First we need to distuingish between form representation and physiological aspects of the universe. The deterministic random world generation is in the first sense a means to determine where to place what kind of elements. It is a completely different topic how these elements behave after their creation, which is not in any way determined by the generator. Also, if for example the generator determines that on a certain location in a solar system a planet should be placed then the planet is not necessarily just there but the created element can just be a divine energy in the area of the planet that slowly evolves into a planet - for example by attracting asteroids, fragments of dead planets, ... So the determination that on a certain location a planet should be does not clash with the way how this planet comes into existance.
Another agent, GBE::GReparentAgent,
handles the hierarchical structure of elements. As discussed
earlier, the universe is represented in a highly hierarchical
structure - which is indeed necessary to allow for huge
universes to be functional. Thus a human being may be part of a
house while being inside. But if the human being walks out,
it will be part of the city or the surrounding landscape. In
order to handle the hierarchy change properly, the reparent
agent observes positional changes and notifies all affected
elements when a change occurs. Details can be found in the
API documentation.
Table of Contents
The G System is designed to simulate huge virtual realities. This can include many solar systems and planets, down to small details such as houses, environments, and even individual plants. All these things are not static. This means everything can change, humans (often controlled by real human users) move around, interact with other elements in their environment, ...
To allow for such a large scale simulation, some infrastructure needs to provide the computer processing power, the storage, and the communication. This infrastructure is provided by the GWE - the G World Engine, it is the container framework for all elements.
Elements simply operate. To them, the complete simulated universe is just a huge (hierarchical) collection of elements. They are not concerned with the physical location of other elements when communicating. Providing this seamless network transparency of a distributed server infrastructure is the purpose of the GWE.
As hinted in the previous section, the GWE is a distributed server infrastructure, or in other words, an element container and thus the framework the elements rely on for their operation.
The following UML diagram should give an idea of the GWE structure.
To provide a flexible implementation a general interface for
each functionality is defined. This currently includes the
GweController which is responsible
for element execution management and logic, and the
GDataController which is responsible
for persistent and network transparent data management and
GWE Server network communication.
The abstract interfaces, including the GWE Factory class, are available as public API, thus enabling application developers to control the GWE behaviour without being dependent on the actual implementation classes being used by the different components (for example whether a simple or advanced GWE Controller is used) because the implementation is not part of the API. The Factory class is used to bootstrap a GWE Server which uses the differen implementation classes.
An important part is the complete abstraction of physical data location. This is accomplished by a data controller class that manages all data and data transfer. This is also where the other GWE Servers in the network are known and intelligent load and data balancing is done according to the logical hierarchical structure of the virtual universe.
Currently there is one XML based data controller available. This controller uses a separate serialisation class to convert between C++ objects and XML representation of elements. The C++ objects are required for execution and the XML representation is used for sending element data across the network and managing database content.
Currently there is one Network implementation available, the
GXmppNetwork. This implementation builds
on the standardised XMPP
protocol. XMPP is essentially a real-time XML router, as Justin K.
has put it in an email. And this is exactly what we need. We have
XML data that needs to be routed through the network to some
destination (another GWE Server).
Speaking in XMPP language a GWE Server is an XMPP client and uses an XMPP server to communicate with other GWE Servers which are also XMPP clients (possibly from other XMPP servers). Thus we do not even have hardwired peer-to-peer connections but rather a transportation framework that takes care of message routing and delivering.
The network structure basically looks like the following:
The elements that make up the universe are structured hierarchically. Every element, such as a planet, includes all its subelements and presents itself as one complete unit. This way it does not matter how complex an element is internally. Planets, complete landscapes, cities, or even houses can be represented by one element, altough they consist of many subelements internally. This hierarchical structure can be thought of as logical layering. Communication between elements usually only takes place between adjecant layers. This makes the complete GWE Server infrastructure enormously scalable because each GWE Server only needs to know about adjecant Servers!
For every element exactly one GWE is responsible. Responsibility distribution is usually handled according to communication load between elements. Elements that communicate a lot with each other normally reside on the same physical GWE Server.
GWE Servers that contain elements that send influence to elements not managed on the same GWE Server need to know where to find these elements. This information can be obtained from GWE servers higher in the hierarchy.
The network capabilities of the G System include distributed element management and in particular element responsibility. This means that responsibility can also shift between GWE Servers. This raises the requirement for data transport and for control messages. Data messages can contain any element data, either complete elements or just partial data which can then be updated on the receiving GWE Server. Elements that are known on multiple GWE Servers can only have one GWE Server that actually is responsible for a particular element. Passing on the responsibility can only originate from a server that currently possesses the responsibility of a particular element.
Network latency implicitly requires save transactions and no influence loss. Here is the general procedure for shifting element responsibility described:
First, the elements themselves need to be made known to the target GWE Server. This is a simple data transfer. The source GWE Server needs to keep track of all received influences after the element data has been transfered, since influences are only forwarded to the currently responsible GWE Server. This can be done by sequentially numbering received influences and remembering the last number that the elements have received before the last data transfer to the target host.
As soon as the data has been sent to the target host, the source GWE Server asks the target host if it has enough capacity to take the responsibility for the specified elements. If the target GWE Server rejects the elements, then no further actions are taken. All influences are further delivered to the elements on the source GWE Server.
In case the target GWE Server accepts the responsibility for the elements all influences that were received by the source GWE Server in the meantime are forwarded to the target host, the responsibility is implicitly shifted and all GWE Servers that are directly higher in the hierarchy of the shifted elements are notified about the responsibility shift. Note that this shift implies that the new responsible GWE Server activates influence emitting of the taken elements while the source GWE Server disables influence emitting for these elements.
It is also possible that other GWE Servers do execute elements they are not responsible for to get adequate predictions of states. Such predicted elements are called secondary elements wheras the real elements are called primary elements. Each GWE Server can thus have primary and secondary elements.
Secondary elements are usually found on a GWE to which clients are connected. Such secondary elements never send out influences but still can receive them from local influence sources to improve prediction accuracy. Such influences are then multiplexed to the primary element and the corresponding secondary element. Influences are never sent to remote secondary elements but only to the primary elements and local secondary elements. The data of predicted elements are constantly updated from the GWE Server that hosts the corresponding primary element.
A client is a GWE Server with an user interface. To the network a client is thus nothing special. In general a GWE Server that acts as a client has few elements it is responsible for and has a lot of secondary (predicted) elements.
All information about required GWE behaviour in terms of service for the G Core System as well as network protocol can be found in this section. It should provide enough information to build your own compatible G World Engine. You are still advised to know about the general GWE design discussed in earlier sections.
This specification is work in progress. Information will be added while the specifications are being discussed and implemented by the developers. You can of course join the development team to add valuable considerations for this important part of the G System.
Although it is a very important aspect to fully document the functionality requirements, the current focus is to provide an implementation on the basis of C++ and the Qt toolkit.
This specification is definitely not as normative as it could or should be. If you have suggestions on improving the style of the specification you are very welcome to contact us on the mailinglists and discuss your ideas. Productive criticism is certainly welcome.
The GCS, which defines the elements, works almost by itself. Still, it requires the GWE to provide some services, in particular the communication infrastructure between elements.
The GWE and the GCS implementation must work together and be of a compatible implementation. If you plan to implement a GWE according to this specification, you must also provide a GCS that is compliant to the GCS specification.
The network protocol builds directly on the XMPP Core protocol. You can find additional information for this on http://www.xmpp.org. The protocol is thus completely XML based. Here, all valid XML messages are defined as well as the roles of the various parts of the G System.
Other protocols might be implemented. In such cases GWE Servers capable of both the XMPP based protocol and the other protocol must act as gateway. It is highly recommended to contact the G System team to discuss the integration of new protocols. The rest of this section only describes the official XMPP based protocol.
The G World Engine Servers represent an XMPP client as defined by the XMPP protocol. XMPP Servers are used to establish a communication between the GWE Servers, the XMPP Servers are out of scope for this project, they are only used as a transport mechanism.
Although the GWE basically adhers to the XMPP-Core (RFC 3920) specification some elements of the XMPP-IM (RFC 3921) specifications are implemented to allow presence handling and basic message exchange with so called Jabber instant messaging clients. This allows for interesting interaction between the GWE and pure instant messaging applications.
GWE communication is totally accomplished through message stanzas. Each message stanza uses the appropriate to attribute for destination identification. In every such stanza additional xml data can be transported which is specified by the actual protocol.
Network communication is initialized according to the rules specified in the XMPP-Core protocol. Additionally, after authentication a presence stanza is transmitted with an empty show element. This changes the state of the GWE Server to online in the sense of XMPP and actually enables message forwarding to the GWE Server from the XMPP Server.
After XMPP initialization the master server of the established GWE Server infrastructure is contacted to register the GWE Server and integrate it into the hierarchy. The master server usually stores the registered GWE Server as a child server.
After server registration basic element data is exchanged. When element information is exchanged for the first time normal communication is taken up.
Normal GWE communication is taken up after successfull initialization and registration. In this state only relevant element data is exchanged between servers.
Further cases to be described in the future:
Server hierarchy restructuring
Element responsibility exchange
Proper shutdown
This section describes all known XML constructs that the GWE server must be able to understand. A short description of the expected behaviour is also given.
We refer to each such XML construct as message. Be careful not to confuse it with the message stanza kind of XMPP. Still, it is true that the messages described here can only be contained in message stanzas, so there is some connection at least.
Remember that particularly this section is subject to changes, especially extensions. The G System developers are not bound to ensure compatibility between different versions of development releases. Only with the release of version 1.0 can this protocol specification be considered stable and compatibility will be ensured at least through all 1.x releases.
See the GCS XML Schema for the structure of a valid GElement XML message.
Additional to the given structure in the XML Schema, the following attribute is allowed in the <GElement> tag:
Additional attributes in <GElement> tag:
owner (string, XMPP JID), defines the GWE Server that manages this element as primary element.
A GElement message is used to exchange information about elements. The additional owner attribute can be used to find the GWE Server that manages this element, which is very important for sending influences and other element related messages to the correct server. This kind of message is probably the most commonly used.
See the GCS XML Schema for the structure of a valid GElementInfluence XML message.
Additional to the given structure in the XML Schema, the following attribute must be included in the <GElementInfluence> tag:
Additional attributes in <GElementInfluence> tag:
target (GCS::GElementID, target element), defines the target element to which the influence should be forwarded.
A GElementInfluence message is used to transport influencing information. The additional target attribute must specify the actual target element since the influence itself does not contain target information.
<reparent> <element> element id which is reparented (GCS::GElementID) </element> <from> old parent GElementID (GCS::GElementID) </from> <to> new parent GElementID (GCS::GElementID) </to> <transformation> transformation matrix (GCS::GMatrix44) </transformation> </reparent>
The IDs are in the form of GCS::GElementID XML representations, the transformation matrix is in the form of GCS::GMatrix44 XML representation. The <transformation> tag is optional, if it is not present, an identity matrix is assumed (usually the case with formless elements).
For reparenting notifications this message is used. The same construct is sent to all servers that manage one of the three affected elements. Each server modifies the parent-child relations of its primary elements.
Whenever a server managing a particular element detects that reparenting is necessary, this message is used to notify the old and new parent. Such a message should only be generated by a server which manages the reparented element (as primary element).
<register> descriptive text </register>
The registration construct is simply empty or can contain some description. The sender can be identified through the XMPP protocol, so it doesn't need to be provided through this message.
The use of this message is to register a new child server at a master GWE server. This is usually done after server startup.
<unregister> descriptive text </unregister>
Like the register message, the unregister message is also an empty element with optional descriptive text in the body.
The use of this message is to unregister a GWE server that is about to shutdown from a master server.
<requestfreeids> unsigned long number, holding the amount of requested free element IDs </requestfreeids>
The requestfreeids message only holds the amount of requested IDs.
If a G World Engine server is running low on available element IDs it usually sends a requestfreeids message to its master server to request more free element IDs. Such a request is usually answered by a <freeids> message.
<freeids> <from> range 1, lower bound </from> <to> range 1, upper bound </to> <from> range 2, lower bound </from> <to> range 2, upper bound </to> <!-- and so on, any number of ranges allowed --> </freeids>
The freeids message ranges of element IDs that can be used for additional element creation.
The <freeids> message is usually an answer to a <requestfreeids> message. These two messages make it possible to distribute free element IDs within the server infrastructure.
Note that it is not uncommon for a GWE server to send free IDs to its master server in case many IDs get available because elements are removed. This way used element IDs get reused.
In order to allow user interaction an user interface has to be provided - this is the job of the G Client Engine. It allows for the rendering of elements (that is, the forms of the elements) and should forward user input to the elements controlled by the user.
The GCE builds directly on the GWE, this means that the user actually has a local GWE Server running which in turn connects to the other servers.
GWE Server can handle primary and secondary elements. Clients will naturally have many secondary elements.
As the current implementation is mostly for testing purposes, we would appreciate it very much if some volunteer with some experience in 3D programming would assist in writing a good client framework.
We target at a very reusable client library. That means everyone should be able to write his/her own client or extend the given standard client to his own needs. The protocol will be completely open - as the whole software is. Thus everyone is able to create clients to personal preferences. Even clients without a GWE seem possible if the GWE protocol is implemented all the way.