WorldCreation
Contents
Description
This page describes how to create your own worlds (ie, content) for WorldForge. This focuses on the Cyphesis server and Ember client.
Audience
This is intended for experienced coders (any language) who are interested in creating new content for WorldForge (which includes creating entire worlds).
I think there is a definite need for tutorials for non-technical content creators but that is outside of the scope of this document. Hopefully coders reading this document can help build the tools necessary for non-technical folks to easily add content.
I have focused on Linux development because that's what I'm using myself.
Introduction
So you have heard of WorldForge, found the web page, and are now poking around and kicking the tires to see what this system can do. Congratulations! You've found a pretty powerful toolset for building worlds. Here I'll walk through some of the steps necessary to create content.
The first thing you should do is to get your own running instance of Cyphesis (the server) and Ember (the client). See the Main_Page for links to downloading those binaries. Take a few minutes and do that now. In my experience (running Fedora Core 5) the setup was pretty painless and worked out-of-the-box.
For the purposes of this page, you do not need to build your own versions of those binaries. You just need working server and client binaries.
Once you have a local copy of cyphesis and ember, you can test your setup by following these steps:
- Start cyphesis. On linux, the binary should be in your ~/.local/bin directory. When you run it, you'll see a few informational lines go by (and a few complaints about the lack of a database, no problem), and finally a line that says "INFO Running". That means cyphesis is in good shape.
- Run cyclient. This binary is also in your ~/.local/bin directory. This will result in a lot of text output (information about how it is populating the world). It takes a few seconds to run but should finally stop after creating a bunch of objects.
- Run ember, also in your ~/.local/bin directory. This will take a few seconds to load up all the media, then prompt you for a server. At the edit box in the bottom left, type in "localhost" and then click "Connect". Ember will connect to your local cyphesis server. It will take a few seconds, but you should see trees, water and finally terrain as ember downloads your environment.
At this point, you have your own world server and a client to explore it. To start with, the world comes with a default setup, including terrain, trees, and chickens. It is a pretty interesting world, with lots of good terrain and buildings, and many creatures and other items. But as a world creator you want to see your terrain, your trees, and your chickens.
Before we start tweaking content, you'll need to understand how the client and server know about the world at all.
Rough Architecture
If you're like me, you were expecting a pretty simple design: a server that knows about the world, and a client that knows how to render it. In fact the reality is a little more complicated! But the actual architecture is quite powerful, and as a world creator you can use that to your advantage.
You may have already noticed some of the complexity above. You had cyphesis (the server) and ember (the client). What was the cyclient program you had to run?
In fact, cyphesis is a very "dumb" server, in the positive computer science sense of the word. Cyphesis exists to host objects and let many clients interact with them. But the details about the world (terrain, items, etc.) are added to the server at runtime.
That is the role of cyclient. It connects to the local cyphesis server, and uploads the world's content. So the terrain, trees, buildings, and everything else in the world is specified by cyclient.
There is still what I call the "binding" problem. Suppose you want to add a new kind of object to the world using cyclient. How will the client (ember) know how to render it? And if there are interesting interaction issues (object is or is not an obstacle to movement, has its own behaviors, etc.) how does cyphesis know what those are?
Someday, all of that (client 3D models, server behavior code) may also be specified at runtime. But for now, both the server and client require some configuration so that they know how to render objects and deal with behaviors.
define_world.py
One of the most important files we'll deal with at first is define_world.py. This is the python script that cyclient uses to populate the world. Hacking define_world.py is a starting point for world creation.
The default install location for define_world.py is ~/.local/share/cyphesis/rulesets/mason. You might want to take a quick look through that file. (Also, you might want to save the original define_world.py so you've got it backed up in case you break it later.)
Terrain
One of the first things to play with is your world's terrain. For WorldForge, terrain is handled by the Mercator engine. See its documentation for exact details on how it works.
But the basic idea is this: as a world creator, you provide some high-level terrain information (height values at a few x,y positions) and Mercator fills in the rest. It is very easy to use!
Let's hack the default terrain to create our own landscape.
The terrain is specified in define_world.py. If you look in that file, around line 180 you'll see the function default defined. This is where world population starts, and some of the first lines in that function concern the terrain.
You can see the default implementation:
points = { }
for i in range(-8, 7):
for j in range(-6, 7):
if i>=5 or j>=5:
points['%ix%i'%(i,j)] = [i, j, uniform(100, 150)]
elif i<=-5 or j <= -5:
points['%ix%i'%(i,j)] = [i, j, uniform(-30, -10)]
elif (i==2 or i==3) and (j==2 or j==3):
points['%ix%i'%(i,j)] = [i, j, uniform(20, 25)]
elif i==4 or j==4:
points['%ix%i'%(i,j)] = [i, j, uniform(30, 80)]
elif i==-4 or j==-4:
points['%ix%i'%(i,j)] = [i, j, uniform(-5, 5)]
else:
points['%ix%i'%(i,j)] = [i, j, 1+uniform(3, 11)*(abs(i)+abs(j))]
points['-4x-1'] = [-4, -1, 12.4]
points['-4x-2'] = [-4, -2, -8.3]
points['-3x-2'] = [-3, -2, -6.2]
points['-3x-1'] = [-3, -1, -5.3]
points['-2x-1'] = [-2, -1, -4.1]
points['-1x-1'] = [-1, -1, -16.8]
points['0x-1'] = [0, -1, -3.8]
points['-1x0'] = [-1, 0, -2.8]
points['-1x1'] = [-1, 1, -1.8]
points['-1x2'] = [-1, 2, -1.7]
points['0x2'] = [0, 2, -1.6]
points['1x2'] = [1, 2, -1.3]
points['1x3'] = [1, 3, -1.1]
points['1x4'] = [1, 4, -0.6]
points['1x-1'] = [1, -1, 15.8]
points['0x0'] = [0, 0, 12.8]
points['1x0'] = [1, 0, 23.1]
points['0x1'] = [0, 1, 14.2]
points['1x1'] = [1, 1, 19.7]
You can see that the default terrain has some random mountains and seafloor (the large i,j values) and then has custom, exactly-specified height values around the origin. In practice, you'll often want to do something like that: have much of the world be generated for you randomly, but take over control of the landscape near important points (villages, castles, chicken coops, etc.).
Later, around line 250, you can see where the terrain points are used to actually create the world terrain (the m.set(world.id, terrain=...) line).
Just to test the engine, I tried a simple mathematical formula to see how the client and server would perform. I replaced all of the above code with my own:
width = 64
w2 = width / 2
w1 = w2 - 1
for i in range(-w2, w1):
for j in range(-w2, w1):
r = sqrt((i * i) + (j * j))
points['%ix%i'%(i,j)] = [i, j, 5.0 + 5.0 * r * sin(8192.0 / (r + 1))]
I've got a radially-varying world, which is mostly flat near the origin but then has larger oscillations near the edges. I don't think this is a terribly interesting world to explore, but it shows how Mercator works. I found that ember and cyphesis were fine with this world! It is much bigger than the default world, but I didn't see any real performance issues. (Once I jacked up the width to something like 96 or 128, I did start noticing rendering delays, but a 64x64 world is pretty big). Because my random world didn't account for any of the other objects, I've got weird results like buildings underwater and boats (and fishes!) on the land, but you get the idea.
You can play with the terrain generation a bit, and see how things work. The usual workflow is this:
- tweak the define_world.py file with your updated terrain
- restart cyphesis
- run cyclient (so your new terrain gets uploaded)
- run ember and explore your new terrain
Objects
So now you are able to create your own terrain. The next step is to add your own items.
If you look through define_world.py, you can see where many objects are added to the world. For instance, around line 270 you can see where the village is added:
m.make('house3',type='house3',pos=(158,150,22),orientation=directions[1])
m.make('house3',type='house3',pos=(158,158,22),orientation=directions[4])
m.make('house3',type='house3',pos=(150,158,22),orientation=directions[0])
m.make('house3',type='house3',pos=(142,158,22),orientation=directions[7])
m.make('house3',type='house3',pos=(142,150,22),orientation=directions[3])
m.make('house3',type='house3',pos=(142,142,22),orientation=directions[6])
m.make('house3',type='house3',pos=(150,125,22),orientation=directions[2])
m.make('house3',type='house3',pos=(171,142,22),orientation=directions[5])
8 houses are added, all close to each other, and all pointing one of the 8 compass directions.
[An aside on the make() command. The m object is the mapeditor, created near the top of the default() function. The first argument to make is the name of the object you want to make. This name is just for public description (it isn't a unique identifier). After the name comes a list of key/value pairs, such as type='house3', pos=(150,158,22), etc. When making an object you have to define the name, type, and position. Other parameters are optional.]
Test case: an Inn
Suppose we wanted to add a new building? After the houses above, add the line:
m.make('Public House', type='inn', pos=(100,100,22))
Then restart cyphesis, run cyclient, and run ember. Volia! As you run towards the village, you should see an inn.
How did that work? It worked because both cyphesis and ember already knew what an inn was.
How did cyphesis know what an inn is? Objects are defined in the mason.xml file, in ~/.local/etc/cyphesis. Around line 1400 in mason.xml you can see the block of configuration where an inn is defined:
<map>
<map name="attributes">
<map name="bbox">
<list name="default">
<float>-4.25</float>
<float>-6</float>
<float>-2.8</float>
<float>10</float>
<float>5</float>
<float>8.2</float>
</list>
<string name="visibility">public</string>
</map>
<map name="burn_speed">
<float name="default">0.15</float>
<string name="visibility">public</string>
</map>
<map name="mass">
<float name="default">10000</float>
<string name="visibility">public</string>
</map>
</map>
<string name="id">inn</string>
<string name="objtype">class</string>
<list name="parents">
<string>structure</string>
</list>
</map>
Cyphesis doesn't know what the inn looks like at all, but it knows it has a large mass and a bounding box, and it knows its object type id is "inn" and it is a type of structure (inherits from that parent).
Ember uses a simple rules engine to determine what media to show. The rules can be found in ~/.local/share/ember/media/user/modeldefinitions/modelmappings.modelmap.xml. Here we find this line
<automodelmapping name="inn" />
This tells Ember to use the model named "inn" for all entities of type "inn". The definitions for the models can be found in the same directory as the mappings file, in this case in ~/.local/share/ember/media/user/modeldefinitions/buildings.modeldef file. Around line 200 in that file, you can see where the inn object is defined for ember:
<model name="inn" usescaleof="width" showcontained="true">
<translate x="4.000000" y="0.000000" z="-2.000000" />
<rotation x="1.000000" y="0.000000" z="0.000000" degrees="0.000000" />
<submodels>
<submodel mesh="3d_objects/items/building/models/castle/inn/inn.mesh">
<parts>
<part name="main" show="true">
<subentities>
<subentity index="0" material="/global/items/building/castle/inner_walls/stained" />
<subentity index="1" material="/global/items/building/castle/inner_walls/stained" />
<subentity index="2" material="/global/items/building/castle/floors/tile_floor" />
<subentity index="3" material="/global/items/building/castle/floors/tile_floor" />
<subentity index="4" material="/global/items/building/castle/floors/tile_floor" />
<subentity index="5" material="/global/items/building/castle/inner_walls/stained" />
<subentity index="6" material="/global/items/building/castle/outer_walls" />
<subentity index="7" material="/global/items/building/castle/outer_walls" />
<subentity index="8" material="/global/items/building/castle/outer_walls" />
<subentity index="9" material="/global/items/building/castle/outer_walls" />
<subentity index="10" material="/global/items/building/castle/outer_walls" />
<subentity index="11" material="/global/items/building/castle/roof/roof_shingle" />
<subentity index="12" material="/global/items/building/castle/roof/roof_shingle" />
</subentities>
</part>
</parts>
</submodel>
</submodels>
<actions />
</model>
This tells ember to use a 3d mesh located at ~/.local/share/ember/media/shared/common/3d_objects/items/building/models/castle/inn/inn.mesh. That mesh file is an Ogre .mesh file. You can convert back to the XML used to describe the model by running OgreXMLConverter.
It also tells Ember to use certain materials for different parts of the mesh. The definitions for the materials can be found in the file .local/share/ember/media/user/materials/scripts/buildings.material.
It further tells Ember to render all entities that are contained within this entity, to move (translate) the mesh a little (this can be needed to make the mesh fit better with the server side representation) and to solely use the width of the server side bounding box when determining how to scale the mesh.
The model definition can be edited in Ember by using the console command "/show_modelEdit". This will bring up the Model Editor. After you've altered the definition you can save it. This will write a file to ~/ember/user-media/modeldefinitions. The next time you start up Ember the files found in the aforementioned directory will be read first and take precedence to the default files. This allows you to experiment with different models configurations.
Deep Dive: Wolf
Here's a specific example to dive a little deeper. It shows how a single entity (a Wolf) is defined, and how multiple pieces (behavior, attributes, 3D model) are put together to render an instance of that entity.
Wolf: on the server
From the server's point of view (cyphesis), the root of the wolf definition is in the ruleset xml file. This is where all entities are defined.
In particular, look at etc/cyphesis/mason.xml. In that file, around line 2750, you'll see the top-level definition of the wolf entity:
<map>
<map name="attributes">
<map name="bbox">
<list name="default">
<float>-0.6</float>
<float>-0.26</float>
<float>0</float>
<float>1.3</float>
<float>0.26</float>
<float>1</float>
</list>
<string name="visibility">public</string>
</map>
<map name="mass">
<float name="default">20</float>
<string name="visibility">public</string>
</map>
<map name="maxmass">
<float name="default">30</float>
<string name="visibility">private</string>
</map>
</map>
<string name="id">wolf</string>
<map name="mind">
<string name="language">python</string>
<string name="name">mind.WolfMind</string>
</map>
<string name="objtype">class</string>
<list name="parents">
<string>character</string>
</list>
</map>
There are a couple of interesting definitions there!
- The ID ("wolf") and type of entity ("class")
- Bounding box for collision detection (I think)
- mass for physics calculations (not sure what the maxmass is for)
- Specification of a mind, which in this case is a python script at the location mind.WolfMind
- The parent entity ("character")
Scripts are (apparently) specified relative to share/cyphesis/rulesets/<ruleset-name>/, which in this case is share/cyphesis/rulesets/mason. If you look at the mind subdirectory, you'll see the WolfMind defined (WolfMind.py).
The wolf mind is pretty basic:
class WolfMind(NPCMind):
def touch_operation(self, op):
return Operation("talk", Entity(say="Grrrr"))
If you try to talk to it, it will just growl. But that's enough for cyphesis to work with! It can handle the creation of wolf objects, knows how big and massive they are, and will make them growl if you try to talk to them. At this point, it's still pretty uninteresting. But that's where the world populator (cyclient) comes in.
Wolf: the populator
Now that cyphesis knows what a wolf is, they can be created. Our world populator script (cyclient) does just that. Take a look at share/cyphesis/rulesets/mason/define_world.py and see how the wolf is created. There are multiple components.
First, there is a static definition of things our wolves can know about:
wolf_knowledge=[('w1','location',(90,-90,settlement_height)),
('w2','location',(110,-90,settlement_height)),
('w3','location',(110,90,settlement_height)),
('w4','location',(90,90,settlement_height))]
Our wolves know about 4 points, defining a narrow rectangle running north/south (assuming Y is aligned on the north/south axis).
Then there is a definition of simple AI strategy:
wolf_goals=[(il.forage,"forage('ham')"),
(il.hunt,"predate('pig',30.0)"),
(il.hunt,"predate('crab',20.0)"),
(il.hunt,"predate('squirrel',10.0)"),
(il.patrol,"patrol(['w1', 'w2', 'w3', 'w4'])")]
[My knowledge here gets very fuzzy--hopefully someone else can improve this]
Here the "interlinguish" object (il) is invoked. The wolf is told to forage for ham (find it and consume it), to hunt (actively chase those types of object and attack them), and then, if all else fails, to patrol its rectangle.
Finally, you can see where cyclient instructs cyphesis to create a wolf:
wolf = m.make('wolf', type='wolf', pos=(90,-90,settlement_height))
m.learn(wolf,wolf_goals)
m.know(wolf,wolf_knowledge)
m.tell_importance(wolf,il.forage,'>',il.hunt)
m.tell_importance(wolf,il.forage,'>',il.patrol)
m.tell_importance(wolf,il.hunt,'>',il.patrol)
Here cyphesis invokes the map editor (m) and asks it to create a single wolf with the public name 'wolf', of type 'wolf', and gives its position. It takes the return value (the ID of the new wolf instance) and puts it in a local variable named 'wolf'.
Then cyphesis gives the new wolf object its AI strategy (wolf_goals), knowledge of the world (wolf_knowledge). Then it specifies the relative ranking of the goals. As you might suspect, it puts foraging at the top: at any time, if a wolf sees a ham it will eat it. The next set of priorities are hunting. And finally, if there is no ham to be eaten and no animals to hunt, it will just patrol.
You can see it would be possible to create other wolves with different territories and even different hunting preferences.
With the two server-side components (cyphesis and cyclient) it is possible to specify wolf classes, create instances, and give specific instances unique behaviors.
Wolf: on the client
Okay, so now the server can create wolves. How do you see them on the client?
For the client, the root of all rendering is the share/ember/media/user/modeldefinitions/modelmappings.modelmap.xml file. Similarly to how the server uses the mason.xml file, ember uses the modelmappings.modelmap.xml file as the root of how rendering works.
If you look in the modelmappings.modelmap.xml file, you can see where the wolf is specified (around line 460):
<automodelmapping name="wolf" />
That "automodelmapping" key is just convenient shorthand that actually expands to this:
<modelmapping name="wolf">
<entitymatch>
<case equals="wolf">
<action type="display-model">wolf</action>
</case>
</entitymatch>
</modelmapping>
That is a modelmapping entry named "wolf". In fact, the name of the modelmapping entry is not important, it just has to be unique within the file.
Within the modelmapping entry is an entitymatch section, used to map object types to their models. In this case, there is a single entry. If the object has a type "wolf", then there is a "display-model" action which points to the wolf model. So that line:
<case equals="wolf">
is what really pulls everything together: that's how ember takes the object class provided by the server (cyphesis) and maps it to something local.
So that maps the "wolf" object type to a local model named "wolf". Where is this local model defined?
If you look in the creatures.modeldef.xml file in the same directory, around line 260 you'll see where the wolf model is defined:
<model name="wolf" usescaleof="height" showcontained="false" >
<submodels>
<submodel mesh="3d_objects/creatures/quadruped_v2/models/wolf/normal_wolf/normal_wolf.mesh" >
<parts>
<part name="main" show="true" />
</parts>
</submodel>
</submodels>
<actions>
<action name="__movement_walk">
<animations>
<animation iterations="1">
<animationpart name="walk" weight="1.0" />
</animation>
</animations>
</action>
<action name="__movement_idle">
<animations>
<animation iterations="1">
<animationpart name="idle" weight="1.0" />
</animation>
</animations>
</action>
</actions>
</model>
That model definition tells ember the following:
- There is a single 3D submodel, located at share/ember/media/common/3d_objects/creatures/quadruped_v2/models/wolf/normal_wolf/normal_wolf.mesh.
- That is the main submodel, and by default it is displayed. (There are more complicated models with multiple submodels, only some of which are displayed by default.)
- There are two animations: one for walking, and one for standing idle.
If you go to the share/ember/media/common/3d_objects/creatures/quadruped_v2/models/wolf/normal_wolf/ directory, you can see there are two files: normal_wolf.mesh (the model used for rendering) and normal_wolf.skeleton (which contains definitions of bones and how they move during animations).
If you run OgreXMLConverter, you can see the raw XML for each file. The mesh file maps each of its vertices to bones in the skeleton file. The skeleton file defines the bones, and defines how the bones move for the two animations "walk" and "idle".
Wolf: nutshell
To summarize:
- The ruleset xml file (mason.xml in this case) defines the object type ("wolf") and binds it to its mind. This is how cyphesis knows what a wolf object is.
- The world populator (cyclient, using define_world.py) creates instances wolf objects and binds their knowledge and AI strategy.
- The client (ember) maps from server-provided object types ("wolf") to local models using the modelmappings.modeldef.xml file.
- A given model can have multiple Ogre submeshes, and each submesh can be attached to skeletons which also provide animations.
Objects: summary
Instantiating objects is straightforward at a high level, plus one interesting twist. Like you'd expect, there are two obvious components.
First, you declare an object type in such a way that the server and client can handle instances of that type. For instance, the necessary entries in the ruleset and modelmapping xml files.
Second, you actually create an instance of the type that was defined. Above we created a new instance of an inn, and saw where a wolf was instantiated.
The unusual third component is runtime binding. Not only can you create new objects, you can also give them new knowledge and AI strategy at runtime. For instance, we created a wolf above, and then added its AI strategies and knowledge, and specified AI priorities.
More to Come
I think this is enough for experienced coders to dig in and start creating new content.
To do:
- show how to create new objects
- show how to create and modify behaviors + goals
Most importantly, it should be easy to separate objects/behaviors in isolated packages so they can be easily created and shared by anyone creating content.
At the moment, I am working on WorldCompiler that should make it easier to create worlds (that is, server packages that run a world and client packages that can connect and contain the media to display them). As part of that effort, I have been decoupling the server (DecouplingCyphesis) and client (DecouplingEmber) from the embedded rulesets and media.
