DecouplingEmber

Description

Similar to DecouplingCyphesis, this is a log of my attempts to get ember running within a self-contained directory tree, and with some autogenerated media.

See DecouplingUseCases for the motivation. Basically, I'm trying to build a WorldCompiler that will make it easy for world authors to take a bunch of existing worldforge binaries and media and put together playable worlds.

Feature Requests

As I go, I'll try to keep track of feature requests that would help decoupling efforts. I'm trying to keep this list prioritized. My intention is to suggest feature enhancements that make decoupling easier but don't impact current functionality at all.

1. Get rid of the ember script that wraps ember.bin! All necessary configuration should be explicitly passed to the binary at startup, or a config file explicitly specified. The current wrapper script attempts to infer a lot of information and sets important parameters in a way that is invisible to users and world authors.
2. Like cyphesis, don't have the binary know about any implicit path information. For instance, when internally constructing paths to config files, don't hardcode concatenations like "ember". Instead, the paths should be fully specified by the caller (or installer) without any path mangling by the binary.
3. Get rid of all the shared objects! Just have a statically linked executable. Yes, it can save a bit of disk space by keeping shared objects shared, but then users of the software get exposed to "DLL hell" (multiple applications possibly stomping on each other's dependencies). Gamers don't want to debug shared library configuration--they'll just give up.
4. Pull the dynamic media config out of the ember wrapper script and make it easily user-settable (explicit parameter).
5. Pull the skybox behavior out of Caelum.zip and make it first-class objects. At the moment the Caelum package has knowledge of objects outside of itself (such as sphere.mesh) so it isn't really self-contained, and its status as a zip'd package makes it difficult to examine or modify.
6. Pull any path information out of ember.conf. At the moment, the file references some media (such as JPEGs used for shader textures). All of the other parameters are purely operational, and can be set/unset independent of the media. But with the [shadertextures] section, suddenly there is an implicit dependency on the media used. This should probably be configured some other way.
7. Are the "carpenter" or "jesus" directories in share/ember used? I don't think they are.

Log

This is a quick log of my attempts to decouple.

Media Tree

At the moment, the local media is quite extensive. Here is just the share/ember/media directory tree (I have truncated the tree heavily, there are many more branches!):

media
|-- shared
|   |-- common
|   |   |-- 3d_objects
|   |   |   |-- body_parts
|   |   |   |-- clothes
|   |   |   |-- creatures
|   |   |   |-- environment
|   |   |   |-- items
|   |   |   `-- plants
|   |   |-- 3d_skeletons
|   |   `-- junkyard
|   |-- gui
|   |   |-- cegui
|   |   |   `-- datafiles
|   |   `-- icons
|   |-- models
|   |   `-- environment
|   |-- packs
|   `-- sounds
`-- user
    |-- materials
    |   |-- base_scripts
    |   |-- programs
    |   |-- scripts
    |   |-- splatting
    |   `-- textures
    |-- modeldefinitions
    |-- models
    |   `-- environment
    |-- packs
    `-- particle

Within the media/user/modeldefinitions directory are the modeldef files that are the heart of the ember rendering (at least for my purposes of binding server-side objects to client media).

My job now is to figure out how to make it easy for an automated script to generate entries in modeldef files and properly populate the other sections of the directory tree. For instance, based on spelunking at WorldCreation, a modeldefinition entry is created first (in media/user/modeldefinitions), then a model is declared (often in the same directory), and that model will refer to a 3D mesh (which I think has to be specified to the media/common directory).

Partial Success

Today (9/30) I was able to get a compiled version of ember mostly working!

Again, by "compiled" I probably mean something more like "packaged." Basically:

1. A single content directory containing all class definitions, model definitions, behaviors, meshes, etc, was created.
2. From that content directory tree, a WorldCompiler assembled two complete packages: a server package (cyphesis and all necessary config) and a client package (ember with all necessary config and media). For ember, this meant autogenerating the modelmapping and modeldefinition files.
3. I ran the server package as-is (see DecouplingCyphesis).
4. I tar'd and gzip'd the client package, copied it to a remote machine, unzipped it and ran it.

It pretty much works! There are a few textures I need to chase down (some are missing in the rendering) but a lot is showing up.

One step back, two steps forward

[6 oct 07] Actually, it turns out I was cheating with my 9/30 success. My newly-created ember client package was still picking up random configuration in the ~[user]/.ember directory.

Ember is designed to handle dynamic updates to media (images, meshes, gui, etc.) so it has two directory trees. One tree is the static tree that the user installs, and the other tree is a dynamic tree which is designed to change over time. Ember typically looks for media in the dynamic tree, and falls back to the static tree if it needs to.

This behavior is spread over multiple places instead of being centralized. That is, multiple components know about and set up this situation, so it is tricky to fix.

First of all, the ember wrapper script sets the location of the user override file, and copies media over to it if need be. The wrapper script puts the dynamic directory in a hidden directory in the user's home directory, which clearly doesn't scale if the user has multiple client installs. So I've moved the dynamic directory into the client install directory. However, I changed a few directory names in the process, and that broke ember. Ember has built-in assumptions about the directory structure within the dynamic directory. So I had to get the user's directory set up correctly.

Now I believe I have correctly set everything up, and I have clean installs that are mostly working. I'm still missing a few meshes to define the skybox (easy to add), and a few other environment materials aren't defined.

Done

[6 oct 07] Now the world compiler is working! It isn't 100% feature-complete, but it is now correctly compiling objects (and the rest of the world!) and creating usable client + server packages. That is, directory trees that can be copied to other machines and run without modification.

Moving now to the general project, DecouplingWorldForge.