Ember: Add better multi threading support

From WorldForgeWiki
Jump to: navigation, search


Original Proposal

Project proposal by Manuel A. Fernandez Montecelo (User:Mafm) for GSoC 2009.

Synopsis

Currently Ember works single-threadedly, and there are areas where it could take advantage of multi-threading for performance reasons, and more important, to enhance the player experience by not having momentary freezes (when performing some actions or loading/processing data).

The goal is to use multi-threading in Ember, starting by well-defined subsystems (such as network, sound, resource processing...) where independent execution makes sense, and always putting thread-safety first.

Benefits for WorldForge

The benefits for WorldForge are mainly:

  • Enhanced user experience in situations were the UI could become less responsive.
  • Enhanced overall performance, taking advantange of more processing cores that desktop systems offer today (and in the foreseeable future).
  • Allowing to perform new tasks such as unintrusively generating or animating content in the world -- without fear of stalling more important tasks/threads.

Deliverables

  • The modified Ember client with areas working in multi-threaded mode.
  • If decided so (by other WorldForge developers) and needed, modified WorldForge intermediate libraries needed for things to work properly, or to work better.

Detailed Description

The goal of this project is to enable multi-threading for large areas of the game, so this implies investigating which parts should be modified, then implement the solution, and then checking that the result is satisfactory. In detail:

  • First, the project will involve identifying subsystems and main areas where independent execution makes sense from the design point of view (due to not needing continuous interactions with many other parts of the game, where they have enough weight to justify using threads, etc). Brief interviews with people working in the client, especially main designers, will come in handy. The result would be a listing of interesting parts, some estimation of the benefits of using one (or several) threads for each of those parts, as well as identifying spots where problems might arise.
  • After identifying those areas, design some detailed plan for each of them, and subsequently proceed to implementation. Each part should be tested separately after the implementation, to ensure that things continue to work and there are no side-effects, trying to stress-test those parts. Do the same for WorldForge libraries that would need to be modified (if decided so), but probably not too much time should be spent in this part (to maintain the focus).
  • Lastly, test thoroughly the whole new system, and try to measure the result of the modifications from several points of view (performance, growth in complexity and risks created while doing this, etc) to see whether the benefits are there or not (and their magnitude).

The library used would be Boost::Threads, because it's already being considered by WorldForge people, and also because of my own experience with it (GSoC 2007, in example).

Project Schedule

Plan:

  • Prior to the beginning of the coding phase, getting acquainted with the codebase and development tools and having things ready to start when the official period starts (or when school duties permit).
  • The first step (probably 2 weeks) should be to identify the different areas where threading techniques would be applicable.
  • The following weeks (3-9) should be dedicated to plan and implement multi-threading in each area, and testing the results. It depends on the difficulty of these areas and the problems that will arise, but probably an average of 1 week per area is a good target, leaving room for aronud 7 areas of the code (each of which would be a milestone).
  • The next remaining weeks (10-12) would be dedicated to testing the whole system, fixing remaining problems, integration and closing the project. This will be used also as a "buffer", should I need to take some free time (school schedules or small vacation).

Development of the Project

Design Documents

Sound

Date: 28 May 2009

Sound with OpenAL works in a non-blocking manner, and the processing is not very intensive, so there's probably very little benefit in adding multi-threading.

One of the problems to use several threads here is that the structure is not very straightforward: there are separate classes for samples, sources, bindings, instances of sounds, you name it, as well as subclasses and interfaces like ISoundMotionProvider -- some of which are called during the part of updates, which would be probably the only good candidate for threading (due to being called in loops). And all this is used from a different set of classes of the Ogre version of Ember, calling and called back.

The scenario would be very different if, in example, there was a kind of SoundServer (Service) interfacing with OpenAL, where a few sounds were asked to play at a given time (e.g. background music looping, steps of our character walking in the world, sounds of animals). In that case, the game would command the server to play a background sound sporadically, and then sounds for the immediate environment, maybe also GUI interactions. The server would have an entry point accepting commands of sound samples to play, then it could operate by itself scheduling the commands and performing any other related actions such as loading resources, etc.

It becomes apparent that this would need a thorough rewrite of the sound support in Ember, taking several weeks/months, so at the moment this idea is put in the freezer.

Likewise, multi-threading in this area is put in the end of the queue, only to look at it again if there are no better areas to look at.

Erik's comments

The nice thing about the sound service is that through the use of OpenAL the main part of the sound playing is already multithreaded: OpenAL provides its own thread through which it plays all sounds. So the application only needs to do three things: 1) start and stop the sounds 2) provide updates on position, pitch, volume etc. 3) provide filled data buffers for the sounds

No. 3 is the one that immediately presents itself as something which should be done in a separate thread. Filling the buffers is a purely data intensive task. There are basically two kinds of sounds: those that fit into the memory at once, and as such only need one buffer which can be filled when the sound starts playing, and those that are too large to fit into memory at once, and needs to be streamed (i.e. the buffers needs to be filled from disk as the sound is played). Currently Ember only support the former kind, but we should of course also support the second one. It's also worth mentioning how the sound resources are handled. Since Ogre already has a very nice resource system we reuse this for sound resource loading. However, we don't want to expose Ogre to the sound service, so we instead wrap it in an instance of IResourceWrapper. If we put loading of the resources and filling of the buffers (here also ogg decoding is involved) into the system we have to make sure that we call the Ogre resource system in a thread safe way.

No. 1 and no. 2 are however things that needs to be done on the same thread as the main execution thread (as long as we plan to keep the Eris polling in the main thread, which I think we will want to for now at least). The updates of the sounds positions comes from Eris, when it's polled (i.e. entities move etc.) and from the input updates (camera movement mainly, which also happens on the main thread). Likewise the starting and stopping of the sounds happens due to Eris events.

The current SoundInstance::update() method currently does both #2 and #3, which it shouldn't. The call to mBinding->update(), which is what makes it update its buffers, should probably happen in a separate method so that it can be called from another thread. We might also want to look into the lifecycle of the instances, how they are destroyed mainly so there's no issue there. --Erik 16:52, 30 May 2009 (UTC)

Manuel's comments

Date: 15 June 2009

I don't quite grasp the whole design of the Sound Service, which I just revisited briefly for continuing to work on this issue. For example, I don't understand very well the difference and why is it needed to have different classes for SoundInstances, SoundBinding, SoundSource etc if Sources aren't shared by different Instances. In my view, SoundInstance (or Source, or whatever name you choose) could encapsulate all this things without being an incoherent or too-fat abstraction.

This doesn't mean that I'm telling you that you're (or Romulo, or whoever the designer) wrong in this respect, or that I want you to explain it to me right now, I'm just telling this because I might be saying something stupid in the paragraphs below.

However, and taking on your comment of SoundInstance::update(), my limited understanding on this area is that you have a bit of an entanglement when you update the position of the listener from the camera into the SoundService in the main loop (OGRE component commanding base framework), but then SoundService::cycle() also called in the main loop but later (or before) of the previous call, commands SoundInstance to ask for updates to the OGRE component, which is SoundAction, which calls SoundEntity, which calls EmberPhysicalEntity to know the position, which is defined in EmberEntity, which is defined in Eris::Entity.

Wouldn't it just be much simpler if Eris/Ember entities would have different representations, and along with the visual representation and its updates, the acoustic representation (or more than one: sounds of movements, originating in the throat, tools/weapons which the entity is wielding can happen at once) and maybe others (like it would be out-of-world on-screen representation, showing on a window stats about an entity) was also updated in the same step?

That is, Ember at some point (which I haven't investigated) learns (either it's notified or asks for the info) that Entity's new position for this frame is point A in 3D space. It could then command all related classes depending on this information to update its location via an event, or directly calling a special method, or directly telling them to do a generic per-frame update where the related classes would pull information from the main Entity class.

The sound instances (or sources or whatever the name) would be created and destroyed on demand when a sound is required or it's not required anymore, the position updates would be called predictably as they're now, and it would perform data-buffer filling if needed for streamed sounds in another thread (which currently it's not implemented, anyway -- nor the threading in streamed sounds, nor even the streamed sounds themselves).

I think that this approach would eliminate a whole series of cascading abstractions while hopefully not damaging a good design, and maybe making it better by means of simplification.

But well, since streamed sounds are not implemented at all yet, and it would be the only part where multi-threading would be worth having in the sound area, I take it that my work in this area is mostly finished -- or do you want me to investigate or implement something else?

Log

August

August 18:

  • Trying to solve remaining problems, chiefly the cracks in the terrain which are probably due to misalignment of the edges in neighbor tiles. I tried to find out differences with the original code and tried several ways to enforce setting neighbor tiles once they're loaded (so their renderables are made neighbors too, thus hopefully aligning edges) without luck. However, I fired the current "official" Ember and I have the same problem, so I conclude that either the problem was present but went unnoticed before, or it was introduced in "official" Ember and then passed down to my repository via merge. Therefore I'd say that my job with GSoC is finished, just needing to merge it in trunk if desired.
  • Sent weekly/final GSoC report to Ember's mailing list.

August 17:

  • Firm pencils down date. However I couldn't work due to problems with the ISP.

August 14:

  • Communicating with Erik about remaining issues.
  • Fixing some remaining issues, especially improving documentation of the newly created class for background processing.
  • Testing a new, bigger terrain, without observing anything abnormal; and verifying that the terrain pages are created in the background thread orderly, and not all at once -- so both the background thread and the PLSM seem to work properly.

August 11-13:

  • Connectivity problems due to ISP mistakes for a couple of days.
  • Taking care of mandatory bureaucratic issues, so between this and the connectivity problems I could accomplish very little done.

August 10:

  • Sending weekly report.
  • Reviewing code of EmberOgre::Terrain, mostly adding const'ness where possible in different files, so the dangerous zones to look for are reduced.
  • Reorganizing Makefile.am in the dir of EmberOgre::Terrain, since there were duplicated files and stuff out of place.

August 9:

  • Taking updateEntityPositions() out of the background thread after creating the page, it's not thread safe (entities might be removed) and the time taken by the function is negligible, typically 0.000073s.
  • Moving the background thread class to its own file.

August 6/7:

  • Finally getting the program to work, creating terrain pages in the background, mainly by taking the code of loading the texture for the shadow to the main thread (suggested by Erik).
  • Optimizing the parts of the page that can be processed in the background thread, since in the first rounds were throwing suboptimal results (half of the time in the background thread, another half in the main thread). For this several changes took place, with the main effect of creating the shadow data also in the background (loading the shadow has to be done in the main thread, and generating terrain materials after loading the shadow).
  • Current measurements related with the creation of pages in the background thread are as follows:
    • The time for creating the pages in the background (without the two steps mentioned above) in my system are: 0.226746s, 0.232994s, 0.226035s, 0.247494s.
    • The time of post-processing that formerly belonged to creating the page (loading shadow into the texture and generating terrain materials) are: 0.082837s, 0.071303s, 0.087059s, 0.129147s; of which the second step typically takes 0.005231s (so I didn't consider into trying harder to perform that part in the background thread). This would be hard to improve, since it's basically something that happens in Ogre.
    • The complete time of post-processing (including the processing back at the PLSM and the two steps above) are: 0.126788s, 0.079426s, 0.094186s, 0.136089s. As we can see, these values have more variance than the previous ones.
    • As a summary, now:
      • We do about 1/4th of the processing in the main thread, 3/4ths in the background thread, excluding the PLSM part (which we decided a while ago to not be put in threads, it would be too complex).
      • Starting with a stall in the main (only) thread of 0.50s to create a page (not counting time in PLSM), Erik had decreased that to about 0.30s, and now it's reduced to a stall of 0.07 to 0.13s that happens after preparing the rest of the page in the background thread. That's a speedup of 400-700% compared to the original (14-23% of the time needed originally), a speedup of 230-430% (23-43% of the time needed after Erik's modifications); and as we can see in this example only in one page the time approaches to 0.13s, so theoretically the values should be more close to the maximum improvement.
      • And of course, more than numerical improvements in performance, this is important because the end user will feel that the stalls produced during page creation are much less pronounced, so the overall experience is enhanced; very important when the terrains start to be much bigger (as planned).

August 5:

  • Compiling Ember for new Ogre version
  • Merging from the main trunk after commit of Erik from an experimental branch. Not many conflicts but a bit messy, it took quite a while to get into shape.
  • Updates to doxygen doc as requested by Erik
  • Continuing investigating about the crashes when using the background thread.

August 4:

  • Still recovering from illness in the past few days
  • Talking a bit with Erik about current status and future work in the remaining time for GSoC
  • Updating, compiling and installing new Ogre 1.6.3.

August 1:

  • Trying to solve the problems of exceptions during rendering causing not showing the terrain pages and plain crashes, after some advice from Erik, but still no luck.
  • Sending weekly report in advance, because maybe I won't be available on Monday.

July

July 31:

  • Scattered work through the day trying to find what's the problem, with the help of Erik. Still no luck.

July 30:

  • Updating and compiling Mercator and Ember, now that my patch to Mercator (needed for my current branch to work) was applied by Al.
  • Working on the problems encountered yesterday, finding no suitable solution.
  • Preparing to push to my remote repository in WF servers, which were almost 30 commits away from my current development branch after two weeks without pushing.
  • Preparing and sending documentation for GSoC, which I hadn't done yet.

July 29:

  • Trying to find out what's the problem causing the aborts when using the background thread to create pages, and trying many combinations to avoid it (all of them unsuccessful), like loading the image of the texture in the main thread, failing and trying to create the shadow for the page instead in the main thread, failing and trying to create everything from that point in the main thread, etc. The crash often involves failure when allocating memory (to load the image for the texture); and according to the manuals, man malloc, "Crashes in malloc(), calloc(), realloc(), or free() are almost always related to heap corruption, such as overflowing an allocated chunk or freeing the same pointer twice", and the debugging environment vars that they advice to set don't cause any effects. It seems that the fear that I expressed in mails and past log entries is becoming real, and either the calls from the background thread to Ogre (to create new memory, load images, etc) is not safe and causes these awkward problems, or the interactions between the many lines of code crossing many classes in Ember::Terrain are interdependent and non-thread safe, or a combination of both.
  • I also tried to work around this problem by using full thread support in Ogre, which despite maybe existing bugs (it says that it's a experimental feature) and loss in performance (due to lock contention even when not needed) should allow the application to call Ogre from many threads, in a safe way. I haven't observed any difference in the behavior. At this point, I'm running out of ideas.

July 28:

  • Finished the implementation, but the program doesn't work well because it aborts when loading the texture of the shadow of the page:
 terminate called after throwing an instance of 'std::bad_alloc'                                                                           
 what():  std::bad_alloc
 
 at:
 #7  0xb7db0b63 in Ogre::Texture::loadImage (this=0xb3ed99c0, img=...) at OgreTexture.cpp:116

July 27:

  • Sending weekly report
  • Reworking the implementation, still some problems due to abstract classes when creating a thread

July 23/24:

  • Implementing most of the solution devised to create pages in the background. I'm having a few problems because of boost::thread idiosyncrasies, but in principle the solution should work and I'm almost there. Another question is if when everything is in place it will work fine and without crashes, because the code to create the page accesses a lot of code from many classes and I'm not sure if everything is thread-safe, but well...

July 22:

  • I had lots of stability issues with X again today, so I took the opportunity to update drivers and recompile Ogre with semi-threaded more. But then it spits the following message shortly after starting and before creating the window: "ember.bin: /usr/include/boost/thread/pthread/recursive_mutex.hpp:72: void boost::recursive_mutex::unlock(): Assertion `!pthread_mutex_unlock(&m)' failed."; with non-threaded Ogre runs fine. So one more problem for loading Ogre resources in the background...

July 21:

  • Starting to implement the solution mentioned before, especially in the area of asynchronous loading. For this, a lot of code had to be shoveled around, because PLSM2 assumes that the loading would be synchronous, with some predefined interval between each page operation of loading/preloading. Another problem is that there are a lot of delegations from one class to another, so for this to work the asynchronous notifications have to be added to all the chain of calling classes. Once the support is in place for the full hierarchy, I'll the actual loading of terrain data from a parallel thread.

July 20:

  • Sending weekly report
  • Designing a solution about the approach mentioned before.

July 17:

  • Continuing to investigate and discussing by mail with Erik a possible solution, which is to use only the part in EmberOgre::Terrain to load terrain pages in the background (the rest of the areas in PLSM2 are too intricate, so multithreading in this area would be quite thorny, full of locks everywhere with the inherent peril of deadlocks and degraded performance at best).

July 16:

  • Researching about background threads to load terrain pages, especially the interactions inside PLSM2 (Paging LandScape Scene Manager) code.

July 14/15:

  • Code analysis to try to find out whether loading terrain pages (OgrePagingLandScapePage) asynchronously in a background thread is doable. Apparently it's very difficult inside the PLSM, because loading a page touches almost every other area of the 20k LoC of the PSLM (loading textures, materials, setting properties on tiles of neighbor pages which might be being unloaded in parallel, etc), so adding locks all around causes contention or worse, deadlocks.

July 13:

  • Sending weekly report.
  • Filling midterm evaluation survey.
  • No changes worth to merge since last merges in the middle of the past week.

July 10:

  • Continue improving OgrePagingLandScapePage and the Manager, removing unused stuff and encapsulating data better, with the intent of simplifying everything to be able to load pages asynchronously.

July 9:

  • Completing the rewriting of OgrePagingLandScapePage. From the most significant log entry: "Complete rewrite of this class, trying to maintain functionality intact for the outside but removign unused parts, factoring out code from functions, using logging to facilitate debugging, simplifying a bit API for classes using it, enhancing design with the use of states of the page (loaded, inited, etc) and in general much improved readability and ease of maintenance, with the idea of making things clear for being able to load pages in background threads. Result: from 719 to 594 LoC, 125 lines or almost 18% less, and there's still some room for improvement."

July 8:

  • Performing measures of several steps in the process of loading pages, in OgrePagingLandScapePage. loadTexture() usually takes as maximum a bit above 0.03s (significant), other times much less than 0.001s (insignificant), preload() is the most demanding step requiring up to 0.3s.
  • Rewriting OgrePagingLandScapePage because it's in a very messy state (unused methods and variables, a mix of modifications over the original designs, etc). Still WIP and not commited.

July 7:

  • Preparing weekly report. Merging from main Ember repository.
  • Communicating with Erik about project's progress.
  • Checking what things have changed in code in the last commits of main repository. At first sight, the time needed to load a terrain page is much smaller, since times of different parts to create pages have changed substantially (using typical times):
    • The operation to create a new page increased from 0.000181s to 0.044629s
    • The operation to add shaders decreased from 0.286488s to 0.030581s
    • The operation to create the shadow is the same 0.08s
    • The operation to generate terrain materials for the page remains in the same 0.003894s
    • The operation to show foliage continues with about 0.062913s
    • All in all, creating a page now means 0.25 to 0.30s, much lower than the previous ~0.5s


July 1-3:

  • Busy with work, and couldn't work normally because of power outages due to unannounced repairs in local electrical grid of the street (including losing my working sessions, UPS ran out of batteries...).

June

June 29:

  • Preparing weekly report.
  • E-mailing Erik, discussing strategies about topics related with terrain loading.

June 26:

  • Look at how terrain pages are loaded, from the PSLM. Initially one page is told to load, then subsequent pages (starting from neighbors) are told to either load, preload, or nothing at all (it depends on the visibility from the camera); and there's a elapse of time (defaulting to 30 frames I think) to load each of those in turn, probably with the intent of not stalling the game initially (or later, when loading new sets of pages) while doing so. In our case, since all the terrain comprises 4 pages (512x512 m each, IIRC), all of them are neighbors to each other, and all of them have to be loaded. If the pages where smaller, many of them wouldn't probably be loaded initially because they are in the back side of the camera, outside the field of view. Also, processing them would imply much less resources (256x256 is 1/4th the size of 512x512) and probably would be less noticeable. Anyway, the intent is to load them asynchronously, so the player won't experience freezes while loading them.

June 25:

  • Finishing and perfecting the class created yesterday.
  • Testing it with real code from Ember. The tested point was the method TerrainGenerator::createPage(), in a point where it adds shaders to a page, in a loop with a single sentence page->addShader(shader). The code seems to be essentially parallel with no interdependencies with the outside. The original code takes around 0.30-0.36s in my machine, and with simple modifications (instantiating the class to process in parallel, creating the derived class with essentially the same page->addShader(shader) sentence, adding instances of this class as jobs in the same loop, and finally telling it to run) the resulting loop takes 0.18-0.20s in my machine with 2 cores, so it's almost a linear speedup. Of course this can be used anywhere where there are parallel things to compute with no interdependencies, there's nothing terrain specific in this approach.

June 24:

  • Playing with boost::thread to create a simple abstraction to perform somewhat heavy computations in parallel in a very easy way. The idea is to have a class to perform the processing, just telling it how many threads are desired (defaults to the number of processing cores in the machine), adding jobs to be processed (classes derived from an abstract class, performing the computations in a single entry point/method), and then telling that class to process the added jobs as fast as it can.

June 23:

  • Revisiting TerrainGenerator thoroughly (still WIP), to try to isolate internal pieces as much as possible, so then we're able to use background threads for processing parts of terrain when building it up or in updates.
  • Sync with main Ember repository.

June 22:

  • Preparing weekly report.
  • Performing some profiling, to know where Ember is having bottlenecks/stuttering. So far the call in the main loop to Eris poll and the handling of input, apart from rendering, are two places where this happens (even if apparently it shouldn't), but times are much higher than 0.01s, sometimes getting to 0.5s.

June 19:

  • Removing references all around to TerrainGenerator and Mercator (and sometimes to other related classes). The goal is to have as few places interacting with these classes as possible, so we can then isolate and protect the data being modified in parallel threads (for building heightmaps, converting terrain format from Mercator to Ogre, etc).

June 18:

  • Continuing to study how the terrain system works, and discussing with Erik and Al about ways to improve how things work. Part of the problem is that there's all around accessing EmberOgre::Terrain classes, sometimes even the "internal" Mercator classes in them, so it's dangerous and very messy to use threads with this -- even if we leave things working now, small slips from new developers (or even ourselves) in accessing data as it's modified can wreak havoc.
  • Reduced the time to run a loop with 256K iterations (performing to Mercator to Ogre data conversion) to ~60% of the time (from a little over 0.06s to a little less than 0.03s in in my machine, which instead of crippling the rendering down to a maximum of 16FPS in that frame, it raises the maximum to more than 35FPS), if the terrains grow substantially (as it's the idea) it will be a nice optimization. Hopefully we can break the terrain up in smaller pieces, and/or make them run in separate threads.

June 17:

  • Learning more about terrain, and discussing with Erik and Al about alternatives to the current situation.

June 16:

  • Learning about Terrain generation, to know if and how could benefit from multi-threading.
  • Tons of modifications of files while at it, mainly for const correctness, but also for other reasons (header cleanup, indentation of parts where they were quite unreadable, removing unused macros or variables/parameters, etc).

June 15:

  • Preparing weekly report. Main Ember repository didn't change for the last week, so no merges in my branch.
  • New follow-up comments about Sound (see above).
  • Fixing some issues with commits from past week, as noted by Erik (method naming, unintended effects with changes in code loading resources).

June 12:

  • Putting up a new desktop machine and studying the possibility to use with a new capable graphics card (I'm planning to buy nVidia GeForce 6800 AGP). This should be enough to work when the integrated Intel graphics card of my laptop give the problems mentioned in the previous days.

June 10:

  • Trying to debug the problems when implementing loading of materials with listeners, but the program crashes and when using debuggers the machine just freezes X leaving the computer unusable until you reboot (killing X and the usual workarounds are not enough).

June 9:

  • Investigating and discussing with Erik about the best ways to take advantage of multi-threading when loading resources (meshes, materials... in models). Starting to implement loading materials with listeners (instead of polling).
  • Enhancements all around in code that I found along the way: avoiding duplicated code, added some code for safety (e.g. constructors for mandatory initialization), using const or const& in many places (both for safety and to avoid copies), cleanup of headers. Some of this is necessary (or helps) for my project, since it's quicker to compile, safer and easier to reason about code, etc.

June 8:

  • Prepared weekly report, and merging from main Ember repository.
  • Reading pending mail from Erik and continuing working on the project, now focusing on Model loading in the background.

June 1-7:

  • Traveling because of work in the first part of the week, announced to mentor many days ago.
  • Sick during the mid-end of the week, with fever and so on.
  • Result: no productive work for GSoC this week.

May

May 31:

  • Preparing weekly summary.
  • Using ccache to speed up compilations, it takes now 1/4th of time for a full compilation (after filling the cache, and without modifications of files).

May 29:

  • Reverting some of the formatting changes to doxygen documentation and so on, it seems that it's the style of WF to do it that way.
  • Spent some time talking with Erik about coding/documentation styles, and other collaboration-related practical issues.
  • Starting to look into Model loading (taking advantage of OGRE semi-threaded way of working).

May 28:

  • Removing parts with pthread, since they're not used or probably not correct, or not working. If desired, now they can be reimplemented with Boost::Thread easily with a few lines of code (probably badly, since they didn't offer safety not using mutexes, etc).
  • Spent quite a lot of time seeing Ember blocking the input of my machine and using 100% of one processor, I had to ssh from another one and kill Ember manually. It seemed to have some issue, and I spent a lot of time recompiling parts trying to see if it was a problem with my newly introduced Boost::Thread linking option, or the removal of -pthread code. A full recompilation (though lengthy) fixed it, the real problem is unknown.

May 27:

  • Enable Boost::Thread and remove pthread support in the configure file, still not working fully.
  • Continue studying the sound, if it's worth to substitute pthreads with Boost::Thread or if it can't be avoided at all (calls to OpenAL library seem to be non-blocking and operate independently, so the gain in multi-threading this part is marginal).
  • Minor cleanups and enhancements while looking at the code.

May 26:

  • Looking at the feasibility of multi-threading sound services, one of the initial areas that I had in mind, but it turns out that it's not very interesting.
  • Peer-reviewing part of the files that I crossed with, enhancing them in some way or another (fixing bugs, adding/completing documentation, removing dead code, ...).

May 25:

  • Speaking briefly with Erik about some practical aspects, like what happens with the integration with the building system.
  • Preparing/submitting weekly report.


May 21:

  • Compiling OGRE with semi-threaded mode as suggested by Erik (so Ogre uses threads internally to load resources, but it's called from only one thread in Ember), having to also compile CEGUI from sources and then Ember after them.
  • Prepared my repository with the last changes before I started with GSoC (it was set up several days ago and some important changes were missing).

May 20:

  • Studying Ember's code.
  • Continue learning about git.

May 19:

  • Learning about Eris and Ember architecture.

May 18:

  • Program starts on 25th, but I started to work today, ahead of schedule, because I might need some free days (for personal and professional reasons) in the next weeks.
  • Preparing this wiki page and creating new categories for GSoC and GSoC years in the wiki, might be useful for other students and in general (as means to organize information).
  • Contact Erik asking for guidance, about which areas of Ember are to be tackled first, in his opinion. I already got things ready for running during the past weeks.
Retrieved from "https://wiki.worldforge.org/w/index.php?title=Ember:_Add_better_multi_threading_support&oldid=5181"