Update I
THANK YOU!
to all my faithful supporters :-D
This little fundraiser has already reached its second goal, currently we’re at 1521€ ! Special thanks and a big hug to each of these people for their very generous donations:
Vandre Brunazo
Nicola Jelmorini
Benjamin Dansie
Kajimba (assume this is you guys? consider donating to them too, their work is hilarious!)
Most work is currently being done on further design parts and investigating implementation details. Naturally there is not much visual stuff to show at this point (i like scribbling on paper for getting my ideas straight), but i’d like to create some more design diagrams on these issues. The basic particle system is in place as a modifier, just like other simulation systems such as cloth and fluids, and accessible in the physics button panel. The overall system for physical simulation and object data definitely needs a larger overhaul, but this can only be done properly as part of a general “scene graph” implementation. So for the time being this is an acceptable solution and in line with other feature designs.
The implementation of particle node execution will be done in C++ rather than C code, since this is much more convenient for the kind of polymorphic node types we’re dealing with. The new compositor and the cycles render engine also use this approach, so i’m in good company here. We may choose to share some code between these projects, but on the other hand the parts that can be used in the same way are not very complicated and can easily be recoded.
The first step when “executing” a particle node tree (which includes time steps as well as render geometry creation) is to make a localized copy of the node tree. This is to prevent changes made to the tree during execution from interfering with the nodes in use, which can lead to memory corruption and crash. Also the localized copy can be optimized for efficiency in contrast to the nodes in the editor, which are tailored to display and editing operators (e.g. all the lists of nodes and sockets can use constant arrays, since they’re not changing during execution).
Error message directly visible as node button (mockup)
One feature i want to implement early on is a debugging and report system for errors and warnings. These can be associated to specific nodes or the tree as a whole and should be visible directly in the editor. This way users can quickly identify the source of problems, which is not so easy when errors are just printed as text messages in a console (which will still be possible though for logging purposes).
The next step will be the implementation of the actual execution code. The details of implementation will be worked out over the next days, but these are the core points:
- All execution code will be implementing a generic “device” interface. While at this time only CPU execution will be implemented, this abstraction will hopefully make it possible to later extend the system with an OpenCL implementation.
- The work for each node should be split into “batches” (comparable to tiles in the compositor), which will avoid too large buffers with minimal allocation overhead. Choice of optimal batch size and allocation of memory is the responsibility of the device implementation.
- For parallel CPU implementation each batch can be calculated by either a single worker thread or split further to allow multiple threads to work on the same batch, depending on its size and the available threads (cores).
- All data should be stored in the form of aligned arrays of primitive data (“Struct of Arrays”, SoA) for efficiency.
Beside the work on node trees there are a few other parts of the base particle system implementation that need to be done. This includes usable drawing modes (currently only point drawing) and especially efficient point caching. The current implementation of the point cache is not well suited to dynamic buffers, all other point cache systems rely on a fixed point set that can be loaded efficiently by plainly iterating over the 0..n point index range. For dynamic buffers that start at some higher particle id this is a very inefficient method. Luckily i have collected enough experience with this problem when implementing paged buffers for the old particle system, so i’ve got some head start on that.
Further plans for the near future include adding a FAQ page, so i don’t have to answer the same questions over and over in comments ;-) and as soon as some useful code is in the git repository i will add a build instructions page too.
Cheers
Lukas
