New features and future plans

I apologize for the long silence here, my excuse is that i’ve been working on internal stuff lately, which is not well suited for producing eye-popping videos ;) Part of this work is better parallelization support and integration of the OpenCL API. My original goal for this blog post was to make a more detailed description of how this all works internally, but it turned out to be a rather long article, which should better be part of the design document and needs a little more work. Like most coders i’m not too fond of writing documentation (a necessary evil, i know), so please be patient. :) If you have any specific questions regarding OpenCL or CPU multithreading, feel free to ask in the comments or IRC or mailing list. Let me talk about some fun stuff and new features instead now.

New quaternion and matrix socket types

These usually describe rotations and transformations. There are currently no math nodes to modify these data types, but they can be selected and copied from data nodes. Note that euler angles and axis-angle representations are also converted to quaternions! This might be a bit of a problem when using euler angles, since the order of rotations matters for these. It’s best to directly use real quaternion rotation data where available.

ForGroup node and the ‘self’ data context

As described in the previous post, there are different data contexts associated to each socket in the node tree. An powerful feature is the use of the “self” context. This is a special keyword that can be used in path inputs to refer to the context a data node is evaluated in. At the moment the places in which node trees exist are limited (in my branch they are still part of a ‘NodeTree’ modifier for particle sets), so the ‘self’ context on the base level isn’t all that interesting. But now there is a special loop node called “ForGroup”, which makes use of this feature:

The nodes left of 'ForGroup' are applied to every object in the group

This simple tree moves all objects in group “Group” by a small amount along the x-axis by adding to the location vectors. Note that the GetData and SetData nodes have no explicit input path set: they work in the object in the ‘self’ context. The ‘ForGroup’ node executes the subtree on the left side for each object in the group and sets the ‘self’ context accordingly. The path inputs of the data nodes could also use a relative path like “self.data.vertices” to access the vertices of the self-context object.

Plans and ideas

  • Make shortcuts to common data nodes: current GetData/SetData nodes are a little cumbersome to use and many of the properties should not be accessed (partly this is due to missing flags in the RNA property settings). Making restricted data nodes for the most common data types (objects, meshes, etc.) will make this easier.
  • Memory management is an important issue in node trees (this has been noted with compositor trees for some time). The idea is to activate nodes “from left to right”, so that the tree does not allocate all node buffer memory at one time. Instead the nodes should be activated successively, so that buffer memory allocation stays below a threshold to avoid problems and free memory as soon as possible.
  • Another possible feature for saving memory could be the “concatenation” of certain nodes: Many node kernels (esp. math nodes) only work on one single work item at a time. When several of these nodes are chained together (as is often the case in complex math expressions), the internal buffers between them might be avoided completely.
  • Many people have been asking for specific node tree uses. While there are many good ideas among these, you will have to accept that my time and capabilities are limited. Here are some of the things that will unfortunately not be part of my work (which doesn’t mean somebody else couldn’t implement them some day using the features i added):
    • Node tree drivers (as an alternative to python expressions)
    • Video compositing
    • L-Systems
    • Mesh modifier replacement
    • Rigging systems
  • What i will concentrate on instead is the implementation of particle simulation features using nodes. Next step for creating more interesting particle effects will be a mesh sampling node for generating points on a mesh surface/volume. This will allow the implementation of common particle emitters.

28 Comments

  1. gustav says:

    Seems like you’re making good progress! Can’t wait for those eye-popping videos :wink:

  2. maces says:

    Sounds very good. Can’t wait to see opncl boosted particle simulations in “eye-popping videos”.

  3. Roger says:

    This is fun and exciting enough, at least for coders like us! looking forward to the OpenCL implementation details, very very very exciting work here.

  4. Germano Martins says:

    You will replace OpenGL by OpenCL or it will be an alternative renderer to the internal renderer?

    • phonybone says:

      These are 3 different things:
      * OpenGL is used for realtime rendering of the UI and viewports. It is purely a realtime graphics API.
      * OpenCL is a more generic programming system that _can_ make use of the graphics card. There are ways of making it communicate with OpenGL to avoid unnecessary data copying in video memory, but the systems still do very different things.
      * The internal renderer is completely unrelated to OpenGL and OpenCL. LuxRender as an external renderer does make use of OpenCL though: http://www.luxrender.net/wiki/index.php?title=Luxrender_and_OpenCL

  5. Scott says:

    Hooray, sounds all awesome! Waiting for particles!

  6. Moolah says:

    I decided to not dig in coding but I’ll be using your system for constructing some cool particle effects for sure!
    Thank you for your great work!

    One question: what is the “quaternion rotation”? I’m guessing something and a possibility to search in “Coder’s part” of the BlenderArtists’ forum scares me :oops:

  7. Let me know If you want help making videos? showcasing use of the node tree.

    One more thing that could be done with nodes is loading external data and use it to drive animations.

    awesome news to find another blender developer /enja is also exploring opencl/ making ground in opencl.

    just recently I’ve been doing fluid sims, maybe there’s to advance algorithms to be lifted onto the cores of the GPU.

    but the softbodies calculations must be more straight forward and should be able to be calculated by the gpu.

    even so that’s beyond your scope doing nodificiation of everything ;D just wanted to throw it in there for people stuck in that opencl is for visual calculations. it’s perfect for particles. I’ve seen a video om vimeo calculating 1million particles in realtime w opencl.

  8. Blenderificus says:

    great new developments, thanks for your hard work on this new branch! cant wait to test it out

  9. tstscr says:

    hi phonybone,
    i would have asked you in irc but where i am atm irc is blocked for some unknown reason.
    The thought i had was about instancing Nodes, or even NodeGroups. Do you think that could be possible? I think that could be quiet handy at times. (with the option to make single attributes local?)

    • phonybone says:

      I think this is not necessary and kind of duplicate design: Nodes should have as little internal state as possible (non-socket options). So there would be very few things that distinguish nodes from each other and would have to be copied (in my case e.g. data paths). Everything else can be done using a few further nodes. When you want two nodes to use the same default input value, you should create a value node, which provides the input to both. Having another reference to a common “actual node type” would just make things complicated and not increase possibilities.

  10. tstscr says:

    yeah, guess that’s a valid argument :)
    I just had the idea from compositing apps.

  11. Dan says:

    This could prove very useful if integrated into Blender well!

    I think Blender would benefit from a Node + scripting modeling tools similar to Grasshopper 3D for Rhino, and this great work looks like a step in the right direction!

  12. Great work! Keep it up! I very excited to see more nodes in Blender, since compositor first started.

  13. Stephen says:

    phonybone is your build now in the main blender trunk?

    Commit by lukastoenne :: r32740 /branches/particles-2010/source/blender/ (10 files in 5 dirs): (link)
    Particle properties can now be accessed from data nodes. Particle properties currently still have to be added in the particle details panel before being used in the node tree, this could be changed so that properties are added automatically.
    Brought back the AddParticle node.

  14. Stephen says:

    ahh no, just noticed, its in the particles branch DOH!

  15. Luciano says:

    so basically you want to make everything in blender a node?, just like in maya… right?::

    id love blender even more!!

  16. gustav says:

    have anyone manage to compile the branch on window recently?

    makesrna.exe crashes for me with MSVC+scons (command output http://www.pasteall.org/17121)

    • phonybone says:

      Sorry for that, but the build log just doesn’t contain enough info to be of any help here. I personally use cmake on linux for coding, so the build files for msvc/scons maybe outdated. I will happily apply any patches for these though and i’ll try to get scons working at least on linux when i find the time.

      • gustav says:

        Compiling once more give this error:

        scons: *** [C:\dev\build\particles-2010\source\blender\makesrna\intern\rna_actio
        n_gen.c] Error -1073741819

        Don’t now it that’s more helpful… anyhow I tried managed to compile with cmake so I’m happy =)

  17. teldredge says:

    Do you have any .blend file examples handy? I’ve compiled your branch and it runs fine but can’t seem to get anything moving as in the examples. I’m very excited about your work! Thanks a lot.

    • gustav says:

      Check the previous blog posts, there are some videos showcasing the functionality. But I agree more examples would be great (especially about those new mesh sampling and data interpolation nodes), however I suspect that there’s still stuff to do before the system is functional enough create cool stuff.

  18. Ejnaren says:

    Hi Lukas.
    We are starting to use grasshopper at my school, which sparked my curiosity…
    I know that the nurbs system of blender isnt anyway near that of rhino and i know the node system isnt meant to be used for geometry etc.
    BUT.. hypothetically (I love that word) would it be possible to make blenders nodes system interact with the any property of the RNA system and program with nodes.
    I ask you because I am very fascinated with your video where you program the rotating box with nodes…
    ??
    Best regards.

  19. Stephen says:

    Just wanted to say happy new year, hope it treats you well.

  20. Hey! I could have sworn I’ve been to this blog before but after browsing through some of the post I realized it’s
    new to me. Nonetheless, I’m definitely glad I found it and I’ll be book-marking and checking
    back frequently!

  21. We give a thorough examination to make sure we find hidden mold as well as
    visible mold. You can take help of mold remediation Charleston SC
    service providers to get rid of these unwanted guests.

    When mold is found, a homeowner must contact certified mold remediation
    companies to find a competent contractor to handle the situation.

Leave a Reply