Maya Alembic Export

Whilst this blog is mainly concerned with setting up alembic maya export within the University network, most of this is also applicable to other systems. You just need to change all the paths to the corresponding ones on your system.

First you need to download and build your alembic maya plugins. These come as part of the Alembic package and you should follow the build instructions. Once everything is built you will have a directory within the source tree called maya/plug-ins.

On the University system this is in the directory /public/bin/alembic/maya/plug-ins and you should see the two plugins AbcExport.so  AbcImport.so

University Setup
In the root of your home directory (type cd and press enter) execute the following commands

cd
mkdir MayaPlug
mkdir MayaScript
cp /public/bin/alembic/maya/plug-ins/* ~/MayaPlug

This will copy the Alembic plugins into a pre-defined directory which we will tell maya to search when starting up. This is controlled by the file Maya.env. Again the location of this file will differ depending upon the install, but in the University this is located here $HOME/maya/2011-x64/ If this file doesn't exist in the directory you can create your own using the following command

cd $HOME/maya/2011-x64/
touch Maya.env
gedit Maya.env

This will open the file and allow us to edit the maya environment variables used when maya starts. We need to add to this file the following

MAYA_PLUG_IN_PATH=/home/jmacey/MayaPlug
MAYA_SCRIPT_PATH=/home/jmacey/MayaScripts
PYTHONPATH=MAYA_SCRIPT_PATH

In the above example you will need to change the /home/jmacey to your own home directory path. This will then setup two areas that maya will search when looking for plugins ( $HOME/MayaPlug ) and scripts ( $HOME/MayaScript ) when you now start maya you should get the following list when opening the Menu Windows->Settings / Preferences -> Plugin-Manager

You should now see the AbcExport.so ( this screen shot from my mac is different as it uses a .bundle) and AbcImport.so. If you click on the Loaded button it will load the plugin and you should be able to type AbcExport -h in the Mel tab of the script editor as shown below

For more info on this read the blog post here. As the command line is a little bit complex, I decided to create a simple GUI to make life easier. The main design for this came from the output of AbcExport -h and all the options printed in the help that actually work have been translated into gui items.

AlembicExport.py

The AlembicExport.py script can be downloaded from here and it should be saved in the $HOME/MayaScripts directory.

When using the script you will need to select all the geometry you wish to export (if you select all Alembic will attempt to export all it can ) and type the following in the python script editor

from AlembicExport import *
AlembicExport()

This will give you the following GUI

The current frame range is selected and by default uv's and normals will be exported. Other options are available and you should read the AlembicExport help for more details. 

The actual alembic jobstring and command line is placed in the job string text field so you can copy this if you wish to use the command line at a later date.

Code outline
The code is fairly self explanatory, however I will outline a couple of areas.
First we check to see if the AlembicExport plugin is installed. This is done with the following code

# check to see if plugin is loaded
plugs=cmds.pluginInfo( query=True, listPlugins=True )
if "AbcExport" not in plugs :
  print "AbcExport not loaded please load it"

To build up the jobstring for the actual export we use the following code

jobstring="AbcExport "
if self.verbose == True :
  jobstring+=" -v "
jobstring+="-j \" -fr %d %d -s %d" %(self.start,self.end,self.steps)
.....

This will build up a complete export command which we will then execute using the eval command as follows

mel.eval(jobstring)

Maya standalone python

I've just had an email from an ex student asking about automating the export process from maya as at present they load each scene by hand and then use a particular plugin to export in a new format.

My suggestion was to use the standalone maya python interpretor and try to semi automate the process, to do this I wrote a simple proof of concept as follows.

The following script scans the current directory for any maya ASCII files, opens the file and selects all. Then exports this as an obj file.

First we need to enable the maya standalone system

import maya.standalone
import maya.cmds as cmds
import os

maya.standalone.initialize(name='python')

The line above imports the maya.standalone module, next we need to initialise this and tell it which interpretor we are using with the name='python' command. We now have an empty maya environment which should have read our Maya.env so all the paths etc are setup. However none of our default auto-loaded plugins are loaded. In this case I wish to use the obj export plugin so need to load it. As this is a simple proof of concept I don't do any checking to make sure it is loaded etc.

cmds.loadPlugin("objExport")

The next batch of code scans the current directory and checks for .ma files then does the export

files = os.listdir(".")
for mayafile in files :
  if mayafile.endswith(".ma") :
    cmds.file(mayafile,o=True)
    cmds.select(all=True)
    newFile="%s.obj" %(mayafile)
    cmds.file(newFile,type="OBJexport",pr=True,es=True)

The rest of the code uses the standard maya.cmds module to first open the file then select all elements.

Next I create a new filename by adding ".obj" to the end of the file loaded and export with the file command.

To run the script we need to use the mayapy command. This should be in the same directory as the rest of the maya executables. On my mac this is /Applications/Autodesk/maya2011/Maya.app/Contents/bin/mayapy but you will need to add it to your path

To run I've saved the file as export.py and run mayapy export.py

ngl::Matrix vs Imath::Matrix44

As I've been working with the Alembic file I/O system for a while I've been using some of the IMath functions as Alembic is build upon IMath / OpenEXR base code. IMath is a templated maths library that, to quote the website, "Imath, a math library with support for matrices, 2d- and 3d-transformations, solvers for linear/quadratic/cubic equations, and more".

Half way through using this I started wanting to use my own ngl::Matrix library as this is integrated into my code base, it when that I discovered that the two were not fully compatible.

Whilst both have very similar functions, the one core difference was how the matrix*matrix multiplication worked, (pre / post multiplication of values). In the end I decided to modify how the ngl::Matrix * operator worked so that it is compatible with IMath::Matrix44

Using IMath in ngl::
To start using IMath in ngl (or other programs) we need to set the compiler include paths to the correct place. By default they are installed in /usr/local (with the headers being in a directory OpenEXR)

A number of the classes in IMath are templated header only files so we don't need to add any additional libs, however some of the functions may also need the additional libImath.so (or static .a version)

To add these in a Qt project file add the following lines

INCLUDEPATH+=/usr/include/
LIBS +=-lImath

Once these have been added to the project we need to add the following header to the program

#include <OpenEXR/ImathMatrix.h>
#include <OpenEXR/ImathVec.h>

The IMath::Matrix44 class is a templated class so we need to construct it to be compatible with the ngl::Matrix class using a float. The following code is going to construct both an ngl::Matrix and a IMath::Matrix44

float xRotation=45.0f;
// Imath
Imath::Matrix44 <float> iXMatrix;
iXMatrix.setAxisAngle(Imath::Vec3<float>(1,0,0),ngl::radians(xRotation));
// ngl
ngl::Matrix nXMatrix;
nXMatrix.rotateX(xRotation);

In the above example the Imath matrix is constructed and will be set to the identity matrix as default. We then use the setAxisAngle method to set the matrix as a rotation around the x axis by xRotation degrees. This method is passed a vector for the axis to rotate around and a value for the rotation which must be converted into radians.

The ngl::Matrix class is also set to the identity when it's constructed, and to set the rotation value we use the rotateX method (which expects the rotation values in degrees).

We can check the output of this by using the overloaded << operators as shown

std::cout<<"X rotation "<<xRotation<<"\n"<<nXMatrix<<"\n"<<iXMatrix<<"\n";


X rotation 45
[+1.0000000000000000,+0.0000000000000000,+0.0000000000000000,+0.0000000000000000]
[+0.0000000000000000,+0.7071067690849304,+0.7071067690849304,+0.0000000000000000]
[+0.0000000000000000,-0.7071067690849304,+0.7071067690849304,+0.0000000000000000]
[+0.0000000000000000,+0.0000000000000000,+0.0000000000000000,+1.0000000000000000]

(  +1.0000000000000000   +0.0000000000000000   +0.0000000000000000   +0.0000000000000000
   +0.0000000000000000   +0.7071067690849304   +0.7071067690849304   +0.0000000000000000
   +0.0000000000000000   -0.7071067690849304   +0.7071067690849304   +0.0000000000000000
   +0.0000000000000000   +0.0000000000000000   +0.0000000000000000   +1.0000000000000000)

Another useful feature of the ngl::Matrix class is that is can be constructed from a float [4][4] as shown here

Matrix::Matrix(Real _m[4][4])
{
  for(int y=0; y<4; ++y)
  {
    for(int x=0; x<4; ++x)
    {
      m_m[y][x]=_m[y][x];
    }
  }

}

This means we can construct an ngl::Matrix from an Imath matrix as shown

  
Imath::Matrix44 <float> iZMatrix;
iZMatrix.setAxisAngle(Imath::Vec3<float>(0,0,1),ngl::radians(zRotation));
// alternativly we can construct from an Matrix.x array
ngl::Matrix nZMatrix(iZMatrix.x);

The listing below show the complete program and the output

int main()
{
  float xRotation=45.0f;
  float yRotation=35.0f;
  float zRotation=15.0f;


  Imath::Matrix44 <float> iXMatrix;
  iXMatrix.setAxisAngle(Imath::Vec3<float>(1,0,0),ngl::radians(xRotation));
  ngl::Matrix nXMatrix;
  nXMatrix.rotateX(xRotation);

  Imath::Matrix44 <float> iYMatrix;
  iYMatrix.setAxisAngle(Imath::Vec3<float>(0,1,0),ngl::radians(yRotation));
  ngl::Matrix nYMatrix;//(iYMatrix.x);
  nYMatrix.rotateY(yRotation);

  Imath::Matrix44 <float> iZMatrix;
  iZMatrix.setAxisAngle(Imath::Vec3<float>(0,0,1),ngl::radians(zRotation));
  // alternativly we can construct from an Matrix.x array
  ngl::Matrix nZMatrix(iZMatrix.x);

  std::cout<<"X rotation "<<xRotation<<"\n"<<nXMatrix<<"\n"<<iXMatrix<<"\n";
  std::cout<<"y rotation "<<yRotation<<"\n"<<nYMatrix<<"\n"<<iYMatrix<<"\n";
  std::cout<<"z rotation "<<zRotation<<"\n"<<nZMatrix<<"\n"<<iZMatrix<<"\n";

  std::cout<<"ngl mult y*x \n"<<nYMatrix*nXMatrix<<"\n";
  std::cout<<"iMath mult y*x \n"<<iYMatrix*iXMatrix<<"\n";
  ngl::Matrix nxyz=nXMatrix*nYMatrix*nZMatrix;
  std::cout<<"ngl mult x*y*z \n"<<nxyz<<"\n";
  Imath::Matrix44 <float> ixyz=iXMatrix*iYMatrix*iZMatrix;
  std::cout<<"iMath mult x*y*z \n"<<ixyz<<"\n";

  Imath::Matrix44 <float> iInverse=ixyz.inverse();
  std::cout<<"inverse \n"<<iInverse<<"\n";
  ngl::Matrix nInverse=nxyz.inverse();
  std::cout<<"inverse \n"<<nInverse<<"\n";

  Imath::Vec3<float> iPos(1,2,3);
  std::cout<<"i V*M "<<iPos*ixyz<<"\n";
  ngl::Vector nPos(1,2,3,1);
  std::cout<<"n V*M"<<nPos*nxyz<<"\n";
  std::cout<<"n M*V"<<nxyz*nPos<<"\n";
}

Which gives the following output

X rotation 45
[+1.0000000000000000,+0.0000000000000000,+0.0000000000000000,+0.0000000000000000]
[+0.0000000000000000,+0.7071067690849304,+0.7071067690849304,+0.0000000000000000]
[+0.0000000000000000,-0.7071067690849304,+0.7071067690849304,+0.0000000000000000]
[+0.0000000000000000,+0.0000000000000000,+0.0000000000000000,+1.0000000000000000]

(  +1.0000000000000000   +0.0000000000000000   +0.0000000000000000   +0.0000000000000000
   +0.0000000000000000   +0.7071067690849304   +0.7071067690849304   +0.0000000000000000
   +0.0000000000000000   -0.7071067690849304   +0.7071067690849304   +0.0000000000000000
   +0.0000000000000000   +0.0000000000000000   +0.0000000000000000   +1.0000000000000000)

y rotation +35.0000000000000000
[+0.8191520571708679,+0.0000000000000000,-0.5735764503479004,+0.0000000000000000]
[+0.0000000000000000,+1.0000000000000000,+0.0000000000000000,+0.0000000000000000]
[+0.5735764503479004,+0.0000000000000000,+0.8191520571708679,+0.0000000000000000]
[+0.0000000000000000,+0.0000000000000000,+0.0000000000000000,+1.0000000000000000]

(  +0.8191520571708679   +0.0000000000000000   -0.5735764503479004   +0.0000000000000000
   +0.0000000000000000   +1.0000000000000000   +0.0000000000000000   +0.0000000000000000
   +0.5735764503479004   +0.0000000000000000   +0.8191520571708679   +0.0000000000000000
   +0.0000000000000000   +0.0000000000000000   +0.0000000000000000   +1.0000000000000000)

z rotation +15.0000000000000000
[+0.9659258127212524,+0.2588190436363220,+0.0000000000000000,+0.0000000000000000]
[-0.2588190436363220,+0.9659258127212524,+0.0000000000000000,+0.0000000000000000]
[+0.0000000000000000,+0.0000000000000000,+1.0000000000000000,+0.0000000000000000]
[+0.0000000000000000,+0.0000000000000000,+0.0000000000000000,+1.0000000000000000]

(  +0.9659258127212524   +0.2588190436363220   +0.0000000000000000   +0.0000000000000000
   -0.2588190436363220   +0.9659258127212524   +0.0000000000000000   +0.0000000000000000
   +0.0000000000000000   +0.0000000000000000   +1.0000000000000000   +0.0000000000000000
   +0.0000000000000000   +0.0000000000000000   +0.0000000000000000   +1.0000000000000000)

ngl mult y*x 
[+0.8191520571708679,+0.4055798053741455,-0.4055798053741455,+0.0000000000000000]
[+0.0000000000000000,+0.7071067690849304,+0.7071067690849304,+0.0000000000000000]
[+0.5735764503479004,-0.5792279839515686,+0.5792279839515686,+0.0000000000000000]
[+0.0000000000000000,+0.0000000000000000,+0.0000000000000000,+1.0000000000000000]

iMath mult y*x 
(  +0.8191520571708679   +0.4055798053741455   -0.4055798053741455   +0.0000000000000000
   +0.0000000000000000   +0.7071067690849304   +0.7071067690849304   +0.0000000000000000
   +0.5735764503479004   -0.5792279839515686   +0.5792279839515686   +0.0000000000000000
   +0.0000000000000000   +0.0000000000000000   +0.0000000000000000   +1.0000000000000000)

ngl mult x*y*z 
[+0.7912400960922241,+0.2120121568441391,-0.5735764503479004,+0.0000000000000000]
[+0.2087472975254059,+0.7879844307899475,+0.5792279839515686,+0.0000000000000000]
[+0.5747727155685425,-0.5780408978462219,+0.5792279839515686,+0.0000000000000000]
[+0.0000000000000000,+0.0000000000000000,+0.0000000000000000,+1.0000000000000000]

iMath mult x*y*z 
(  +0.7912400960922241   +0.2120121568441391   -0.5735764503479004   +0.0000000000000000
   +0.2087472975254059   +0.7879844307899475   +0.5792279839515686   +0.0000000000000000
   +0.5747727155685425   -0.5780408978462219   +0.5792279839515686   +0.0000000000000000
   +0.0000000000000000   +0.0000000000000000   +0.0000000000000000   +1.0000000000000000)

inverse 
(  +0.7912400960922241   +0.2087472826242447   +0.5747727155685425   +0.0000000000000000
   +0.2120121717453003   +0.7879844903945923   -0.5780409574508667   +0.0000000000000000
   -0.5735764503479004   +0.5792279243469238   +0.5792278647422791   +0.0000000000000000
   +0.0000000000000000   -0.0000000000000000   +0.0000000000000000   +1.0000000000000000)

inverse 
[+0.7912402153015137,+0.2087473124265671,+0.5747727751731873,+0.0000000000000000]
[+0.2120122015476227,+0.7879846096038818,-0.5780410170555115,+0.0000000000000000]
[-0.5735765099525452,+0.5792279839515686,+0.5792279243469238,+0.0000000000000000]
[+0.0000000000000000,+0.0000000000000000,+0.0000000000000000,+1.0000000000000000]

i V*M (+2.9330530166625977 +0.0538582801818848 +2.3225636482238770)
n V*M[+2.9330530166625977,+0.0538582801818848,+2.3225636482238770,+1.0000000000000000]
n M*V[-0.5054649114608765,+3.5224001407623291,+1.1563749313354492,+1.0000000000000000]

The main upshot from these changes occur in the ngl demos where the code to load the matrices to the shader have been modified as shown in the code here

ngl::Matrix MV;
ngl::Matrix MVP;
ngl::Mat3x3 normalMatrix;
ngl::Matrix M;

// load matrix to shader before changes to ngl::Matrix

M=_tx.getCurrentTransform().getMatrix();
MV=m_cam->getViewMatrix() *_tx.getCurrAndGlobal().getMatrix();
MVP=m_cam->getProjectionMatrix()*MV*;

// new version with compatible matrix
M=_tx.getCurrentTransform().getMatrix();
MV=_tx.getCurrAndGlobal().getMatrix()*m_cam->getViewMatrix() ;
MVP=MV*m_cam->getProjectionMatrix();

As you can see when we calculated the previous MVP matrix it was done using the P*V*M calculation order, now we use M*V*P instead. If you are having any issues just swap the matrix order in these type of functions.

Pipeline Stuff

As we are starting the group project section of the Masters term I thought it would be a good idea to do some basic pipeline stuff.

This example is going to write some simulation data created in a C++ program and then we will write some simple Python scripts to load this data into Maya and Houdini.

The following video shows the program in action as well as the Maya and Houdini versions.

As this is an ad-hoc system the output file format is very simple, the C++ program writes out the following data :-


NumParticles 500
Frame 0
P0 64.301 -57.7284 49.8516

....

Frame 1

....

Where the particle data consists of The name of the Particle (in this case P0,P1....Pn) and the x,y,z positions of the particle per frame.

Maya Version

The maya version of the program will popup a dialog box to prompt the user to select an input file, it will then parse the file and create a locator for each of the particle names found.

Then for every frame a keyframe is created for the locator replicating the animation from the C++ simulation.

import maya.OpenMaya as OM
import maya.OpenMayaAnim as OMA
import maya.cmds as cmds

The code above loads in the maya elements we need, OpenMaya contains the core maya elements we need, the OpenMayaAnim namespace has the controls for all the animation transports such as setting the current frame and maya.cmds gives us access to all the maya cmds similar to the mel commands printed in the console when we generate things.

def createLocator(_name,_x,_y,_z) :
    cmds.spaceLocator( name=_name)
    cmds.move(_x,_y,_z)
    cmds.setKeyframe()

The createLocator function creates a simple space locator then moves it to the current x,y,z position we then set the keyframe to make the initial key at the currently set frame (more on this later)

def moveLocator(_name,x,y,z) :
    cmds.select(_name)
    cmds.move(x,y,z)
    cmds.setKeyframe()

The move locator function, first selects the locator (based on the name passed in), it then calles the move command which will move the currently selected object, finally we set the keyframe.

def importParticleFile() :
    basicFilter = "*.out"

    fileName=cmds.fileDialog2(caption="Please select file to import",fileFilter=basicFilter, fm=1)
    if fileName[0] !=None :

 file=open(str(fileName[0]))
 frame=0
 numParticles=0
 #set to frame 0
 animControl=OMA.MAnimControl()
 animControl.setCurrentTime(OM.MTime(frame))

 for line in file :
  line=line.split(" ")
  if line[0]=="NumParticles" :
   numParticles=int(line[1])
  elif line[0]=="Frame" :
   frame=int(line[1])
   animControl.setCurrentTime(OM.MTime(frame))
  else :
   name=line[0]
   x=float(line[1])
   y=float(line[2])
   z=float(line[3])
   if frame==0 :
    #we need to create our initial locators
    createLocator(name,x,y,z)
   else :
    moveLocator(name,x,y,z)

Houdini Version
The houdini version of the script will generate a null which is the houdini equivalent of a locator. By default a houdini null doesn't contain any geometry so we also need to parent this to some axis (houdini call these controls).


To start with we need to get the name of the file, this is done using the getAbsoluteFilename function described here http://jonmacey.blogspot.com/2011/01/houdini-python-ascode.html

def createNull(parent,_name,x,y,z) :
 #create a null this will set loads of default values
 null = parent.createNode("null", _name, run_init_scripts=False, load_contents=True)
 # set the x,y,z values
 null.parm("tx").set(x)
 null.parm("ty").set(y)
 null.parm("tz").set(z)
 # now add a control to the null so we have something to visualise
 null.createNode("control", "ctrl"+_name, run_init_scripts=False, load_contents=True)
 # now grab the keyframe
 setKey = hou.Keyframe()
 # set to frame 0
 setKey.setFrame(0)
 # now key the tx/y and z values
 setKey.setValue(x)
 null.parm("tx").setKeyframe(setKey)
 setKey.setValue(y)
 null.parm("ty").setKeyframe(setKey)
 setKey.setValue(z)
 null.parm("tz").setKeyframe(setKey)
 # now add to our network node

This function is passed the parent node, which in this case will be the houdini path "/obj/" from this we create a new node called a "null" and set it's parameters. In Houdini all parameters can be access using the parm(...) method of the object passing in the name of the parameter we wish to access, in this case "tx/ty/tz". Next we set the keyframes using the hou.keyframe class.

def moveNull(_name,frame,x,y,z) :
 null=hou.node("/obj/"+_name)
 setKey = hou.Keyframe()
 setKey.setFrame(frame)
 setKey.setValue(x)
 null.parm("tx").setKeyframe(setKey)
 setKey.setValue(y)
 null.parm("ty").setKeyframe(setKey)
 setKey.setValue(z)
 null.parm("tz").setKeyframe(setKey)

In this function we grab the object by name then set the keyframes using the same method as above. Finally we are going to create our nodes and import the file, the file reading code is exactly the same, however we create a subNetwork first to make the houdini scene neater.

def importParticleFile() :

 fileName=GetAbsoluteFileName("Select particle File","*.out",hou.fileType.Any)
 # get the the object leve as our parent
 if locals().get("hou_parent") is None:
  parent = hou.node("/obj")

 # make sure we got a filename
 if fileName !=None :
  subNetName=hou.ui.readInput("Please Enter name for the subnet")
  ## @brief we now copy this to a new string
  subNetName=subNetName[1]

  subNet=parent.createNode("subnet", subNetName, run_init_scripts=False, load_contents=True)
  #open the file (could do a proper check here)
  file=open(fileName)
  frame=0
  numParticles=0
  # now process the file and create the nulls
  for line in file :
   line=line.split(" ")
   if line[0]=="NumParticles" :
    numParticles=int(line[1])
   elif line[0]=="Frame" :
    frame=int(line[1])
   else :
    name=line[0]
    x=float(line[1])
    y=float(line[2])
    z=float(line[3])
    if frame==0 :
     #we need to create our initial locators
     createNull(subNet,name,x,y,z)
    else :
     moveNull(subNetName+"/"+name,frame,x,y,z)

To grab the full code use the following

DemoProgram (requires NGL)
particles.out (the output of the program which the python files read)
Houdini Script
Maya Script