Staring out the Windoze

My usual day to day development environment is a mac, and I usually write code using Qt / g++ so it's easy to port it to Linux which is the main development system at the University. However most of the students seem to use Windows as their main computer system so I had to really support windows with my NGL library.

Luckily the Windows version of the QtSDK comes with a g++ compatible compiler called mingw and once the other libraries were installed in the correct places the library seems to compile and link into a DLL.

Problems with GLEW


The first problem I encountered was with the GLEW library. The versions available for download are compiled with the visual studio compiler and are not compatible with the mingw compiler / linker. To make this easier to use and integrate into the ngl library I added the source of the latest GLEW tree to the ngl project and this is included into the windows branch of ngl automatically in the Qt project file. This is done using the following code

win32: {
        LIBS += -lmingw32
        LIBS += -lstdc++
        DEFINES += WIN32
        DEFINES += USING_GLEW
        DEFINES +=GLEW_STATIC
        INCLUDEPATH += C:/boost_1_44_0
        INCLUDEPATH += C:/boost
        LIBS += -L/lib/w32api
        DESTDIR = c:/
        SOURCES+=$$BASE_DIR/Support/glew/glew.c
        INCLUDEPATH+=$$BASE_DIR/Support/glew/
        DEFINES+=_WIN32
        CONFIG+=dll
        DEFINES+=BUILDING_DLL

}

Once this code is in place, the windows version of Qt will set the win32 specific flags and compile correctly and also include the glew elements statically into my library and hence reducing a dll dependancy.

Cross Platform Development
Some other conditional code was also required to make things compile correctly on all platforms. NGL uses 3 flags to set specific platform dependent includes and flags. These are found in the .pro file as part of the qmake process and the following shows these

linux-g++:{
          DEFINES += LINUX
          LIBS+=-lGLEW
}

linux-g++-64:{
          DEFINES += LINUX
          LIBS+=-lGLEW
}
macx:DEFINES += DARWIN

So not the compiler will pass on the command line -D[DARWIN][LINUX][WIN32] dependent upon the platform being used. This is then used to switch in the main header files for the libs, and to place this all in one include file the <ngl/Types.h> is used and has the following conditional compilation code

#if defined (LINUX) || defined (WIN32)
  #include <gl/glew.h>
  #include <gl/gl.h>
  #include <gl/glu.h>
#endif
#ifdef DARWIN
  #include <unistd.h>
  #include <opengl/gl.h>
  #include <opengl/glu.h>
#endif

With all this in place the library built with no problems and a .dll / .a file was generated under windows and a simple demo application would run.  I left the library under windows at this stage and continued the development under mac, however I began to get reports of things not working under windows. Initially I thought it was a driver / gpu problem as it seem to occur in the shader manager class.

When things go wrong

The first indications of things not working was when a lookup to a shader returned the null program which is a safety mechanism. The following code shows the method and the call to the method.

ShaderProgram * ShaderManager::operator[](
                      const std::string &_programName
                     )
{
  std::map <std::string, ShaderProgram * >::const_iterator program=m_shaderPrograms.find(_programName);
  // make sure we have a valid  program
 if(program!=m_shaderPrograms.end() )
  {
  return  program->second;
  }
  else
  {
    std::cerr<<"Warning Program not know in [] "<<_programName.c_str();
    std::cerr<<"returning a null program and hoping for the best\n";
    return m_nullProgram;
  }
}

// in application
ngl::ShaderManager *shader=ngl::ShaderManager::instance();
(*shader)["Blinn"]->use();

It appeared at first that the map find method was at fault as it was not finding a shader that I knew was in the class.  However I started to print out the size and contents of the map I noticed that in some cases the map contained the correct values and in other cases it was empty. Why would this be?

This shouldn't happen!

It seemed weird that this was happening as the class was based on a singleton pattern so there should only be one instance of the class, also this bug only appeared in the windows version so I was unsure what was going on.

I placed a number of debug statements in the singleton template class used for the shader manager and discovered that the constructor was being called twice WTF!

My initial response was that I was having a weird threading issue and my singleton wasn't thread safe, so I added a QMutex / QMutexLocker construct around the singleton and also made it inherit from boost::noncopyable so that it could not be copied.

This however was not the solution to the problem as it still continued.

Digging Deeper

After digging deeper into the code I traced exactly where the rouge call to the singleton ctor came from. In my TransformStack there is a convenience method to load the current Transform into a parameter of the shader, the code is actually in the ngl::Transform class and looks like this.

void Transformation::loadMatrixToShader(
                                        const std::string &_shader,
                                        const std::string &_param,
                                        const ACTIVEMATRIX &_which
                                       )
{
  computeMatrices();
  ShaderManager *shader=ngl::ShaderManager::instance();
  switch (_which)
  {
    case NORMAL :
    {
      shader->setShaderParamFromMatrix(_shader,_param,m_matrix);
    }
    break;
    case TRANSPOSE :
    {
      shader->setShaderParamFromMatrix(_shader,_param,m_transposeMatrix);
    }
    break;
    case INVERSE :
    {
      shader->setShaderParamFromMatrix(_shader,_param,m_inverseMatrix);
    }
    break;
  }
}

This method actually lives in the library, and hence under windows in the DLL, and this is where the problem begins.

Separate compilation units
It seems in the windows DLL the library data is in a different scope when constructed unlike the unix / mac shared libraries. In this case when the dll invokes the shader manager singleton, it doesn't exist in the DLL memory space even though it has been called in the main application linking to the DLL. In other words the singleton is not shared by default. There is a good post here about it.

To test this was the case I quickly created a static global variable, within the transform class which could be set outside the DLL module by a quick method call. This basically passed in the application side instance of the Shader manager class, and used that in the call, and it fixed the problem. However Rob the Bloke pointed out the error of my ways by saying

"That's a critical bug and refactor waiting to happen. If you need to share something across DLL boundaries, there is exactly one way to achieve that. Your solution fails the second anyone adds another DLL that links to ngl (i.e. you've just offset the problem for your students to deal with later). s_globalShaderInstance *needs* to be DLL exported."

Making it a "proper" dll

It was now time to bite the bullet and do a major re-factor of the code to make all the classes export from the DLL properly, I've never done this before and the initial attempts were somewhat problematic, however the basic approach is as Rob said

"if you need to access something from a DLL, it needs to be DLL_EXPORTED. End of story" 

This code would have to be in the windows version, and depending upon wether we are building the library (DLL) or using the library we would need to either __dllimport or __dllexport our classes.

So we need some pre-compilation glue to make this work, first under windows only then dependent upon if we are building the dll or using the dll. The following code was added to the Types.h class

#ifdef WIN32
  #ifdef BUILDING_DLL
    #define NGL_DLLEXPORT __declspec(dllexport)
  #else
    #define NGL_DLLEXPORT __declspec(dllimport)
  #endif
#else
    #define NGL_DLLEXPORT
#endif

Once this has been added to the types file we need to add the BUILDING_DLL define to each of the classes. This is shown below.

class NGL_DLLEXPORT Vector
{
 ...
};

Once this was added it seemed to work for all the classes in the basic demo. However some of the more advance demos still failed to compile. I have several functions in the library which also needed to be exported, this was done as follows

// Util.h
extern NGL_DLLEXPORT ngl::Real radians(
                                        const Real _deg
                                       );
// Util.cpp
NGL_DLLEXPORT Real radians(
                            const ngl::Real _deg
                            )
{
  return (_deg/180.0f) * M_PI;
}

And finally any extraction / insertion operators as they are friend classes will also need to be exported

friend NGL_DLLEXPORT std::istream& operator>>(std::istream& _input, ngl::Vector &_s);

NGL_DLLEXPORT std::istream& operator>>(
                                                 std::istream& _input,
                                                 ngl::Vector& _s
                                                )
{
  return _input >> _s.m_x >> _s.m_y >> _s.m_z;
}

Finally it's working !

Or so I thought,  some of the simple programs that only used the ngl library compiled and ran correctly, however one of the programs which made a call to glUseProgram(0) in the main application failed to compile, due to a linker error, this error said that the method was not found and gave an imp__glew....  style error which means that the actual function call could not be found.

I thought that I had statically bound GLEW into my lib so it would work, however GLEW basically binds method calls to the driver once the glewInit function is called and in this case this is called in the DLL but no my application, to overcome this problem I need to put the glew.c source code into my applications as well the library.

To do this the following is added to the application .pro file (as the glew.c file is shipped with ngl)

win32: {
        DEFINES+=USING_GLEW
        INCLUDEPATH+=-I c:/boost_1_44_0
        INCLUDEPATH+=-I c:/boost

        INCLUDEPATH+= -I C:/NGL/Support/glew
        LIBS+= -L C:/NGL/lib
        LIBS+= -lmingw32
        DEFINES += WIN32
        DEFINES += USING_GLEW
        DEFINES +=GLEW_STATIC
        DEFINES+=_WIN32
        SOURCES+=C:/NGL/Support/glew/glew.c
        INCLUDEPATH+=C:/NGL/Support/glew/
}

And finally we need to initialise glew in both the DLL and Application

ngl::NGLInit *Init = ngl::NGLInit::instance();
#ifdef WIN32
  glewInit();
#endif
Init->initGlew();

This is not an ideal solution but is quite easy to do and saves the hassle of building the glew library using Qt / mingw. I will at some stage do this but for now this quick hack will suffice.
And for my next trick
Now that I've sullied my mac by installing windows I think I need to do a bit more windows support, the next stage is going to be building a Visual Studio version of ngl. Once this is done I think I may also add a DirectX back end to it as well however for now I will stick to the mac ;-)

Using Qt GUI components and NGL

This blog post is going to be my first Video blog post, it show how to start from scratch and create a gui program using Qt and NGL, the videos have been posted to youtube as the quality is much better than on blogspot, and you can grab the code using bzr from here

bzr branch http://nccastaff.bmth.ac.uk/jmacey/Code/QtNGL

For my students I've also copied the original movie files onto the public directory here /public/mapublic/jmacey/QtNGL/

Part 1 Setup

This video show how to setup the initial project and directories, it uses the ncca coding standard for project setup



Part 2 Adding source files to the project

This video shows how to add our source files as well as an initial QMainWindow class and a main.cpp file



Part 3 Setting up the .pro project file

This move shows how to setup the .pro file for qmake and we write a simple QApplication within the main.cpp file



Part 4 Creating a GUI using the Designer

In this Video I create the inital part of the GUI using the QDesigner module of QT Creator, I also go into quite a lot of detail about using the QGridLayout and the use of Layouts in general




Part 5 Adding the rest of the GUI components

In this video I add the rest of the UI components and finish off the rest of the GUI.



Part 6 Creating the OpenGL Window

In this video we extend the QGLWidget class to build our own GLWindow which we will put in the NGL classes to draw a teapot / sphere and cube and adjust the properties of the ngl::Transform Class.  In this initial part of this video we add the GLWindow to the grid layout. I've added an empty GLWindow.cpp/h file here



Part 7 Adding NGL to the .pro file

Next we add lots of flags to the .pro qmake file to configure the paths used for the ngl library.



Part 8 Adding a Camera

Now we add an ngl::Camera class and a simple ngl::Primitive drawing routine



Part 9 Adding glsl Shaders

Using the ngl::ShaderManager class we load the vertex and fragment shaders and give them a quick test



Part 10 Adding Signals and Slots 

We now use the Qt Signals and Slots mechanism to wire up the GUI components to our GLWindow class



Part 11 connecting the rest of the GUI components

Using the signals and slots mechanism I wire up the rest of the components





Part 12 Connecting the combo box


Using signals and slots I connect the QComboBox to select the different drawing modes



Part 13 Lights and Colour

In the final instalment I add an ngl::Light and Material to the program to light and set the initial material values, next I use the QColorDialog to select the colour I wish to use for the object

Game Style Object control in Qt

It is quite typical in a game to need to control an object by keeping a key pressed and for it to respond to these pressed key continuously. The following example demonstrates this in action as shown in the following video

Space Ship class
The following class diagram shows the basic space ship class we are going to move

The constructor of the class is very simple, it loads in the mesh and sets the member variables to default values as shown

SpaceShip::SpaceShip(
                     ngl::Vector _pos,
                     std::string _fname
                     )
 {
   m_pos=_pos;
   m_mesh = new ngl::Obj(_fname);
   m_mesh->createVBO(GL_STATIC_DRAW);
   m_rotation=0;
 }

To draw the ship we move our transform stack position to the correct location and rotation and draw the VBO from the mesh

void SpaceShip::draw()
{
  ngl::ShaderManager *shader=ngl::ShaderManager::instance();
  (*shader)["Blinn"]->use();
  m_transform.getCurrentTransform().setPosition(m_pos);
  m_transform.setRotation(0,m_rotation,0);
  m_transform.loadGlobalAndCurrentMatrixToShader("Blinn","ModelMatrix");
  m_mesh->draw();
}

To update the ships position m_pos and rotation we use the following methods

const static float s_xExtents=40;
const static float s_yExtents=30;

void SpaceShip::move(
                      float _x,
                      float _y
                    )
{
 float currentX=m_pos.m_x;
 float currentY=m_pos.m_y;
 m_pos.m_x+=_x;
 m_pos.m_y+=_y;
 if(m_pos.m_x<=-s_xExtents || m_pos.m_x>=s_xExtents)
 {
  m_pos.m_x=currentX;
 }


 if(m_pos.m_y<=-s_yExtents || m_pos.m_y>=s_yExtents)
 {
  m_pos.m_y=currentY;
 }
}

void SpaceShip::rotate(
                        float _rot
                      )
{
 m_rotation+=_rot;
}

The move method, first stores the current x and y values of m_pos, we then increment the values based on the parameter passed in and check agains the extents of the screen. If we are not at the edge of the screen as set in these extents we add the values passed in to the value of the m_pos attribute.
For the rotation the value is just added.

Processing Key presses

Each subclass of the QWidget class has access to the following virtual protected methods

void QWidget::keyPressEvent ( QKeyEvent * event ) ;
void QWidget::keyReleaseEvent ( QKeyEvent * event ); 

Which ever QtWindow has focus will receive these events and the current key can be queried via the event parameter passed in. In this example the MainWindow class processes the key events as we wish to allow the window to become fullscreen and this method can only be called from a class that inherits from QMainWindow The following code shows how the key press and release methods are implemented

void MainWindow::keyPressEvent(
                               QKeyEvent *_event
                              )
{
  // this method is called every time the main window recives a key event.
  switch (_event->key())
  {
  case Qt::Key_Escape : QApplication::exit(EXIT_SUCCESS); break;
  case Qt::Key_W : glPolygonMode(GL_FRONT_AND_BACK,GL_LINE); break;
  case Qt::Key_S : glPolygonMode(GL_FRONT_AND_BACK,GL_FILL); break;
  case Qt::Key_F : showFullScreen(); break;
  case Qt::Key_N : showNormal(); break;
  default : break;
  }
 // once we have processed any key here pass the event
 // onto the GLWindow class to do more processing
  m_gl->processKeyDown(_event);
}

void MainWindow::keyReleaseEvent(
                 QKeyEvent *_event
                 )
{
 // once we have processed any key here pass the event
 // onto the GLWindow class to do more processing
  m_gl->processKeyUp(_event);

}

Once any keys required in the MainWindow class are processed the key events are then passed onto the GLWindow class for more processing.
GLWindow key processing
To process the keys in the GLWindow class we are going to create a QSet to contain the active key strokes, for each keyPress event (key down) we will add the key to the QSet using the += operator. Every time is released the key will be removed from the QSet using the -= operator. This is shown in the following code

// in GLWindow.h
/// @brief the keys being pressed
QSet< Qt::Key> m_keysPressed;

// in GLWindow.cpp
void GLWindow::processKeyDown(
                QKeyEvent *_event
               )
{
 // add to our keypress set the values of any keys pressed
 m_keysPressed += (Qt::Key)_event->key();

}


void GLWindow::processKeyUp(
                QKeyEvent *_event
               )
{
 // remove from our key set any keys that have been released
 m_keysPressed -= (Qt::Key)_event->key();
}

Now we will create an method to process the key values stored in the set and update the ship.

void GLWindow::moveShip()
{
 /// first we reset the movement values
 float xDirection=0.0;
 float yDirection=0.0;
 float rotation=0.0;
 // now we loop for each of the pressed keys in the the set
 // and see which ones have been pressed. If they have been pressed
 // we set the movement value to be an incremental value
 foreach(Qt::Key key, m_keysPressed)
 {
  switch (key)
  {
   case Qt::Key_Left :  { xDirection=s_shipUpdate; break;}
   case Qt::Key_Right : { xDirection=-s_shipUpdate; break;}
   case Qt::Key_Up :   { yDirection=s_shipUpdate; break;}
   case Qt::Key_Down :  { yDirection=-s_shipUpdate; break;}
   case Qt::Key_R :     { rotation=1.0; break;}
   default : break;
  }
 }
 // if the set is non zero size we can update the ship movement
 // then tell openGL to re-draw
 if(m_keysPressed.size() !=0)
 {
  m_ship->move(xDirection,yDirection);
  m_ship->rotate(rotation);
 }
}

First we set the x,y and rotation values to 0 to indicate there is no movement to the ship, next we loop through all the keys in the QSet using the foreach iterator provided by Qt. We then check to see if a key is active and update the direction variable based on the static variable s_shipUpdate. Depending upon the direction we set this to +/-s_shipUpdate.
Finally we check to see if there are any keys active and if so update the ship direction by calling the move function.

Tying it all together
Finally we want to separate the movement and draw commands so we can get a fixed update for drawing. To do this we create two timers one which will trigger the ship move update and one for the re-draw. This is done with the following code, first in the constructor we start the timer.

m_updateShipTimer=startTimer(2);
m_redrawTimer=startTimer(10);

Next in the timerEvent in GLWindow we process and dispatch if we get the correct timer event as follows

void GLWindow::timerEvent(
                          QTimerEvent *_event
                         )
{
 // the usual process is to check the event timerID and compare it to
 // any timers we have started with startTimer
 if (_event->timerId() == m_updateShipTimer)
 {
  moveShip();
 }
 if (_event->timerId() == m_redrawTimer)
 {
  updateGL();
 }

}

The full code for this demo can be downloaded from here using the command bzr branch http://nccastaff.bmth.ac.uk/jmacey/Code/GameKeyControl 

Anonymous Union / Struct Weirdness

I've been having a look at clang++ / llvm as this will be the new default compiler for Mac, when I came across the following error in some of my code.

In file included from src/Vector.cpp:17:

include/ngl/Vector.h(476) : error: 'm_x' is a protected member of 'ngl::Vector'

return Vector(_k*_v.m_x, _k*_v.m_y, _k*_v.m_z,_v.m_w);

^

include/ngl/Vector.h(423) : note: declared protected here

Real m_x;

^

This is fair enough however this compiled fine with g++, after turning on the pedantic flag in g++ I got the following error

In file included from src/Vector.cpp:17:

include/ngl/Vector.h:437: error: ISO C++ prohibits anonymous structs

Now this is where the plot thickens and I learnt something new.

So clang++ says I have a visibility problem, whereas g++ running normally says this is fine, and g++ in full pedantic mode say no way, who is correct and what is going on?

A Simple Test Class

To test the problems I was encountering I decided to write a simple class as shown below

class Vector
{

public :
Vector(
          const Vector& _v
         ) :
            m_x(_v.m_x),
            m_y(_v.m_y),
            m_z(_v.m_z),
            m_w(_v.m_w){;}

Vector(
         const float& _x=0.0,
         const float& _y=0.0,
         const float& _z=0.0,
         const float& _w=1.0f
         ):
           m_x(_x),
           m_y(_y),
           m_z(_z),
           m_w(_w){;}


private :
  union
  {
    struct
    {
      float m_x;
      float m_y;
      float m_z;
      float m_w;
    };
  float m_openGL[4];
  };

};

So here we have a simple union of 4 floats and an array of 4 floats which if we look at the memory layout would look like this

I then wrote a simple test program which looked like this

int main()
{
 Vector v(1.0,2.0,3.0,1.0);
 v.m_x=2.0;
 std::cout<< v.m_x<<"\n";
}

You will notice in the above program, that I access the member m_x directly. However in the class this is declared in the private section. How does this work?

Well i'm not really sure, according to clang++ "anonymous structs are a GNU extension" so are allowed, however this doesn't account for the breaking of encapsulation.

After further tests I moved the union into the protected area and still it works and I can access the member directly. So it would appear that anonymous struct / unions are always public.

Testing this with Visual Studio 2005 I get the same results, however with the new version of clang++ that ships with Xcode 4 (Apple clang version 2.0 (tags/Apple/clang-137) (based on LLVM 2.9svn)) I get the following



clang++ -pedantic -fdiagnostics-fixit-info VecTest.cpp -o Test
VecTest.cpp:30:5: warning: anonymous structs are a GNU extension [-Wgnu]
    struct
    ^
VecTest.cpp:46:4: error: 'm_x' is a protected member of 'Vector'
        v.m_x=2.0;
          ^
VecTest.cpp:32:13: note: declared protected here
      float m_x;
            ^
VecTest.cpp:47:15: error: 'm_x' is a protected member of 'Vector'
        std::cout<<v.m_x<<"\n";
                     ^
VecTest.cpp:32:13: note: declared protected here
      float m_x;
            ^
1 warning and 2 errors generated.

So it seems clang++ deals with the visibility / scope correctly unlike the other two compilers, for now I think I'm going to move the structs in my library to public anyway as this will give me more speed / access for the basic math classes.