Showing posts with label OpenGL. Show all posts
Showing posts with label OpenGL. Show all posts

Friday, 21 March 2014

Using emscripten to port NGL to the Web

Introduction

I have been using the ngl:: library for many years as part of teaching various graphic programming courses. I decided recently it would be interesting to port the core library and many of the demos to work interactively on the web using WebGL so started investigating a number of ways to do this. The main library is written in C++ and uses either Qt or SDL to create the OpenGL context. 
I had a number of choices as to which approach to take, I could learn Java Script and  three.js however this would mean porting all my codebase to Java Script which seemed like too much work.
In the end I came across the emscripten system which is a LLVM-to-JavaScript Compiler which can convert my C++ code into LLVM and then into JavaScript and then into asm.js the process was quite a steep learning curve, however I manage to get quite a lot of demos ported very quickly which can be seen here the rest of the blog will outline the process and how the webngl system was developed.

Installing Emscripten

My main development environment is a mac, however I have also tested the files under linux and it also works well. To get started I followed the tutorial here and all worked first time. The next stage was to try a simple WebGL demo that is provided with the examples. There are several WebGL demos using different libraries for OpenGL context creation however as I'm most familiar with SDL I chose to use this as the basis of the framework.

A Simple SDL demo program

The following code is a simple SDL program (very similar to a normal SDL program) the only difference is the call to emscripten_set_main_loop.

#include "SDL.h"
#include <GLES2/gl2.h>
#define GL_GLEXT_PROTOTYPES 1
#include <GLES2/gl2ext.h>
#include <emscripten.h>
#include <iostream>
#include <cstdlib>


void process()
{
  // as we don't have a timer we need to do something here
  // using a static to update at an interval
  static int t=0;
  if(++t > 100)
  {
    float r=(double)rand() / ((double)RAND_MAX + 1);
    float g=(double)rand() / ((double)RAND_MAX + 1);
    float b=(double)rand() / ((double)RAND_MAX + 1);
  
    glClearColor(r,g,b,1);
    t=0;
  }
  glClear(GL_COLOR_BUFFER_BIT);
  // this is where we draw
  SDL_GL_SwapBuffers();
}

int main(int argc, char *argv[])
{

 SDL_Surface *screen;

 // Init SDL
 if ( SDL_Init(SDL_INIT_VIDEO) != 0 ) 
 {
  std::cerr<<"Unable to initialize SDL: "<<SDL_GetError();
  return EXIT_FAILURE;
 }

 SDL_GL_SetAttribute( SDL_GL_DOUBLEBUFFER, 1 );

 screen = SDL_SetVideoMode( 720, 576, 16, SDL_OPENGL  | SDL_RESIZABLE); 
 if ( !screen ) 
 {
  std::cerr<<"Unable to set video mode: "<<SDL_GetError();
  return EXIT_FAILURE;
 }

 glEnable(GL_DEPTH_TEST);

 // let emscripten process something then 
 // give control back to the browser
 emscripten_set_main_loop (process, 0, true);

 SDL_Quit();
 return EXIT_SUCCESS;
}

The emscripten_set_main_loop function is explained very well here basically we create a function that is called asynchronously to allow the browser to regain control after every iteration of the function. It is important that this function does exit else the browser will hang up and I have had several times when I get a complete lockup of the system.

Compiling the program

To compile the program we use the following command line (in this case I'm using c++ )
em++ -s FULL_ES2=1 SDL1.cpp -o SDL1.html
The flag -s FULL_ES2 tells emscripten to use full OpenGL ES 2 specification when compiling the code, the -o SDL1.html will generate an html file as well as the javascript file for the canvas to use. The html file can now be opened in the browser an in this case you will see a screen that changes colour every 100 cycles of the main loop. In the next post I will begin to discuss how I ported the rest of ngl to use emscripten.

Thursday, 20 February 2014

GamePlay 3D 101

Another Guest post on my Blog, this time from Callum James and Ramesh Balachandran from the BSc. Software Development for Animation, Games and Effects course at the NCCA

What is Gameplay 3D? 

 Gameplay 3D is an open-source, code based engine that is aimed at being cross-platform compatible. We decided to use this engine over others as we would have the most control over the features and optimisations of the program / game we create. Whilst giving you the tools to produce a simple piece of high quality in relatively little time, other game engines such as the Unreal Development Kit , are restrictive to their scripting functionality and UI interaction. Therefore, by using an open-source engine that has an established rendering engine, we are able to extend upon and create a new version of the engine in our own desired way.

Lack of Documentation
One of the greatest obstacles we are still striving to overcome whilst using the Gameplay 3D engine is its lack of clear and precise documentation for its use. Whilst there exists basic documentation on the engine and classes, a lot of features found within the engine do not have any form of detailed information regarding their uses or function. The sections are ways we have found to set-up the game engine and use its features to hopefully serve as a 101 guide to getting started with Gameplay 3D. 

Our Preferred Gameplay 3D Setup
There are multiple ways you can use the Gameplay 3D engine on different platforms. By default, the engine comes with source and project files for Visual Studio on Windows and an XCode project for MacOSX.  It is then up to you to get an environment set up for a Linux build. However this makes it a little cumbersome to be easily platform independant for efficient development. We however have come up with a simple solution to make it as easy as possible to pass your project between all three major platforms and quickly get up and running for development.
Whilst it would be ideal to have a single development project for all three platforms, due to our desire to support not only Windows 7 but also Windows 8, we have had to separate the Windows build to its own Visual Studio project. For Mac and Linux, we then have a single Qt Creator project (.pro file) that very easily compiles and runs on both of the Unix flavours.
As the VS project file and the Qt project files all read from the same sources, cross platform development is still quick, easy and affects all platforms. To ensure this works however it is important to keep a strict directory hierarchy to your project so that no file gets lost and you keep independant local file paths to a minimum. 

Following is a quick overview into how to set up Gameplay 3D on each of the three major platforms in the way we have discussed here. 

Setting up Gameplay 3D on Unix - Linux and MacOSX
Setting up Gameplay 3D to work on a Unix platform (either Linux or MacOSX) using Qt Creator, there are a few things you need to make sure are set and linked correctly. Here we will discuss what you need to ensure is set correctly and included in the .pro file so that your project will build and run successfully.
The first thing you need to look at is your include and library paths. You need to ensure that your include paths are at least set to ./include to pick up on your header files. 
Other directories to include are:


INCLUDEPATH += $$GAMEPLAY_DIR/gameplay/src 
INCLUDEPATH += $$GAMEPLAY_DIR/gameplay/src/lua
INCLUDEPATH += $$GAMEPLAY_DIR/external-deps/bullet/include
INCLUDEPATH += $$GAMEPLAY_DIR/external-deps/lua/include
INCLUDEPATH += $$GAMEPLAY_DIR/external-deps/oggvorbis/include
INCLUDEPATH += $$GAMEPLAY_DIR/external-deps/png/include
Libraries to link up to for your projects are as follows for Linux:
unix:!macx{
   LIBS += -L$$GAMEPLAY_DIR/external-deps/lua/lib/linux/x64/ -llua
   LIBS += -L/usr/local/lib -lBulletCollision -lBulletDynamics -lLinearMath
   LIBS += -L$$GAMEPLAY_DIR/external-deps/glew/lib/linux/x64/ -lGLEW
   QMAKE_CXXFLAGS += $$system(pkg-config --cflags gtk+-2.0)
   LIBS += -lgtk-x11-2.0 -lglib-2.0 -lgobject-2.0
   LIBS += -L$$GAMEPLAY_DIR/external-deps/oggvorbis/lib/linux/x64/ -lvorbis -logg
   LIBS+=  -lrt -ldl -lpng -lopenal -lz -pthread -lGL -lX11
}
And for MacOSX:
macx:{
   INCLUDEPATH+=/usr/local/include/
   LIBS+= -L/usr/local/lib -lBulletDynamics  -lBulletCollision -lLinearMath
   LIBS+= -L/usr/local/lib -lpng
   LIBS += -L/ext-deps/fmod/lib/macosx -lfmodex
   LIBS += -framework Cocoa -framework GameKit -framework IOKit -framework QuartzCore -framework OpenGL -framework OpenAL
   LIBS+= -llua -lvorbis -lvorbisfile -logg -lz  -ldl -lpthread
}
These are all the include paths and libraries you will need to get started. It is however important to note on Mac, you need to make sure you are linking against the right architecture for your build. The external dependency libraries supplied with the Gameplay 3D library are all 32 bit so if you want to build 64 bit projects you will need to recompile the external libraries as 64 bit and then link to these. You will the need to include in your .pro file the following:
macx:QMAKE_CXXFLAGS+= -arch x86_64
macx:CONFIG+=x86_64
Secondly, local path set-up is vital to ensuring your project will run on all platforms without having to go through and change all of the paths to files and assets. The paths differ due to the way that the game is built on different platforms. On Linux (and Windows) it is built as a simple executable. Gameplay 3D built on OSX however utilises the MacOSX app bundle feature. When compiling on Mac, the application will be built into the directory ./application_name.app/Contents/MacOS. This is also where it will be run from. However because the executable is run from here and not the root directory, any assets it requires are no longer visible to it. As such, all the required assets need to be copied into the correct directory for the application to find them. On Linux and Windows, this is simple as the executable will look from assets in a res/ folder from the root directory. So within your root directory as long as your assets are stored within a res folder, you will be fine. However on Mac, the files you require for the game need to be copied into the Resources folder into a folder called res. The gameplay engine will then pick up on these resources. The config files also go into the Resources folder (but not the res) folder. These can then be picked up and read by the application when run. In the .pro file, you just need to ensure all the files you require are passed to QMAKE_BUNDLE_DATA. So for example, to copy some splash screen images to the correct directory and use them with a local path, you need the following in your .pro file:
SPLASHIMAGES += res/splash/logo_powered_white.png

macx:{
 APP_SPLASH_IMAGES.path = Contents/Resources/res/splash
     APP_SPLASH_IMAGES.files += $$SPLASHIMAGES
 
 QMAKE_BUNDLE_DATA +=  … \
        APP_SPLASH_IMAGES \
 … \ 
}

Any other files you need also need to be added to QMAKE_BUNDLE_DATA in this manner, so for example to also add audio files, use:
QMAKE_BUNDLE_DATA +=  APP_AUDIO_FILES \
                        APP_SPLASH_IMAGES \
   … \ 
The Gameplay engine will automatically pick up on them when located in this directry, but anything you write yourself using paths needs to be prefixed. Adding ../Resources/ to the front of any local paths used in your code will ensure your game looks into the right folder on Mac. To automatically do this, we have developed a very simple function as below:
#ifdef DARWIN
    #define PATH_PREFIX "../Resources/"
#else
    #define PATH_PREFIX ""
#endif

std::string GeneralUtils::PLATFORM_FILE_PATH(std::string _path)
{
    return (PATH_PREFIX+_path);
}
The DARWIN define is set within the .pro file if the macx platform is detected, as follows:
unix:!macx: DEFINES += LINUX
macx: DEFINES += DARWIN
If this is all set up correctly, then local paths will work on all platforms without any need to change them.

Setting up Gameplay 3D on Windows

The setup for programming with Gameplay 3D on Windows differs slightly from the Unix build. Due to the current operating system used for development on Windows (Windows 8.1 64-bit), there were issues in setting up Qt Creator to enable complete cross-platform programming. The IDE of choice was shifted to Visual Studio 2010. In order to set up the project to work with Unix build, the Visual Studio project settings need to be set up to correctly link to the Gameplay3D libraries as well as the correct Windows external dependencies. To make sure this is done correctly, the engine needs to be built using the Visual Studio project file accompanying the engine. The Gameplay3D files are compiled for a 64-bit architecture of Windows to work alongside the 64-bit architecture setup of the OSX and Linux builds. Once this is compiled, the Visual Studio project needs to correctly link to the engine header files as well as the necessary external dependency files. This is done under Project -> Properties -> Configuration Properties -> C/C++ in the ‘Additional Include Directories’ section. In this section, the paths to all of the header files would be added:
../../external-deps/lua/lib/windows/x64
../../external-deps/bullet/lib/windows/x64
../../external-deps/openal/lib/windows/x64
../../external-deps/oggvorbis/lib/windows/x64
../../external-deps/glew/lib/windows/x64
../../external-deps/png/lib/windows/x64
../../external-deps/zlib/lib/windows/x64
../../gameplay/windows/x64/$(Configuration)
ext-deps/fmod/lib/windows
In addition to this the .lib files need to be explicitly linked under Linker -> Input in the ‘Additional Dependencies’ section:
lua.lib;OpenAL32.lib;OpenGL32.lib;GLU32.lib;glew32.lib;libpng14.lib;zlib.lib;gameplay.lib;libogg.lib;libvorbis.lib;libvorbisfile.lib;BulletDynamics.lib;BulletCollision.lib;LinearMath.lib;fmodex64_vc.lib;%(AdditionalDependencies)
Once this is set, all that is necessary is to include the source files we were currently using to the project and make sure that it compiles. At this point, the project folder is transferrable between platforms as it contains both the Qt .pro file as well as the Visual Studio .vcxproj file, keeping the order of source and include files as they were.

Using Config Files
When you run your Gameplay 3D project or game, the window it creates will be generated from a game.config file located within the root directory of your project. If no config file is found, then it will default to standard values to best fit the situation. The basic options you can specify in the game.config file within the window scope (window {}) are:
  • title The text you wish to appear at the top of the window if full screen is false
  • width  The resolution width you wish to use
  • height  The resolution height you wish to use
  • fullscreen  This can either be true or false and will set the game either full screen or windowed
The .config file can also be used to set up aliases. These aliases can be used across your project, in both material, scene and physics files and also within the code when specifying paths and passing them to the engine. To set up an alias, simply first specify the scope aliases

aliases
{

}
Within this scope, declare a name and set what it is equal to. For example:
aliases
{
  komodo = res/textures/komodo.png
}
This will create an alias to the texture file called komodo.png. If you then wanted to use this alias within another file, all you need to do is reference it as follows:
{what_to_set_to} = @komodo
The engine will then replace @komodo with the right path. You can have as many aliases as you like within one project. Another use for the .config file is in the gamepad scope, declared as:
gamepads
{

}
This will declare aliases for gamepad forms, which can then be used on touch devices as a way to control the game, for example:
gamepads
{
   form = res/common/gamepad.form

}
It is important to remember that whilst on Windows and Linux the game.config file can remain in the root directory, on Mac, it must be copied into the .app/Contents/Resources folder when building the game. If it is not, then the executable will not find the file and so will resort to default values.

Encoding Files


Encoding files is an important aspect of using the engine, as Gameplay3D works with .gpb files, which are a Gameplay3D binary file format. This is then read in by the engine for assets used in the game. To encode files, we need to use the encoder that is compiled upon compiling the Gameplay source files. The encoder takes in a .fbx file exported from Autodesk Maya and encodes it into the .gpb file format. Using the encoder at its basic level is just running the command in terminal:

gameplay-encoder ‘name of .fbx file’
This will create a .gpb file of the same name as the input .fbx file which can then be read into the engine either by passing the reference to the file in the .scene file used by the engine or explicitly loading the file in the source code. The encoder will also pick up on any animation present in the .fbx file, prompting the user when encoding if they wish to group the animations. If you intend to use the animations for a character, for example if the character has multiple animation cycles, you would run a different version of the encoder command in the terminal:
gameplay-encoder -g ‘name of root joint in Maya’  ‘name of animation group’ ‘name of .fbx file’
This would group the animations for the specified joint hierarchy under the name of the input animation group from the .fbx file. The animation group name is the name of the animation set specified in the .animation file following the structure:
animation name of animation group
{
  frameCount = 1100
 clip idle
 {
  begin = 27
  end = 167
  repeatCount = INDEFINITE
  loopBlendTime = 250
 }
}

Using the Scene Files

Using scene files is a matter of specifying the contents of the encoded .gpb file to easily load them into the game without having to specify each asset and its respective material and/or physics profile in the code. A simplistic .scene file would follow the format:
scene
{
 path = res/models/boy.gpb

 node boycharacter
 {
   collisionObject = res/common/demo.physics#boy
 }

 node boymesh
 {
   material = res/common/demo.material#boy
 }

 node boyshadow
 {
   material = res/common/demo.material#demo
   tags
   {
     transparent
     dynamic
   }
 }

 node camera
 {
   collisionObject = res/common/demo.physics#camera
 }
}

This file is then loaded into the scene using ‘Scene::load("res/common/demo.scene");’. Given the specified .scene file and all the nodes in the file, we are able to use the findNode(“asset name”); to get the necessary asset and manipulate it as necessary. Static objects in the scene, as mentioned, don’t need to be found from the scene file and are loaded into the game scene based on the material and physics tags given in the .scene file. Note in the example above that the nodes are in reference to the .gpb file given under ‘path’.

Setting up Physics

In order to get physics working within the game, there are two methods that can be taken. The first is the more straightforward approach, using a .physics file which contains an outline of the physics objects that can be assigned to assets in the .scene file. An example of a physics object from the .physics file is as follows:
collisionObject staticMesh
{
 type = RIGID_BODY
 shape = MESH
 static = true
 mass = 0
 restitution = 0.01
 linearDamping = 1.0
 angularDamping = 1.0
}
This can then be referenced in the .scene file for a node using the # symbol after the reference to location of the .physics file in collisionObject. For example as seen in the example .scene file for instance, if you were to set the boyCharacter node as a static object using the above collision object the path would read as:
node boycharacter
{
collisionObject = res/common/demo.physics#staticMesh
}

The second approach to physics in the game engine, would be to attach a physics character node to the asset in code. This would mean instantiating a PhysicsCharacter that has the properties of the node’s collision object, but this would allow you to manipulate the node such as updating it’s velocity, step height, etc.

Using Materials

One easy way of controlling and using materials within your project is through the use of material files (.material). You can have multiple material files per project, and they simply need to be referenced with the correct local path when using them. So if you had a foo.material and a foo2.material file, you can extract materials from either by simply quoting the correct path when setting a material in the engine. A material file has a fairly similar structure to both the scene and physics files described previously. It makes use of a scope to define a single material. Within that scope, a number of attributes are then defined. For example, to create a material called demo:
material demo
{

}
Within this material you can then specify certain techniques, such as:
material demo
{
 technique Default
 {

 }

 technique Edges
 {

 }
}
This will allow you to specify a single material but with multiple techniques. To access these in the code, use the following:
{__model__}->setMaterial("res/common/demo.material#demo");
{__model__}->setTechnique("Default");
This will look into the demo.material file and find the demo material (denoted by the #). The next line will then set the active technique to Default. If there are no techniques specified within the material, it will simply use the information supplied within the material scope. To specify which shaders and any uniforms to set in the shader, simply declare these within this scope. So for example:
material demo
{

technique Default
{
  pass
  {
  // shaders
    vertexShader = res/shaders/colouredVertex.glsl
    fragmentShader = res/shaders/colouredFragment.glsl

    // uniforms
    u_worldViewProjectionMatrix = WORLD_VIEW_PROJECTION_MATRIX
    u_inverseTransposeWorldViewMatrix = INVERSE_TRANSPOSE_WORLD_VIEW_MATRIX
    u_diffuseColor = 1.0, 1.0, 1.0, 1.0

    renderState
    {
      cullFace = true
      depthTest = true
    }
  }
 }
}

Within the technique, the pass scope declares a single render pass and everything declared within the pass scope will be used in a single pass. More passes can be defined by defining more pass scopes. The vertexShader and fragmentShader paths simply declare the shader file to use for each. Below this are some uniforms being set. WORLD_VIEW_PROJECTION_MATRIX and INVERSE_TRANSPOSE_WORLD_VIEW_MATRIX are Gameplay 3D built in values. However if you wish to upload your own values to these uniforms, you can do through setting the material within the main code. The render state scope is optional and simply sets some OpenGL options for rendering of this pass. If you are using a shader that requires a texture or image path, this can be set by using the keyword path = and then setting either a declared path or an alias such as @komodo. This is the basic use of a material file for your project.

Animations

Animation in the game engine works based on animations that are done in Maya and encoded in the .gpb file as stated above. In order to get animations working in your scene you would need the .animation file that defines the different animation clips as mentioned in the explanation of encoding files. The animation is loaded using code such as:
m_animation = node->getAnimation("animations");
m_animation->createClips("res/common/boy.animation");
From here, you can use the loaded animation to store and/or play specific clips to be used in the scene when needed. For instance, in the initialize function, if you needed the character in your game to start playing the idle animation when it loads you would use:
play("idle", true);
Using this, the animation clips specified in the .animation file can be called explicitly to make the character animate with the chosen animation clip. The loop blending time can be specified in code or in the .animation file to determine the smoothness of the blending between different animation clips when switching animations, for example from idle, to walking to running.The different clips can also be linked to character velocities in order to simulate changes in animation based on the speed at which the character is moving.

First Person Control System

In order to set up a first-person camera system in a Gameplay3D scene, a few things need to be determined. Firstly, you would need to include a camera in the encoded .gpb file. This camera node is then found using the aforementioned findNode function in order to work with the camera is the scene. Secondly, you would need to ensure that the mouse event is set to capture the mouse within the game window using:
Game::setMouseCaptured(true);
The reason for this is to make sure that you are able to have full control of camera movement in the scene without having the mouse leave the game window. Once this is all set up, it is a matter of determining how to link the values from the mouse event function to the movement and rotation of the camera. If there isn’t a character set up in the scene for your game, the linking is much simpler. The y change of the mouse in the mouse event function, gives you a value with which you can determine how much to rotate the camera in the x-axis (look up and down). The x value in this function would then be used to determine the y-axis rotation for the camera (look left and right). If a character was used in the scene, you would then need to make sure that the PhysicsCharacter assigned to the character looks where the camera looks and vice-versa. The easiest method to doing this is to parent the camera to the character in the scene in Maya, but could potentially result in orientation issues. As such, what needs to be done, is that using the x mouse value, you would rotate character and ensure the the camera is similarly oriented using the character’s forward vector. This would handle the ‘looking’ aspect of the first-person camera system. In order to move the camera, you would register key-inputs and respectively use the key inputs to determine the direction of movement. This can then be used to affect and update the camera’s position in the update function. Once again, if a character was used, you would update the camera’s position not based on key inputs but by getting the character’s position using the getTranslation function and moving the camera to that location. Note that the camera would thus be correctly oriented using the forward vector of the character as mentioned above. The basis of the first-person camera system is now set, but can be further developed should the need arise for different camera actions, such as leaning around corners or peeking over obstacles.

Use of external GL calls

Even though Gameplay 3D has its own GL context and its own rendering engine (used through the render() function) you are able to use your own calls to GL functions and GL draw calls if you wish. These will apply to the same GL context as every other Gameplay 3D draw call. This can give you a good deal of extra control over certain rendering aspects and GL attributes used within your project.

Saturday, 11 May 2013

GLSL tessellation Shaders under Mac OSX

So this interesting post appeared the other day saying that tessellation shaders were working  on Mac OSX. According to the documents and other things like GLView this is not the case.

I decided to spend the day investigating the claims to a) see if they were true, and b) see how I could replicate this in my own library.

The following video shows how I used the Mac OpenGL profiler to dig into the source code and find out how to do this.


 The main code elements you need are the following defines

#ifdef DARWIN
    #ifndef GL_TESS_CONTROL_SHADER
        #define GL_TESS_CONTROL_SHADER 0x00008e88
    #endif
    #ifndef GL_TESS_EVALUATION_SHADER
        #define GL_TESS_EVALUATION_SHADER 0x00008e87
    #endif
    #ifndef GL_PATCHES
        #define GL_PATCHES 0x0000000e
    #endif
#endif
Once this has been defined somewhere you can use the default OpenGL commands to create a shader, I chose to use the demo code / shader here the main thing to remember is that you must use GL_PATCHES to draw to the tessellation units.

The other problem is that code like
glPatchParameteri(GL_PATCH_VERTICES, 16); 
is not available as the glPatchParam functions are not exposed, the good news is that you can set all of this in the shader so it is not too much of an issue. The updates have now been rolled into the core ngl library, and I will add some demos soon.

Monday, 11 March 2013

Adding Movement in ngl

This video blog shows how to add movement using the Model, View, Project matrix in ngl you can get the demo program here and should also read the lecture notes here


Monday, 12 November 2012

Sponza Demo Pt 3 The GroupedObj class

In the previous post I described the Mtl class. This video blog will show the design and ideas behind the the GroupedObj class as shown in the following diagram










You can get the code from here

Sponza Demo Pt 2 Mtl class

In the previous post I discussed the basic overview of the system this post will explain how the Mtl class works and how is was developed.

The mtl file has the following basic structure, where each element may or may not be present.

newmtl leaf
  Ns 10.0000
  Ni 1.5000
  d 1.0000
  Tr 0.0000
  Tf 1.0000 1.0000 1.0000
  illum 2
  Ka 0.5880 0.5880 0.5880
  Kd 0.5880 0.5880 0.5880
  Ks 0.0000 0.0000 0.0000
  Ke 0.0000 0.0000 0.0000
  map_Ka textures\sponza_thorn_diff.tga
  map_Kd textures\sponza_thorn_diff.tga
  map_d textures\sponza_thorn_mask.tga
  map_bump textures\sponza_thorn_ddn.tga
  bump textures\sponza_thorn_ddn.tga

The newmtl keyword is used to indicate that a new material is being specified and the rest of the elements are part of that material. To store this information I decided to use a std::map using a std::string as the key which will be the name following the newmtl (in the above example this is "leaf"). The map will then store the following structure.
typedef struct
{
  float Ns;
  float Ni;
  float d;
  float Tr;
  int illum;
  ngl::Vec3 Tf;
  ngl::Vec3 Ka;
  ngl::Vec3 Kd;
  ngl::Vec3 Ks;
  ngl::Vec3 Ke;
  std::string map_Ka;
  std::string map_Kd;
  std::string map_d;
  std::string map_bump;
  std::string bump;
  GLuint map_KaId;
  GLuint map_KdId;
  GLuint map_dId;
  GLuint map_bumpId;
  GLuint bumpId;
}mtlItem;
You will notice that this replicates the mtl format shown above and also have several extra items which have the extra postfix ID, these will be used to store the OpenGL texture ID's of the textures once loaded. I also made the design decision not to follow my usual coding standard for the structure as this would make it easier to follow what is happening in the mtl file.

Class functional design


The main elements of the class are shown in the following class diagram.
The main functional design of the Mtl class is split into two areas, first the loading and parsing of the original mtl file. This can either be done in the constructor or using the load method. Next is the actual use of the class data. To allow easy access to this data iterators have been exposed for the std::map as well as other methods. Finally we have the ability to save and load the data in a binary format to save the parse time of reading the original files.

Parsing the mtl file


As the structure of the mtl file is quite simple I decided a full blown parser was not required. Instead I decided to use the boost::tokenizer template to process the data.  The file is opened and the data read a line at a time. The tokenizer splits the data and looks for the keywords, which are then processed one at a time. This is shown in the following code.
  // this is the line we wish to parse
  std::string lineBuffer;
  // say which separators should be used in this
  // case Spaces, Tabs and return \ new line
  boost::char_separator<char> sep(" \t\r\n");
  // loop through the file
  while(!fileIn.eof())
  {
    // grab a line from the input
    getline(fileIn,lineBuffer,'\n');
    // make sure it's not an empty line
    if(lineBuffer.size() >1)
    {
      // now tokenize the line
      tokenizer tokens(lineBuffer, sep);
      // and get the first token
      tokenizer::iterator  firstWord = tokens.begin();
      // now see if it's a valid one and call the correct function
      if( *firstWord =="newmtl")
      {

        // add to our map it is possible that a badly formed file would not have an mtl
        // def first however this is so unlikely I can't be arsed to handle that case.
        // If it does crash it could be due to this code.
        //std::cout<<"found "<<m_currentName<<"\n";
        parseString(firstWord,m_currentName);
        m_current= new mtlItem;
        // These are the OpenGL texture ID's so set to zero first (for no texture)
        m_current->map_KaId=0;
        m_current->map_KdId=0;
        m_current->map_dId=0;
        m_current->map_bumpId=0;
        m_current->bumpId=0;

        m_materials[m_currentName]=m_current;
      }
      else if(*firstWord =="Ns")
      {
        parseFloat(firstWord,m_current->Ns);
      }
      else if(*firstWord =="Ni")
      {
        parseFloat(firstWord,m_current->Ni);
      }
      else if(*firstWord =="d")
      {
        parseFloat(firstWord,m_current->d);
      }
      else if(*firstWord =="Tr")
      {
        parseFloat(firstWord,m_current->Tr);
      }
      else if(*firstWord =="Tf")
      {
        parseVec3(firstWord,m_current->Tf);
      }
      else if(*firstWord =="illum")
      {
        parseInt(firstWord,m_current->illum);
      }
      else if(*firstWord =="Ka")
      {
        parseVec3(firstWord,m_current->Ka);
      }
      else if(*firstWord =="Kd")
      {
        parseVec3(firstWord,m_current->Kd);
      }
      else if(*firstWord =="Ks")
      {
        parseVec3(firstWord,m_current->Ks);
      }
      else if(*firstWord =="Ke")
      {
        parseVec3(firstWord,m_current->Ke);
      }

      else if(*firstWord == "map_Ka")
      {
        parseString(firstWord,m_current->map_Ka);
      }
      else if(*firstWord == "map_Kd")
      {
        parseString(firstWord,m_current->map_Kd);
      }
      else if(*firstWord == "map_d")
      {
        parseString(firstWord,m_current->map_d);
      }
      else if(*firstWord == "map_bump")
      {
        parseString(firstWord,m_current->map_bump);
      }
      else if(*firstWord == "bump")
      {
        parseString(firstWord,m_current->bump);
      }


   } // end zero line
 } // end while

// as the trigger for putting the meshes back is the newmtl we will always have a hanging one
// this adds it to the list
 m_materials[m_currentName]=m_current;

The individual parse functions then use the boost::lexical_cast template to convert the values as shown in the following example
void Mtl::parseFloat(tokenizer::iterator &_firstWord, float &io_f)
{
  // skip first token
    ++_firstWord;
    // use lexical cast to convert to float then increment the itor
    io_f=boost::lexical_cast<float>(*_firstWord++);
}

A Subtle Bug

On problem I had when initially using this system was that the texture files were not found when loading. It turned out that as this was a windows mtl  file it was using \ in the file names and I was running under mac osx and linux which expected /. To overcome this problem the filename paths are parsed and / converted to \ and visa-versa depending upon operating system.

void Mtl::parseString(tokenizer::iterator &_firstWord, std::string &io_s)
{
  ++_firstWord;
  // there is a chance that we have either windows or linux slashes
  // need to process file name for either
  io_s=*_firstWord;

  #ifdef WIN32
  std::replace(io_s.begin(), io_s.end(), '/', '\\');
  #else
    std::replace(io_s.begin(), io_s.end(), '\\', '/');
  #endif
}

Designing for efficiency

One of the many things to think about when designing the class is the efficiency of the data storage / texture usage. It is quite possible that the maps used are loaded by several materials and only the multipliers are changed.  To ensure that the data is not replicated, the textures are processed when loaded. First I step through each of the materials and load them into a std::list. then the std::list::unique method is called to remove any duplicates. Once this is done the textures are loaded and the ID's stored in a std::vector. Finally these are re-associated with the mtlItem data to store all the values.
void Mtl::loadTextures()
{
  m_textureID.clear();
  std::cout<<"loading textures this may take some time\n";
  // first loop and store all the texture names in the container
  std::list <std::string> names;
  std::map<std::string, mtlItem *>::const_iterator end=m_materials.end();
  std::map<std::string, mtlItem *>::const_iterator i = m_materials.begin();
  for( ; i != end; ++i )
  {
    if(i->second->map_Ka.size() !=0)
      names.push_back(i->second->map_Ka);
    if(i->second->map_Kd.size() !=0)
      names.push_back(i->second->map_Kd);
    if(i->second->map_d.size() !=0)
      names.push_back(i->second->map_d);
    if(i->second->map_bump.size() !=0)
      names.push_back(i->second->map_bump);
    if(i->second->map_bump.size() !=0)
      names.push_back(i->second->bump);
  }

  std::cout<<"we have this many textures "<<names.size()<<"\n";
  // now remove duplicates
  names.unique();
  std::cout<<"we have "<<names.size()<<" unique textures to load\n";
  // now we load the textures and get the GL id
  // now we associate the ID with the mtlItem

  BOOST_FOREACH(std::string name , names)
  {
    std::cout<<"loading texture "<<name<<"\n";
    ngl::Texture t(name);
    GLuint textureID=t.setTextureGL();
    m_textureID.push_back(textureID);
    std::cout<<"processing "<<name<<"\n";
    i=m_materials.begin();
    for( ; i != end; ++i )
    {
      if(i->second->map_Ka == name)
        i->second->map_KaId=textureID;
      if(i->second->map_Kd == name)
        i->second->map_KdId=textureID;
      if(i->second->map_d == name)
        i->second->map_dId=textureID;
      if(i->second->map_bump == name)
        i->second->map_bumpId=textureID;
      if(i->second->bump == name)
        i->second->bumpId=textureID;
    }
  }
  std::cout <<"done \n";
}

Serialisation

Whilst the parsing of the mtl file is relatively quick it was decided to allow for both binary read and write of the data. As there is a lot of text data to save the process is not quite as simple as it could be. For most of the data we need to determine the length of the string to write out then I write out the size of the string followed by the string data. I also decided to write out a unique ID for the file header so we can check that the file being loaded is the correct one.

The code to write the data is as follows

bool Mtl::saveBinary(const std::string &_fname) const
{
  std::ofstream fileOut;
  fileOut.open(_fname.c_str(),std::ios::out | std::ios::binary);
  if (!fileOut.is_open())
  {
    std::cout <<"File : "<<_fname<<" could not be written for output"<<std::endl;
    return false;
  }
  // write our own id into the file so we can check we have the correct type
  // when loading
  const std::string header("ngl::mtlbin");
  fileOut.write(header.c_str(),header.length());

  unsigned int size=m_materials.size();
  fileOut.write(reinterpret_cast<char *>(&size),sizeof(size));
  std::map<std::string, mtlItem *>::const_iterator start=m_materials.begin();
  std::map<std::string, mtlItem *>::const_iterator end=m_materials.end();
  for(; start!=end; ++start)
  {
    //std::cout<<"writing out "<<start->first<<"\n";
    // first write the length of the string
    size=start->first.length();
    fileOut.write(reinterpret_cast<char *>(&size),sizeof(size));
    // now the string
    fileOut.write(reinterpret_cast<const char *>(start->first.c_str()),size);
    // now we do the different data elements of the mtlItem.
    fileOut.write(reinterpret_cast<char *>(&start->second->Ns),sizeof(float));
    fileOut.write(reinterpret_cast<char *>(&start->second->Ni),sizeof(float));
    fileOut.write(reinterpret_cast<char *>(&start->second->d),sizeof(float));
    fileOut.write(reinterpret_cast<char *>(&start->second->Tr),sizeof(float));
    fileOut.write(reinterpret_cast<char *>(&start->second->illum),sizeof(int));

    fileOut.write(reinterpret_cast<char *>(&start->second->Tf),sizeof(ngl::Vec3));
    fileOut.write(reinterpret_cast<char *>(&start->second->Ka),sizeof(ngl::Vec3));
    fileOut.write(reinterpret_cast<char *>(&start->second->Kd),sizeof(ngl::Vec3));
    fileOut.write(reinterpret_cast<char *>(&start->second->Ks),sizeof(ngl::Vec3));
    fileOut.write(reinterpret_cast<char *>(&start->second->Ke),sizeof(ngl::Vec3));

    // first write the length of the string
    size=start->second->map_Ka.length();
    fileOut.write(reinterpret_cast<char *>(&size),sizeof(size));
    // now the string
    fileOut.write(reinterpret_cast<const char *>(start->second->map_Ka.c_str()),size);

    // first write the length of the string
    size=start->second->map_Kd.length();
    fileOut.write(reinterpret_cast<char *>(&size),sizeof(size));
    // now the string
    fileOut.write(reinterpret_cast<const char *>(start->second->map_Kd.c_str()),size);

    // first write the length of the string
    size=start->second->map_d.length();
    fileOut.write(reinterpret_cast<char *>(&size),sizeof(size));
    // now the string
    fileOut.write(reinterpret_cast<const char *>(start->second->map_d.c_str()),size);

    // first write the length of the string
    size=start->second->map_bump.length();
    fileOut.write(reinterpret_cast<char *>(&size),sizeof(size));
    // now the string
    fileOut.write(reinterpret_cast<const char *>(start->second->map_bump.c_str()),size);

    // first write the length of the string
    size=start->second->bump.length();
    fileOut.write(reinterpret_cast<char *>(&size),sizeof(size));
    // now the string
    fileOut.write(reinterpret_cast<const char *>(start->second->bump.c_str()),size);
  }


  fileOut.close();
  return true;
}
Loading in the data is almost the reverse of writing it, we first read in the header bytes and check to see if it is the correct file type then read in the data re-sizing the strings to we have enough room to read the data into it.
bool Mtl::loadBinary(const std::string &_fname)
{
  std::ifstream fileIn;
  fileIn.open(_fname.c_str(),std::ios::in | std::ios::binary);
  if (!fileIn.is_open())
  {
    std::cout <<"File : "<<_fname<<" could not be opened for reading"<<std::endl;
    return false;
  }
  // clear out what we already have.
  clear();
  unsigned int mapsize;


  char header[12];
  fileIn.read(header,11*sizeof(char));
  header[11]=0; // for strcmp we need \n
  // basically I used the magick string ngl::bin (I presume unique in files!) and
  // we test against it.
  if(strcmp(header,"ngl::mtlbin"))
  {
    // best close the file and exit
    fileIn.close();
    std::cout<<"this is not an ngl::mtlbin file "<<std::endl;
    return false;
  }


  fileIn.read(reinterpret_cast<char *>(&mapsize),sizeof(mapsize));
  unsigned int size;
  std::string materialName;
  std::string s;
  for(unsigned int i=0; i<mapsize; ++i)
  {
    mtlItem *item = new mtlItem;

    fileIn.read(reinterpret_cast<char *>(&size),sizeof(size));
    // now the string we first need to allocate space then copy in
    materialName.resize(size);
   fileIn.read(reinterpret_cast<char *>(&materialName[0]),size);
    // now we do the different data elements of the mtlItem.
   fileIn.read(reinterpret_cast<char *>(&item->Ns),sizeof(float));
   fileIn.read(reinterpret_cast<char *>(&item->Ni),sizeof(float));
   fileIn.read(reinterpret_cast<char *>(&item->d),sizeof(float));
   fileIn.read(reinterpret_cast<char *>(&item->Tr),sizeof(float));
   fileIn.read(reinterpret_cast<char *>(&item->illum),sizeof(int));

   fileIn.read(reinterpret_cast<char *>(&item->Tf),sizeof(ngl::Vec3));
   fileIn.read(reinterpret_cast<char *>(&item->Ka),sizeof(ngl::Vec3));
   fileIn.read(reinterpret_cast<char *>(&item->Kd),sizeof(ngl::Vec3));
   fileIn.read(reinterpret_cast<char *>(&item->Ks),sizeof(ngl::Vec3));
   fileIn.read(reinterpret_cast<char *>(&item->Ke),sizeof(ngl::Vec3));
  // more strings
   fileIn.read(reinterpret_cast<char *>(&size),sizeof(size));
   // now the string we first need to allocate space then copy in
   s.resize(size);
   fileIn.read(reinterpret_cast<char *>(&s[0]),size);
   item->map_Ka=s;

   fileIn.read(reinterpret_cast<char *>(&size),sizeof(size));
   // now the string we first need to allocate space then copy in
   s.resize(size);
   fileIn.read(reinterpret_cast<char *>(&s[0]),size);
   item->map_Kd=s;

   fileIn.read(reinterpret_cast<char *>(&size),sizeof(size));
   // now the string we first need to allocate space then copy in
   s.resize(size);
   fileIn.read(reinterpret_cast<char *>(&s[0]),size);
   item->map_d=s;

   fileIn.read(reinterpret_cast<char *>(&size),sizeof(size));
   // now the string we first need to allocate space then copy in
   s.resize(size);
   fileIn.read(reinterpret_cast<char *>(&s[0]),size);
   item->map_bump=s;

   fileIn.read(reinterpret_cast<char *>(&size),sizeof(size));
   // now the string we first need to allocate space then copy in
   s.resize(size);
   fileIn.read(reinterpret_cast<char *>(&s[0]),size);
   item->bump=s;

   m_materials[materialName]=item;
  }
  m_loadTextures=true;
  loadTextures();
  return true;
}

Using the class

It is quite easy to use the class as the following code demonstrates

Mtl *m_mtl = new Mtl("models/sponza.mtl",true);
m_mtl->saveBinary("sponzaMtl.bin");

bool loaded=m_mtl->loadBinary("sponzaMtl.bin");
if(loaded == false)
{
 std::cerr<<"error loading mtl file ";
 exit(EXIT_FAILURE);
}

m_mtl->debugPrint();

That is about it for the Mtl class, the next post will describe the design of the GroupedObj class.

Sponza Demo Pt 1 Initial Design

The Sponza model is quite popular for real-time graphics visualisation, It can be downloaded from the CryTek website, and you can see many demos of it being used for different lighting tests on youtube.

I decided it would be a good example to use in a bigger system of how to load and process meshes / textures in OpenGL and also as part of a bigger modelling / game pipeline.

The main model comes in two parts, a wavefront obj file (obj) and a Material template library file (mtl). These files are simple text files and can be exported from all major animation packages.

Initial Design
My initial design for the program is as follows
I will split the Mtl and Obj files into two different classes, the Mtl class will be responsible for loading the textures and storing OpenGL texture ID's. The ObjG (GroupedObj) class will load the Obj and then determine and process the groups in the file and store all the information required to draw all the individual grouped elements.

The mesh itself will be uploaded as a single OpenGL Vertex Array Object (VAO) and elements will be drawn as sub meshes with the correct textures enabled.

I decided to design / write the Mtl class first. This design can be seen in the next post.

Tuesday, 6 March 2012

Setting OpenGL Formats in Qt

Got asked how to enable multi-sampling in my demos the other day, and realised that I hadn't shared this information in any of my lectures so I thought I would write it up here.

OpenGL has a number of extensions which allow a number of different rendering features to be enabled. For example we can do stereo, accumulation  multisampling and much more.

In Qt we do this by using the QGLFormat class and we can enable it for a specific QGLWidget, or for all widgets we create. For this example I will generate a default format and then create a widget to use the format. This code would be put in main.cpp in my demos (will update them soon) before the creation of the MainWindow class.

QGLFormat glf = QGLFormat::defaultFormat();
glf.setSampleBuffers(true);
glf.setSamples(4);
QGLFormat::setDefaultFormat(glf);
Now when we create the GLWindow in the ngl:: demos this will be used for all windows created. Finally we need to enable GL_MULTISAMPLE when rendering which can be done using the following code
glEnable(GL_MULTISAMPLE);
This method can also be used to enable things such as the new OpenGL core profile (under linux and windows for Qt 4.7 and with Qt 4.8 this will also work on the mac eventually, as long as you have Lion). You should be able to see from the documentation all the other features which can be enabled in this way.

Thursday, 20 October 2011

Text Rendering using OpenGL 3.2

I've been working on porting the ngl:: library to be completely OpenGL core profile 3.2. One of the main issues I've had is the rendering of text. In the last version I used the QGLWidget::renderText method, however this is not available when using core profile OpenGL and also has a habit of crashing when using certain modern OpenGL elements.

To overcome this problem I have designed and implemented my own OpenGL text renderer and the following post will explain the process / design of this and my approach.  The approach I use follows the standard methods used in other packages and the following survey gives a good overview of these. My main concern with the design was that it only uses Fonts from Qt and that it works with OpenGL 3.2 core profile and above.

Initial Process
The usual process of text rendering is to create a billboard for each character in the text string and apply a texture to that quad for the given character. The image below shows a basic set of billboards and a (not to scale) set of text glyphs


The process we need to follow to create the text to render is as follows
  1. Load a font and generate all the characters required
  2. Generate OpenGL textures for each individual character
  3. Generate billboards for each glyph depending upon the char height and width
  4. Store this data for later text rendering
As the design of this system evolved It was decided to foce the user of the fonts to decide in advance the size and emphasis of the the font.  This was done so the texture / billboard generation only happens once at the start of the program, however once this is done any text can be rendered using this font set. This makes any programs using the text much quicker as the data is effectively cached as OpenGL textures / Vertex Array Objects

Loading the Font
The initial design decision was to create a single texture image containing all the visible ASCII characters, however this was soon dropped as there are issues with the ways that fonts are constructed to have different kerning values. So for the final design an individual image / texture is created for each font, with the character calculated to be at an origin of (0,0) Top left of the screen. Using this we will need to then calculate the height of the font once and for each character generated store the width of the different text types. 

As we are using Qt we can use the QFont class and the QFontMetrics class to gather all of this information. For each class the initial design was to store the following :-
struct FontChar
{
    int width; /// @brief the width of the font
    GLuint textureID; /// @brief the texture id of the font billboard
    ngl::VertexArrayObject *vao; /// a vao for the font
};


This class stores the individual width of each character's glyph gathered from the method QFontMetric::width() as well as the id for a generated texture and a pointer to a vertex array object which will contain the Vertices and UV data for the billboard.

The following code shows the basic process of generating each of the font textures.

  // so first we grab the font metric of the font being used
  QFontMetrics metric(_f);
  // this allows us to get the height which should be the same for all
  // fonts of the same class as this is the total glyph height
  float fontHeight=metric.height();

  // loop for all basic keyboard chars we will use space to ~
  // should really change this to unicode at some stage
  const static char startChar=' ';
  const static char endChar='~';
  // Most OpenGL cards need textures to be in powers of 2 (128x512 1024X1024 etc etc) so
  // to be safe we will conform to this and calculate the nearest power of 2 for the glyph height
  // we will do the same for each width of the font below
  int heightPow2=nearestPowerOfTwo(fontHeight);

  // we are now going to create a texture / billboard for each font
  // they will be the same height but will possibly have different widths
  for(char c=startChar; c<=endChar; ++c)
  {
    QChar ch(c);
    FontChar fc;
    // get the width of the font and calculate the ^2 size
    int width=metric.width(c);
    int widthPow2=nearestPowerOfTwo(width);
    // now we set the texture co-ords for our quad it is a simple
    // triangle billboard with tex-cords as shown
    //  s0/t0  ---- s1,t0
    //         |\ |
    //         | \|
    //  s0,t1  ---- s1,t1
    // each quad will have the same s0 and the range s0-s1 == 0.0 -> 1.0
    ngl::Real s0=0.0;
    // we now need to scale the tex cord to it ranges from 0-1 based on the coverage
    // of the glyph and not the power of 2 texture size. This will ensure that kerns
    // / ligatures match
    ngl::Real s1=width*1.0/widthPow2;
    // t0 will always be the same
    ngl::Real t0=0.0;
    // this will scale the height so we only get coverage of the glyph as above
    ngl::Real t1=metric.height()*-1.0/heightPow2;
    // we need to store the font width for later drawing
    fc.width=width;
    // now we will create a QImage to store the texture, basically we are going to draw
    // into the qimage then save this in OpenGL format and load as a texture.
    // This is relativly quick but should be done as early as possible for max performance when drawing
    QImage finalImage(nearestPowerOfTwo(width),nearestPowerOfTwo(fontHeight),QImage::Format_ARGB32);
    // set the background for transparent so we can avoid any areas which don't have text in them
    finalImage.fill(Qt::transparent);
    // we now use the QPainter class to draw into the image and create our billboards
    QPainter painter;
    painter.begin(&finalImage);
    // try and use high quality text rendering (works well on the mac not as good on linux)
    painter.setRenderHints(QPainter::HighQualityAntialiasing
                   | QPainter::TextAntialiasing);
    // set the font to draw with
    painter.setFont(_f);
    // we set the glyph to be drawn in black the shader will override the actual colour later
    // see TextShader.h in src/shaders/
    painter.setPen(Qt::black);
    // finally we draw the text to the Image
    painter.drawText(0, metric.ascent(), QString(c));
    painter.end();

First we need to create a QImage to render the character to a glyph. As most OpenGL implementations required power of 2 textures we also need to round our image size to the nearest power of two. To do this I found a handy function here which is called with the image width and height to make sure we get the correct texture sizes.

Next we set the texture fill to be Qt::transparent and the image pen colour to be Qt::black. As the actual text rendering is done by a special shader this value is unimportant, what we are actually after is the alpha  values in the image which are used to indicate the "coverage" of the text (more of this later).
After this is done we use the QPainter class to render the text into our QImage.

This stage is all Qt specific, as long as you have a Font library which allows you to save the glyphs into and image format and gather the font metrics it should be easy to port to other libraries.

Generating the Billboards
Now we have the font dimensions and the glyph as a QImage we can generate both the billboard and the  OpenGL textures.

To generate the textures we use the following code
 // now we create the OpenGL texture ID and bind to make it active
    glGenTextures(1, &fc.textureID);
    glBindTexture(GL_TEXTURE_2D, fc.textureID);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    // QImage has a method to convert itself to a format suitable for OpenGL
    // we call this and then load to OpenGL
    finalImage = QGLWidget::convertToGLFormat(finalImage);
    // the image in in RGBA format and unsigned byte load it ready for later
    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, finalImage.width(), finalImage.height(),
       0, GL_RGBA, GL_UNSIGNED_BYTE, finalImage.bits());




The image above show the original sketch for the Billboard as two triangles, with the UV cords and the sequence of generation. The ngl library has a simple class for storing this kind of data called the VertexArrayObject all we have to do is pass the data to the class and then use it to draw. The following code does this.
 // this structure is used by the VAO to store the data to be uploaded
    // for drawing the quad
    struct textVertData
    {
    ngl::Real x;
    ngl::Real y;
    ngl::Real u;
    ngl::Real v;
    };
    // we are creating a billboard with two triangles so we only need the
    // 6 verts, (could use index and save some space but shouldn't be too much of an
    // issue
    textVertData d[6];
    // load values for triangle 1
    d[0].x=0;
    d[0].y=0;
    d[0].u=s0;
    d[0].v=t0;

    d[1].x=fc.width;
    d[1].y=0;
    d[1].u=s1;
    d[1].v=t0;

    d[2].x=0;
    d[2].y=fontHeight;
    d[2].u=s0;
    d[2].v=t1;
    // load values for triangle two
    d[3].x=0;
    d[3].y=0+fontHeight;
    d[3].u=s0;
    d[3].v=t1;


    d[4].x=fc.width;
    d[4].y=0;
    d[4].u=s1;
    d[4].v=t0;


    d[5].x=fc.width;
    d[5].y=fontHeight;
    d[5].u=s1;
    d[5].v=t1;


    // now we create a VAO to store the data
    ngl::VertexArrayObject *vao=ngl::VertexArrayObject::createVOA(GL_TRIANGLES);
    // bind it so we can set values
    vao->bind();
    // set the vertex data (2 for x,y 2 for u,v)
    vao->setData(6*sizeof(textVertData),d[0].x);
    // now we set the attribute pointer to be 0 (as this matches vertIn in our shader)
    vao->setVertexAttributePointer(0,2,GL_FLOAT,sizeof(textVertData),0);
    // We can now create another set of data (which will be added to the VAO)
    // in this case the UV co-ords
    // now we set this as the 2nd attribute pointer (1) to match inUV in the shader
    vao->setVertexAttributePointer(1,2,GL_FLOAT,sizeof(textVertData),2);
    // say how many indecis to be rendered
    vao->setNumIndices(6);

    // now unbind
    vao->unbind();
    // store the vao pointer for later use in the draw method
    fc.vao=vao;
    // finally add the element to the map, this must be the last
    // thing we do
    m_characters[c]=fc;
  }

Text rendering 
To render the text we need to convert from screen space (where top left is 0,0) to OpenGL NDC space, effectively we need to create an Orthographic project for our billboard to place it on the correct place.

The following diagrams show the initial designs for this.


As each of the billboards is initially calculated with the top right at (0,0) this transformation can be simplified. The following vertex shader is used to position the billboard vertices.

#version 150 
in vec2 inVert; 
in vec2 inUV; 
out vec2 vertUV; 
uniform vec3 textColour; 
uniform float scaleX; 
uniform float scaleY; 
uniform float xpos;  
uniform float ypos; 

void main() 
{ 
 vertUV=inUV; 
 gl_Position=vec4( ((xpos+inVert.x)*scaleX)-1.0,((ypos+inVert.y)*scaleY)+1.0,0.0,1.0);
}

This shader is passed the scaleX and scaleY values from the Text class these values are calculated in the method setScreenSize as shown below.

 
void Text::setScreenSize(
                          int _w,
                          int _h
                        )
{

  float scaleX=2.0/_w;
  float scaleY=-2.0/_h;
  // in shader we do the following code to transform from
  // x,y to NDC
  // gl_Position=vec4( ((xpos+inVert.x)*scaleX)-1,((ypos+inVert.y)*scaleY)+1.0,0.0,1.0); "
  // so all we need to do is calculate the scale above and pass to shader every time the
  // screen dimensions change
  ngl::ShaderLib *shader=ngl::ShaderLib::instance();
  (*shader)["nglTextShader"]->use();

  shader->setShaderParam1f("scaleX",scaleX);
  shader->setShaderParam1f("scaleY",scaleY);
}

This method must be called every time the screen changes size so the x,y position of the font are correctly calculated.

The xpos / ypos uniforms are the x,y co-ordinates of the current text to be rendered and the actual billboard vertices are passed to the shader as the inVert attribute. This is shown in the renderText method below.

void Text::renderText(
                      float _x,
                      float _y,
                      const QString &text
                     ) const
{
  // make sure we are in texture unit 0 as this is what the
  // shader expects
  glActiveTexture(0);
  // grab an instance of the shader manager
  ngl::ShaderLib *shader=ngl::ShaderLib::instance();
  // use the built in text rendering shader
  (*shader)["nglTextShader"]->use();
  // the y pos will always be the same so set it once for each
  // string we are rendering
  shader->setShaderParam1f("ypos",_y);
  // now enable blending and disable depth sorting so the font renders
  // correctly
  glEnable(GL_BLEND);
  glDisable(GL_DEPTH_TEST);
  glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
  // now loop for each of the char and draw our billboard
  unsigned int textLength=text.length();

  for (unsigned int i = 0; i < textLength; ++i)
  {
    // set the shader x position this will change each time
    // we render a glyph by the width of the char
    shader->setShaderParam1f("xpos",_x);
    // so find the FontChar data for our current char
    FontChar f = m_characters[text[i].toAscii()];
    // bind the pre-generated texture
    glBindTexture(GL_TEXTURE_2D, f.textureID);
    // bind the vao
    f.vao->bind();
    // draw
    f.vao->draw();
    // now unbind the vao
    f.vao->unbind();
    // finally move to the next glyph x position by incrementing
    // by the width of the char just drawn
    _x+=f.width;

  }
  // finally disable the blend and re-enable depth sort
  glDisable(GL_BLEND);
  glEnable(GL_DEPTH_TEST);

}
Before we can render the text we need to enable the OpenGL alpha blending and we use the SRC_ALPHA, ONE_MINUS_SRC_ALPHA so the text will be rendered over the top of any other geometry. I also disable depth sorting so if the text is rendered last it should always appear over any geometry.

Text Colour
To set the text colour we use the fragment shader below.
#version 150 

uniform sampler2D tex; 
in vec2 vertUV; 
out vec4 fragColour; 
uniform vec3 textColour; 

void main() 
{ 
  vec4 text=texture(tex,vertUV.st); 
  fragColour.rgb=textColour.rgb; 
  fragColour.a=text.a; 
}

This shader is quite simple, it takes the input texture (the glyph) and grabs the alpha channel (which we can think of as the coverage of the ink). We then set the rest of the colour to be the user defined current colour and this will get rendered to the screen as shown in the following screen shot

Future Work
As an initial proof of concept / working version this works quite well. It is reasonably fast and most fonts I have tried seem to work ok (including Comic Sans!)

The next tests / optimisations are to determine if we have any billboards of the same size and only store ones we need. This should same VAO space and make things a little more efficient.

Update 
I've actually added code to do the billboard optimisation, all it needed was
QHash <int,vertexarrayobject * > widthVAO;
 ......
// see if we have a Billboard of this width already
if (!widthVAO.contains(width))
{
      // do the billboard vao creation
}
else
{
  fc.vao=widthVAO[width];
}
For Times font we now only create 15 Billboards, for Arial 16 unique billboards, for Courier only one as it's a mono spaced font.

Now for Unicode support!

Wednesday, 1 December 2010

GLSL Shader Manager design Part 3

(for some more refinements see this post)

In the previous post I discussed the design of the ShaderProgram class, in this final instalment I will discuss the overall ShaderManager class and it's basic usage.

The basic design consideration for the  manager is it will allow for the creation of Shaders and ShaderPrograms that will be independent of each other, these will be stored by name as a std::string and basic methods may be called to configure and load these.

It will also have the ability to access the ShaderPrograms via a lookup using the [] operator passing in the name of the shader.

The class diagram is as follows


The std::map containers for the shader programs and Shaders contain pointers to the classes so each of them can live as separate entities and Shader Programs may share multiple shaders.

Construction of the class will also create a nullProgram object as discussed in the previous post. The motivation for this is to allow the return of a class which will be a glUseProgram(0) but still be an object capable of being callable for methods without crashing.
ShaderManager::ShaderManager()
{
 m_debugState=true;
 m_nullProgram = new ShaderProgram("NULL");
}
To create a shader program we use the following method to construct a new ShaderProgram class

void ShaderManager::createShaderProgram(std::string _name)
{
 std::cerr<<"creating empty ShaderProgram "<<_name.c_str()<<"\n";
 m_shaderPrograms[_name]= new ShaderProgram(_name);
}
Next we can add shaders using the following methods
void ShaderManager::attachShader(
                                  std::string _name,
                                  SHADERTYPE _type
                                )
{
  m_shaders[_name]= new Shader(_name,_type);
}
At this stage the shader may not have any source attached to it so we have a method to allow the loading of the shader source.

void ShaderManager::loadShaderSource(std::string _shaderName, std::string _sourceFile)
{
  std::map <std::string, Shader * >::const_iterator shader=m_shaders.find(_shaderName);
  // make sure we have a valid shader and program
 if(shader!=m_shaders.end() )
  {
    shader->second->load(_sourceFile);
  }
  else {std::cerr<<"Warning shader not know in loadShaderSource "<<_shaderName.c_str();}
}

Once the source is loaded to compile the Shader we do the following
void ShaderManager::compileShader(std::string _name)
{
  // get an iterator to the shaders
  std::map <std::string, Shader * >::const_iterator shader=m_shaders.find(_name);
  // make sure we have a valid shader
 if(shader!=m_shaders.end())
  {
    // grab the pointer to the shader and call compile
    shader->second->compile();
  }
  else {std::cerr<<"Warning shader not know in compile "<<_name.c_str();}
}

Next we need to attach the shader to a program, as both classes are contained in maps we have to search for both and then call the relevant methods as show in the following code

void ShaderManager::attachShaderToProgram(std::string _program,std::string _shader)

{

  // get an iterator to the shader and program
  std::map <std::string, Shader * >::const_iterator shader=m_shaders.find(_shader);
  std::map <std::string, ShaderProgram * >::const_iterator program=m_shaderPrograms.find(_program);

  // make sure we have a valid shader and program
 if(shader!=m_shaders.end() && program !=m_shaderPrograms.end())
  {
    // now attach the shader to the program
    program->second->attatchShader(shader->second);
    // now increment the shader ref count so we know if how many references
    shader->second->incrementRefCount();

    if (m_debugState == true)
    {
      std::cerr<<_shader.c_str()<<" attached to program "<<_program.c_str()<<"\n";
    }
  }
  else {std::cerr<<"Warning cant attach "<<_shader.c_str() <<" to "<<_program.c_str()<<"\n";}
}

To overload the [] operator to work with a std::string we use the following
ShaderProgram * ShaderManager::operator[](const std::string &_programName)
{
  std::map <std::string, ShaderProgram * >::const_iterator program=m_shaderPrograms.find(_programName);
  // make sure we have a valid  program
 if(program!=m_shaderPrograms.end() )
  {
    return  program->second;
  }
  else
  {
    std::cerr<<"Warning Program not know in [] "<<_programName.c_str();
    std::cerr<<"returning a null program and hoping for the best\n";
    return m_nullProgram;
  }
}
For brevity the rest of the methods can be seen in the source download here (lecture 8)

Using the ShaderManager
The file GLWindow.cpp demonstrates the use of the ShaderManager, we must first initialise GLEW if we are using linux or windows (Mac OS X is fine)

The following example program is based on the code here and is basically OpenGL 3.x compatible, however I use GLSL Version 120 as the basis as mac OSX doesn't support higher at present (come on APPLE !)

We uses very basic shaders that allow the basic vertex and colour attributes to be named in the client program and the shader.
#version 120
attribute vec3 inPosition;
attribute vec3 inColour;
varying vec3 vertColour;

void main()
{
 gl_Position = vec4(inPosition, 1.0);
 vertColour = inColour;
}


#version 120

varying vec3 vertColour;

void main()
{
  gl_FragColor = vec4(vertColour,1.0);
}
Generating Vertex data
The vertex data is created using a vertex buffer object and a vertex array object these will then be bound to attributes in the shaders

void GLWindow::createTriangle()
{
 // First simple object
 float* vert = new float[9]; // vertex array
 float* col  = new float[9]; // color array

 vert[0] =-0.3; vert[1] = 0.5; vert[2] =-1.0;
 vert[3] =-0.8; vert[4] =-0.5; vert[5] =-1.0;
 vert[6] = 0.2; vert[7] =-0.5; vert[8]= -1.0;

 col[0] = 1.0; col[1] = 0.0; col[2] = 0.0;
 col[3] = 0.0; col[4] = 1.0; col[5] = 0.0;
 col[6] = 0.0; col[7] = 0.0; col[8] = 1.0;

 // Second simple object
 float* vert2 = new float[9]; // vertex array

 vert2[0] =-0.2; vert2[1] = 0.5; vert2[2] =-1.0;
 vert2[3] = 0.3; vert2[4] =-0.5; vert2[5] =-1.0;
 vert2[6] = 0.8; vert2[7] = 0.5; vert2[8]= -1.0;

 // Two VAOs allocation
  glGenVertexArrays(2, &m_vaoID[0]);

 // First VAO setup
  glBindVertexArray(m_vaoID[0]);

 glGenBuffers(2, m_vboID);

 glBindBuffer(GL_ARRAY_BUFFER, m_vboID[0]);
 glBufferData(GL_ARRAY_BUFFER, 9*sizeof(GLfloat), vert, GL_STATIC_DRAW);
  m_shaderManager["Simple"]->vertexAttribPointer("inPosition",3,GL_FLOAT,0,0);
  m_shaderManager["Simple"]->enableAttribArray("inPosition");
 glBindBuffer(GL_ARRAY_BUFFER, m_vboID[1]);

 glBufferData(GL_ARRAY_BUFFER, 9*sizeof(GLfloat), col, GL_STATIC_DRAW);
  m_shaderManager["Simple"]->vertexAttribPointer("inColour",3,GL_FLOAT,0,0);
  m_shaderManager["Simple"]->enableAttribArray("inColour");

 // Second VAO setup
  glBindVertexArray(m_vaoID[1]);

 glGenBuffers(1, &m_vboID[2]);

 glBindBuffer(GL_ARRAY_BUFFER, m_vboID[2]);
  ceckGLError(__FILE__,__LINE__);
 glBufferData(GL_ARRAY_BUFFER, 9*sizeof(GLfloat), vert2, GL_STATIC_DRAW);
 glVertexAttribPointer((GLuint)0, 3, GL_FLOAT, GL_FALSE, 0, 0);
 glEnableVertexAttribArray(0);
  glBindVertexArray(0);

 delete [] vert;
 delete [] vert2;
 delete [] col;
}
Next we load and build the Shader program using the ShaderManager class
void GLWindow::initializeGL()
{
  ngl::NGLInit *init = ngl::NGLInit::instance();
  init->initGlew();
  glClearColor(0.4f, 0.4f, 0.4f, 1.0f);

  m_shaderManager.createShaderProgram("Simple");

  m_shaderManager.attachShader("SimpleVertex",VERTEX);
  m_shaderManager.attachShader("SimpleFragment",FRAGMENT);
  m_shaderManager.loadShaderSource("SimpleVertex","shaders/Vertex.vs");
  m_shaderManager.loadShaderSource("SimpleFragment","shaders/Fragment.fs");

  m_shaderManager.compileShader("SimpleVertex");
  m_shaderManager.compileShader("SimpleFragment");
  m_shaderManager.attachShaderToProgram("Simple","SimpleVertex");
  m_shaderManager.attachShaderToProgram("Simple","SimpleFragment");


  m_shaderManager.bindAttribute("Simple",0,"inPosition");
  m_shaderManager.bindAttribute("Simple",1,"inColour");

  m_shaderManager.linkProgramObject("Simple");
  m_shaderManager["Simple"]->use();

  createTriangle();
}
You will notice before we link the shader program we assign the inPosition and inColour attributes to the attribute locations 0 and 1 respectively. This will be used in the drawing routine later.

void GLWindow::paintGL()
{

  glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
  m_shaderManager["Simple"]->use();

  glBindVertexArray(m_vaoID[0]);  // select first VAO
  glDrawArrays(GL_TRIANGLES, 0, 3); // draw first object

  glBindVertexArray(m_vaoID[1]);  // select second VAO
  m_shaderManager["Simple"]->vertexAttrib3f("inColour",1.0,0.0,0.0);
  //glVertexAttrib3f((GLuint)1, 1.0, 0.0, 0.0); // set constant color attribute
  glDrawArrays(GL_TRIANGLES, 0, 3); // draw second object
}

Next Step
One of the initial goals of this design was for a standalone version of a shader manager that people could use in their own project, with very few external library dependancies (at present we only need OpenGL, GLEW and Qt for the project and some string IO).

The next step is to integrate this into my existing ngl:: library and replace the existing ShaderManager class. This is going to happen very soon and I will post an update on the integration and a fuller critique of the design once this integration has happened, and finally a teapot

Sunday, 28 November 2010

GLSL Shader Manager design Part 2

In the previous post I discussed the initial design of the Shader Manager class and outlined the design of the Shader Class. This post will concentrate on the design of the ShaderProgram class which contains the actual Shader sources and be linked and ready to use with GLSL.

The Orange book states

“Each shader object is compiled independently. To create a program,  applications need a mechanism for specifying a list of shader objects to be linked. You can specify the list of shaders objects to be linked by creating a program object and attaching to it all the shader objects needed to create the program”. (Rost 2009)

To this end we need to create an empty program object then attach the existing shaders to the program. The attachment can be made before the shader is compiled, however the Program object can't be linked unless the shaders are compiled. As shaders may be attached to more than one Program we basically need to hold a list of pointers to the shader objects rather than the shaders themselves.

The ShaderProgram will also be the main access point to the shaders once loaded on the GPU so it will have all the methods for accessing the attributes in the program. As OpenGL is a C based library there is no method overloading so we need to implement a method for each of the different accessors, in this cas the class have over 50 methods, however for brevity the class diagram only show the basic ones as seen in the following class diagram.


As part of the design it was also decided to allow attributes in the shader to be bound using a std::string so they could be referenced by name and not the numeric id.

The constructor for the class is as follows

ShaderProgram::ShaderProgram(std::string _name)
{
  // we create a special NULL program so the shader manager can return
  // a NULL object.
 if (_name !="NULL")
  {
    m_programID = glCreateProgram();
  }
  else
  {
    m_programID=0;
  }
  std::cerr <<"Created program id is "<<m_programid><<"\n";
  m_debugState=true;
  m_programName=_name;
  m_linked=false;
  m_active=false;
}
In the constructor we check for a special name called "NULL" this integrates with the ShaderManager class so we can create an empty default shader object with an ID of 0. GLSL uses the 0 shader to represent the "fixed functionality" pipeline. By default the ShaderManager class will create a NULL shader so that if the named one passed by the user is not found a fixed shader will be returned and calls to this object will not crash the system.

To attach a shader we use the following code

void ShaderProgram::attatchShader(Shader *_shader)
{
  m_shaders.push_back(_shader);
  glAttachShader(m_programID,_shader->getShaderHandle());
}


Any number of shaders may be attached to the ShaderProgram, once we are ready we can link the shader

void ShaderProgram::link()
{
  glLinkProgram(m_programID);
  if(m_debugState==true)
  {
    std::cerr <<"linking Shader "<< m_programName.c_str()<<"\n";
  }
  GLint infologLength = 0;

  glGetProgramiv(m_programID,GL_INFO_LOG_LENGTH,&infologLength);
  std::cerr<<"Link Log Length "<<infologLength<<"\n";

  if(infologLength > 0)
  {
    char *infoLog = new char[infologLength];
    GLint charsWritten  = 0;

    glGetProgramInfoLog(m_programID, infologLength, &charsWritten, infoLog);

    std::cerr<<infoLog<<std::endl;
    delete [] infoLog;
    glGetProgramiv(m_programID, GL_LINK_STATUS,&infologLength);
    if( infologLength == GL_FALSE)
    {
      std::cerr<<"Program link failed exiting \n";
      exit(EXIT_FAILURE);
    }
  }
  m_linked=true;
}
At present if the link fails the program will exit, I'm not sure if this is actually the best approach but will do for now as this a developmental system.

If we wish to bind the attributes in the shader we can do it before the link of the programs or after however each approach needs a different coding approach. At present binding is only available before linking, however this will be modified in the next version to check the link state and use the appropriate method. Attributes must be specified by the user using a numeric value and the name to be bound. Once this has been done the user may then specify attributes by name only.

void ShaderProgram::bindAttrib(GLuint _index, std::string _attribName)
{
  if(m_linked == true)
  {
    std::cerr<<"Warning binding attribute after link\n";
  }
  m_attribs[_attribName]=_index;
  glBindAttribLocation(m_programID,_index,_attribName.c_str());
  std::cerr<<"bindAttribLoc "<<m_programID<<" index "<<_index<<" name "<<_attribName<<"\n";
  ceckGLError(__FILE__,__LINE__);
}
Once an attribute is bound we can access it via name using the following functions

bool ShaderProgram::vertexAttribPointer(
                                        const char* _name,
                                        GLint _size,
                                        GLenum _type,
                                        GLsizei _stride,
                                        const GLvoid *_data,
                                        bool _normalise
                                       ) const
{

  std::map <std::string, GLuint >::const_iterator attrib=m_attribs.find(_name);
  // make sure we have a valid  program
 if(attrib!=m_attribs.end() )
  {
    glVertexAttribPointer(attrib->second,_size,_type,_normalise,_stride,_data);
    return  true;
  }
  else
  {
   return false;
  }
}

void ShaderProgram::vertexAttrib1f(
                                  const char * _name,
                                  GLfloat   _v0
                                  ) const
{
  std::map <std::string, GLuint >::const_iterator attrib=m_attribs.find(_name);
  // make sure we have a valid  program
 if(attrib!=m_attribs.end() )
  {
    glVertexAttrib1f(attrib->second, _v0);

  }

}

Accessing Uniforms
To access the uniform data within the shader we need to query the linked program object to get the numeric location of the variable, once this is found we can modify the variable using the OpenGL functions. The following method returns the numeric ID for a uniform and is used in the other functions

GLuint ShaderProgram::getUniformLocation(
                                          const char* _name
                                        ) const
{
  GLint loc = glGetUniformLocation( m_programID ,_name);
  if (loc == -1)
  {
    std::cerr<<"Uniform \""<<_name<<"\" not found in Program \""<<m_programName<<"\"\n";
  }
  return loc;
}

Now for every access variable type in OpenGL we can write a method to change the uniform, for example to change a uniform float we use the following code

void ShaderProgram::setUniform1f(
                                  const char* _varname,
                                  float _v0
                                ) const
{
  glUniform1f(getUniformLocation(_varname),_v0);
}

The rest of the class is rather repetitive code to allow the different attributes and uniforms to be accessed, the next post will look at the overall management class for the Shader system with examples of how it's used.

A Note on std::map
Whilst writing this I found the way I'd been using the std::map could lead to errors. If you consider the following code

#include <iostream>
#include <map>

int main()
{

  std::map<std::string,int> mymap;
  mymap["a"]=1;
  mymap["b"]=2;
  std::cout<<mymap.size()<<"\n";
}
Will print out a size of 2 as expected and we can access the map values using the ["a"] type syntax. However if we do the following
#include <iostream>
#include <map>

int main()
{

  std::map<std::string,int> mymap;
  mymap["a"]=1;
  mymap["b"]=2;
  std::cout<<mymap.size()<<"\n";
  std::cout<<mymap["c"]<<"\n";
  std::cout<<mymap.size()<<"\n";
}

The program still works but as the key "c" is not know it will output a value of 0 for the line mymap["c"] however it will also insert a new map entry and the 2nd call to mymap.size will return the size 3.
To overcome this behaviour we must use the find method as shown in some of the examples above.
References
Rost, R, Licea-Kane B (2009). OpenGL Shading Language. 3rd. ed. New York: Addison Wesley.