Text Rendering using OpenGL 3.2

I've been working on porting the ngl:: library to be completely OpenGL core profile 3.2. One of the main issues I've had is the rendering of text. In the last version I used the QGLWidget::renderText method, however this is not available when using core profile OpenGL and also has a habit of crashing when using certain modern OpenGL elements.

To overcome this problem I have designed and implemented my own OpenGL text renderer and the following post will explain the process / design of this and my approach.  The approach I use follows the standard methods used in other packages and the following survey gives a good overview of these. My main concern with the design was that it only uses Fonts from Qt and that it works with OpenGL 3.2 core profile and above.

Initial Process
The usual process of text rendering is to create a billboard for each character in the text string and apply a texture to that quad for the given character. The image below shows a basic set of billboards and a (not to scale) set of text glyphs

The process we need to follow to create the text to render is as follows

1. Load a font and generate all the characters required
2. Generate OpenGL textures for each individual character
3. Generate billboards for each glyph depending upon the char height and width
4. Store this data for later text rendering

As the design of this system evolved It was decided to foce the user of the fonts to decide in advance the size and emphasis of the the font.  This was done so the texture / billboard generation only happens once at the start of the program, however once this is done any text can be rendered using this font set. This makes any programs using the text much quicker as the data is effectively cached as OpenGL textures / Vertex Array Objects

Loading the Font

The initial design decision was to create a single texture image containing all the visible ASCII characters, however this was soon dropped as there are issues with the ways that fonts are constructed to have different kerning values. So for the final design an individual image / texture is created for each font, with the character calculated to be at an origin of (0,0) Top left of the screen. Using this we will need to then calculate the height of the font once and for each character generated store the width of the different text types. 

As we are using Qt we can use the QFont class and the QFontMetrics class to gather all of this information. For each class the initial design was to store the following :-

struct FontChar
{
    int width; /// @brief the width of the font
    GLuint textureID; /// @brief the texture id of the font billboard
    ngl::VertexArrayObject *vao; /// a vao for the font
};

This class stores the individual width of each character's glyph gathered from the method QFontMetric::width() as well as the id for a generated texture and a pointer to a vertex array object which will contain the Vertices and UV data for the billboard.

The following code shows the basic process of generating each of the font textures.

  // so first we grab the font metric of the font being used
  QFontMetrics metric(_f);
  // this allows us to get the height which should be the same for all
  // fonts of the same class as this is the total glyph height
  float fontHeight=metric.height();

  // loop for all basic keyboard chars we will use space to ~
  // should really change this to unicode at some stage
  const static char startChar=' ';
  const static char endChar='~';
  // Most OpenGL cards need textures to be in powers of 2 (128x512 1024X1024 etc etc) so
  // to be safe we will conform to this and calculate the nearest power of 2 for the glyph height
  // we will do the same for each width of the font below
  int heightPow2=nearestPowerOfTwo(fontHeight);

  // we are now going to create a texture / billboard for each font
  // they will be the same height but will possibly have different widths
  for(char c=startChar; c<=endChar; ++c)
  {
    QChar ch(c);
    FontChar fc;
    // get the width of the font and calculate the ^2 size
    int width=metric.width(c);
    int widthPow2=nearestPowerOfTwo(width);
    // now we set the texture co-ords for our quad it is a simple
    // triangle billboard with tex-cords as shown
    //  s0/t0  ---- s1,t0
    //         |\ |
    //         | \|
    //  s0,t1  ---- s1,t1
    // each quad will have the same s0 and the range s0-s1 == 0.0 -> 1.0
    ngl::Real s0=0.0;
    // we now need to scale the tex cord to it ranges from 0-1 based on the coverage
    // of the glyph and not the power of 2 texture size. This will ensure that kerns
    // / ligatures match
    ngl::Real s1=width*1.0/widthPow2;
    // t0 will always be the same
    ngl::Real t0=0.0;
    // this will scale the height so we only get coverage of the glyph as above
    ngl::Real t1=metric.height()*-1.0/heightPow2;
    // we need to store the font width for later drawing
    fc.width=width;
    // now we will create a QImage to store the texture, basically we are going to draw
    // into the qimage then save this in OpenGL format and load as a texture.
    // This is relativly quick but should be done as early as possible for max performance when drawing
    QImage finalImage(nearestPowerOfTwo(width),nearestPowerOfTwo(fontHeight),QImage::Format_ARGB32);
    // set the background for transparent so we can avoid any areas which don't have text in them
    finalImage.fill(Qt::transparent);
    // we now use the QPainter class to draw into the image and create our billboards
    QPainter painter;
    painter.begin(&finalImage);
    // try and use high quality text rendering (works well on the mac not as good on linux)
    painter.setRenderHints(QPainter::HighQualityAntialiasing
                   | QPainter::TextAntialiasing);
    // set the font to draw with
    painter.setFont(_f);
    // we set the glyph to be drawn in black the shader will override the actual colour later
    // see TextShader.h in src/shaders/
    painter.setPen(Qt::black);
    // finally we draw the text to the Image
    painter.drawText(0, metric.ascent(), QString(c));
    painter.end();

First we need to create a QImage to render the character to a glyph. As most OpenGL implementations required power of 2 textures we also need to round our image size to the nearest power of two. To do this I found a handy function here which is called with the image width and height to make sure we get the correct texture sizes.

Next we set the texture fill to be Qt::transparent and the image pen colour to be Qt::black. As the actual text rendering is done by a special shader this value is unimportant, what we are actually after is the alpha  values in the image which are used to indicate the "coverage" of the text (more of this later).
After this is done we use the QPainter class to render the text into our QImage.

This stage is all Qt specific, as long as you have a Font library which allows you to save the glyphs into and image format and gather the font metrics it should be easy to port to other libraries.

Generating the Billboards
Now we have the font dimensions and the glyph as a QImage we can generate both the billboard and the  OpenGL textures.

To generate the textures we use the following code

 // now we create the OpenGL texture ID and bind to make it active
    glGenTextures(1, &fc.textureID);
    glBindTexture(GL_TEXTURE_2D, fc.textureID);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    // QImage has a method to convert itself to a format suitable for OpenGL
    // we call this and then load to OpenGL
    finalImage = QGLWidget::convertToGLFormat(finalImage);
    // the image in in RGBA format and unsigned byte load it ready for later
    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, finalImage.width(), finalImage.height(),
       0, GL_RGBA, GL_UNSIGNED_BYTE, finalImage.bits());

The image above show the original sketch for the Billboard as two triangles, with the UV cords and the sequence of generation. The ngl library has a simple class for storing this kind of data called the VertexArrayObject all we have to do is pass the data to the class and then use it to draw. The following code does this.

 // this structure is used by the VAO to store the data to be uploaded
    // for drawing the quad
    struct textVertData
    {
    ngl::Real x;
    ngl::Real y;
    ngl::Real u;
    ngl::Real v;
    };
    // we are creating a billboard with two triangles so we only need the
    // 6 verts, (could use index and save some space but shouldn't be too much of an
    // issue
    textVertData d[6];
    // load values for triangle 1
    d[0].x=0;
    d[0].y=0;
    d[0].u=s0;
    d[0].v=t0;

    d[1].x=fc.width;
    d[1].y=0;
    d[1].u=s1;
    d[1].v=t0;

    d[2].x=0;
    d[2].y=fontHeight;
    d[2].u=s0;
    d[2].v=t1;
    // load values for triangle two
    d[3].x=0;
    d[3].y=0+fontHeight;
    d[3].u=s0;
    d[3].v=t1;


    d[4].x=fc.width;
    d[4].y=0;
    d[4].u=s1;
    d[4].v=t0;


    d[5].x=fc.width;
    d[5].y=fontHeight;
    d[5].u=s1;
    d[5].v=t1;


    // now we create a VAO to store the data
    ngl::VertexArrayObject *vao=ngl::VertexArrayObject::createVOA(GL_TRIANGLES);
    // bind it so we can set values
    vao->bind();
    // set the vertex data (2 for x,y 2 for u,v)
    vao->setData(6*sizeof(textVertData),d[0].x);
    // now we set the attribute pointer to be 0 (as this matches vertIn in our shader)
    vao->setVertexAttributePointer(0,2,GL_FLOAT,sizeof(textVertData),0);
    // We can now create another set of data (which will be added to the VAO)
    // in this case the UV co-ords
    // now we set this as the 2nd attribute pointer (1) to match inUV in the shader
    vao->setVertexAttributePointer(1,2,GL_FLOAT,sizeof(textVertData),2);
    // say how many indecis to be rendered
    vao->setNumIndices(6);

    // now unbind
    vao->unbind();
    // store the vao pointer for later use in the draw method
    fc.vao=vao;
    // finally add the element to the map, this must be the last
    // thing we do
    m_characters[c]=fc;
  }

Text rendering 
To render the text we need to convert from screen space (where top left is 0,0) to OpenGL NDC space, effectively we need to create an Orthographic project for our billboard to place it on the correct place.

The following diagrams show the initial designs for this.

As each of the billboards is initially calculated with the top right at (0,0) this transformation can be simplified. The following vertex shader is used to position the billboard vertices.

#version 150 
in vec2 inVert; 
in vec2 inUV; 
out vec2 vertUV; 
uniform vec3 textColour; 
uniform float scaleX; 
uniform float scaleY; 
uniform float xpos;  
uniform float ypos; 

void main() 
{ 
 vertUV=inUV; 
 gl_Position=vec4( ((xpos+inVert.x)*scaleX)-1.0,((ypos+inVert.y)*scaleY)+1.0,0.0,1.0);
}

This shader is passed the scaleX and scaleY values from the Text class these values are calculated in the method setScreenSize as shown below.

 
void Text::setScreenSize(
                          int _w,
                          int _h
                        )
{

  float scaleX=2.0/_w;
  float scaleY=-2.0/_h;
  // in shader we do the following code to transform from
  // x,y to NDC
  // gl_Position=vec4( ((xpos+inVert.x)*scaleX)-1,((ypos+inVert.y)*scaleY)+1.0,0.0,1.0); "
  // so all we need to do is calculate the scale above and pass to shader every time the
  // screen dimensions change
  ngl::ShaderLib *shader=ngl::ShaderLib::instance();
  (*shader)["nglTextShader"]->use();

  shader->setShaderParam1f("scaleX",scaleX);
  shader->setShaderParam1f("scaleY",scaleY);
}

This method must be called every time the screen changes size so the x,y position of the font are correctly calculated.

The xpos / ypos uniforms are the x,y co-ordinates of the current text to be rendered and the actual billboard vertices are passed to the shader as the inVert attribute. This is shown in the renderText method below.

void Text::renderText(
                      float _x,
                      float _y,
                      const QString &text
                     ) const
{
  // make sure we are in texture unit 0 as this is what the
  // shader expects
  glActiveTexture(0);
  // grab an instance of the shader manager
  ngl::ShaderLib *shader=ngl::ShaderLib::instance();
  // use the built in text rendering shader
  (*shader)["nglTextShader"]->use();
  // the y pos will always be the same so set it once for each
  // string we are rendering
  shader->setShaderParam1f("ypos",_y);
  // now enable blending and disable depth sorting so the font renders
  // correctly
  glEnable(GL_BLEND);
  glDisable(GL_DEPTH_TEST);
  glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
  // now loop for each of the char and draw our billboard
  unsigned int textLength=text.length();

  for (unsigned int i = 0; i < textLength; ++i)
  {
    // set the shader x position this will change each time
    // we render a glyph by the width of the char
    shader->setShaderParam1f("xpos",_x);
    // so find the FontChar data for our current char
    FontChar f = m_characters[text[i].toAscii()];
    // bind the pre-generated texture
    glBindTexture(GL_TEXTURE_2D, f.textureID);
    // bind the vao
    f.vao->bind();
    // draw
    f.vao->draw();
    // now unbind the vao
    f.vao->unbind();
    // finally move to the next glyph x position by incrementing
    // by the width of the char just drawn
    _x+=f.width;

  }
  // finally disable the blend and re-enable depth sort
  glDisable(GL_BLEND);
  glEnable(GL_DEPTH_TEST);

}

Before we can render the text we need to enable the OpenGL alpha blending and we use the SRC_ALPHA, ONE_MINUS_SRC_ALPHA so the text will be rendered over the top of any other geometry. I also disable depth sorting so if the text is rendered last it should always appear over any geometry.

Text Colour
To set the text colour we use the fragment shader below.

#version 150 

uniform sampler2D tex; 
in vec2 vertUV; 
out vec4 fragColour; 
uniform vec3 textColour; 

void main() 
{ 
  vec4 text=texture(tex,vertUV.st); 
  fragColour.rgb=textColour.rgb; 
  fragColour.a=text.a; 
}

This shader is quite simple, it takes the input texture (the glyph) and grabs the alpha channel (which we can think of as the coverage of the ink). We then set the rest of the colour to be the user defined current colour and this will get rendered to the screen as shown in the following screen shot

Future Work

As an initial proof of concept / working version this works quite well. It is reasonably fast and most fonts I have tried seem to work ok (including Comic Sans!)

The next tests / optimisations are to determine if we have any billboards of the same size and only store ones we need. This should same VAO space and make things a little more efficient.

Update 

I've actually added code to do the billboard optimisation, all it needed was

QHash <int,vertexarrayobject * > widthVAO;
 ......
// see if we have a Billboard of this width already
if (!widthVAO.contains(width))
{
      // do the billboard vao creation
}
else
{
  fc.vao=widthVAO[width];
}

For Times font we now only create 15 Billboards, for Arial 16 unique billboards, for Courier only one as it's a mono spaced font.

Now for Unicode support!