The mtl file has the following basic structure, where each element may or may not be present.
newmtl leaf Ns 10.0000 Ni 1.5000 d 1.0000 Tr 0.0000 Tf 1.0000 1.0000 1.0000 illum 2 Ka 0.5880 0.5880 0.5880 Kd 0.5880 0.5880 0.5880 Ks 0.0000 0.0000 0.0000 Ke 0.0000 0.0000 0.0000 map_Ka textures\sponza_thorn_diff.tga map_Kd textures\sponza_thorn_diff.tga map_d textures\sponza_thorn_mask.tga map_bump textures\sponza_thorn_ddn.tga bump textures\sponza_thorn_ddn.tga
The newmtl keyword is used to indicate that a new material is being specified and the rest of the elements are part of that material. To store this information I decided to use a std::map using a std::string as the key which will be the name following the newmtl (in the above example this is "leaf"). The map will then store the following structure.
typedef struct
{
float Ns;
float Ni;
float d;
float Tr;
int illum;
ngl::Vec3 Tf;
ngl::Vec3 Ka;
ngl::Vec3 Kd;
ngl::Vec3 Ks;
ngl::Vec3 Ke;
std::string map_Ka;
std::string map_Kd;
std::string map_d;
std::string map_bump;
std::string bump;
GLuint map_KaId;
GLuint map_KdId;
GLuint map_dId;
GLuint map_bumpId;
GLuint bumpId;
}mtlItem;
You will notice that this replicates the mtl format shown above and also have several extra items which have the extra postfix ID, these will be used to store the OpenGL texture ID's of the textures once loaded. I also made the design decision not to follow my usual coding standard for the structure as this would make it easier to follow what is happening in the mtl file.
Class functional design
The main elements of the class are shown in the following class diagram.
The main functional design of the Mtl class is split into two areas, first the loading and parsing of the original mtl file. This can either be done in the constructor or using the load method. Next is the actual use of the class data. To allow easy access to this data iterators have been exposed for the std::map as well as other methods. Finally we have the ability to save and load the data in a binary format to save the parse time of reading the original files.
Parsing the mtl file
As the structure of the mtl file is quite simple I decided a full blown parser was not required. Instead I decided to use the boost::tokenizer template to process the data. The file is opened and the data read a line at a time. The tokenizer splits the data and looks for the keywords, which are then processed one at a time. This is shown in the following code.
// this is the line we wish to parse
std::string lineBuffer;
// say which separators should be used in this
// case Spaces, Tabs and return \ new line
boost::char_separator<char> sep(" \t\r\n");
// loop through the file
while(!fileIn.eof())
{
// grab a line from the input
getline(fileIn,lineBuffer,'\n');
// make sure it's not an empty line
if(lineBuffer.size() >1)
{
// now tokenize the line
tokenizer tokens(lineBuffer, sep);
// and get the first token
tokenizer::iterator firstWord = tokens.begin();
// now see if it's a valid one and call the correct function
if( *firstWord =="newmtl")
{
// add to our map it is possible that a badly formed file would not have an mtl
// def first however this is so unlikely I can't be arsed to handle that case.
// If it does crash it could be due to this code.
//std::cout<<"found "<<m_currentName<<"\n";
parseString(firstWord,m_currentName);
m_current= new mtlItem;
// These are the OpenGL texture ID's so set to zero first (for no texture)
m_current->map_KaId=0;
m_current->map_KdId=0;
m_current->map_dId=0;
m_current->map_bumpId=0;
m_current->bumpId=0;
m_materials[m_currentName]=m_current;
}
else if(*firstWord =="Ns")
{
parseFloat(firstWord,m_current->Ns);
}
else if(*firstWord =="Ni")
{
parseFloat(firstWord,m_current->Ni);
}
else if(*firstWord =="d")
{
parseFloat(firstWord,m_current->d);
}
else if(*firstWord =="Tr")
{
parseFloat(firstWord,m_current->Tr);
}
else if(*firstWord =="Tf")
{
parseVec3(firstWord,m_current->Tf);
}
else if(*firstWord =="illum")
{
parseInt(firstWord,m_current->illum);
}
else if(*firstWord =="Ka")
{
parseVec3(firstWord,m_current->Ka);
}
else if(*firstWord =="Kd")
{
parseVec3(firstWord,m_current->Kd);
}
else if(*firstWord =="Ks")
{
parseVec3(firstWord,m_current->Ks);
}
else if(*firstWord =="Ke")
{
parseVec3(firstWord,m_current->Ke);
}
else if(*firstWord == "map_Ka")
{
parseString(firstWord,m_current->map_Ka);
}
else if(*firstWord == "map_Kd")
{
parseString(firstWord,m_current->map_Kd);
}
else if(*firstWord == "map_d")
{
parseString(firstWord,m_current->map_d);
}
else if(*firstWord == "map_bump")
{
parseString(firstWord,m_current->map_bump);
}
else if(*firstWord == "bump")
{
parseString(firstWord,m_current->bump);
}
} // end zero line
} // end while
// as the trigger for putting the meshes back is the newmtl we will always have a hanging one
// this adds it to the list
m_materials[m_currentName]=m_current;
The individual parse functions then use the boost::lexical_cast template to convert the values as shown in the following example
void Mtl::parseFloat(tokenizer::iterator &_firstWord, float &io_f)
{
// skip first token
++_firstWord;
// use lexical cast to convert to float then increment the itor
io_f=boost::lexical_cast<float>(*_firstWord++);
}
A Subtle Bug
On problem I had when initially using this system was that the texture files were not found when loading. It turned out that as this was a windows mtl file it was using \ in the file names and I was running under mac osx and linux which expected /. To overcome this problem the filename paths are parsed and / converted to \ and visa-versa depending upon operating system.void Mtl::parseString(tokenizer::iterator &_firstWord, std::string &io_s)
{
++_firstWord;
// there is a chance that we have either windows or linux slashes
// need to process file name for either
io_s=*_firstWord;
#ifdef WIN32
std::replace(io_s.begin(), io_s.end(), '/', '\\');
#else
std::replace(io_s.begin(), io_s.end(), '\\', '/');
#endif
}
Designing for efficiency
One of the many things to think about when designing the class is the efficiency of the data storage / texture usage. It is quite possible that the maps used are loaded by several materials and only the multipliers are changed. To ensure that the data is not replicated, the textures are processed when loaded. First I step through each of the materials and load them into a std::list. then the std::list::unique method is called to remove any duplicates. Once this is done the textures are loaded and the ID's stored in a std::vector. Finally these are re-associated with the mtlItem data to store all the values.
void Mtl::loadTextures()
{
m_textureID.clear();
std::cout<<"loading textures this may take some time\n";
// first loop and store all the texture names in the container
std::list <std::string> names;
std::map<std::string, mtlItem *>::const_iterator end=m_materials.end();
std::map<std::string, mtlItem *>::const_iterator i = m_materials.begin();
for( ; i != end; ++i )
{
if(i->second->map_Ka.size() !=0)
names.push_back(i->second->map_Ka);
if(i->second->map_Kd.size() !=0)
names.push_back(i->second->map_Kd);
if(i->second->map_d.size() !=0)
names.push_back(i->second->map_d);
if(i->second->map_bump.size() !=0)
names.push_back(i->second->map_bump);
if(i->second->map_bump.size() !=0)
names.push_back(i->second->bump);
}
std::cout<<"we have this many textures "<<names.size()<<"\n";
// now remove duplicates
names.unique();
std::cout<<"we have "<<names.size()<<" unique textures to load\n";
// now we load the textures and get the GL id
// now we associate the ID with the mtlItem
BOOST_FOREACH(std::string name , names)
{
std::cout<<"loading texture "<<name<<"\n";
ngl::Texture t(name);
GLuint textureID=t.setTextureGL();
m_textureID.push_back(textureID);
std::cout<<"processing "<<name<<"\n";
i=m_materials.begin();
for( ; i != end; ++i )
{
if(i->second->map_Ka == name)
i->second->map_KaId=textureID;
if(i->second->map_Kd == name)
i->second->map_KdId=textureID;
if(i->second->map_d == name)
i->second->map_dId=textureID;
if(i->second->map_bump == name)
i->second->map_bumpId=textureID;
if(i->second->bump == name)
i->second->bumpId=textureID;
}
}
std::cout <<"done \n";
}
Serialisation
Whilst the parsing of the mtl file is relatively quick it was decided to allow for both binary read and write of the data. As there is a lot of text data to save the process is not quite as simple as it could be. For most of the data we need to determine the length of the string to write out then I write out the size of the string followed by the string data. I also decided to write out a unique ID for the file header so we can check that the file being loaded is the correct one.
The code to write the data is as follows
bool Mtl::saveBinary(const std::string &_fname) const
{
std::ofstream fileOut;
fileOut.open(_fname.c_str(),std::ios::out | std::ios::binary);
if (!fileOut.is_open())
{
std::cout <<"File : "<<_fname<<" could not be written for output"<<std::endl;
return false;
}
// write our own id into the file so we can check we have the correct type
// when loading
const std::string header("ngl::mtlbin");
fileOut.write(header.c_str(),header.length());
unsigned int size=m_materials.size();
fileOut.write(reinterpret_cast<char *>(&size),sizeof(size));
std::map<std::string, mtlItem *>::const_iterator start=m_materials.begin();
std::map<std::string, mtlItem *>::const_iterator end=m_materials.end();
for(; start!=end; ++start)
{
//std::cout<<"writing out "<<start->first<<"\n";
// first write the length of the string
size=start->first.length();
fileOut.write(reinterpret_cast<char *>(&size),sizeof(size));
// now the string
fileOut.write(reinterpret_cast<const char *>(start->first.c_str()),size);
// now we do the different data elements of the mtlItem.
fileOut.write(reinterpret_cast<char *>(&start->second->Ns),sizeof(float));
fileOut.write(reinterpret_cast<char *>(&start->second->Ni),sizeof(float));
fileOut.write(reinterpret_cast<char *>(&start->second->d),sizeof(float));
fileOut.write(reinterpret_cast<char *>(&start->second->Tr),sizeof(float));
fileOut.write(reinterpret_cast<char *>(&start->second->illum),sizeof(int));
fileOut.write(reinterpret_cast<char *>(&start->second->Tf),sizeof(ngl::Vec3));
fileOut.write(reinterpret_cast<char *>(&start->second->Ka),sizeof(ngl::Vec3));
fileOut.write(reinterpret_cast<char *>(&start->second->Kd),sizeof(ngl::Vec3));
fileOut.write(reinterpret_cast<char *>(&start->second->Ks),sizeof(ngl::Vec3));
fileOut.write(reinterpret_cast<char *>(&start->second->Ke),sizeof(ngl::Vec3));
// first write the length of the string
size=start->second->map_Ka.length();
fileOut.write(reinterpret_cast<char *>(&size),sizeof(size));
// now the string
fileOut.write(reinterpret_cast<const char *>(start->second->map_Ka.c_str()),size);
// first write the length of the string
size=start->second->map_Kd.length();
fileOut.write(reinterpret_cast<char *>(&size),sizeof(size));
// now the string
fileOut.write(reinterpret_cast<const char *>(start->second->map_Kd.c_str()),size);
// first write the length of the string
size=start->second->map_d.length();
fileOut.write(reinterpret_cast<char *>(&size),sizeof(size));
// now the string
fileOut.write(reinterpret_cast<const char *>(start->second->map_d.c_str()),size);
// first write the length of the string
size=start->second->map_bump.length();
fileOut.write(reinterpret_cast<char *>(&size),sizeof(size));
// now the string
fileOut.write(reinterpret_cast<const char *>(start->second->map_bump.c_str()),size);
// first write the length of the string
size=start->second->bump.length();
fileOut.write(reinterpret_cast<char *>(&size),sizeof(size));
// now the string
fileOut.write(reinterpret_cast<const char *>(start->second->bump.c_str()),size);
}
fileOut.close();
return true;
}
Loading in the data is almost the reverse of writing it, we first read in the header bytes and check to see if it is the correct file type then read in the data re-sizing the strings to we have enough room to read the data into it.
bool Mtl::loadBinary(const std::string &_fname)
{
std::ifstream fileIn;
fileIn.open(_fname.c_str(),std::ios::in | std::ios::binary);
if (!fileIn.is_open())
{
std::cout <<"File : "<<_fname<<" could not be opened for reading"<<std::endl;
return false;
}
// clear out what we already have.
clear();
unsigned int mapsize;
char header[12];
fileIn.read(header,11*sizeof(char));
header[11]=0; // for strcmp we need \n
// basically I used the magick string ngl::bin (I presume unique in files!) and
// we test against it.
if(strcmp(header,"ngl::mtlbin"))
{
// best close the file and exit
fileIn.close();
std::cout<<"this is not an ngl::mtlbin file "<<std::endl;
return false;
}
fileIn.read(reinterpret_cast<char *>(&mapsize),sizeof(mapsize));
unsigned int size;
std::string materialName;
std::string s;
for(unsigned int i=0; i<mapsize; ++i)
{
mtlItem *item = new mtlItem;
fileIn.read(reinterpret_cast<char *>(&size),sizeof(size));
// now the string we first need to allocate space then copy in
materialName.resize(size);
fileIn.read(reinterpret_cast<char *>(&materialName[0]),size);
// now we do the different data elements of the mtlItem.
fileIn.read(reinterpret_cast<char *>(&item->Ns),sizeof(float));
fileIn.read(reinterpret_cast<char *>(&item->Ni),sizeof(float));
fileIn.read(reinterpret_cast<char *>(&item->d),sizeof(float));
fileIn.read(reinterpret_cast<char *>(&item->Tr),sizeof(float));
fileIn.read(reinterpret_cast<char *>(&item->illum),sizeof(int));
fileIn.read(reinterpret_cast<char *>(&item->Tf),sizeof(ngl::Vec3));
fileIn.read(reinterpret_cast<char *>(&item->Ka),sizeof(ngl::Vec3));
fileIn.read(reinterpret_cast<char *>(&item->Kd),sizeof(ngl::Vec3));
fileIn.read(reinterpret_cast<char *>(&item->Ks),sizeof(ngl::Vec3));
fileIn.read(reinterpret_cast<char *>(&item->Ke),sizeof(ngl::Vec3));
// more strings
fileIn.read(reinterpret_cast<char *>(&size),sizeof(size));
// now the string we first need to allocate space then copy in
s.resize(size);
fileIn.read(reinterpret_cast<char *>(&s[0]),size);
item->map_Ka=s;
fileIn.read(reinterpret_cast<char *>(&size),sizeof(size));
// now the string we first need to allocate space then copy in
s.resize(size);
fileIn.read(reinterpret_cast<char *>(&s[0]),size);
item->map_Kd=s;
fileIn.read(reinterpret_cast<char *>(&size),sizeof(size));
// now the string we first need to allocate space then copy in
s.resize(size);
fileIn.read(reinterpret_cast<char *>(&s[0]),size);
item->map_d=s;
fileIn.read(reinterpret_cast<char *>(&size),sizeof(size));
// now the string we first need to allocate space then copy in
s.resize(size);
fileIn.read(reinterpret_cast<char *>(&s[0]),size);
item->map_bump=s;
fileIn.read(reinterpret_cast<char *>(&size),sizeof(size));
// now the string we first need to allocate space then copy in
s.resize(size);
fileIn.read(reinterpret_cast<char *>(&s[0]),size);
item->bump=s;
m_materials[materialName]=item;
}
m_loadTextures=true;
loadTextures();
return true;
}
Using the class
It is quite easy to use the class as the following code demonstrates
Mtl *m_mtl = new Mtl("models/sponza.mtl",true);
m_mtl->saveBinary("sponzaMtl.bin");
bool loaded=m_mtl->loadBinary("sponzaMtl.bin");
if(loaded == false)
{
std::cerr<<"error loading mtl file ";
exit(EXIT_FAILURE);
}
m_mtl->debugPrint();
That is about it for the Mtl class, the next post will describe the design of the GroupedObj class.
No comments:
Post a Comment