13 September 2008

STL: A Love-and-Hate Relationship

I'm a big fan of the STL and specifically the containers. Usually. The interface generally makes sense, code is written around them fairly quickly and they have good interoperability. However, there are some problems that really start to become apparent on larger projects. Consider these search results on EQII's codebase:
  • "std::map" - 2460
  • "std::set" - 1574
  • "std::list" - 393

The EQII codebase has nearly 3 million lines of just C/C++/C# (client, server, tools, not counting server lua script and data files) and uses STL containers liberally. If I had to do it all over again, I believe I'd forgo using STL containers for a few reasons:
  • Memory Usage - STL allocators typically hold on to memory forever and reuse it as necessary. However, EQII has a very good memory manager with a constant-time free-able small block allocator (written by yours truly), reporting, leak detection, etc. Writing custom allocators for STL is a major pain (especially on an established codebase) and I'd really rather not hack STLport or the gnu implementation to use our memory manager.
  • Compile Times - The more templated code you have the longer the compile and link times. I've talked previously about how we replaced std::vector with our own class to save compile times. Also, have you looked at STL implementation code? It's bloated and ugly, which brings me to my next point...
  • Debugging - Trying to dig through a container in a debugger watch window (or GDB) is a pain. Sometimes it's even a matter of obscure memory offsets and you have to calculate addresses to see values in a map.
  • Additional Functionality - If you have a vector that you're not using anymore and you want to free the memory for it, clear() doesn't cut it. The code to do it is neither trivial nor intuitive: std::vector< T >().swap( myVec ). Adding functions isn't easy as the STL containers are purposely non-virtual.

I'll probably look for a different solution for containers for my next project. Home-grown containers can emulate the STL containers and provide all that I desire above. They can even be compatible with STL functors and algorithms too.


Mark Storer said...

Given how important memory management can be to fast execution, I'm surprised SOMEONE at SOE hasn't written a custom allocator yet.

Meyer's "Effective STL", items 10 and 11 (in the Containers section) seem to cover the gotchas. 10 even points to a couple samples with all the boilerplate code it says you'll need.

Mark Storer said...

PS: One of the sample code URLs is to www.cuj.com, which is now camped by a domain squatter. The article has been moved to DDJ's site: