Latest news on the nVidia 8800 problem

We just did an update to our Test Server that has a new option for players suffering from the nVidia 8800 stuttering issue. The new option turns on a vertex buffer cache that will hold onto and reuse vertex and index buffers without trying to free them all the time.

We plan to hotfix this code live soon.

Incidentally, this Test update also has a fix for the Veeshan's Peak particle leak.

Leaking memory by the kilo[byte]

I finally got around to looking into reports of a memory leak in Veeshan's Peak mostly dealing with lava bombs on a bridge.

Sure enough, there's a leak. After a 30 minute meeting my test client had grown half a gig. Just while watching lava bombs bash a bridge. (Well, this is a debug-build client which uses more memory and doesn't have any optimizations, so that number is a little inflated)

Knowing that you have a leak is just the beginning. Sometimes, finding it (or fixing it) can be a pain.

We have some pretty decent tools built into the EQ2 framework for dealing with memory issues. I'm pretty proud of them. Given that, it didn't take long to find out that we were leaking particles, hundreds of them per frame.

EQ2 uses a lot of reference-counted objects to reduce memory leaks. Ironically, they can also contribute to memory leaks when you have a reference cycle (or circular reference). A reference cycle is basically when two objects hold references to each other. Unless something breaks the cycle, the reference counts won't decrease allowing both objects to be destroyed. Finding reference cycles can be difficult too since it's easy to determine that something is holding a reference to an object, but it's not always so easy to know which object is holding the reference.
... Until I instrumented our smart-pointer class to do that without having to change the many thousands of places where smart-pointers are used. The concept is pretty simple: The smart-pointer remembers the line of code (code address) where it increments the reference. The base object (with the reference counting support) remembers which smart-pointers are pointing to it. This allows me to know which line of code caused the excess reference counts. In this case, the smart-pointer is no longer a simple class that adds and removes references; it gets a little heavier. The good news is that you only need that extra weight when you're trying to find memory leaks. I'll leave this as an exercise for the reader.

So, case in point, we definitely had a reference cycle with some particles. Basically particles would reference scenegraph nodes that owned them. The fix is essentially to not hold a reference if it would cause a cycle, but we still need to hold a reference if there wouldn't be a cycle.

Therefore, without further ado, the fix:

Now, any excuse to use placement new gets me a little giddy. Plus I want to keep the Particle object size small because we always have a lot of them.

union
{
    SceneGraphNode* pRawNodePointer;
    char space[ sizeof( SmartPointer< SceneGraphNode > ) ];
} m_nodePointer;

Since these two members are bound by a union, I can only use one at a time. If I want to use a smart pointer, I have to construct it using placement new:

new (m_nodePointer.space) SmartPointer< SceneGraphNode >( pNode );

If I want to switch and use the raw pointer, I have to destruct the smart pointer:

reinterpret_cast< SmartPointer< SceneGraphNode >* >( m_nodePointer.space )->~SmartPointer< SceneGraphNode >();
m_nodePointer.pRawNodePointer = pNode;

Voíla! All the particles get cleaned up now. This fix should be making its way onto live servers pretty soon.

Update 04/17/08: This fix is now on live servers!

nVidia 8800 Update

If you're following the current EQ2/nVidia 8800 continuing saga, here's an update:

nVidia finally contacted us (after seeing my previous blog post i might add). We're working with a technology evangelist there to get the problem resolved.

Since some people mentioned not having issues with Vanguard: Saga of Heroes, I started chatting with the VG team about ideas. Unfortunately the ideas that they've given me so far haven't panned out, but there's more to try.

Between both of these, we're definitely working on a solution. I'll keep you posted.

Update 4/16/2008: See latest update

Interview up on Stratics

My good friend Ransom of BiC is working for Stratics now and asked me if she could do an interview of me.

Head on over to Stratics and check it out! (And the forum thread)

April Fools' Memories

Every April Fools' day it reminds me of my first 4/1 in the game industry. I had been working on Ultima Online for about six months and it was still run by Origin Systems in Austin, TX. The players had rose-colored glasses of the era of UO that came with the first expansion (The Second Age) just prior to the second expansion (Renaissance). There was always a post on the forums calling for a pre-Renaissance server.

So on April Fools' I decided to give them just that. I tweaked some flags on the test server to turn on the much more difficult "Siege Perilous" ruleset and renamed it "UO Pre-Ren".

I wish I still had a link to the post on the forums with the player responses after that.

In retrospect, while it may seem a little mean, it was well within the spirit of the holiday. And you've just got to expect stuff like that on April Fools' day.

And, in case you're wondering what we did in EQ2, Kendricke found it by surprise.


ALSO: I just checked the Rock Band song list to see if there were any new songs worth downloading. At first I thought the new song was just an April Fools' joke, but it's for real! I'm jammin' away on it right now! Best of all, it's free!