Deconstructing the XBox security system

This is a great video that talks about the original XBox, its security shortcomings, and how hackers eventually got it to run Linux. Over an hour, but very entertaining for geeks like myself.

Getting Handsy

Grandma and Grandpa are in town, so somehow, crazy videos with my son turn up. Here's the latest:

Humanized Software

A friend showed me an interesting piece of software today over at Humanized. It's called Enso, and has been described by some as "The Command Line Comeback".

It's a very intriguing idea. Highlight a block of text that says "132/4", hold down Caps Lock and type "calculator". Presto-changeo, the highlighted text gets replaced with 33. There are several other commands to open programs, browse the Internet, or control windows. Head on over there and check out the demo!

The install was seamless. Click a web control and it's as easy as installing the Google Toolbar. Unfortunately, as a technogeek, I didn't want it installed where it automatically ended up (and I didn't see any custom install options). Obviously a very minor note.

The other problem that I had with it is that it's overly resource-hungry. This might be due to the fact that it's almost entirely written in Python, but for as simple as it seems, I don't think it needs 30MB of memory and 17 threads.

The possibilities for this piece of software are abundant. You can 'teach' it new commands and future versions are supposed to be extensible with Python. As a programmer, I'd love to have things like conversions between hex/decimal/binary or even a calculator that does bitwise operations.

This is definitely something I'll be keeping my eye on.

Don't buy an iPod

For a while now I've been having a consistent problem with my 3rd generation iPod: the last 15-20% of some songs get cut off. Unfortunately, there seems to be nothing that Apple is doing to address the problem (or even acknowledge what it is). Several people have reported this problem in various forms. Now, I only use iTunes to import my songs, and as others mention, they play fine in iTunes. And the kicker: this is even happening with songs that I purchased from the iTunes store!

It's definitely a problem with the iPod: the progress bar gets within 15-20% of the end of the song, and all of a sudden, the iPod switches to the next song. If I manually set the progress bar into the part that gets cut off, it immediately skips to the next song without playing anything. It's not every song, but it's always the same songs. It seems like every so often a new song starts exhibiting this problem.

If Apple would address the problem, that would be great. Any hope of getting that to happen? Probably not.

My wife's 1st generation iPod mini also has a problem: In less than 3 years (it's newer than my 3rd generation iPod) the battery won't hold a charge for even 30 minutes. Is it in warranty? Not anymore! Is the battery user-serviceable? Never! Last time I checked, Apple wants $100 to swap out the battery.

By now you should know what I'm going to say: my next MP3 player selection will NOT be of the Apple variety.

Update, August 2007: I decided to go with a Zune; read the whole post

Hackalicious

The more I beta-test/play/develop MMOs, the more I realize that developers still cut corners when it comes to security. If you're working on an MMO, especially before launch, it's very important to take a look at your game through the eyes of a hacker.

I've worked on hardening some games to the hacker types (and let's admit it: I was one myself once), so here's my short primer on MMOs and hacking:

Don't ship your game with the .PDB file! Obvious? Maybe, but not to some.

Turn off RTTI. Why are you using dynamic_cast anyways? It supports crazy things like casting horizontally in multiple inheritance. Not to mention that it's slow. Hackers love RTTI though, as it causes the executable to include mangled names of pretty much everything. If you need a dynamic typesafe casting solution, search the web or get a book. Or use virtual functions, though it's ugly as sin.

Take a look at your client in a hex editor, especially strings. You might be surprised at what you find. Those assert lines can actually contain a ton of useful information, like source file names and line numbers, not to mention C++ code itself. Or you might just find that you're including the class names of every message type in the game. Yes, it's been done.

Beware large areas of [un]initialized memory. Hackers love it when they already have space allocated in your program that they are free to write their code into. UO had a bunch of these before I cleaned them up, the largest of which being a 640x480 graphics buffer that was never fully used. Use the heap and make it lazy allocate. If possible, delete it as soon as you're done with it.

Change message IDs frequently. If you can do it right (and unpredictably), this can work well to foil those who like to read the network traffic.

Log stuff to the server, but don't take action on it right away. You don't want to do anything that lets a hacker know that you're onto them. Check with your legal department to find out what you can report, but it generally shouldn't be bad to do something like CRC the client in memory and send back a pass/fail response to the server. Or report if they're using a debugger, though this can be challenging. Unfortunately, uploading their modifications for review is anathema.

Watch your encryption keys. Yes, EQ2 actually held the encryption key in plaintext in a static buffer after negotiating it. It's fixed now, but hackers rejoiced!

Verify user data. UO has an immediate request/response targeting system. The server sends a request to the client with an ID which the client returns unmolested to the server. But what happens when a hacker figures out how to make use of changing that ID? You get the insurance bug.

Beware of unreleased/undocumented content. Again with UO, there were quite a few "gumps" that were in half-implemented features that still got pushed to live servers. A hacker figured out how to make use of one of these and ended up teleporting half the players on the server to a single X,Y location. Use source control branches or some configuration system to make sure this untested stuff can't be used. Period.

Don't make assumptions. You've heard the clichés so i'll spare you. Just because a normal user can't see something (like a UI window or network message format) doesn't mean they can't exploit it.

Movement. A notoriously bad topic for MMOs, especially 3D ones, there should really be more seriousness here, at least in the way of logging. Servers can at least sanity check movement without having to run simulations of every client.

Buffer overruns/remote code execution. Yes, it can happen to you. Watch how you're using the string functions, especially the varargs ones (sprintf, sscanf, etc) and wide functions. The discussion on this one topic alone could be (and has been) a lecture in itself, so I'll just say: be aware of it.

You can never fully stop people from hacking on your game, but you can make it difficult (and you should). Then there's the other side of the coin: banning the people you catch hacking. Usually this is financially advantageous: if a hacker is turning people off of your game, it's probably not just one.

Unfortunately, maintaining hack resistance and detection can be a full-time job, especially if your game is well established and there are many hack programs out for it. In such cases, it might be better to look at using something like PunkBuster.

This is definitely an area of MMOs that should be collaborated on, since all MMO developers don't want the hackers :)

Look what Keating found...

Good ol' MrTact conjured up a caricature of me over on his blog.

Needless to say, I was amused :)

Remember Showbiz Pizza?

My old boss on UO, Anthony "SunSword" Castoro was apparently a child actor. Who knew? You can see a quality 80's Showbiz Pizza commercial and read all about it here.

Anthony has apparently just moved back to Austin to start Heatwave Interactive.

Fun with C++ template specialization

Today I had some fun with C++ template specialization.

The EQ2 codebase doesn't use std::vector anymore in an effort to reduce compile and link times. Instead, we have our own templated array type that is, for all intents and purposes, equivalent to std::vector but without some of the template overhead for allocators and such. Call it ArrayType< T >.

Now, ArrayType relies upon a static traits structure to do certain things for it, such as construct a range of T objects or copy T objects. Template specialization is used, especially for built-in types such as int or float, to perform the necessary operation in the most efficient way. For a built-in type, a copy could typically be implemented as a memcpy() or a memmove(). That wouldn't work very well for most custom classes.

STL libraries such as STLport use complicated type traits to answer questions like is_POD_type. Ours is much simpler and specialized to remove that template overhead.

In any case, back to my use of specialization. Consider what happens when you do something like this:

std::vector< std::vector< int > > vec;
vec.push_back( std::vector< int >( 1 ) );
vec.push_back( std::vector< int >( 2 ) );

vec.insert( vec.begin() + 1, std::vector< int >( 3 ) );

If you have vectors of vectors, each reallocate of the underlying data structure is going to cause the contained vectors to be copied. That's potentially a LOT of allocations. Likewise, the insert is going to move all of the elements including and after the insert position to make room for the new element. More allocations. Theoretically, there is a way to overcome this. Vectors (and most other STL containers) have a nifty little function called swap which swaps the contents of two same-type containers in O(1). To handle the insert, all you'd need to do is construct a default object (or copy-construct the new object) at the end and swap with previous starting at the end. The default object ends up right where you want your inserted element to go, so you can use the assignment operator to set it to the new element. No deep copies of all of the vectors, just nice-and-tidy memory management.

The rub is that there doesn't seem to be a tidy, generic way to do this with the STL. If you know how the traits work for STLport, you might be able to extend it to get what you want. That's far from a generic solution. This is where I get to brag about our ArrayType. All I had to do to make this work the way it should was define the swap semantics in a partially specialized ArrayType_traits:

template< typename U >
struct ArrayType_traits {
    ...
    static void rev_copy( VeArray<U>* dest, VeArray<U>* src, unsigned n )
        { dest += n, src += n;
          while ( n-- ) { (*--dest).swap( *--src ); } }
    ...
};

Neener, neener.

LEGO + MMO = LEGMMO?

Ah Legos. I never grew up. I still build all the Legos I can find. So, an MMO built with Legos? We'll have to wait and see how this turns out, but i've got my fingers crossed on this one.

The start of a legacy or just another waste of time?

Well, after several unsuccessful attempts at starting a blog, hopefully this one will stick. I imagine it will contain thoughts on a variety of topics.

On the game programming front, my good friend Tim "MrTact" Keating just left Electronic Arts to work for a startup in Austin. As Palpatine said in Episode 1, "We shall be watching your career with great interest."

On the topic of family, here are a few videos of my 3-month-old daughter, who just discovered that she could laugh. Enjoy!