Don't Repeat Yourself

Something that I've been working with the Planetside 2 team lately has been a more strategic direction in how we write maintainable code for the future. Part of my direction for this has been adherence to the D.R.Y. principle--Don't Repeat Yourself. Namely, instead of distributing authority out by the expression of Interfaces (and writing a lot of repetitive code), we are architecting unambiguously authoritative systems that can handle all functionality in a common way.

One of our most recent realizations of this principle is in our new UI data-source layer.

Previously, Planetside 2 (and previous Forgelight Engine ancestors) had a concept of a DataSource interface. This was a simple interface that allowed sub-systems (such as Skills or Items) to express data in a string-based row/column format to the UI layer (Planetside 2 uses Scaleform GFx, a Flash-based interpreter/renderer). The Actionscript 3 UI code could find a DataSource by name and query it in the ways expressed by the interface (it could also register as a listener to determine if anything changed).

class IUIDataSourceTable {
public:
    virtual int GetData(int row, int column, String& output) = 0;
    virtual int GetRowCount() = 0;
    virtual int GetColumnCount() = 0;
    virtual String GetColumnName(int column) = 0;
    virtual void AddListener(IUIListener*) = 0;
    virtual void RemoveListener(IUIListener*) = 0;
    virtual void NotifyChanges() = 0;
};

Seems great so far, right?

Because this is an interface, each system is responsible for surfacing internal data to the UI through the GetData call. But each system can implement it completely different. As long as 'output' receives the value of a given cell, where that data comes from isn't important to the UI layer. Also, detecting when that data changed (and notifying listeners) became the responsibility of each system.

Okay... So?

Well, now let's say that you want to do more complex operations with that data, such as sorting, filtering or joining with another IUIDataSourceTable. Sure we could add additional functions to the IUIDataSourceTable, but as our codebase grows, we're faced with a quandary: how do shrinking development teams maintain ever-growing codebases??

Let's look at it a different way. Say I wanted to add a Filter function to my IUIDataSourceTable. Planetside 2 has on the order of about 130 implementations of this interface (give or take) in very disparate systems such as Character Select, Skills, Items, Marketplace, Social, Experience, etc. If I want to add another function to my interface, I'm looking at actually writing 130 functions and doing a lot of research into those systems to find out how they actually surface the data.

This is the problem with distributed authority: it exponentially extends a simple change. Work-arounds in the past involving just two things that we always want to do--filtering and sorting--required either a C++ engineer to add to a specific DataSource implementation, or a UI engineer to gather everything and sort it in Actionscript 3. This is not a very productive or sustainable model and it's ridiculously time intensive (both on the CPU and in development time).

Ok, now let's adopt a don't-repeat-yourself strategy.

Instead of declaring an Interface that each system must adhere to, let's design a system that is unambiguously authoritative in terms of UI data. We want one system that we can tune and add functionality to. And since we're talking about UI datasources, can you think of anything that is designed to store row/column data and has sorting and filtering capabilities? Maybe... like... a database?! Turns out there is such a beast that we can use as the core of our new authoritative UI data system: sqlite3! This gives us a light-weight, in-memory SQL database that we can use to store our data and slice it any way we need! And better yet, browsers (and even mobile devices) have been using it for years!

After getting an engineer excited about this prospect and putting her on the project, she turned out an awesome system called uiDB that exposed game data to the Flash-based UI through SQL and tables. This has some really cool features:

• Adding new functionality is trivial. We just added some very broad functionality in less than a day because we changed one authoritative system instead of 130 interface implementations.
• sqlite has a callback mechanism, so we can use Actionscript 3's event mechanism to notify the UI when data changes--automatically. C++ just updates the game data in the uiDB system as it changes, and the authoritative uiDB system notifies any listeners. No extra code required.
• The AS3 code can create a view to slice, sort, filter, or join data however it wants and can still get notified when any of that data changes. Without having to write any additional C++ code.
• The AS3 code can also perform any SQL query on the in-memory UI database whenever it wants. Need data one time? No problem. Need to display data and refresh when it changes? Equally trivial.
• We wrote a DataProvider implementation for uiDB tables and views. This is the standard DataProvider that Flash widgets use, so our UI engineers don't have to write any code to handle change events or anything--the DataProvider can do all of that and the UI just updates automatically. Talk about code reduction! 

Another surprise out of changing from the interface to authoritative model is that our UI data performance increased, even though we were now using a SQL-interpreting database! This is because of a few different reasons:

• In the Interface model, each system retrieved its data through different means. Some could be woefully inefficient.
• sqlite3 is a well-tested, highly optimized, tightly-packed, small C library. It has a development team that cares only about its performance and functionality in a variety of settings. As such, we are gaining the wealth of knowledge and experience of another team focusing intently on one piece of technology, allowing us to focus on making a game. (This could branch into another discussion about utilizing as much third-party technology as makes sense).
• By consolidating authority in one system instead of spreading it out, we had one target to focus on for performance testing and optimization.

We are continuing to provide additional functionality through this system to our UI engineers in order to make their lives easier (and we're doing this across the board for all disciplines). This is looking forward to the future of Daybreak Games and the Forgelight Engine as much as the future of Planetside2.

All in all, work smarter, save time across the board, and don't repeat yourself!