August 22, 2010

HTTP Server

In my previous post, I mentioned the goal of separating protocols from the client object. As an experiment, I started writing a HTTP service using the new code. This would let me run a light web server as a service in conjunction with something like WebSockets on the same, single-threaded server.

You definitely don't want to waste time serving big files in a single threaded application because it blocks but for little things like a login page it could be handy. Plus, I've written web based apps for years but never fully understood what was happening under the hood and here was a chance to find out.

The actual HTTP protocol is pretty simple. It only has eleven commands (or methods); HEAD, GET, POST, PUT, DELETE, TRACE, OPTIONS, CONNECT, and PATCH. Currently, I only support one, GET (which is easily 99.9% of the web). All other methods are responded to with a status code 501 Not Implemented.

Here's my index.html rendered on Chromium:

This actually represents three separate HTTP GET requests; the index.html file, the skull image, and the tiny skull favicon shown on the tab.

The skull is a compressed SVG, to get Chromium to display it I had to add Content-encoding parameter to my HTTP Response;


I've started the total re-write of my asynchronous network library Miniboa. The new code, in addition to being cleaner, is different in several ways.

What's New
  • No longer Telnet-centric; my aim is to keep connections generic and wrap them in protocol handlers.
  • Services; you can now run multiple services under a single server. Each service listens on a different port and can have unique protocols and events.
  • Added hooks for coroutine-based, character-at-a-time protocol handlers.
  • No longer line-based; the new API has get_ch() to get one character or get_input() to retrieve the entire input buffer. This is support users that want to write features like text editors and command completion.
  • New event; on_input() that is called when the data is received from the client.
  • When run under Linux, it uses the Epoll API introduced in Linux kernel 2.5.44. All other platforms user Select().
I kept the server polled from within the user's game loop. I know that event and callback driven designs are more efficient, but I really like the approachability of that pattern. It can be hard to think in terms of callbacks.

May 2, 2010

The Plague that struck Azeroth

I ran across some old screen-shots from an event that occurred in the MMO World of Warcraft on September, 13 2005 after the opening of a dungeon called Zul'Gurub. One of the encounters involved a damage-over-time spell called Corrupted Blood that spread from the victim to anyone nearby. It was designed to be contained within the instance but players found that they could dismiss infected pets and later recall them with the effect still running, and massively contagious, regardless of where they went.

This is what the main Horde city of Orgrimmar looked like;

Like Morpheus said about The Matrix, this world was built on rules and it was fascinating to see those rules play out in completely unintended ways. Low level players running into town would die in a tick or two. Anyone high enough to survive the intitial damage only helped to spread it farther, including NPCs.

April 19, 2010

Not Actually Dead

To the 1.2 people who actually follow this blog -- I've taken a new job as a county IT director which I'm enjoying but there's sooooo much stuff to unravel and mentally digest that it may be some time before my head is clear enough to code on my free time again.

March 21, 2010

Thinking about Miniboa 2.0

In the early 90s, my hobby was running a computer bulletin board system (BBS) in Virginia called Grendel's Place. I started with a free package called RBBS and then bought a license for the more spiffy TriBBS. Like the joke said; a sysop is someone who spends thousands of dollars to let other people use his computer.

I spent a lot of time writing utilities and experimental games with Borland's Turbo C compiler. Man, I loved that program. Eventually, I bought their C++ compiler for Windows but the combination of the awful C++ language and Microsoft's hideous Foundation Class Library killed my hobby programming for years.

BBSes had a wealth of Door games, which were external programs that the BBS passed control of the serial port to while the user played them. Some of these were, like Tradewar 2000 and Solar Realms Elite were incredibly compelling. Since the typical BBS had only a single phone line, these games tended to be turn-based with only so many moves allotted to each player per day.

Having started to work with Telnet, I immediately lamented the absence of two features from my days coding for a BBS. The first was IBM's extended character set that let you draw lines and blocks. Check out this screen from Global War:

The second thing I missed was ANSI.SYS, a DOS driver from Microsoft that controlled character color and cursor placement. Most BBS clients either used ANSI.SYS directly or as the standard against their own implementation. You could safely make assumptions about the capability of your visitor's terminal and do neat things.

Telnet clients, on the other hand, are horrible.

They are horrible in two ways -- actually implementing the Telnet protocol and supporting those same ANSI control codes. Feature support across the various clients is so awful and quirky that it's impossible to rely on anything but the shallowest of implementations -- even little things like setting the background color.

While writing Miniboa, I scaled back many of my aspirations and focused on two areas; things that mostly worked regardless of client and things that were harmless if they failed. If your client supported NAWS, you got 'on-the-fly' terminal sizing. If not, you got 80 columns. This is one of the reasons Miniboa is line-based (as in, it gives you an entire line of client input rather than keystrokes). It worked by default on every client I tested.

Another reason I went line-based was I'm toying with the idea of writing a Jabber/XMPP server to support play from instant messaging and wanted to keep my interface to the clients duck-typed; client.get_cmd() and client.send().

Whew, that's too much background eh?

OK, OK -- Miniboa 2.0. I had a request to add character support to Miniboa and realized there was no way to do it without some really ugly hacks and the right thing(tm) was to decouple, decouple, decouple. Currently, Telnet is handled via a do-it-all class that stores the Socket object as a member property and performs a lot of state juggling. I think I want to switch things over to nested coroutines. Where the inner ones perform progressively lower level IO.

David Beazley, author of the excellent Python Essential Reference, gave a lecture entitled A Curious Course on Coroutines and Concurrency at PyCon 2009. You can watch a video of it here:

Part 1
Part 2
Part 3

Another development comes from Confuto and Stendec who have kindly donated code for implementing WebSocket and Flash servers with Miniboa. This is pretty exciting because of the potential for rich clients that both represent. Plus if I do write a Jabber server, it would benefit from the decoupling process as well.

February 16, 2010

Character Skills

Now that I've covered Stats, I wanted to talk about Character Skills but let's start with a clear definition since skill has multiple meanings in the context of computer RPGs;
In Bogboa, Skill represents the actor's measure of talent when it comes to performing abilities within given categories.
So a player attempting to disarm a trap might rely on an comprehend_traps skill to provide the base number for the success check. Building on the stat post, that base number might be bumped up a few percent if they possess lots of cunning.

Skill Modifiers

The idea is to maintain a list of skill modifiers that vary per guild (character class). The rogue guild might have a comprehend_traps modifier of 1.0 while the warrior guild has .01. This results in a warrior having 1/10th of the trap removing potential of a rogue. If we set it to zero, warriors will always fail regardless of level and cunning.

You can imagine these modifiers like the sliders on an equalizer as we use them to control the relative power of each guild. Creating hybrid classes would be a simple matter of setting a ratio for primary skill modifiers -- a paladin might have a healing skill of .65 and a combat skill of .85 as compared to a cleric's (1.0, 0.5) and a warrior's (0.0, 1.0). If a guild does not list a given skill we treat the modifier as zero.

The potential power of a skill modifier is immense and, like a thermostat that goes to a billion, should be used great care. It's my intention to stick very close to 1.0 as the maximum modifier value*. This keeps power ratios cleanly defined across guilds and relative to mob level. Let's contrive an example to illustrate why this is probably a good idea. We have a designer that wants to create a Cavalier Guild and decides they're like warriors but better with swords. Assuming the warrior had 1.0 sword skill modifier, he gives his cavalier 1.2. Now they are 20% deadlier than mobs their level -- a level 40 cavalier attacks like he's 8 levels above his opponent. This makes them too easy, so our designer bumps up NPCs to compensate and now every other player guild has a rougher time of it.

The designer could try to balance this offense power with lower defense but it's tricky. Sustained damage output sets the rate of experience gain for the character. Unless you implement some form of downtime, almost dying is meaningless.

* An exception to this would be Wizard Guild defense and health modifiers, which will be something like 100000.0 to make them immortal. Conversely, their offensive skills should be 0.0 to discourage abuse.

Skill Caps

Skills are capped at;
actor_level * 10 * guild_modifier
Not much needs explained here; a level 20 warrior with a 1.0 combat modifier will have a maximum combat skill of 200.

Raising Skills

There are many ways to handle this but I've always been partial to raising skills through using them. Most of the skill rating functions (see below) have an optional tutor argument. If this is set to True and the player's current skill is not maxed, he gets a chance to increase his current skill during the check. The chance becomes progressively lower as the player closes in on his current max skill.

Rating Skills

This is the meat and potatoes of the skill system. Using a group of functions, we rate an actor's skill by looking up the current value and applying any stat related bonuses or penalties. The result is a rating: a numeric value used for comparisons and success/failure checks. Some ratings are derived by combining other ratings. For example, a player's hit rating is the average of his melee combat rating and the specific weapon rating for the type of weapon he's carrying. This is to accommodate scenarios like a level 20 warrior who has never touched a mace suddenly picking one up. Even though his skill in blunt weapons is zero, his melee combat skill provides some chance of landing a hit.

It's a small shift it perspective, but thinking in terms of skill ratings was something I found helpful.


We've already covered that we'd use skill ratings for deciding success but not how those rolls will be calculated. Here's an example for striking with the player's primary weapon. Basically, we pick a random number in the range of 0 to (defender's defense rating + attacker's hit rating ). If the number is higher than the defender's defense, it is a hit. You can see how either value grows, probability will shift accordingly but the attack always has a chance to hit.

def primary_weapon_test(src, tar):
Test for a hit using Source's primary melee weapon.
hit = skill.primary_hit_rating(src)
defend = skill.defense_rating(tar)
return bool(random.randrange(defend + hit) > defend)

One of my goals is to avoid hard-coding rules as much as possible. So rather than telling a magician he cannot use plate armor, I'd rather use his guild's skill modifiers to provide more benefit and AC from a robe but, if he really wants to, he can dress up in heavy plate. Same thing for weapons.

Similarly, learned abilities wont be guild specific but will require certain skill values to obtain. A potent healing spell might be available to a cleric at level 10, a druid at 20, and a templar at 40 depending on their healing_magic modifiers.

January 30, 2010

Character Stats

I have finally settled on a set of character stats to use for the game. I've been torn over using a generic (some may say tired) but immediately recognizable D&D inspired set like STR, DEX, INT, WIS, CON, CHR versus making up completely different ones with the risk they'd seem needlessly exotic and labored.

I wanted to split them evenly between physical and mental attributes with as little overlap as possible. For example, I've always found dexterity and agility awkwardly close to each other.

  • Brawn - strength, heartiness, and balance.
  • Vigor - stamina, pain tolerance, and body recuperation rate.
  • Precision - dexterity, grace, accuracy, and physical reaction time.

Mental Stats
  • Knowledge - memory, education, and life experience (street smarts).
  • Faith - confidence, willpower, zealotry, and mental recuperation rate.
  • Cunning - intelligence, ingenuity, and mental reaction time.
An even split meant either 4, 6, or 8 stats -- 4 was too few and 8 was too many. We want different character guilds/classes to prize different stat combinations without things getting out of hand. Let's also try to make mental attributes appealing to melee classes. Monks should want faith and rogues cunning. Faith might seem like an odd word but it will fit in the setting I'm planning. Lastly, let's drop the wisdom versus intelligence caster meme.


For every 10 points in a given stat, the character will receive a 1% bonus towards related skills. In the case of negative stats, characters will suffer a 1% penalty per negative stat point.

January 27, 2010

Key-Value Data Loading

I wanted to follow up with an example of loading an avatar's persistent dictionaries from a key-value store in the SQLite database;

def load_account(client, name, uuid):
load a previously created account.
client.send('\nWelcome back, %s. Your last visit was %s.\n\n' %
(name, last_on(name)))
## Create an in-game Avatar from stored data
avatar = Avatar(client)
## Read from database
avatar.profile = fetch_kv_dict(uuid, 'profile')
avatar.resources = fetch_kv_dict(uuid, 'resources')
avatar.skills = fetch_kv_dict(uuid, 'skills')
avatar.worn = fetch_kv_dict(uuid, 'worn')
avatar.carried = fetch_kv_dict(uuid, 'carried')
avatar.abilities.update(fetch_kv_set(uuid, 'abilities'))

Just in case it wasn't clear. You may notice an absence of a 'stats' dictionary. Character stats will be computed based on race + gear, so aren't saved.


I've had a few questions about how I plan to handle persistence, as in saving the states of game objects in case the server is restarted. The quick answer is I'm not -- with the exception of player character data. Bogboa is using two forms of data storage; YAML files and an SQLite database. I'll discuss each.

YAML Files

Currently, YAML is the only third-party library used in Bogboa. I didn't want to use any really, but YAML is such a human friendly format that I went with it.

On startup, the world is created from one or more directory trees filled with YAML files that define individual rooms, items, races, etc. This is a one-way conversion where each YAML file is parsed into a Python dictionary that is used to set attributes of game classes. One of my long-term goals is to support reading these YAML trees from zip files so that designers could release adventure modules that contain complete dungeons with new monsters and items that load seamlessly into existing game worlds. This is one of the reasons I'm using UUID's for keys, since they don't require multiple developers to coordinate or negotiate around potential collisions like vnums do.

So restarting the server cleans every floor of dropped loot and every dead NPC is alive again. This appeals to me because it's simple and troubleshooting scripts is easier from known states.

SQLite Database

Starting in Python 2.5, SQLite3 is part of the standard library (and a kick-ass little SQL database) which makes using it a non-brainer.

My early tables were pretty vanilla with columns; (name, race, level, guild, stat1, stat2, etc) and I was always changing them. Also, I was using a lot of sub-tables to hold information like skills, inventory, and character flags. These sub-tables all had nearly identical structure; (character_uuid, skill_name) and (character_uuid, inventory_slot).

At the same time, I was making structural changes to how I split player from their avatars (which is why the codebase is currently b0rked). I knew that the closer I handled avatars like NPCs the easier things would be down the road. Moving all the various traits into dictionaries was helpful and this transition to key-value thinking migrated to how I stored character data as well.

What I wanted was a way to load whole dictionaries at a time but update individual key-value pairs on the fly. When you log in, your character's profile dictionary is retrieved containing (name, race, guild, level, ...) and when you gain a level, your characters (profile + level) record is incremented by one and if the server crashes a moment later you, hopefully, didn't lose it.

One of the nuisances of SQL is you often have to do a SELECT before you can save data because sometimes you cannot be sure if you need to use the INSERT or the UPDATE command. SQLite supports a non-standard but convenient variation of this called INSERT OR UPDATE. The catch is the key has to be unique.

What I did was make the key = (Character UUID + Some Catergory Name + Key Name);

uuid TEXT,
category TEXT,
key TEXT,
value TEXT,
PRIMARY KEY(uuid, category, key)

In the above example, the category name would be 'profile'.

Reading all the rows of a category would look like;

SELECT key, value
FROM key_value
WHERE uuid = ? AND category = ?;

And saving an individual key-value would look like;

INSERT OR REPLACE INTO key_value (uuid, category, key, value)
VALUES (?, ?, ?, ?);

January 16, 2010

Everything I need to know I learned from D&D

Clever presentation by a guy who compares software development to playing Dungeons and Dragons:

All I Need To Know About Life I learned From Dungeons and Dragons. An IgniteOKC Talk. from Chad Henderson on Vimeo.

January 7, 2010

MUD Game Programming

I believe the only book ever written as howto for creating a MUD was MUD Game Programming by Ron Pelton. Originally published in late 2003, it's been out of print for a while and I finally managed to snag a used copy for a reasonable price. It even came with an unopened CD.

I don't expect to glean a great deal from it having already written an asynchronous server, plus Pelton is using C++ where I'm doing everything in Python. Still, it seems like a quality book with a lot of example code. Books on game programming tend to suffer a poor signal to noise ratio. There's a ton of dreck out there, especially garbage written by industry insiders who have no clue about programming but feel like they can help you with their experience interviewing other non-programmers. Ugh. Even worse are the ones aimed at teens -- clearly targeting the well-intentioned but useless gift from Mom category.

Pardon the rant.

Anyways, I've read the first couple chapters on networking and socket programming and Pelton is delivering the goods. He is specific and writes in an friendly, engaging manner. He gets extra points for covering Linux in addition to Windows. Recommended (if you can find it).

January 2, 2010

Work Around

--start of edit

Well, it turned out that the approach listed below does not find methods from base classes. I revised it to use __getattribute__() instead which returns a bound instance of the class but that only lives for the scope of the cmd_driver() method:

def cmd_driver(self):
## call the driver method for current state
---end of edit

My last post detailed a garbage collection issue I was having when an instance of the User class referenced itself using a bound variable. I got around it changing my state property to a string variable holding the name of the method I wanted to call and then calling said method via some class introspection;
def cmd_driver(self):
## call the driver method for current state via class introspection

That kind of introspection always feels like duct tape.

I should point out that Python's garbage collector probably would have deleted those User instances eventually and that I could have manually ended the connection by calling socket.close(). I prefer the immediate garbage collection closure because it's an indicator that everything is running smoothly.