The Architecture of
Open Source Applications
Amy Brown and Greg Wilson (eds.)
License / Buy / News / Contribute / FAQ
A vast star empire encompasses a hundred worlds, stretching a thousand parsecs across space. Unlike some other areas of the galaxy, few warriors live here; this is an intellectual people, with a rich cultural and academic tradition. Their magnificent planets, built turn by turn around great universities of science and technology, are a beacon of light to all in this age of peace and prosperity. Starships arrive from the vast reaches of the quadrant and beyond, bearing the foremost researchers from far and wide. They come to contribute their skills to the most ambitious project ever attempted by sentient beings: the development of a decentralized computer network to connect the entire galaxy, with all its various languages, cultures, and systems of law.
Thousand Parsec is more than a video game: it is a framework, with a complete toolkit for building multiplayer, turn-based space empire strategy games. Its generic game protocol allows diverse implementations of client, server, and AI software, as well as a vast array of possible games. Though its size has made planning and execution challenging, forcing contributors to walk a thin line between excessively vertical and excessively horizontal development, it also makes it a rather interesting specimen when discussing the architecture of open source applications.
The journalist's label for the genre Thousand Parsec games inhabit is "4X"—shorthand for "explore, expand, exploit, and exterminate," the modus operandi of the player controlling an empire1. Typically in the 4X genre of games, players will scout to reveal the map (explore), create new settlements or extend the influence of existing ones (expand), gather and use resources in areas they control (exploit), and attack and eliminate rival players (exterminate). The emphasis on economic and technological development, micromanagement, and variety of routes to supremacy yield a depth and complexity of gameplay unparalleled within the greater strategy genre.
From a player's perspective, three main components are involved in a game of Thousand Parsec. First, there is the client: this is the application through which the player interacts with the universe. This connects to a server over the network—communicating using the all-important protocol—to which other players' (or, in some cases, artificial intelligence) clients are also connected. The server stores the entire game state, updating clients at the start of each turn. Players can then perform various actions and communicate them back to the server, which computes the resulting state for the next turn. The nature of the actions a player may perform is dictated by a ruleset: this in essence defines the game being played, implemented and enforced on the server side, and actualized for the player by any supporting client.
Because of the diversity of possible games, and the complexity of the architecture required to support this diversity, Thousand Parsec is an exciting project both for gamers and for developers. We hope that even the serious coder with little interest in the anatomy of game frameworks might find value in the underlying mechanics of client-server communication, dynamic configuration, metadata handling, and layered implementation, all of which have grown rather organically toward good design over the years in quintessential open source style.
At its core, Thousand Parsec is primarily a set of standard specifications for a game protocol and other related functionality. This chapter discusses the framework mostly from this abstract viewpoint, but in many cases it is much more enlightening to refer to actual implementations. To this end, the authors have chosen the "flagship" implementations of each major component for concrete discussion.
The case model client is
tpclient-pywx, a relatively mature
wxPython-based client which at present supports the largest set of
features and the latest game protocol version. This is supported by
libtpclient-py, a Python client helper library providing
caching and other functionality, and
libtpproto-py, a Python
library which implements the latest version of the Thousand Parsec
protocol. For the server,
tpserver-cpp, the mature C++
implementation supporting the latest features and protocol version, is
the specimen. This server sports numerous rulesets, among which the
Missile and Torpedo Wars milestone ruleset is exemplary for
making the most extensive use of features and for being a
"traditional" 4X space game.
In order to properly introduce the things that make up a Thousand Parsec universe, it makes sense first to give a quick overview of a game. For this, we'll examine the Missile and Torpedo Wars ruleset, the project's second milestone ruleset, which makes use of most of the major features in the current mainline version of the Thousand Parsec protocol. Some terminology will be used here which will not yet be familiar; the remainder of this section will elucidate it so that the pieces all fall into place.
Missile and Torpedo Wars is an advanced ruleset in that it implements all of the methods available in the Thousand Parsec framework. At the time of writing, it is the only ruleset to do so, and it is being quickly expanded to become a more complete and entertaining game.
Upon establishing a connection to a Thousand Parsec server, the client probes the server for a list of game entities and proceeds to download the entire catalog. This cataloger includes all of the objects, boards, messages, categories, designs, components, properties, players, and resources that make up the state of the game, all of which are covered in detail in this section. While this may seem like a lot for the client to digest at the beginning of the game—and also at the end of each turn—this information is absolutely vital for the game. Once this information has been downloaded, which generally takes on the order of a few seconds, the client now has everything it needs to plot the information onto its representation of the game universe.
When first connected to the server, a random planet is generated and assigned as the new player's "home planet", and two fleets are automatically created there. Each fleet consists of two default Scout designs, consisting of a Scout Hull with an Alpha Missile Tube. Since there is no Explosive component added, this default fleet is not yet capable of fleet-to-fleet or fleet-to-planet combat; it is, in fact, a sitting duck.
At this point, it is important for a player to begin equipping fleets with weaponry. This is achieved by creating a weapon design using a Build Weapon order, and then loading the finished product onto the target fleet through a Load Armament order. The Build Weapon order converts a planet's resources—of which each planet has amounts and proportions assigned by a random distribution—into a finished product: an explosive warhead which is planted on the creating planet's surface. The Load Armament order then transfers this completed weapon onto a waiting fleet.
Once the easily accessible surface resources of a planet are used up, it is important to obtain more through mining. Resources come in two other states: mineable and inaccessible. Using a Mine order on a planet, mineable resources may be converted over time into surface resources, which can then be used for building.
In a Thousand Parsec universe, every physical thing is an object. In fact, the universe itself is also an object. This design allows for a virtually unlimited set of elements in a game, while remaining simple for rulesets which require only a few types of objects. On top of the addition of new object types, each object can store some of its own specific information that can be sent and used via the Thousand Parsec protocol. Five basic built-in object types are currently provided by default: Universe, Galaxy, Star System, Planet, and Fleet.
The Universe is the top-level object in a Thousand Parsec game, and it is always accessible to all players. While the Universe object does not actually exert much control over the game, it does store one vastly important piece of information: the current turn number. Also known as the "year" in Thousand Parsec parlance, the turn number, naturally, increments after the completion of each turn. It is stored in an unsigned 32-bit integer, allowing for games to run until year 4,294,967,295. While not impossible in theory, the authors have not, to date, seen a game progress this far.
A Galaxy is a container for a number of proximate objects—Star Systems, Planets and Fleets—and provides no additional information. A large number of Galaxies may exist in a game, each hosting a subsection of the Universe.
Like the previous two objects, a Star System is primarily a container for lower-level objects. However, the Star System object is the first tier of object which is represented graphically by the client. These objects may contain Planets and Fleets (at least temporarily).
A Planet is a large celestial body which may be inhabited and provide resource mines, production facilities, ground-based armaments, and more. The Planet is the first tier of object which can be owned by a player; ownership of a Planet is an accomplishment not to be taken lightly, and not owning any planets is a typical condition for rulesets to proclaim a player's defeat. The Planet object has a relatively large amount of stored data, accounting for the following:
The built-in objects described above provide a good basis for many rulesets following the traditional 4X space game formula. Naturally, in keeping with good software engineering principles, object classes can be extended within rulesets. A ruleset designer thus has the ability to create new object types or store additional information in the existing object types as required by the ruleset, allowing for virtually unlimited extensibility in terms of the available physical objects in the game.
Defined by each ruleset, orders can be attached to both Fleet and Planet objects. While the core server does not ship with any default order types, these are an essential part of even the most basic game. Depending on the nature of the ruleset, orders may be used to accomplish almost any task. In the spirit of the 4X genre, there are a few standard orders which are implemented in most rulesets: these are the Move, Intercept, Build, Colonize, Mine, and Attack orders.
In order to fulfill the first imperative (explore) of 4X, one needs to be able to move about the map of the universe. This is typically achieved via a Move order appended to a Fleet object. In the flexible and extensible spirit of the Thousand Parsec framework, Move orders can be implemented differently depending on the nature of the ruleset. In Minisec and Missile and Torpedo Wars, a Move order typically takes a point in 3D space as a parameter. On the server side, the estimated time of arrival is calculated and the number of required turns is sent back to the client. The Move order also acts as a pseudo-Attack order in rulesets where teamwork is not implemented. For example, moving to a point occupied by an enemy fleet in both Minisec and Missile and Torpedo Wars is almost certain to be followed by a period of intense combat. Some rulesets supporting a Move order parameterize it differently (i.e. not using 3D points). For example, the Risk ruleset only allows single-turn moves to planets which are directly connected by a "wormhole".
Typically appended to Fleet objects, the Intercept order allows an object to meet another (commonly an enemy fleet) within space. This order is similar to Move, but since two objects might be moving in different directions during the execution of a turn, it is impossible to land directly on another fleet simply using spatial coordinates, so a distinct order type is necessary. The Intercept order addresses this issue, and can be used to wipe out an enemy fleet in deep space or fend off an oncoming attack in a moment of crisis.
The Build order helps to fulfill two of the 4X imperatives—expand and exploit. The obvious means of expansion throughout the universe is to build many fleets of ships and move them far and wide. The Build order is typically appended to Planet objects and is often bound to the amount of resources that a planet contains—and how they are exploited. If a player is lucky enough to have a home planet rich in resources, that player could gain an early advantage in the game through building.
Like the Build order, the Colonize order helps fulfill the expand and exploit imperatives. Almost always appended to Fleet objects, the Colonize order allows the player to take over an unclaimed planet. This helps to expand control over planets throughout the universe.
The Mine order embodies the exploit imperative. This order, typically appended to Planet objects and other celestial bodies, allows the player to mine for unused resources not immediately available on the surface. Doing so brings these resources to the surface, allowing them to be used subsequently to build and ultimately expand the player's grip on the universe.
Implemented in some rulesets, the Attack order allows a player to explicitly initiate combat with an enemy Fleet or Planet, fulfilling the final 4X imperative (exterminate). In team-based rulesets, the inclusion of a distinct Attack order (as opposed to simply using Move and Intercept to implicitly attack targets) is important to avoid friendly fire and to coordinate attacks.
Since the Thousand Parsec framework requires ruleset developers to define their own order types, it is possible—even encouraged—for them to think outside the box and create custom orders not found elsewhere. The ability to pack extra data into any object allows developers to do very interesting things with custom order types.
Resources are extra pieces of data that are packed into Objects in the game. Extensively used—particularly by Planet objects—resources allow for easy extension of rulesets. As with many of the design decisions in Thousand Parsec, extensibility was the driving factor in the inclusion of resources.
While resources are typically implemented by the ruleset designer, there is one resource that is in consistent use throughout the framework: the Home Planet resource, which is used to identify a player's home planet.
According to Thousand Parsec best practices, resources are typically used to represent something that can be converted into some type of object. For example, Minisec implements a Ship Parts resource, which is assigned in random quantities to each planet object in the universe. When one of these planets is colonized, you can then convert this Ship Parts resource into actual Fleets using a Build order.
Missile and Torpedo Wars makes perhaps the most extensive use of resources of any ruleset to date. It is the first ruleset where the weapons are of a dynamic nature, meaning that they can be added to a ship from a planet and also removed from a ship and added back to a planet. To account for this, the game creates a resource type for each weapon that is created in the game. This allows ships to identify a weapon type by a resource, and move them freely throughout the universe. Missile and Torpedo Wars also keeps track of factories (the production capability of planets) using a Factories resource tied to each planet.
In Thousand Parsec, both weapons and ships may be composed of various components. These components are combined to form the basis of a Design—a prototype for something which can be built and used within the game. When creating a ruleset, the designer has to make an almost immediate decision: should the ruleset allow dynamic creation of weapon and ship designs, or simply use a predetermined list of designs? On the one hand, a game using pre-packaged designs will be easier to develop and balance, but on the other hand, dynamic creation of designs adds an entirely new level of complexity, challenge, and fun to the game.
User-created designs allow a game to become far more advanced. Since users must strategically design their own ships and their armaments, a stratum of variance is added to the game which can help to mitigate otherwise great advantages that might be conferred on a player based on luck (e.g., of placement) and other aspects of game strategy. These designs are governed by the rules of each component, outlined in the Thousand Parsec Component Language (TPCL, covered later in this chapter), and specific to each ruleset. The upshot is that no additional programming of functionality is necessary on the part of the developer to implement the design of weapons and ships; configuring some simple rules for each component available in the ruleset is sufficient.
Without careful planning and proper balance, the great advantage of using custom designs can become its downfall. In the later stages of a game, an inordinate amount of time can be spent designing new types of weapons and ships to build. The creation of a good user experience on the client side for design manipulation is also a challenge. Since design manipulation can be an integral part of one game, while completely irrelevant to another, the integration of a design window into clients is a significant obstacle. Thousand Parsec's most complete client, tpclient-pywx, currently houses the launcher for this window in a relatively out-of-the-way place, in a sub-menu of the menu bar (which is rarely used in-game otherwise).
The Design functionality is designed to be easily accessible to ruleset developers, while allowing games to expand to virtually unlimited levels of complexity. Many of the existing rulesets allow for only predetermined designs. Missile and Torpedo Wars, however, allows for full weapon and ship design from a variety of components.
One might say that the Thousand Parsec protocol is the basis upon which everything else in the project is built. It defines the features available to ruleset writers, how servers should work, and what clients should be able to handle. Most importantly, like an interstellar communications standard, it allows the various software components to understand one another.
The server manages the actual state and dynamics of a game according to the instructions provided by the ruleset. Each turn, a player's client receives some of the information about the state of the game: objects and their ownership and current state, orders in progress, resource stockpiles, technological progress, messages, and everything else visible to that particular player. The player can then perform certain actions given the current state, such as issuing orders or creating designs, and send these back to the server to be processed into the computation of the next turn. All of this communication is framed in the Thousand Parsec protocol. An interesting and quite deliberate effect of this architecture is that AI clients—which are external to the server/ruleset and are the only means of providing computer players in a game—are bound by the same rules as the clients human players use, and thus cannot "cheat" by having unfair access to information or by being able to bend the rules.
The protocol specification describes a series of frames, which are hierarchical in the sense that each frame (except the Header frame) has a base frame type to which it adds its own data. There are a variety of abstract frame types which are never explicitly used, but simply exist to describe bases for concrete frames. Frames may also have a specified direction, with the intent that such frames need only be supported for sending by one side (server or client) and receiving by the other.
The Thousand Parsec protocol is designed to function either standalone over TCP/IP, or tunnelled through another protocol such as HTTP. It also supports SSL encryption.
The protocol provides a few generic frames which are ubiquitous in
communication between client and server. The previously mentioned
Header frame simply provides a basis for all other frames via
its two direct descendants, the
frames. The former is the basis for frames which initiate
communication (in either direction), and the latter for frames which
are prompted by these. The
Fail frames (both
Response frames) provide the two values for Boolean logic in
the exchange. A
Sequence frame (also a
indicates to the recipient that multiple frames are to follow in
response to its request.
Thousand Parsec uses numerical IDs to address things. Accordingly, a
vocabulary of frames exists to push around data via these IDs. The
Get With ID frame is the basic request for things with such an
ID; there is also a
Get With ID and Slot frame for things which
are in a "slot" on a parent thing which has an ID (e.g., an order on
an object). Of course, it is often necessary to obtain sequences of
IDs, such as when initially populating the client's state; this is
Get ID Sequence type requests and
Sequence type responses. A common structure for requesting multiple
items is a
Get ID Sequence request and
response, followed by a series of
Get With ID requests and
appropriate responses describing the item requested.
Before a client can begin interacting with a game, some formalities
need to be addressed. The client must first issue a
frame to the server, to which the server might respond with
Connect frame includes the client's
protocol version, one reason for failure might be a version
mismatch. The server can also respond with the
for moves or server pools. Next, the client must issue a
frame, which identifies and possibly authenticates the player; players
new to a server can first use the
Create Account frame if the
server allows it.
Because of the vast variability of Thousand Parsec, the client needs
some way to ascertain which protocol features are supported by the
server; this is accomplished via the
Get Features request and
Features response. Some of the features the server might
respond with include:
Get Games request and sequence of
responses informs the client about the nature of the active games on
the server. A single
Game frame contains the following
information about a game:
It is, of course, important for a player to know who he or she is up
against (or working with, as the case may be), and there is a set of
frames for that. The exchange follows the common item sequence pattern
Get Player IDs request, a
List of Player IDs
response, and a series of
Get Player Data requests and
Player Data responses. The
Player Data frame contains
the player's name and race.
Turns in the game are also controlled via the protocol. When a player
has finished performing actions, he or she may signal readiness for
the next turn via the
Finished Turn request; the next turn is
computed when all players have done so. Turns also have a time limit
imposed by the server, so that slow or unresponsive players cannot
hold up a game; the client normally issues a
Get Time Remaining
request, and tracks the turn with a local timer set to the value in
Time Remaining response.
Finally, Thousand Parsec supports messages for a variety of purposes:
game broadcasts to all players, game notifications to a single player,
player-to-player communications. These are organized into "board"
containers which manage ordering and visibility; following the item
sequence pattern, the exchange consists of a
Get Board IDs
List of Board IDs response, and a series of
Get Board requests and
Once the client has information on a message board, it can issue
Get Message requests to obtain messages on the board by slot
Get Message uses the
Get With ID and Slot base
frame); the server responds with
Message frames containing the
message subject and body, the turn on which the message was generated,
and references to any other entities mentioned in the message. In
addition to the normal set of items encountered in Thousand Parsec
(players, objects, and the like), there are also some special
references including message priority, player actions, and order
status. Naturally, the client can also add messages using the
Post Message frame—a vehicle for a
frame—and delete them using the
Remove Message frame (based
The bulk of the process of interacting with the universe is accomplished through a series of frames comprising the functionality for objects, orders, and resources.
The physical state of the universe—or at least that part of it that
the player controls or has the ability to see—must be obtained upon
connecting, and every turn thereafter, by the client. The client
generally issues a
Get Object IDs request (a
Sequence), to which the server replies with a
List of Object
IDs response. The client can then request details about individual
Get Object by ID requests, which are answered
Object frames containing such details—again subject to
visibility by the player—as their type, name, size, position,
velocity, contained objects, applicable order types, and current
orders. The protocol also provides the
Get Object IDs by
Position request, which allows the client to find all objects
within a specified sphere of space.
The client obtains the set of possible orders following the usual item
sequence pattern by issuing a
Get Order Description IDs request
and, for each ID in the
List of Order Description IDs response,
Get Order Description request and receiving a
Order Description response. The implementation of the orders
and order queues themselves has evolved markedly over the history of
the protocol. Originally, each object had a single order queue. The
client would issue an
Order request (containing the order type,
target object, and other information), receive an
response detailing the expected result of the order, and, after
completion of the order, receive a
Result frame containing the
In the second version, the
Order frame incorporated the
contents of the
Outcome frame (since, based on the order
description, this did not require the server's input), and the
Result frame was removed entirely. The latest version of the
protocol refactored the order queue out of objects, and added the
Get Order Queue IDs,
List of Order Queue IDs,
Order Queue, and
Order Queue frames, which work similarly to
the message and board functionality2. The
Get Order and
Remove Order frames (both
GetWithIDSlot requests) allow
the client to access and remove orders on a queue, respectively. The
Insert Order frame now acts as a vehicle for the
payload; this was done to allow for another frame,
which is used by the client in some cases to obtain information for
Resource descriptions also follow the item sequence pattern: a
Get Resource Description IDs request, a
List of Resource
Description IDs response, and a series of
Description requests and
Resource Description responses.
The handling of designs in the Thousand Parsec Protocol is broken down into the manipulation of four separate sub-categories: categories, components, properties, and designs.
Categories differentiate the different design types. Two of the most
commonly used design types are ships and weapons. Creating a category
is simple, as it consists only of a name and description; the
Category frame itself contains only these two strings. Each
category is added by the ruleset to the Design Store using an
Add Category request, a vehicle for the
frame. The remainder of the management of categories is handled in the
usual item sequence pattern with the
Get Category IDs request
List of Category IDs response.
Components consist of the different parts and modules which comprise a
design. This can be anything from the hull of a ship or missile to the
tube that a missile is housed in. Components are a bit more involved
than categories. A
Component frame contains the following
Requirementsfunction, in Thousand Parsec Component Language (TPCL).
Of particular note is the
Requirements function associated with
the component. Since components are the parts that make up a ship,
weapon, or other constructed object, it is necessary to ensure that
they are valid when adding them to a design. The
function verifies that each component added to the design conforms to
the rules of other previously added components. For example, in
Missile and Torpedo Wars, it is impossible to hold an Alpha
Missile in a ship without an Alpha Missile Tube. This verification
occurs on both the client side and the server side, which is why the
entire function must appear in a protocol frame, and why a concise
language (TPCL, covered later in the chapter) was chosen for it.
All of a design's properties are communicated via
frames. Each ruleset exposes a set of properties used within the
game. These typically include things like the number of missile tubes
of a certain type allowed on a ship, or the amount of armor included
with a certain hull type. Like
Property frames make use of TPCL. A
contains the following information:
Requirementsfunctions, in Thousand Parsec Component Language (TPCL).
The rank of a property is used to distinguish a hierarchy of
dependencies. In TPCL, a function may not depend on any property which
has a rank less than or equal to this property. This means that if one
had an Armor property of rank 1 and an Invisibility property of rank
0, then the Invisibility property could not directly depend on the
Armor property. This ranking was implemented as a method of curtailing
circular dependencies. The
Calculate function is used to define
how a property is displayed, differentiating the methods of
Missile and Torpedo Wars uses XML to import game
properties from a game data file. Figure 21.2 shows an
example property from that game data.
<prop> <CategoryIDName>Ships</CategoryIDName> <rank value="0"/> <name>Colonise</name> <displayName>Can Colonise Planets</displayName> <description>Can the ship colonise planets</description> <tpclDisplayFunction> (lambda (design bits) (let ((n (apply + bits))) (cons n (if (= n 1) "Yes" "No")) ) ) </tpclDisplayFunction> <tpclRequirementsFunction> (lambda (design) (cons #t "")) </tpclRequirementsFunction> </prop>
Figure 21.2: Example Property
In this example, we have a property belonging to the Ships category,
of rank 0. This property is called Colonise, and relates to the
ability of a ship to colonize planets. A quick look at the TPCL
Calculate function (listed here as
reveals that this property outputs either "Yes" or "No" depending
on whether the ship in question has said capability. Adding properties
in this fashion gives the ruleset designer granular control over
metrics of the game and the ability to easily compare them and output
them in a player-friendly format.
The actual design of ships, weapons, and other game artifacts are
created and manipulated using the
Design frame and related
frames. In all current rulesets, these are used for building ships and
weaponry using the existing pool of components and properties. Since
the rules for designs are already handled in TPCL
functions in both properties and components, the creation of a design
is a bit simpler. A
Design frame contains the following
This frame is a bit different from the others. Most notably, since a design is an owned item in the game, there is a relation to the owner of each design. A design also tracks the number of its instantiations with a counter.
A server administration protocol extension is also available, allowing for remote live control of supporting servers. The standard use case is to connect to the server via an administration client—perhaps a shell-like command interface or a GUI configuration panel—to change settings or perform other maintenance tasks. However, other, more specialized uses are possible, such as behind-the-scenes management for single-player games.
As with the game protocol described in the preceding sections, the
administration client first negotiates a connection (on a port
separate from the normal game port) and authenticates using
Login requests. Once connected, the client
can receive log messages from and issue commands to the server.
Log messages are pushed to the client via
frames. These contain a severity level and text; as appropriate to the
context, the client can choose to display all, some, or none of the
log messages it receives.
The server may also issue a
Command Update frame instructing
the client to populate or update its local command set; supported
commands are exposed to the client in the server's response to a
Get Command Description IDs frame. Individual command
descriptions must then be obtained by issuing a
Description frame for each, to which the server responds with a
Command Description frame.
This exchange is functionally quite similar to (and, in fact, was originally based on) that of the order frames used in the main game protocol. It allows commands to be described to the user and vetted locally to some degree, minimizing network usage. The administration protocol was conceived at a time when the game protocol was already mature; rather than starting from scratch, the developers found existing functionality in the game protocol which did almost what was needed, and added the code to the same protocol libraries.
Thousand Parsec games, like many in the turn-based strategy genre, have the potential to last for quite some time. Besides often running far longer than the circadian rhythms of the players' species, during this extended period the server process might be prematurely terminated for any number of reasons. To allow players to pick up a game where they left off, Thousand Parsec servers provide persistence by storing the entire state of the universe (or even multiple universes) in a database. This functionality is also used in a related way for saving single-player games, which will be covered in more detail later in this section.
The flagship server,
tpserver-cpp, provides an abstract
persistence interface and a modular plugin system to allow for various
database back ends. At the time of writing,
with modules for MySQL and SQLite.
Persistence class describes the functionality
allowing the server to save, update, and retrieve the various elements
of a game (as described in the Anatomy of a Star Empire section). The
database is updated continuously from various places in the server
code where the game state changes, and no matter the point at which
the server is terminated or crashes, all information to that point
should be recovered when the server starts again from the saved data.
The Thousand Parsec Component Language (TPCL) exists to allow clients to create designs locally without server interaction—allowing for instant feedback about the properties, makeup, and validity of the designs. This allows the player to interactively create, for example, new classes of starship, by customizing structure, propulsion, instrumentation, defenses, armaments, and more according to available technology.
TPCL is a subset of Scheme, with a few minor changes, though close enough to the Scheme R5RS standard that any compatible interpreter can be used. Scheme was originally chosen because of its simplicity, a host of precedents for using it as an embedded language, the availability of interpreters implemented in many other languages, and, most importantly to an open source project, vast documentation both on using it and on developing interpreters for it.
Consider the following example of a
Requirements function in
TPCL, used by components and properties, which would be included with
a ruleset on the server side and communicated to the client over the
(lambda (design) (if (> (designType.MaxSize design) (designType.Size design)) (if (= (designType.num-hulls design) 1) (cons #t "") (cons #f "Ship can only have one hull") ) (cons #f "This many components can't fit into this Hull") ) )
Readers familiar with Scheme will no doubt find this code easy to
understand. The game (both client and server) uses it to check other
component properties (
Num-Hulls) to verify that this component can be added to a
design. It first verifies that the
Size of the component is
within the maximum size of the design, then ensures that there are no
other hulls in the design (the latter test tips us off that this is
Requirements function from a ship hull).
In war, every battle counts, from the short skirmish in deep space between squadrons of small lightly-armed scout craft, to the massive final clash of two flagship fleets in the sky above a capital world. On the Thousand Parsec framework, the details of combat are handled within the ruleset, and there is no explicit client-side functionality regarding combat details—typically, the player will be informed of the initiation and results of combat via messages, and the appropriate changes to the objects will take place (e.g., removal of destroyed ships). Though the player's focus will normally be on a higher level, under rulesets with complex combat mechanics, it may prove advantageous (or, at least, entertaining) to examine the battle in more detail.
This is where BattleXML comes in. Battle data is split into two major parts: the media definition, which provides details about the graphics to be used, and the battle definition, which specifies what actually occurred during a battle. These are intended to be read by a battle viewer, of which Thousand Parsec currently has two: one in 2D and the other in 3D. Of course, since the nature of battles are entirely a feature of a ruleset, the ruleset code is responsible for actually producing BattleXML data.
The media definition is tied to the nature of the viewer, and is stored in a directory or an archive containing the XML data and any graphics or model files it references. The data itself describes what media should be used for each ship (or other object) type, its animations for actions such as firing and death, and the media and details of its weapons. File locations are assumed to be relative to the XML file itself, and cannot reference parent directories.
The battle definition is independent of the viewer and media. First, it describes a series of entities on each side at the start of the battle, with unique identifiers and information such as name, description, and type. Then, each round of the battle is described: object movement, weapons fire (with source and target), damage to objects, death of objects, and a log message. How much detail is used to describe each round of battle is dictated by the ruleset.
Finding a public Thousand Parsec server to play on is much like locating a lone stealth scout in deep space—a daunting prospect if one doesn't know where to look. Fortunately, public servers can announce themselves to a metaserver, whose location, as a central hub, should ideally be well-known to players.
The current implementation is
metaserver-lite, a PHP script,
which lives at some central place like the Thousand Parsec
website. Supporting servers send an HTTP request specifying the update
action and containing the type, location (protocol, host, and port),
ruleset, number of players, object count, administrator, and other
optional information. Server listings expire after a specified timeout
(by default, 10 minutes), so servers are expected to update the
The script can then, when called with no specified action, be used to
embed the list of servers with details into a web site, presenting
clickable URLs (typically with the
name). Alternatively, the badge action presents server listings in a
compact "badge" format.
Clients may issue a request to a metaserver using the get action to
obtain a list of available servers. In this case, the metaserver
returns one or more
Game frames for each server in the list to
the client. In
tpclient-pywx, the resulting list is presented
through a server browser in the initial connection window.
Thousand Parsec is designed from the ground up to support networked multiplayer games. However, there is nothing preventing a player from firing up a local server, connecting a few AI clients, and hyperjumping into a custom single-player universe ready to be conquered. The project defines some standard metadata and functionality to support streamlining this process, making setup as easy as running a GUI wizard or double-clicking a scenario file.
At the core of this functionality is an XML DTD specifying the format for metadata regarding the capabilities and properties of each component (e.g., server, AI client, ruleset). Component packages ship with one or more such XML files, and eventually all of this metadata is aggregated into an associative array divided into two major portions: servers and AI clients. Within a server's metadata will typically be found metadata for one or more rulesets—they are found here because even though a ruleset may be implemented for more than one server, some configuration details may differ, so separate metadata is needed in general for each implementation. Each entry for one of these components contains the following information:
Forced parameters are not player-configurable and are typically options which allow the components to function appropriately for a local, single-player context. The player parameters have their own format indicating such details as the name and description, the data type, default, and range of the value, and the format string to append to the main command string.
While specialized cases are possible (e.g., preset game configurations for ruleset-specific clients), the typical process for constructing a single-player game involves selecting a set of compatible components. Selection of the client is implicit, as the player will have already launched one in order to play a game; a well-designed client follows a user-centric workflow to set up the remainder. The next natural choice to make is the ruleset, so the player is presented with a list—at this point, there is no need to bother with server details. In the event that the chosen ruleset is implemented by multiple installed servers (probably a rare condition), the player is prompted to select one; otherwise, the appropriate server is selected automatically. Next, the player is prompted to configure options for the ruleset and server, with sane defaults pulled from the metadata. Finally, if any compatible AI clients are installed, the player is prompted to configure one or more of them to play against.
With the game so configured, the client launches the local server with appropriate configuration parameters (including the ruleset, its parameters, and any parameters it adds to the server's configuration), using the command string information from the metadata. Once it has verified that the server is running and accepting connections, perhaps using the administration protocol extension discussed previously, it launches each of the specified AI clients similarly, and verifies that they have successfully connected to the game. If all goes well, the client will then connect to the server—just as if it were connecting to an online game—and the player can begin exploring, trading, conquering, and any of a universe of other possibilities.
An alternate—and very important—use for the single-player functionality is the saving and loading of games, and, more or less equivalently, the loading of ready-to-play scenarios. In this case, the save data (probably, though not necessarily, a single file) stores the single-player game configuration data alongside the persistence data for the game itself. Provided all appropriate components in compatible versions are installed on the player's system, launching a saved game or scenario is completely automatic. Scenarios in particular thus provide an attractive one-click entry into a game. Although Thousand Parsec does not currently have a dedicated scenario editor or a client with an edit mode, the concept is to provide some means of crafting the persistence data outside of the normal functioning of the ruleset, and verifying its consistency and compatibility.
So far, the description of this functionality has been rather
abstract. On a more concrete level, the Python client helper library,
libtpclient-py, is currently home to the only full realization
of single-player mechanics in the Thousand Parsec project. The library
SinglePlayerGame class, which upon instantiation
automatically aggregates all available single-player metadata on the
system (naturally, there are certain guidelines as to where the XML
files should be installed on a given platform). The object can then be
queried by the client for various information on the available
components; servers, rulesets, AI clients, and parameters are stored
as dictionaries (Python's associative arrays). Following the general
game building process outlined above, a typical client might perform
SinglePlayerGame.rulesets, and configure the object with the chosen ruleset by setting
SinglePlayerGame.list_servers_with_ruleset, prompt the user to select one if necessary, and configure the object with the chosen (or only) server by setting
SinglePlayerGame.list_sparams, respectively, and prompt the player to configure them.
SinglePlayerGame.list_aiclients_with_ruleset, and prompt the player to configure one or more of them using the parameters obtained via
SinglePlayerGame.start, which will return a TCP/IP port to connect on if successful.
Thousand Parsec's flagship client,
tpclient-pywx, presents a
user-friendly wizard which follows such a procedure, initially
prompting instead for a saved game or scenario file to load. The
user-centric workflow developed for this wizard is an example of good
design arising from the open source development process of the
project: the developer initially proposed a very different process
more closely aligned with how things were working under the hood, but
community discussion and some collaborative development produced a
result much more usable for the player.
Finally, saved games and scenarios are currently implemented in
tpserver-cpp, with supporting functionality in
libtpclient-py and an interface in
is achieved through a persistence module using SQLite, a public domain
open source RDBMS which requires no external process and stores
databases in a single file. The server is configured, via a forced
parameter, to use the SQLite persistence module if it is available,
and as usual, the database file (living in a temporary location) is
constantly updated throughout the game. When the player opts to save
the game, the database file is copied to the specified location, and a
special table is added to it containing the single player
configuration data. It should be fairly obvious to the reader how this
is subsequently loaded.
The creation and growth of the extensive Thousand Parsec framework has allowed the developers plenty of opportunity to look back and assess the design decisions that were made along the way. The original core developers (Tim Ansell and Lee Begg) built the original framework from scratch and have shared with us some suggestions on starting a similar project.
A major key to the development of Thousand Parsec was the decision to define and build a subset of the framework, followed by the implementation. This iterative and incremental design process allowed the framework to grow organically, with new features added seamlessly. This led directly to the decision to version the Thousand Parsec protocol, which is credited with a number of major successes of the framework. Versioning the protocol allowed the framework to grow over time, enabling new methods of gameplay along the way.
When developing such an expansive framework, it is important to have a very short-term approach for goals and iterations. Short iterations, on the order of weeks for a minor release, allowed the project to move forward quickly with immediate returns along the way. Another success of the implementation was the client-server model, which allowed for the clients to be developed away from any game logic. The separation of game logic from client software was important to the overall success of Thousand Parsec.
A major downfall of the Thousand Parsec framework was the decision to use a binary protocol. As you can imagine, debugging a binary protocol is not a fun task and this has lead to many prolonged debugging sessions. We would highly recommend that nobody take this path in the future. The protocol has also grown to have too much flexibility; when creating a protocol, it is important to implement only the basic features that are required.
Our iterations have at times grown too large. When managing such a large framework on an open source development schedule, it is important to have a small subset of added features in each iteration to keep development flowing.
Like a construction skiff inspecting the skeletal hull of a massive prototype battleship in an orbital construction yard, we have passed over the various details of the architecture of Thousand Parsec. While the general design criteria of flexibility and extensibility have been in the minds of the developers from the very beginning, it is evident to us, looking at the history of the framework, that only an open source ecosystem, teeming with fresh ideas and points of view, could have produced the sheer volume of possibilities while remaining functional and cohesive. It is a singularly ambitious project, and as with many of its peers on the open source landscape, much remains to be done; it is our hope and expectation that over time, Thousand Parsec will continue to evolve and expand its capabilities while new and ever more complex games are developed upon it. After all, a journey of a thousand parsecs begins with a single step.