One of the things we're working on at Unknown Worlds is the Natural Selection 2 Technology Release (NS2TR). We've described it in more detail elsewhere, but briefly the NS2TR is a set of materials, props, tools and a stripped down version of the game that we'll be releasing prior to shipping NS2. This package should give level designers everything they need to create maps for Natural Selection 2. We're trying to craft the NS2TR in such a way that mappers will be able to get tons of visual variety out of the included materials. The art team has been experimenting with decals lately as a way of making the environments feel more unique and realistic. Decals are texture maps that can stuck on top of walls or objects in Hammer to add extra details not present in the base texture. Since the same decal can be combined with different base textures, you can get a lot more combinations out of the same set of art assets. Some of the things we've been turning into decals are rust stains, control panels and signs.
In Natural Selection 1 the rooms in all of the maps are tagged with a names for easier navigation. These name of the room you're currently in is displayed on the HUD, but why not also include signs in the world with those names to help you get from one area to another?
When we considering this possibility for Natural Selection 2, we realized there was a problem. Every piece of text requires a separate texture map, which on top of being a lot of work, means we'd have to know the names of all the areas ahead of time. As an alternative, someone on the team suggested procedurally generating sign textures at run-time. Besides saving the development time, loading time and disk space, procedurally generating the signs makes it really easy for mappers to place custom signs with their own text. Since they are created at run-time, it also gives us the option of localizing all of the text on the signs so that they appear in the player's native language.
As I began implementing, I realized that by generalizing the system a little we'd be able to do a lot more than create simple signs. My first step was to change the system from working only with decals to hooking directly into the material system. Since decals are drawn using the material system, this change still allows us to create procedural decals. The advantage is that in addition to working with decals, we can also apply procedural textures to any surface or object in the game world.
The second change I made was to use our scripting system to control the drawing of the texture. Contrast this with the original design -- hard coding the system to just draw a single line of text for the sign. Sure you can add parameters to control different aspects of the hard coded approach, like the font, color, size, etc., but say you wanted a sign that looks like old peeling paint? Or a neon sign that flickers? Or a sign with text centered below an icon? Since these behaviors don't have much in common, crafting a reasonable set of parameters to control them would be difficult. The scripting system, on the other hand, allows for all of those (and more) through a simple set of drawing functions. Since we already had a mechanism in place for drawing the UI from script code, I just hooked into the same mechanism and now any of the facilities that we could use to draw the HUD we can also use to render onto textures.
I mentioned earlier that the motivation for these changes was to allow us to do a wider range of things with the system. So what kinds of things can it do? Since the texture generator is in script code, it can use any of the game state to influence the creation of the texture. For example you could create a score board in the world that displays the names of the current leaders (think DM-Morpheus from the original Unreal Tournament). Or you could put an ammo readout on the back of a weapon like in the concept artwork I posted last week. Or you can combine a face texture, a body texture and skin hue modifier at load time to generate a unique player appearance based on the user's preferences. Or you could implement Conway's Game of Life on all of the computer monitors in your space station. As you can see, the possibilities are vast with a flexible system like this. Let's hear your ideas.
I ended the last blog entry hopeful that the remaining work for scripted weapons – the transmission of variables between the server and the client – would proceed smoothly. It seems that my optimism was well placed, since instead of the week I originally scheduled, it only took a day to get working!
Our updated pistol behaves a lot more like a pistol should -- it has limited ammunition and when you exhaust all of the bullets in a clip you can reload it. Here's the updated pistol source code with the interesting bits highlighted.
Since both the server and the client are running the same script code, they both keep track of how much ammo is left and remove a bullet after each shot. Normally you might that that's enough to keep everything working perfectly, but it's possible for the ammo value on the server and the client to get out of sync with each other. This occurs if the client incorrectly executes a weapon firing event which never occurs on the server.
Here's a scenario where that can happen: say you're getting charged by an Onos. Naturally you pull out the most powerful weapon in the TSA arsenal – the pistol – and draw a bead on his left nostril. But a millisecond before you shoot, the Onos does a stomp attack that stuns you. It takes a little while for the message saying you're stunned to arrive from the server, so your client thinks you should be able to fire when you pull the trigger. It happily runs the pistol attack code which plays the attack animation and removes a bullet from your clip. When the server gets the message that you pulled the trigger, it just ignores it since it knows that you're supposed to be immobile. Now the pistol on the client has one fewer bullets than the pistol on the server.
This is where variable transmission comes to the rescue; at least to the rescue of our out of sync problem, it won't help one bit with your Onos problem. Since our pistol script code informed the engine that the ammo variable should be networked, the networking system will periodically transmit its current value from the server to the client. So even though the variable is incorrect on the client, it'll be fixed up soon enough (although in your current predicament with the Onos you may not live to see it).
Keep in mind that this type of scenario is a corner case and doesn't happen too often. Normally the client prediction matches what's happening on the server and a fix up like this isn't necessary. But when you have a high ping to the server, your client tends to be more out-of-date with the server and the situation is more prevalent. That's why when you're lagging in most shooters, you sometimes see things happen on your machine that a few seconds later become undone. This however is a small price to pay for the responsiveness that client-side prediction affords.
The last piece I'm going to add to the pistol before moving on is a secondary fire mode. We're planning on including alternate fire for all of the weapons in Natural Selection 2, although the list currently isn't finalized. Feel free to post your suggestions in the comments for this blog.
One other thing worth mentioning in the pistol code is the use of the balance variable system. Balance variables store important constants that can be changed via an in-game debug interface while the game is running. The balance variables in the pistol code are things like kPistolClip and kPistolDamage, which store the maximum clip size and the damage done by each bullet. As the name suggests, the balance variable system is there so that we can easily adjust the game balance without interrupting a play test session. Just like the scripting approach, balance variables are another facet of our initiative to help us rapidly iterate and fine tune the game.
The balance variable system is an enhancement of a similar system that was in Natural Selection 1, and one of the great new features for Natural Selection 2 is how it integrates with our scripts. In Lua the balance variables automatically appear as global variables and you access their values just like you would any other variables. This isn't directly possible in C++ since all variables must be declared at compile time and the balance variables are read from a text file when the game is started. That's just another way that scripting is making things a little bit easier.
In the client-server network architecture used by the Source engine, all of the important game decisions are made on the server. These “decisions” include things like what happens when a player is killed, when the game ends, and what is the function of a turret factory. To keep things simple, our original plan for scripting in Natural Selection 2 was limited to scripts that executed on the server and controlled those aspects that didn't require any complex networking with the client.
Even with server-side-only scripting there is still a lot of power to alter the game; if you've ever played on Natural Selection 1 servers that have plug-ins like the Lerk lift or Armory healing you've gotten a taste of what can be done with a few server-side tweaks.
However, the ability to easily mod Natural Selection 2 is only one of the reasons we're integrating a scripting system into the game. Perhaps the most important reason is to speed up our own development, and allow us to quickly and easily prototype different ideas.
With that in mind, server-only scripting felt too limited. When we started building Commander Mode, we decided that user interfaces were a perfect candidate for scripting and it was worth the effort to run scripts on the client as well as on the server.
The nice thing about user interfaces is that they exist only on the client. This means that adding scripts to control them was fairly easy since we could still sidestep the issue of networking in script code. But just as we became dissatisfied with the server-only scripting, it didn't take long before these non-networked client scripts seemed a little too limited for our tastes.
Although some stuff happens only on the server, and other stuff happens only on the client, a lot of stuff needs to happen on both the server and the client, and data needs to be shared across the network to keep things in sync. This includes anything where the latency (“lag”) of the network connection is going to become evident. The most prominent example is probably the weapons.
If the light machine gun was implemented so that when you pressed the trigger it had to send a message to the server and wait for a response telling it the the gun fired, you'd feel a noticeable delay and would probably have a pretty hard time hitting the Skulk that's about to bite your head off. Instead, the Source engine runs the weapon firing code instantly on your client (where you see the effect immediately), and once the server receives the message that the gun was fired it also runs the weapon firing code doing the actual damage to your target. The Source networking system does some clever stuff where it rewinds time and does the firing test on the server with the skulk at the position he was when you actually pulled the trigger. This gives the guy with a 1000 ping a chance to make it to another tour of duty.
Since Garry is also using Lua with the Source engine, I imagine that some people probably think that there's a switch you flip that makes the two work together nicely. Unfortunately this isn't the case, which is why we started off with the more limited approach to scripting. But full scripting of weapons became too compelling to pass up, so we bit the bullet (no pun intended) and have been working on getting those objects that exist on both the client and server scriptable.
The Source engine has lots of neat little macros that make it pretty easy to setup weapons in C++ code. These macros are magical single lines of code that expand into a whole mess of complex code when you compile your game. Since a lot of this exists in the publicly available game code, you can check out the Source wiki if you're interested in getting a deeper look at what exactly we're talking about.
With a scripting approach, the weapon code doesn't ever get compiled, which is one of the big time savers with using a scripting language. In fact the game doesn't even know those weapons exist until the the map is loaded. These are good things in terms of development, but it means those handy macros that the Source engine provides don't work at all when building a weapon in script.
So the task that's consumed me for the past week is delving through all of that networking code, figuring out how it works, and building a layer between Lua and Source that replicates the functionality of the C++ macros. This is one of those times that working on a stand-alone game with full access to the engine code has really come in handy, since ultimately we needed to modify how some of the low level networking code works to allows us to build that layer. It's also quite nice be able to fire off an email to Valve and get a quick response from the guy who wrote that networking code.
Right now we're at the stage where we've got our scripted entities hooked into the core network system. With that in-place, the engine will instantiate client-side versions of the entities when the server tells them they've come into the player's view. Those entities are hooked into script files which control how the entity behaves on the client and the server. In practice what that means is we've got a scripted pistol up and running that works as you'd expect with the appropriate client-side prediction and the code running on both the client and the server. The screen shot shows the current code for the pistol inside our Lua IDE, called Decoda.
Sharp readers will notice that the pistol code doesn't include any mention of ammo or reloading. Adding in network propagation of variables – such as the amount of ammo left in the gun – is the next step for our Lua integration. This builds on the core changes we've just finished implementing, so hopefully things will continue smoothly. Smooth or not, I'm sure we'll be writing more about that in a future blog.
Max wrote a very nerdy and interesting blog update on client-side scripts and the code behind the weapons in NS2. Let your inner dork free.
If you aren't a programmer this post might be pretty dry. You've been warned...onward!
In our continuing quest to streamline development and allow our tiny team to work as efficiently as possible, we tackled another problem that dogged us a bit during NS1 development - builds. Building the game so you can release it to a group of playtesters, or so you can release it publicly requires quite a few steps:
1. Get the latest source code from your repository (Subversion in our case) 2. Clean and rebuild everything. MSVC sometimes gets confused and builds wacky binaries, so cleaning first will make sure to avoid wasting time on a temporary "bug". 3. Perform some basic "smoke" testing to make sure all the load, make sure no values snuck into your .cfg files (my config was shipped a few times to our playtesters, resulting in a server full of "Flayras" and mass confusion). 4. Create the dedicated server .zip and the client installer. We also create patches, which are the difference between the last version and the current version, so players and server operators don't need to download the whole thing. 5. Upload both to our site and send out e-mail to mirrors (if a public build) so they can have the files mirrored the minute we release.
For NS1, we did all this manually. Harry "puzl" Walsh and I got pretty good at this, though it ended up taking a good portion of a day. It is stressful because if there are any bugs or problems that appear at any point along the way, you have to fix the bug or include the file and start over at step 1. We always talked about automating this process in some way, but we never found the time.
For NS2, we're making this as painless as possible. We're shipping on Steam so that takes care steps 4 and 5. On the flip-side, builds take a lot longer. NS1 only took about 15 minutes to build from scratch and a complete rebuild of NS2 takes over an hour. So this time we wrote a script that's accessible by web-browser to do steps 1 and 2 on our server. Here's what it looks like:
I can kick off the build by clicking a link and then it e-mails me when the build is done. Some notes on the script implementation if you want to do this yourself:
- I used MagicISO to install MSVC on our server. You can make disc images, upload them, then mount them so it looks like a CD. This is needed if you rent a server that you don't have physical access to.
- You could use cron (or PyCron) and rsync to build your own system that will allow your playtesters to only need to download diffs.
- We save all the output of the builds to build logs so if there are any compilation or system errors they visible. We queue the process via the browser, which is then built by a PyCron job so the browser doesn't "hang".
I hope this gives you some ideas on how to look at your own build process and streamline it. Now the final part is reducing work on step #3 (automated testing), which I'm taking a look at now.