I wanted to address some questions and thoughts that crop up now and then like these:

"Where is the best place to report bugs?"

"Where should I post my ideas?"

"Where should I report problems I found in maps?"

"I am not sure the devs have seen my report?"

"Do my posts get lost?"


So I figured people in the community (you) might be interested in or at least reassured as to how all the data is tracked. There is a lot of it and I do mean a lot of it and I have been writing tools and modifications to manage it all.

The Tools...

Pivotal Tracker
The Core team uses Pivotal Tracker to manage tasks: Bugs, Chores, Features and Release Date Targets. You see a fraction, and I mean a fraction of that data on the Progress Page.


Perforce is the version control manager that handles the builds.


Mantis BT
The Internal Testing Team reports to our Mantis Bug Tracker, Some of you already know what Mantis is, how it works & how confusing it can get.
Our Mantis tracker is very different from the vanilla download from the Mantis website. A lot of it has been automated, it is synchronised with Pivotal Tracker & Perforce, Has a totally different interface and a lot of the options in the reporting screen are automated and changed to be more informative and direct as well as there being other additions added.

getSatisfaction
Where you the community leave a large portion of your feedback.


In game annotation system
A recent addition where players can leave notes in the world where they get stuck, find a hole, prop missing etc for the current version of the game.

To display annotations use the following console commands:

CODE

displayannotations true

To add an annotation:

CODE

annotate Hole here part way up wall next to red pipe

Details & Demo here.

How we view the data...

Bug Tracking
I can't show you Pivotal Tracker, Has too many secrets :)
However a lot of things pass through Mantis that I have modified to have a similar work flow as Pivotal Tracker so we can easily sync the two up meaning should a Dev be tracking the same bug in Pivotal that exists in Mantis then both will be updated. We also needed to customise Mantis so that tracking was a lot simpler and featured some new status of the bugs.
All the Beta Testers talk to each other in a chat program, Mantis alerts our testers whenever Perforce adds a new build to Dropbox, it also alerts the testers to new fixes that need to be tested in the build that was just automatically distributed to them.

Say hello to Mantis:


This is the developer view, the Beta team only see and need to use the Accept/Reject buttons in Stage 3, So as you can see, we can see everything that is going on at the same time and what needs testing, What has been tested. Who should be fixing what, Who is fixing what.
Stage 3 fills up with bugs that have been fixed but is not in the PT's build yet, upon a dev triggering a build, perforce sends a notification to mantis, mantis notifies the team of a new build being uploaded and also updates all the cued bugs to report they are now in the current build and notifies all the testers those fixes need to be tested. A lot of the manual labour has been removed.


Annotation Viewer
Mappers can now quickly view all your annotations you left in levels, Including the ones you left in past builds, some of you where worried they get lost when a new version released, no fear Mantis is here lolz




getSatisfaction Feedback
Now this I am still working on but Pivotal Tracker already can grab any getSatisfaction reports but its an ugly sight for soar eyes.
We can browse every single Praise,Bug, Idea & question from within Mantis and so I hope to make the reports importable by the PT's taking the workload off the Main team who needs to fix the bugs rather than spend a lot of time rolling through the tonnes of reports, Something Brian is doing his best to keep on top of.
We dubbed this browser as getMantisfaction Satisfaction (Thanks Toothy)





getSatisfaction e-mails the team.
Mantis auto generates and e-mails the team daily report digests with priority break downs.

I hope this gives you a good insight on data management and more assurance that we are watching everything you say. We definitely do not have to time to give everyone an acknowledgement because there are so many of you it would eat up too much time from the game development but what you report or say is read and seen more so than the forums though on forums, Posts will get lost in there.

Very informative, however, where does the post-crash debug report go to?

wow great post! very interesting to see the tool's the dev's are using to keep track of everything! Sure looks like your work is cut out for you..

Why isn't this on the front page as a News blog entry, I am sure it would be a very good thing to be on the main page of the website.

That mantisfaction plugin is not nice. 4 different resolution states in one view? Why?

I see the lack of dedicated QA Lead means bug summaries are not in a standardised format and easy to scan. QQ

Why are crashes and 'tweeks' [sic] the same severity? Getting soft-locked in the CC is far worse than not being able to jump as a Skulk on stairs. If the severities aren't consistent they hold little value and make it harder to track progress.

I like the map comments feature, though. That I will probably steal for the next game! So much more useful than having to copy+paste in co-ords, especially if it feeds into the level editing tool. I'm presuming it just edits a text file and you have to manually commit the changes, or maybe it has an automated commit system on game shutdown (problematic if the game crashes), or maybe it's enclosed in its own separate tool independant of version control, that sends the data automatically and collates it all in realtime.

Pivotal Tracker looks like great software for a startup with limited funds.

If you are refering to the "Crashes - Tweeks" dropdown option, that just orders the bugs from crashes to tweaks.

sticky

Good post, Marc. I'll take the previous post's suggestion and sticky this.

I know this thread has been missed by a lot of people, but it is not really one you can reply to. Unless it is a simple 'thank you'. :)

This is a very interesting read. Admittedly, i was one of those thinking getsatisfaction got all but ditched. Thanks for this!

It makes me very happy that the games developers over at UWE work with their community. UWE, you are one a kind, don't change. It's awesome that you do this. Now release a mac version of this game :P

Whoah, thanks for taking the time to do this Sgt Barlow. Answers alot of questions I've been wanting to ask.