Creating a Discord bug-hunting leaderboard
Setting up a pipeline from in-game bug reporting to a Discord bot via a Postgres database.
As part of getting Hexahedra ready to launch, I've added a bug reporting tool to the game. From the Esc menu, players can fill in a form with a description of the bug and some reproduction steps, and this gets submitted to the bugtracker along with a screenshot, their Unity log file, hardware specs, and a dump of the current scene.
As a way of encouraging and thanking players for reporting bugs, I've also set up a leaderboard system using a bot on the Sidequest Ninja Discord server that allows players to "claim" the bugs they've reported, with players ranked by the number of bug reports they've had accepted. And, assuming players are happy being mentioned, the leaderboard will also appear in the credits.
This requires three parts working together - the Unity client, the bugtracker, and the bot. Here I'll discuss each part in turn, and how they team up.
Unity
The form players fill in is very simple, they only have to fill in two fields — a brief description, and a set of reproduction steps. The form also tells the player what other information will be submitted, and shows a screenshot of the current state of the game.
The game gathers all the data it needs, turns it into a JSON string, and sends it via HTTPS POST to the bugtracker webserver. Gathering all the extra bits of data posed varying levels of challenge:
Screenshot
Grabbing a screenshot is straightforward in Unity, using a call to ScreenCapture.CaptureScreenshotAsTexture() followed by ImageConversion.EncodeToJPG which converts the Texture2D to a byte array. We can also turn the Texture2D into a scaled-down Sprite to show on the bug form.
The screenshot is grabbed when the user clicks the "Report Bug" button in the menu, so to avoid the screenshot mostly being of the menu we need to temporarily hide it first. Since we need to be sure Unity has rendered a new frame before trying to grab the screen, we pass this job off to a coroutine:
// (We hide the menu before calling this)
private IEnumerator OpenForm() {
// Get screenshot. Wait for end of frame first.
yield return new WaitForEndOfFrame();
screenGrab = ScreenCapture.CaptureScreenshotAsTexture();
// Scale down to the on-screen size.
Sprite miniScreenshot = Sprite.Create(screenGrab, new Rect(0, 0, screenGrab.width, screenGrab.height), new Vector2(0.5f, 0.5f), 100.0f);
screenshot.sprite = miniScreenshot;
// Make the form appear.
background.SetActive(true);
}
Grabbing a screenshot once a new frame has been rendered.
Unity log file
This is another straightforward bit of info to grab. Unity creates a file called Player.log in whatever directory Application.persistentDataPath points to, so we can load that up asynchronously. This file can get pretty large, particularly if the game is throwing errors (which is likely if the player is reporting a bug!) so to save space, we gzip the file before sending it.
private async Task<byte[]> LoadPlayerLog() {
String path = Path.Combine(Application.persistentDataPath, PLAYER_LOG_FILENAME);
byte[] playerLogBytes;
using (var fileStream = File.OpenRead(path)) {
using (var streamReader = new StreamReader(fileStream)) {
String playerLog = await streamReader.ReadToEndAsync();
playerLogBytes = BugReport.encoding.GetBytes(playerLog);
}
}
using (var outputStream = new MemoryStream()) {
using (var zipStream = new GZipStream(outputStream, CompressionMode.Compress)) {
zipStream.Write(playerLogBytes, 0, playerLogBytes.Length);
zipStream.Close();
return outputStream.ToArray();
}
}
}
Loading and gzipping the log file asynchronously.
Hardware Specs
Unity provides a SystemInfo class that provides all kinds of useful information — graphics card details (including lots of specifics on supported features), number of CPU cores, how much RAM the machine has, and so on. Unfortunately it doesn't provide a neat way to dump all of that into a nice serializable format. However, armed with a bit of reflection we can put all the data in SystemInfo into a JSON object:
var dump = typeof(SystemInfo).GetProperties().ToDictionary(x => x.Name, x => x.GetValue(null));
report.specDump = JsonConvert.SerializeObject(dump);
This dumps all the properties in SystemInfo into a Dictionary<String, Object>, which Newtonsoft.JSON then serializes for us.
One big advantage of this method is that if Unity adds any more properties to SystemInfo they'll be added to the output automatically.
Scene Dump
Finally, we have the dump of the scene at the point the player sends the bug. This turned out to be much more difficult than I expected. Given that Unity is already able to serialize a scene to a text file (that's what a .unity scene file is if it's in text mode, after all), I expected there to be a handy function call I could use to do that for me, but this doesn't exist.
So, instead, I created my own system for this. I made a lot of use of Unity's JsonUtility, which is capable of serializing public and [SerializeField] fields in things like MonoBehaviours, but you can't hand it a GameObject and ask it to serialize all its attached scripts, you have to feed them in one at a time. It also can't serialize Transforms.
So, I got Unity to pass me all the root GameObjects (thankfully there is a function for that, SceneManager.GetActiveScene().GetRootGameObjects()) and built up a JSON hierarchy myself:
public class DumpGameObject {
private GameObject root;
private SerializableTransform transform;
private MonoBehaviour[] components;
private DumpGameObject[] children;
public DumpGameObject(GameObject root) {
this.root = root;
}
public String Serialize() {
// Pull out the data we need.
transform = new SerializableTransform(root.transform);
components = root.GetComponents();
children = new DumpGameObject[root.transform.childCount];
for (int i = 0; i < children.Length; i++) {
children[i] = new DumpGameObject(root.transform.GetChild(i).gameObject);
}
StringBuilder sb = new StringBuilder();
sb.Append("{");
// Do the info on this object
sb.Append("\"transform\": " + JsonUtility.ToJson(transform));
foreach (var c in components) {
sb.Append(",\""+c.GetType().FullName + "\": " + JsonUtility.ToJson(c));
}
// Do the children, if any
sb.Append(", \"children\": [");
bool first = true;
foreach (var c in children) {
if (!first) {
sb.Append(",");
}
sb.Append(c.Serialize());
first = false;
}
sb.Append("]");
sb.Append("}");
return sb.ToString();
}
}
It's not particularly elegant, but it gets the job done.
SerializableTransform is just an object tagged as [Serializable] which contains Vector3s for position, rotation, and scale, i.e. what you see in the Unity inspector.
Note that if you have two GameObjects at the same position in the hierarchy with the same name, this will fail because the JSON will attempt to create a property that already exists, so depending on your setup you may have to do some fancy footwork here.
I could have delved deeper on this. Doing things this way, I'm essentially dumping all the data you'd see in the inspector. There's a huge amount of data here, and trawling it will be a pain, I'm mostly grabbing it as a last resort if I can't track a bug down by other means. I could have used Newtonsoft.JSON and forced it to serialize private fields and got even more out of this, but I didn't particularly want to make the wall of text even bigger.
On a relatively simple scene like the first tutorial level, this dump comes out at around 120KB, so like the log file I'm also gzipping this before sending.
Sending the bug
Once we've got all the data, we POST it to the bugtracker webserver. If all goes well, the bugtracker returns a unique, 4-character code (a-z, 0-9) that the player can use to claim the bug on Discord and appear on the leaderboard if they want to.
Bugtracker
The Bugtracker system is a C# console app running under .NET 5.0 on Linux. It's behind an nginx proxy so that I can put some rate limiting in place in case someone tries to spam bug reports. It receives the bug report, makes sure the required data is present, and then stores it. It's also in charge of maintaining the leaderboard of bughunters who've claimed their bugs on Discord.
I did consider integrating with an existing issue tracking system, such as Trello or Jira or Redmine, but since I'm a solo dev and I don't need any complicated collaboration tools or workflows (I'm managing regular tasks with a spreadsheet) I decided it was easier to just use a Postgres database and whip up a simple web interface as a backend.
Assuming the bug appears to have all the necessary data the first thing the bugtracker does is create the 4-character code for the bug. It creates a code at random and then runs it through a filter to check that we're not about to swear at the player when we show them the code. I based my list on the one available in this GitHub repo (proceed with caution!), trimming out anything too long but also adding in terms to deal with, say, codes that look like racial slurs with the vowels taken out. The filter is regex-based so I can easily deal with 5 looking like s and so on. Let me give a shoutout to one of my regular Twitch viewers, WeirdBeardDev, who did a first pass through the list for me.
Most of the data goes into Postgres, using a parametrized query to prevent Ye Olde SQL Injectionne Attacke. The code is used as the primary key; in the unlikely event that it clashes with an existing row (there are about 1.6 million possible codes, and I'm confident my code isn't that buggy) we generate another code and try again. The screenshot, the log file, and the scene dump (the latter two still gzipped) are saved to the filesystem with names matching the code so they can easily be retrieved later.
One bit of data that's added at this stage is the bug's status, which starts as Triage. Once I've had a look at a new bug, I'll either set it to one of the rejected states (e.g. Duplicate) or I'll set it to Open. The Triage state is important because, unlike an Open bug, when the Discord bot is asked about the state of a bug in Triage it won't print out the bug's description. This means that if someone submits a bug report with something offensive in the description the bot won't happily print it out in Discord on demand. Similarly, bugs marked as Rejected won't print their description, as that's where I'll dump any such bugs.
I also add in a timestamp and three columns which for new bugs will start out null:
reporteris where I put the Discord user's ID when a bug is claimed. Unfortunately, since Postgres doesn't support unsigned numeric types and the Discord user ID is a 64-bit unsigned integer I have to do a bit of conversion in C# to avoid any overflow errors.issuenumberis a column that holds the bug's issue ID in my spreadsheet when it gets assigned one.notesis just a free text field for me to jot anything down.
With this done, I whipped up a couple of web pages, one that lets me view all bugs as a list, and one that shows full details of an individual bug along with the ability to update a bug's status etc. All the user input needs to be HTML-encoded on display to prevent HTML or JavaScript injection attacks when viewing the bugs.
At this point it's possible to submit a bug and then view and update it on the web backend, but there's no way for players to actually claim them, or see their state.
Discord bot
Building the bot took much less time than I expected. I used the open source Discord.Net API wrapper which does most of the heavy lifting, which meant I ended up writing less code to get the bot responding to commands in a Discord channel than I did getting it to talk to the database.
Claiming a bug and viewing status
These just require the bot to submit a couple of simple queries to the bug database. The !claim code command checks that the bug is currently unclaimed and, if so, sets the reporter to the user running the command. The !status code command just SELECTs the relevant bug and returns some information about it (status permitting).
Bug Hunters Leaderboard
To build the leaderboard I created a view that condenses the bug table down to three columns: the reporter's ID; how many bugs they have in the Open, In Progress, Fixed, or Launched states; and the timestamp of their most-recently reported accepted bug. The leaderboard is primarily sorted by bug count, but in the case of ties the reporter whose most recent bug was reported first is sorted higher.
I was nervous about being able to easily ask "what is a particular player's rank?" without fetching the entire view and counting rows in C#, but happily SQL has a ROW_NUMBER function that lets you do that, and it's even 1-indexed which is exactly what you want for a leaderboard.
SELECT r.pos
FROM
(SELECT leaderboard.reporter, ROW_NUMBER() OVER(ORDER BY score DESC, timestamp) AS pos FROM leaderboard) r
WHERE r.reporter=@p1
Thanks, ROW_NUMBER().
As well as being able to get their own rank with a simple !rank command, players can also see the ranks of others by tagging them.
Mission Complete
With that, we have an end-to-end system from Unity to Discord. The Hexahedra demo isn't out yet, so there are no user-submitted bugs, just my test data, but I'm looking forward to getting lots of eyeballs on the game soon, and fixing the inevitable issues I missed. I'd like to thank in advance everyone who submits a bug and helps to make the game better.
Hexahedra has a Steam demo — head to the store page to try it out.