UE4, Protobuf, and Yarn Spinner

A little while ago I embarked on the adventure of porting Yarn Spinner to Unreal Engine. While that task isn't finished, the first stage is and this is the opportune moment to write a blog post about the challenges thus far.

I didn't think porting Yarn Spinner would be easy, but I also didn't anticipate the hardest part (so far) would be reading the yarn files into Unreal.

This is a story of integrating protobuf into UE4.

Please note, this is a fairly dense blog post. This is because it is as much for me to process my thoughts and have a record of my thinking for later on when I forget it all, as it is for others to learn from. If that's not your style: sorry.

If you don't care about the story that led to this blog and are just after the example code please go to Github repo.

⚠️ This was written for Unreal Engine 4.24.3 using Visual Studio 15.9.21: It may not work for any other versions!

For Those That Came In Late

A little bit of backstory, Yarn Spinner uses Google's protocol buffers (also known as protobuf) as its internal storage format.

This might seem a little weird as Yarn is clearly a text format: you definitely write it as text. While Yarn scripts are text you really don't want to be parsing through a giant blob of text every time you need the dialogue, we want something a bit more lightweight. Additionally with the improvements to the localisation support in Yarn Spinner the core of the project no longer even cares about the dialogue itself, all it sees are line identifiers. These line identifiers are then looked up in a strings table (a .csv file) to work out what to show to the user. Under the hood then, Yarn dialogue is nothing but a big list of instructions that Yarn Spinner obeys, some of these instructions show lines to the user, others are responsible for features like reading variables or branching your story.

All of this means that keeping Yarn as text at runtime isn't the best move and we needed a binary format. We could have created our own format but that just invites problems, so we chose to move the internal format to protobuf because it matched our requirements:

  • lightweight
  • portable
  • supports collection based data structures

and had a whole bunch of other benefits that are tricky to achieve already done for us:

  • types and enums
  • verification and error handling
  • serialisation and deserialisation
  • massive documentation and community
  • many different programming environments supported

While I haven't tested this I would say something like FlatBuffers or Cap'n Proto might be an even better fit than protobuf, but they just don't have the supporting community, documentation, and environments that protobuf has. Docs and community trump features and performance every time. Maybe in the future we'll look again at changing the internal format but for now protobuf it is.

Portability

One of the main purposes of protobuf is its portability. Once you have your data serialised into protobuf you can then pass it around anywhere that supports protobuf. This means if we have a story in Yarn compiled down to protobuf we can just share around the protobuf files. In an ideal world this makes porting Yarn Spinner simpler as we don't need to reproduce the entire compiler toolchain, all you need is the ability to read in protobuf files and to port the runtime. Eventually you will probably also want to port the compiler over as well so that all platforms have an equivalent feature set, but for a "v1" of any Yarn Spinner port this approach reduces the workload massively.

What Unity Does

If you have used Yarn Spinner in Unity you don't ever have to worry about this, you just add your .yarn files to the project and that's it. Behind the scenes we are actually running the Yarn Spinner compiler over them, extracting a strings table and spitting out a compiled protobuf yarn file .yarnc, but you never have to worry about this either. At runtime when you ask for a particular piece of dialogue to be run, we load in the .yarnc file and use that to work out what to show. The text form of your dialogue, the .yarn file itself, is never touched at runtime and won't even be baked into the final version of the game.

The Goal

For an initial version of Yarn Spinner in Unreal we decided we'd first take the compiled .yarnc protobuf files, and their string tables, and build a version of Yarn Spinner for Unreal that uses that. It wouldn't be hugely user friendly but it would be enough to start working out kinks and figuring out the best flow to integrate Unreal and Yarn Spinner. The path ahead was clear then:

  1. get protobuf working with UE4
  2. get it reading .yarnc files
  3. build up the runtime around them
  4. add in blueprint support for all of this

The Plugin Pickle

Unreal has a system for extending Unreal called plugins, or maybe modules, or maybe they are the same thing? Who knows? From my current understanding, modules are standalone components that can be formed together into a plugin and this is my assumption for the rest of this blog, but I genuinely don't know.

This is the first issue I found when getting started: there isn't a lot of reliable and up to date information out there about plugins and modules in Unreal. What there is is often contradictory, and most of the example code for this on sites like Github are all on ancient versions of Unreal, and the UE4 wiki (before being taken down) while it did have some truly excellent info on it (luckly saved by the Internet Archive), was likewise filled with posts that were well out of date.

This the first time I can safely say Unreal is doing something worse than Unity. For the most part I have been super impressed with Unreal, and often have been favouring its approach to Unity's, but not for extensions. Some of this is just my inexperience with Unreal, but a lot of it isn't. There is so much knowledge in the Unreal community that you sadly only see glimpses of in the Unreal forums or occassionally popping up on twitter.

From this point onwards I'll be making a new project, integrating protobuf into it, creating some wrappers for a new protobuf message, and showing off using the protobuf message in a blueprint.

Tools

If you want to follow along with this you will need:

While this blog will be describing every step and code block in, I hope, sufficient detail that anyone can follow along I am assuming you are somewhat comfortable with writing code and using the terminal. I did all of this on Windows, in theory most of this is translatable to macOS but I have not tried to do so yet.

Creating the project

First things first, we need a project. While all our code will be going into our plugin we will still need a project so we can see if its working.

  1. Create a new project in Unreal

Screenshot of the Unreal Engine New Project UI

  1. Choose the blank template

Screenshot of the Unreal Engine New Project Template UI

  1. Set the project to use C++ instead of Blueprints
  2. Select No Starter Content
  3. Call the project ProtobufTestProject
  4. Save the project in your preferred location (in my case D:\Development\Unreal\)

Screenshot of the Unreal Engine New Project Settings UI

Even though we've chosen to use a C++ project we can, and will be, using blueprints. With our project created we need to make our plugin.

Making a plugin

The Unreal editor should have launched with our project open, from here we can start building up our plugin.

  1. From the menu bar go to Edit -> Plugins

Screenshot of opening the Plugins window from Unreal Editor

  1. Click on New Plugin

Screenshot of the Unreal Editor plugin window

  1. Select the Blank template
  2. Name the plugin ExamplePlugin

If you want you can name the plugin something else, in my case, for Yarn Spinner, I named the plugin YarnSpinner, but for this blog I'll be sticking with ExamplePlugin.

  1. Click Create Plugin

Screenshot of the Unreal Editor plugin creator

After a brief moment the plugin will be created and Visual Studio will launch with our project and plugin loaded as a solution and ready to modify.

  1. Visual Studio always launches behind Unreal, so alt+tab over to Visual Studio.

When you get Visual Studio to the forefront it will have a warning message about how the solution has been modified outside of Visual Studio and needs to be reloaded.

Screenshot of the Visual Studio warning dialogue

  1. Click Reload

With that done we can take a look at the structure of our plugin.

Screenshot of the Visual Studio Solution Explorer

We've got two main points where our code lives (or will live), the Sources folder and the Plugins folder. The Sources folder is where we put project specific code and we won't be adding anything in there. The Plugins folder is our main point of contact and goes into lower folders for the plugin itself in the ExamplePlugin folder.

Screenshot of the folder structure for the project

If you navigate into the ExamplePlugin folder you'll find the Sources folder for the plugin and inside of that another ExamplePlugin folder (so D:\Development\Unreal\ProtobufTest\Plugins\ExampleProject\Plugins\ExamplePlugin\Sources\ExamplePlugin in my case).

Screenshot of the folder structure for the plugin

This was something that initially threw me off, why is there another folder? Assuming I understand UE4 correctly, this is the folder for the ExamplePlugin module, and where all our code for that will go. Our ExamplePlugin plugin will use this module as part of it, confusing right? I kind of wish Epic would rename the module something like ExamplePluginModule instead of just naming it the same as the plugin, but oh well.

Nonetheless, inside this folder is the ever so important ExamplePlugin.Build.cs file. We'll talk a little more about this file in a moment and what it does and controls but just for now know that this file is what Unreal will use to link our module into the project. If you don't have this Unreal won't see your module and won't know how to build it or present it to the rest of your project.

From my understanding of UE4 we don't need to do this as a plugin, or have this level of folder abstraction but my mind likes the separation. If you aren't big on that you can probably just place everything into the Sources folder of your project (not the plugins) if that's more your jam.

Now we can move onto working with protobuf, but before we do that we need to clean up:

  1. Close Visual Studio
  2. Close Unreal editor

The reason we close both of these is because we are going to be manipulating the folder structure and this is much simpler to do without those open and then refresh them once we've got it all set up.

Building protobuf

Protobuf is a big project, around 150,000 lines of code, and it has its own rules and assumptions. This means building protobuf isn't exactly easy, neither will the protobuf way be directly compatible with the Unreal way. Unreal has some assumptions as to how libraries will be working and these are documented here. We need to meet these requirements, there is no other way around it.

What this means is we are going to have to build protobuf, we can't use any prebuilt versions of, or at least none that I could find. There are heaps of ways of doing this and I tested a few different one and to cut a long story short I settled on vcpkg as the one that works the best for me.

If you've not used vcpkg before it is a library manager and build tool for C++ made by Microsoft. The general philopshy behind it is you describe the triplet you want to build and vcpkg takes care of it from there. Triplet are the config files for vcpkg and describe the CPU architecture, operating system, and runtime settings you want the build to have. As an example a triplet might look something like x86-windows-static which is saying that for whatever package we want to build we want it built for the x86 CPU architecture (x86-windows-static), on Windows (x86-windows-static), and to have static CRT linkage for MSVC (x86-windows-static). If you are curious about triplets the vcpkg website has a good docs writeup about them.

Not all triplet combinations are valid for all projects, but luckily for us the one we need for protobuf is. So let's build protobuf!

  1. If not done already, install vcpkg
  2. Open the Terminal
  3. Navigate to your vcpkg installation folder

In my case this required changing drive, and moving to my development folder:

d:
cd Development\vcpkg\

I imagine you'll have a different installation folder to which you'll navigate, but if not, woo vcpkg-installation-folder-buddy high-five!

⚠️ Because it had been so long since I last used the Windows terminal I had actually forgotten how to change drives!

  1. Build protobuf with the x64-windows-static-md triplet:
vcpkg.exe install protobuf:x64-windows-static-md

This will then shoot off on its own downloading and building the project, giving us the perfect opportunity to talk about the triplet (or should it be called a quadruplet?). The first two parts (x64-windows-) specify we want it to be build for Windows on the x64 architecture which is most of Windows and more importantly what my computer is. The next two components are the more interesting ones.

First, we are specifying (-static-) that we want the CRT to be linked into the library statically. From my understanding of how Unreal works I don't think this should be necessary but when I made builds without it being statically linked I got a whole bunch of linker errors, so static it is. If someone knows why this is the case, do let me know.

The final part (-md) is saying we want the library to be built as a multithreaded specific library, and this is a requirement of linking in 3rd party code into Unreal. If you are after more information on the various build settings around MD, MT, MDd, etc, and the implications therein Microsoft has a decent discussion around what all this means. For us because we need it to integrate with Unreal and Unreal requires MD, so MD it is.

⚠️ This will only work on x64 Windows builds of Unreal: If you want to use protobuf on different operating systems or architectures you will have to build a different version of the library!

Quick segue

There are heaps of ways of building C++ projects, and if you are confident enough just using Visual Studio or CMake you can likely handle all of this yourself without using something like vcpkg but I lack such bravado. There is one really interesting project called UE4cli which has the ability to integrate with the conan build tool and can build targets for Unreal.

I initially started using it for this when I was doing most of my Yarn Spinner development on macOS. I ran into some issues there with UE4cli that due to my lack of knowledge around conan and Unreal meant I couldn't fix them so I moved to Windows and vcpkg as I already knew how to use it. In the future I want to check out UE4cli again because I think it's got the actual potential to avoid ever having to think about building 3rd-party libraries for Unreal.

Also, the dev for it was super nice on Github when he was helping me out!

Adding protobuf to our plugin

With protobuf built it is time to start adding it into our plugin. We are going to be adding it in as a 3rd party library into our plugin, making it its own module.

  1. In Explorer navigate to your plugins' (not the project's) Source folder

In my case that is at D:\Development\Unreal\ProtobufTestProject\Plugins\ExamplePlugin\Source.

  1. Create a ThirdParty folder inside of Source
  2. Create a protobuf folder inside of ThirdParty

Now we need to add in our protobuf build from vcpkg

  1. Open a new Explorer window and navigate to your vcpkg folder

In my case that is at D:\Development\vcpkg.

  1. Navigate into the installed\protobuf_x64-windows-static-md folder

This folder has the version of protobuf we built earlier.

  1. Copy the lib and include folders from here
  2. Back inside our plugins explorer window paste those folders into the protobuf folder we made

You should now have a folder structure so that inside of PrototbufTestProject\Plugins\ExamplePlugin\Source there is both an ExamplePlugin folder and a ThirdParty folder. Inside of that ThirdParty folder you should have a protobuf folder and inside of that folder you should have two folders, lib and include.

This is an awful lot of folders but it does need to be done in this way or else Unreal won't be able to find and link it all together. So it is worth putting in the time to make sure your plugin setup is identical to how I described. With that done we are almost finished with protobuf, and now need to make it so that Unreal can see it.

  1. Inside the protobuf folder create a new text file
  2. Name the file protobuf.Build.cs

⚠️ The name is very important here: If you name it something else it won't work!

  1. Open protobuf.Build.cs inside Visual Studio (or your text editor of choice) and add the following code:
using UnrealBuildTool;
using System;
using System.IO;

public class protobuf : ModuleRules
{
    public protobuf(ReadOnlyTargetRules Target): base(Target)
    {
        Type = ModuleType.External;

        PublicSystemIncludePaths.Add(Path.Combine(ModuleDirectory, "include"));
        PublicIncludePaths.Add(Path.Combine(ModuleDirectory, "include"));

        if (Target.Platform == UnrealTargetPlatform.Win64)
        {
            PublicLibraryPaths.Add(Path.Combine(ModuleDirectory, "lib"));
            PublicAdditionalLibraries.Add(Path.Combine(ModuleDirectory, "lib", "libprotobuf.lib"));
        }
        else
        {
            throw new Exception("Currently " + Target.Platform.ToString() + " is unsupported");
        }

        PublicDefinitions.AddRange(
            new string[]
            {
                "GOOGLE_PROTOBUF_NO_RTTI=1",
                "GOOGLE_PROTOBUF_USE_UNALIGNED=0"
            });
    }
}

This code does a few things so it is worth quickly going through it. At the highest level, it is declaring a new module called protobuf and then configuring how that module works.

Going into a bit more detail, first we are telling Unreal that it will need to make sure to search through the include folder for header includes. Without this you wouldn't be able to use any of the various protobuf header files, and while we won't be using them directly, any protobuf files we generate will still need them. We need to do this because Unreal has full control over the build process and just manually putting the path to files directly in your #include's won't work. I assume under the hood UnrealBuildTool and UnrealHeaderTool are rearranging the folder and file structure as it sees fit, which means we have to tell it about the folders it needs to allow searching upon.

Then we do a quick check to see if we are on Windows or not. Because we only built protobuf as a Windows library if we aren't on Windows we will just throw an exception and halt the process. If we are on Windows though (like I was when I wrote this) then we add the path to our built out libraries. I hope that it shouldn't be too tricky to build protobuf for macOS or linux, and adapt the library loading code from Windows, I just haven't tried yet.

Finally we are setting two flags for protobuf to obey. We are disabling runtime type information, because it uses reflection and that's a no-no. The second flag I don't fully understand, but I had some linker errors without it and I noticed a few people had mentioned in passing to disable it, and disabling it fixed my issues. I'm assuming based on its name we are forcing protobuf to align its data in a particular fashion, to pack out memory in a specific way. Unsure as to why that matters here though. If people have more insight into what that flag does and why its important in this situation let me know.

With this file created and filled out Unreal can now link in protobuf for use in our plugin.

Integrating Protobuf With Our Plugin

Luckily for us getting our ExamplePlugin module to be able to see protobuf is easy.

  1. Navigate to the ExamplePlugins module folder

In my case this was at D:\Development\Unreal\ProtobufTestProject\Plugins\ExamplePlugin\Source\ExamplePlugin

  1. Open the ExamplePlugin.Build.cs file
  2. Add the following line to the PrivateDependencyModuleNames section:
"protobuf"

You need to place it below the others being added as part of the AddRange function. Epic have very kindly put a line that says // ... add private dependencies that you statically link here ... which helps find the right area. Once done your PrivateDependencyModuleNames section should look like the following:

PrivateDependencyModuleNames.AddRange(
    new string[]
    {
        "CoreUObject",
        "Engine",
        "Slate",
        "SlateCore",
        // ... add private dependencies that you statically link with here ...    
        "protobuf"
    });

And just like that we've included protobuf into our module! Of course it doesn't actually do anything yet, but we're getting to it.

It's worth just quickly seguing here to talk about protobuf-lite. Protobuf-lite is a trimmed down runtime of protobuf with less features but designed for more restrictive environments. Protobuf-lite looks like it might be a better fit for the needs of video games than regular protobuf. While I've not tested this protobuf-lite should just work as a drop-in replacement for the full protobuf by changing which lib file it is targetting. Much like with FlatBuffers vs. protobuf, there is just less information around there around UE4 and protobuf-lite and is currently in the "future Tim" basket.

Creating some protobuf files

Protobuf works by defining a schema for your data and then you use the protoc compiler tool (which vcpkg has helpfully installed for us already) to generate code in your desired programming language that can serialise and deserialise data that matches the schema. This means we will need to do a few things, write a data schema, generate some C++ code for that schema, and then finally include that into our plugin.

Our data schema

We need to define a data scheme in the protobuf language that will describe the structure of data we want our game to understand. Because protobuf is such a large and well documented project we won't go into too much detail describing the message structure here. If you are curious as to exactly what all this means, check out the official language specification page.

  1. Create a new text file and name it Doggo.proto
  2. Add the following code to the protobuf file:
syntax = "proto3";

message Doggo
{
    string name = 1;
    int32 age = 2;
    float goodBoyness = 3;
}

message Kennel
{
    repeated Doggo goodBoys = 1;
    string name = 2;
}

This is our schema, we define a new message type (protobuf's term for a data structure) for a Doggo. Each Doggo will contain their name (a string), their age (an integer), and a value to indicate how much of a good boy they are (a float).

⚠️ If you're concerned about dataset being biased towards dude dogs, fear not: The good boyness of a dog is immaterial to their gender.

We then define a new message type which is a Kennel which just contains an array of Doggos and the name of the kennel. This is intentionally kept simple but protobuf can describe pretty much any level of information you so desire.

⚠️ Don't worry about those numbers next to the message fields: While it might look like we are setting a Doggos age here to two, we aren't, this is just part defining protobuf message.

Generating our C++ code

With our schema ready we can now use it and the protobuf compiler to generate some C++ code which will be able to read and understand Doggos and Kennels.

⚠️ Do not treat protobuf generated classes as normal C++ classes: While the code generated contains lots of classes you should treat them more akin to simple structs. Don't inherit or extend them, they are data only!

  1. Open Terminal
  2. Navigate to where vcpkg installed the protobuf compiler

In my case that looked like this:

D:
cd Development\vcpkg\installed\x64-windows-static-md\tools\protobuf

Inside here you will find the protoc.exe protobuf compiler which will be generating our code for us. Now to use this tool we will need to know the absolute paths for our input file (Doggo.proto), it's folder, and where we want our output to go. In my case I saved my Doggo.proto file in the same directory as my module, so D:\Development\Unreal\ProtobufTest\Plugins\ExamplePlugin\Source\ExamplePlugin\Public\Doggo.proto and I will be exporting the generated code to the same folder, so in my case D:\Development\Unreal\ProtobufTest\Plugins\ExamplePlugin\Source\ExamplePlugin\Public

  1. Run the protobuf compiler using the following command:
protoc.exe --cpp_out=D:\Development\Unreal\ProtobufTest\Plugins\ExamplePlugin\Source\ExamplePlugin\Public --proto_path=D:\Development\Unreal\ProtobufTest\Plugins\ExamplePlugin\Source\ExamplePlugin\Public Doggo.proto

There are four parts to this worth just quickly pointing out.

The first protoc.exe is the compiler itself. Next the --cpp_out= is us telling the compiler where we want the C++ files it generates to be saved. This is also where you can configure for which language you want protobuf to generate files, for example if we'd used the --csharp_out flag instead we'd have gotten C# code. --proto_path= tells the compiler the path of where our proto files are stored, and Doggo.proto is telling the compiler we want to generate code for that file.

⚠️ As a rule of thumb you don't want things public unless you've no choice: While writing this I realise in hindsight the generated files don't really need to be public. I just did it that way while figuring all this stuff out, but hindsight is perfect and I had already written the code for this and I'm not rewriting this monster blog!

If you look inside the public folder you'll see two new files, Doggo.pb.h and Doggo.pb.cc, these are our generated C++ files. Normally you shouldn't touch these files unfortunately this is another situation where the Unreal Way and Protobuf Way conflict so we are going to have to tweak the generated files.

  1. Open Doggo.pb.cc
  2. Add the following code to the very top of the file:
#pragma warning (disable : 4800)
#pragma warning (disable : 4125)
#pragma warning (disable : 4647)
#pragma warning (disable : 4668)
#pragma warning (disable : 4582)
#pragma warning (disable : 4583)
#pragma warning (disable : 4946)
#pragma warning (disable : 4577)

#ifdef _MSC_VER
#include "Windows/AllowWindowsPlatformTypes.h"
#endif
  1. At the very bottom of the file add the following code:
#ifdef _MSC_VER
#include "Windows/HideWindowsPlatformTypes.h"
#endif

We just added a fair bit of code so let's quickly go through it. The first part we added (all the #pragma stuff) disables some warning checking on the file. We have to do this because protobuf code generates some warnings and Unreal (correctly in my mind) converts warnings into errors. This halts the compilation and build process. Normally you should be fixing the warnings instead of just suppressing them but because this isn't our code and I don't want to vendor protobuf, suppression it is.

With the other two parts first up in the top one we're creating a quick check if we are on Windows, and if so we need to include the Windows data types. I'm not fully sure why we need these: it could be a protobuf thing, or an Unreal thing, or a combination of both. It might have something to do with Unreal not being huge into the STL and Protobuf being quite good friends with it. Regardless we need them, hence the inclusion. At the bottom though we turn them back off because we only need them for our protobuf code, not all over the place.

With that done we are very close, and all that's left is to quickly add some code so we can make sure they are hooked up correctly.

⚠️ This is test only code: What we are about to write is to make sure we've hooked it all up correctly and we should delete it once we've made sure it all works. Don't leave this code in your project!

  1. Open ExamplePlugin.cpp
  2. Include our protobuf header:
#include "Doggo.pb.h"
  1. Inside the StartupModule method add the following code:
Doggo goodBoy;
goodBoy.set_name("Bork Bork");
goodBoy.set_age(3);
goodBoy.set_goodboyness(1);

FString name = FString(goodBoy.name().c_str());

UE_LOG(LogTemp, Warning, TEXT("Who's a good boy? %s is!"), *name);

Here we are building a new Doggo and then logging it.

  1. Build the solution and launch the project in the Unreal Editor to see it in action.

If you don't have the output log open you can open it from the Unreal Editor menu bar, Window -> Developer Tools -> Output Log

  1. Filter the output log to only show the LogTemp logs

Screenshot of the Unreal Engine output log showing the message we wrote in the above code, "who's a good boy? Bork Bork is"

Tah-dah!

It's at this point in the story where most guides end and that irks me because we've only told half the story. Sure it is technically working, but in my mind you can't say something is useful without actually using it. So we are going to go the last few steps and wrap our protobuf into an asset, and then write some blueprint code that can use those assets. That said, if you are already comfortable in assets and blueprints in Unreal this is a good time to stop.

Using your protobuf

This last few bits are about creating some wrapper code to make using the protobuf a little bit easier. We'll be wrapping the protobuf Kennel into an Unreal asset class, making a factory for it so we can have drag and drop support in the content browser, and writing a blueprint that uses the asset.

Making an asset

To get started we need an asset which will wrap around our protobuf datatype. If we wanted to we could work our way through the protobuf data at instantiation time and convert it into a raw C++ object but this feels unnecessary to me, protobuf is pretty lightweight. The advantage to doing this though is you won't need to write as many wrappers to get access to the data within. If you want to do this however a lot of the steps will be the same.

All of this section is inspired and in some parts based on the TextAsset example asset project, available under the BSD 3-Clause license. You should check it out if you are interested in assets in UE4.

  1. Create a new C++ Class from within the Unreal Editor

Right click on the content browser and select New C++ Class from the menu that appears.

Screenshot of the Unreal Engine showing the create a new class menu item

Once that is a done the new class wizard menu will appear.

  1. Select the Show All Classes toggle
  2. Select the Object class at the top
  3. Click Next

Screenshot of the Unreal Engine showing the new c++ class menu wizard

  1. Name the class KennelAsset
  2. Choose the ExamplePlugin (Runtime) to make sure it goes into our module and not the game itself
  3. Select the Public toggle
  4. Click Create Class

This will create two new files called KennelAsset.h and KennelAsset.cpp, the header file will be inside the ExamplePlugin\Public folder, the other inside the ExamplePlugin\Private folder.

  1. Open KennelAsset.h
  2. Include the protobuf header file:
#include "Doggo.pb.h"

For I'm sure what are good reasons, but are very annoying to me because I keep forgetting, you need to make sure that the last include is the KennelAsset.generated.h UE4 generated file. I tend to just put some space around that include so I can add others above it as needed.

  1. Modify the UCLASS() macro to add in support for blueprints:
UCLASS(BlueprintType, hidecategories = (Object))

I'm still a little unsure exactly what this is doing, but you need it so you can use the asset in blueprints which is our end goal, so in it goes! Next we need to work out what features we want our asset to have so we can start implementing them. For this example we'll keep it pretty simple, we will want to have an array of the dogs names who are in the kennel. We'll also need a variable to hold onto our protobuf data which contains this information.

  1. Add the following code to the UKennelAsset class definition:
public:
    Kennel kennelData;
    TArray<FString> goodBoys();

The last thing we need is we will have to create a parse method so that when we've hooked up drop and drag support it has a method to call to deserialise the protobuf. This method will return a bool indicating the success or failure of the parsing.

bool parse(const uint8*& Buffer, size_t length);

The finished header for our asset will look like the following:

#pragma once

#include "CoreMinimal.h"
#include "UObject/NoExportTypes.h"
#include "Doggo.pb.h"
#include "KennelAsset.generated.h"

UCLASS(BlueprintType, hidecategories = (Object))
class EXAMPLEPLUGIN_API UKennelAsset : public UObject
{
    GENERATED_BODY()

public:
    Kennel kennelData;
    bool parse(const uint8*& Buffer, size_t length);
    TArray<FString> goodBoys();
};

Implementing the asset

Time to start implementing our asset.

  1. Open KennelAsset.cpp
  2. Create the goodBoys method implementation:
TArray<FString> UKennelAsset::goodBoys()
{
    TArray<FString> nodes;

    for (auto& dog : kennelData.goodboys())
    {
        nodes.Emplace(FString(dog.name().c_str()));
    }

    return nodes;
}

Here we are looping through all of the dogs inside the kennel, getting their name, and adding that name to a TArray which we then return at the end.

  1. Create the parse method implementation
bool UKennelAsset::parse(const uint8*& Buffer, size_t length)
{
    if (kennelData.ParseFromArray(Buffer, length))
    {
        UE_LOG(LogTemp, Warning, TEXT("Kennel parsing success"));
        return true;
    }
    else
    {
        UE_LOG(LogTemp, Warning, TEXT("Kennel parsing failed"));
        return false;
    }
}

This method is deceptively simple because protobuf itself is doing all the actual heavy lifting. There is a built in method in protobuf called ParseFromArray which takes in a buffer and the length of the buffer and can then convert that into a valid datatype. If this method fails it returns false, we are using this so that we can then return if this worked or not. I've also chucked in a bit of logging because during development you want to know immediately if your parsing has failed to help track down bugs.

Where does the buffer come from? From the factory we are about to write.

Making a drag and drop factory

Unreal requires you to make a factory to add in drop and drag support but there's a bit of an issue with this, the base factory objects we will need to inherit from are all part of the Unreal editor classes. You can't mix and match runtime modules (like our ExamplePlugin) and editor modules, so we are going to have to make a new module. A special shoutout to Alex Stevens for pointing this out to me, this was a source of a lot of confusion that he immediately cleared up. Turns out people who know more than you are useful to know, who knew? We'll be building this module in a similar fashion to how we made our protobuf module.

  1. Inside of the Sources folder for our plugin create a new folder called ExamplePluginEditor

In my case this means the folder I created went in D:\Development\Unreal\ProtobufTest\Plugins\ExamplePlugin\Source.

  1. Inside of the ExamplePluginEditor folder create two new folders, Private and Public.

These will hold our code for the module.

  1. Inside of the Private folder create a new folder called Factories

This folder will hold the code for our factory.

  1. Inside of the ExamplePluginEditor folder create a new text file called ExamplePluginEditor.Build.cs
  2. Add the following code to the ExamplePluginEditor.Build.cs file:
using UnrealBuildTool;

public class ExamplePluginEditor : ModuleRules
{
    public ExamplePluginEditor(ReadOnlyTargetRules Target) : base(Target)
    {
        PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs;

        DynamicallyLoadedModuleNames.AddRange(
            new string[] {
                "AssetTools",
                "MainFrame",
            });

        PrivateIncludePaths.AddRange(
            new string[] {
                "ExamplePluginEditor/Private",
                "ExamplePluginEditor/Private/Factories",
            });

        PrivateDependencyModuleNames.AddRange(
            new string[] {
                "Core",
                "CoreUObject",
                "Engine",
                "ExamplePlugin",
                "UnrealEd",
                "protobuf"
            });

        PrivateIncludePathModuleNames.AddRange(
            new string[] {
                "AssetTools",
                "UnrealEd",
            });
    }
}

This is a build file much like what we created earlier for our protobuf module, the difference is here we are including a bunch of different libraries and includes because we need access to the editor functionality instead of the runtime. That said, we are also including protobuf and ExamplePlugin because we will need access to their functionality. One thing that is different from before is that inside of our PrivateIncludePaths we are including folders based on a hierarchy. We don't need to go to this level considering we've only got a single factory in here but because this code is based on my code for YarnSpinner where I do want that level of structure we get it here too.

  1. Inside of the Factories folder create two new files KennelAssetFactory.h and KennelAssetFactory.cpp
  2. Open KennelAssetFactory.h and add the following code:
#pragma once

#include "Factories/Factory.h"
#include "UObject/ObjectMacros.h"
#include "KennelAssetFactory.generated.h"

UCLASS(hidecategories=Object)
class UKennelAssetFactory : public UFactory
{
    GENERATED_UCLASS_BODY()

public:

    virtual UObject * FactoryCreateBinary
    (
        UClass * InClass,
        UObject * InParent,
        FName InName,
        EObjectFlags Flags,
        UObject * Context,
        const TCHAR * Type,
        const uint8 *& Buffer,
        const uint8 * BufferEnd,
        FFeedbackContext * Warn,
        bool & bOutOperationCanceled
    ) override;
};

Here we are declaring that we will be making a new subclass of UFactory and will be overriding one factory, FactoryCreateBinary which will be called by Unreal editor when a file of the approriate type is dragged into the content browser. There are a few different FactoryCreate methods to choose from, and I knew I'd need to choose one that provides a binary buffer but sadly the docs for most of these methods they aren't exactly well documented.

Screenshot of the FactoryCreateBinary method showing eight of the ten parameters are without description

Nonetheless this one appears to be the one we want because, if I'm understanding the parameters correctly, we can set the bool input to false if the protobuf parsing fails. I could be wrong about this though, if someone's got more insight into this, let me know.

  1. Open KennelAssetFactory.cpp and add the following code:
#include "KennelAssetFactory.h"
#include "Containers/UnrealString.h"
#include "KennelAsset.h"

UKennelAssetFactory::UKennelAssetFactory( const FObjectInitializer& ObjectInitializer ) : Super(ObjectInitializer)
{
    Formats.Add(FString(TEXT("knl;")) + NSLOCTEXT("UKennelAssetFactory", "FormatKennel", "Compiled Kennel File").ToString());
    SupportedClass = UKennelAsset::StaticClass();
    bCreateNew = false;
    bEditorImport = true;
}

This constructor adds in support for the specific file formats and set up the UKennelAsset class is what this factory will be creating instances of. Specifically we are telling Unreal Editor that anything that has the .knl file extension will be connected into this factory. This isn't an existing format, just one that I've made up that will be used for storing serialised protobuf Kennels.

  1. Add the following code to KennelAssetFactory.cpp:
UObject* UKennelAssetFactory::FactoryCreateBinary(UClass* InClass, UObject* InParent, FName InName, EObjectFlags Flags, UObject* Context, const TCHAR* Type, const uint8*& Buffer, const uint8* BufferEnd, FFeedbackContext * Warn, bool & bOutOperationCanceled)
{
    UKennelAsset* asset = nullptr;

    asset = NewObject<UKennelAsset>(InParent, InClass, InName, Flags);
    bool success = asset->parse(Buffer, BufferEnd - Buffer);

    if (success == true)
    {
        bOutOperationCanceled = false;
    }
    else
    {
        asset = nullptr;
        bOutOperationCanceled = true;
    }

    return asset;
}

This is the real meat of the factory. We create a new UKennelAsset and then call the parse method on it. We pass into the parse the Buffer parameter from the method call. Assuming it parses correctly we return the newly instantiated kennel.

With that done we're almost finished with our factory and asset, we just need to do some work to hook it up into our module and plugin.

  1. Inside the Public folder of the ExamplePluginEditor folder create a new file named ExamplePluginEditor.h
  2. Inside the Private folder of the ExamplePluginEditor folder create a new file named ExamplePluginEditor.cpp

These are the module files for our ExamplePluginEditor module. They'll be almost identical to the ones Unreal editor created for the ExamplePlugin module just with a different name.

  1. Add the following code to the ExamplePluginEditor.h file:
#pragma once

#include "CoreMinimal.h"
#include "Modules/ModuleManager.h"

class FExamplePluginEditorModule : public IModuleInterface
{
public:
    virtual void StartupModule() override;
    virtual void ShutdownModule() override;
};
  1. Add the following code to the ExamplePluginEditor.cpp file:
#include "ExamplePluginEditor.h"

#define LOCTEXT_NAMESPACE "FExamplePluginEditorModule"

void FExamplePluginEditorModule::StartupModule() {}

void FExamplePluginEditorModule::ShutdownModule() {}

#undef LOCTEXT_NAMESPACE

IMPLEMENT_MODULE(FExamplePluginEditorModule, ExamplePluginEditor)

With those created all we've left is to associate our new module with our plugin.

  1. Open the ExamplePlugin.uplugin file

This will be located in the ProtobufTestProject\Plugins\ExamplePlugins folder and is the file that is responsible for controlling how the plugin works. Despite it's file type it is just JSON and can be edited with any old text editor of your choice. So far we've been fine to use the defaults that Unreal created way back when we first made the plugin, but now we need to modify it.

  1. Add the following object to the "Modules" array:
{
    "Name": "ExamplePluginEditor",
    "Type": "Editor",
    "LoadingPhase": "Default"
}

This is telling Unreal that our plugin to use two modules, ExamplePlugin and ExamplePluginEditor, and that one of them is a runtime module and the other an editor module. Now we're ready to test out our code.

  1. Open the project in Unreal editor

It will have to rebuild the plugin but once that is done we can now drag in a .knl file into the content browser and just like that we've got working assets.

Screenshot of the Unreal editor content browser showing three Kernel assets

Now creating these files isn't too tricky because we've got the protobuf schema, but going through the steps to do that is a bit excessive in this already mammoth blog. This is left, as they say, as an exercise for the reader. Instead I've created a few already made a few you can download from the github repo.

Using our assets in Blueprints

The final step is to integrate our new protobuf asset into some blueprints. We are going to be doing this by creating a standalone blueprint library, because this is the easiest way to get it up and running.

  1. If not still open, open the project inside Unreal
  2. Inside Unreal editor create a new C++ class
  3. From the Add C++ Class wizard select Blueprint Function Library
  4. Click Next

Screenshot of the Unreal editor New Class Wizard

  1. Name the class KennelTestLibrary
  2. Select ExamplePlugin (Runtime) from the drop down
  3. Click Create Class

Screenshot of the Unreal editor New Class Wizard

Once this has been created it will open up inside Visual Studio and we can create our blueprint.

  1. Open KennelTestLibrary.h
  2. Include the KennelAsset.h header:
#include "KennelAsset.h"

As with our early includes this needs to go above the include of the generated header or else it won't work.

  1. Inside the class definition add the following code:
UFUNCTION(BlueprintCallable, meta = (DisplayName = "Lists the names of dogs inside a kennel", Keywords = "Kennel doggo list"), Category = "KennelTesting")
static TArray<FString> KennelDoggoList(UKennelAsset* kennel);

This is declaring a new function that will return an array of strings, each string will be the name of a Doggo inside of the Kennel. The UFUNCTION macro with the BlueprintCallable flag set lets us declare that the function is to be presented as a Blueprint.

  1. Open KennelTestLibrary.cpp
  2. Add the following code:
TArray<FString> UKennelTestLibrary::KennelDoggoList(UKennelAsset * kennel)
{
    return kennel->goodBoys();
}

Here we are returning the array of names from inside the UKennelAsset, since we've already set everything up in the asset itself there is very little to do here. Now we can start testing this out. We will be making it so that an Actor will show on the screen the list of dog names inside of a kennel when the Enter key is pressed.

  1. Build the project in Visual Studio
  2. Go back into Unreal editor
  3. Create a new actor, I'll be using a cube for this but you can use anything you feel like.
  4. Add a new blueprint to the cube
  5. Name the blueprint KennelTestBlueprint
  6. Save it in the project Content folder and not the plugin's

Screenshot of creating a new blueprint

  1. Inside the Blueprint editor add a new variable
  2. Name the variable Kennel
  3. Set the variable to be Instance Editable
  4. Save this and open up the Event Graph
  5. From the Event BeginPlay node drag off and connect up the Enable Input node
  6. Add in a Get Player Controller node
  7. Set the Player Index to 0
  8. Connect the return value of the Get Player Contoller Node up to the Player Controller input of the Enable Input node

This will allow our cube to receive input events. We don't have to do it this way but this is probably the easiest way to get input working on non-player controllers. If you have a player controller that might be a better place. The blueprint should look like the following:

Screenshot of the blueprint setting up control

  1. Create an Enter event node
  2. Drag out from the Pressed Exec pin and connect up our Lists the names of dogs inside a kennel node

This is the blueprint function we created earlier. It won't necessarily be easy to find as the list of nodes gets pretty cluttered, the easiest way is to just search for the word kennel.

  1. Drag our Kennel variable into the graph
  2. Choose the get option
  3. Connect the variable node up as in the input into our Lists the names of dogs inside a kennel node
  4. Drag off from the Lists the names of dogs inside a kennel node and connect a For Each Loop node
  5. Connect the Return Value of our Lists the names of dogs inside a kennel node as the Array input for the loop
  6. From the Loop Body exec pin drag off and connect a Print String node
  7. Connect the Array Element value up to the In String of the Print String node

This will loop through all the dogs inside our kennel and print their name onto the screen. The blueprint should look like the following:

Screenshot of the blueprint for logging out dog data

  1. Save and compile the blueprint
  2. If you've not already done so drag a .knl file into the content browser
  3. Select the cube and in the Details connect the kennel asset into the cube's Kennel variable slot.

Screenshot of the details panel with a Kennel asset variable set

  1. Play the project
  2. Press enter and enjoy the sight of dog names appearing on your screen

Screenshot of the Unreal editor showing the dog data from our Kennel, Sean, Jon, Susie, Rachel, Jimmy

and just like that we're done!

The Future

So this seems to be working, or at least working well enough for me currently. The next step with this project is to try and do a build of Unreal out to a standalone game and see if it all still works. I wouldn't be surprised if it turns out there is some sort of magic in builds that means what I have done doesn't work correctly. Once builds are working the next trick will be to get it working for macOS.

Anyways, back to porting Yarn Spinner I guess.

OSCON and Write the Docs Adventuring

So I may have slipped up on the whole write one blog post a month minimum promise I made to myself, but I'll make up for it, I swear.

So this last month I have been in the US for a slew of conferences, mostly OSCON and Write the Docs and I thought might as well write a blog about that. Let's get cracking.

OSCON

My fabulous new OSCON Speaker Pants!

My fabulous new OSCON Speaker Pants!

If you've ever read any of my other posts about OSCON I think it will be obvious that I really enjoy OSCON, and for good reason. It is a crazy ecletic mix of tech, community, clever people, and parties.
This year it moved from Portland (my favourite US city) to Austin, now I had never been to Austin before so I was kinda looking forward to the move.
After spending a week in Austin I still consider Portland to be a nicer place but Austin is pretty darn snazzy, in the spirit of overanalysing everything for future argumentative goals let's break down the Austin OSCON vs the Portland OSCON:

Public Transport

Sorry Austin, Portland crushes you like an egg here, a tiny teensy pathetic weakling egg. Your public transport options are shameful, Hobart should not have better public transport than you! Now while you don't control Uber and Lyft pulling out that only made EVERYTHING worse. If I were in charge of OSCON I would have to be reconsidering going back to Austin on the public transport alone.

Conference Centre Layout

Again the point goes to Portland. The ACC is actually nice and large and had plenty of space for OSCON but the design of the building is a big square you move along the outside of. This meant you had to walk rather long distances with no easy way to cut through the middle. The OCC on the other hand (or at least the part OSCON uses) is roughly a horse shoe shape with paths through the middle. The practical upshot of all this is it took less time moving between rooms in the OCC compared to the ACC.

Conference Centre Food

Ok Austin wins this one, while conference food is still conference food the Austin lunches were just that cut above what the OCC ever provided. The Mexican lunch food and speaker lounge breakfasts were the biggest differences. I am a firm believer in that one of the things that makes OSCON a good conference is that the majority of the attendees can sit down and eat lunch together, the ACC did a better job of this than the OCC.

Err lunch..?

Err lunch..?

Weather

Another point to Portland, I like the cold and the occasional drop of rain, Austin on the other hand was mostly muggy and warm (it doesn't deserve the moniker hot because the roads didn't start melting). This one is far more subjective than the rest though so eh...

Wifi

Hands down Austin, and this one wasn't even a challenge, the ACC wifi held up like a champ never faltering the whole time I was there, something I could never say about the OCC. I would guess this has something to do with ACC also being where they host the abhorrent looking SXSW.

After parties

I'd say this one was a draw, some were better at Austin and others at Portland. Austin probably has nicer bars, in that the staff was a little bit more attentive and the layouts just a bit bigger, but on the other hand Portland has better drinks.

The Content

The talks were probably the single largest difference between the years gone past and this year and that has nothing to do with Austin (at least I don't think it does). Changing the conference from five days to four meant there were less talks than in the past.
From my perspective (which is both biased and probably wrong) the talks that got dropped were what I would call the nerd talks, those silly weird talks that realistically have nothing to do with the core of OSCON (open source and the world surrounding that). These talks were ones that argued maybe we shouldn't teach programming in schools, or how to make a computer for $9. Most of the talks I submitted to OSCON were these sort of talks. Sadly without these useless talks, OSCON feel a little less awesome this year compared to the past. A bit more dull, and lacking that little bit of weird that I really like about OSCON.

With that said, I will still keep submitting nerd talks and I encourage others to do as well, they still have a place and I am pretty sure the talk commitee will come back around to seeing it my way. Other than the lack of nerd talks the talks this year were their regular solid and entertaining selves.

Write the Docs

Snazzy looking badge

Snazzy looking badge

So the other conference I went to was Write the Docs NA, a conference for technical writers (the so-called documentarians, a term I find silly simply because you don't actually need to give everything a name, let alone a silly one). Now I am not a technical writer, I write technical books but the skillset required for that is quite a bit different from one who writes documentation meant to be used constantly by thousands.
Nonetheless their program committee accepted mine and Paris' talk on what we called Interactive Documentation Environments, basically live coding environments that support textual markup and supporting notes, we made the argument that these sort of tools are the future of technical writing.

Now if you are interested in the talk the video of it is here and the slides here (as an aside we made the slides in Deckset which is a pretty damn awesome tool, kinda like reveal.js but not ugly), based on the comments on twitter and in the fleshscape it was well received but I want to talk about the rest of the conference and not my part in it.

Write the docs is easily one the best run small conferences I have ever attended, a single track with an adjacent unconference, and long transitions gave it quite a personal feeling. The talks were all really high quality and felt like they had been rehearsed to within a bee's dick of perfection. There were minimal uses of shibboleths so I always felt like I was in the same group as those around me. The conference had a venue tour, a bingo card, and helpful little posters strewn about the place to make it easier to participate and to feel like you belong.

Marvellous stuff

The venue itself was interesting and I am still in two minds about it, The Crystal Ballroom is primarily a dance and theatre hall, this resulted in two interesting properties:

  • the acoustics really worked great when the speaker was talking
  • the room echoed a lot when everyone was speaking

This meant that while talks were on it was a lovely audible experience, but during speaker changeover and during breaks the room got loud and loud fast.
The other strange thing was the floating dance floor which was a very unique experience for a conference, did make walking around the venue a lot more interesting than a normal floor.

Slightly weird venue aside, the conference truly was excellent and the community and organisers almost scarily friendly, I will definiately try to come back next year.

Righty-oh so that is my quick roundup of the two conferences I've been at this month, I'll try and blog more reguarly, I am working on a few posts I hope to have finished up soon and will post them when I get them done but I do own Battleborn and haven't had any chance to play it while traveling so that will probably take priority.

GDC 2016 - Adventure, Mystery, Intrigue, Games!

So I told myself I was going to write a blog post at least once a month and I left it pretty late this month, but here we go, time for a GDC post-mortem.

Last week I went to the Game Developers Conference in San Francisco. I have wanted to go to GDC for many years but have never managed to quite scrape up the funds. This year was no different and I only got to go due to the kindness of others, but still I got to go! Woo!

I have to turbo-thank the GDAA for giving me a scholarship which purchased the all access pass to the conference, I literally could not have gone without this and I will be eternally thankful!

If you are involved in game development and are an Australian, you want to know the GDAA, they do great work and are pretty much the only voice trying to make game development in Australia more viable.

Thanks!

To GDC!

For those of you who read my blog but don't know what GDC is (I assume there is one of you...), GDC is the Game Developers Conference held annually in San Francisco. It is the largest and most eclectic conference for game developers because it encompasses every stage and role of game development from design, to writing, to programming, to art, to audio, to marketing, and everything else I have missed. GDC also intermingles AAA devs and indies, the person sitting next to you at lunch could be working on a game worth hundreds of millions of dollars, or something they are cooking up alone in their spare time.

This mix of roles leads to what is easily the most enthusasitc bunch of people at a large conference I have ever met. Everyone at GDC was looking forward to making games and being a part of the games industry. The reason for this enthusiasm is, I think, due to the mix of people you meet. As you roam around GDC you will run into people who work in a totally different fields, and they will happily discuss with you their latest song they are composing or the character design they just finished up. Even if you hate your job and the part you play in the giant game making machine, being exposed to new yet familiar information and topics helps keep you enthused for the end result. The general atmosphpere of GDC bodes well for both the future of the games industry and the games it will produce.

This isn't to say that at every other large conference I attend all the attendees are morose dullards who sit around discussing nihilism waiting for the day to end while hiding from the light, but no other conference anywhere near the size of GDC is so damn enthusiastic.

Another curiosity I noticed at GDC from other large conferences I have been to was when it came to question time at the end of talks. The questions asked were always on-topic and well thought out, there were no "I don't have a question, just a comment" or questions where someone was asking just to make themselves look good to the audience. This came as a pleasant surprise to me, as I have to come to associate question time at large conferences with the moment arseholes decide to make themselves known. Now this might be just because the talks I went to happened to be mostly arsehole free but it could also be for whatever reason GDC has a lower percentage of arseholes than other large conferences.

I have two competing theories as to why the arsehole ratio is so low at GDC. The first is related to the Dunning-Kruger effect, and that because game development is pretty damn brutal, requiring insane levels of cooperation and technical expertise (especially in the AAA industry but also true in the indie scene) people rapidly get over the "I AM A GENIUS" stage and rapidly into the "I know nothing" stage and so aren't as willing to make themselves look arrogant in front of a crowd of people.

The second theory has to do with game development's differences from other technical fields. Because game development needs people from so many different fields it is going to rapidly fill up with people of various skill sets and ideas about how games should be made. This blend of people and ideas is very likely going to either force arseholes to either adapt and lose a bit of arseholery or leave the industry. Either way it means the arsehole to not arsehole ratio is rather low in comparison to other large technical conferences I have been a part of (especially programming language conferences).

Another interesting side-effect of all this low arsehole ratio is that everyone I spoke to at GDC assumed I knew what I was doing (despite my business card literally saying otherwise...) and I assumed that they knew what they were doing. The low arseholery meant that you could have a discussion between a luminary of the field and a new developer and everyone involved was taken seriously.

This is something that other conferences I go to NEEDS to work on and I am not sure how to improve this yet, but I feel that GDC with its seeing luminaries as just another dev as a good starting point; hero worship is probably dangerous to a community in the long run.

Sessions and the lure of the Vault

Ok so enough about the people, what about the talks? GDC is interesting in that it is currently the only conference I attend where I am not either an organiser or a speaker, it was also my first time going so I wanted to go to as many talks as possible (something which I doubt will change should I manage to go again). I also knew from previous conferences that trying to go to everything will burn me out super fast, so I did generally pace myself.

Something that I really like about GDC, and other conference also do this, is that by buying an all access pass you also get access to the GDC vault, which is an online collection of this and past years talks. This had an interesting impact on what I went to see, when there was more than one talk I wanted to see on at the same time I tended to prioritise the one which I felt would be better live. As such I went to talks which were audio talks (giant speakers trump headphones any day), workshops (gotta get that human touch), and story sessions (personal impact).

In general the talks were stupidly high quality, every session I went to was well rehearsed, on time, and confidently presented. In the past I'd heard that GDC has a very rigorous review process and it seems to have paid off.

My favourite sessions were the Diablo post-mortem which went delved into the creation of Diablo including showing us the original design doc and had a guilty developer paying for his copy of Diablo he pirated years ago. The Lara Croft Go post-mortem was also brilliant, breaking down the process of distilling an action-adventure game into a turn based mobile app without losing the bit that made the game a Lara Croft game. Although this is only my first year I know already that I am going to go to as many post-mortems as I can in the future should I get to go back.

The session on the audio and music of Star Wars Battlefront was also another favourite, the amount of work that went into making the game audio and new music sound star-warsey was staggering. The game also won a whole bunch of audio awards at GDC, which it totally deserves, the game sounds bloody amazing.

There was a rather brutal but entertaining session on song composition where composers could submit songs to be judged by game audio pros. It was basically if Australian Idol was good, but wow were the judges brutal at times.

On the first two days I went to the game design workshop, which while it took up the first two days of the conference was totally worth it. It covered game design through discussion, paper prototyping, idea distillation, team work, mini-games, and optional micro-sessions. Losing two days was an acceptable trade off. The workshop was not only scarily informative and useful to help you think about game design but because it is all done in teams it is a great way to meet people outside of the lunch time and pre-talk queues. I think it was telling that I met six people who had done the workshop multiple times. If you ever go to GDC, even if you are not interested in game design, do the workshop!

Finally the awards, including the game audio network guild, ceremonies were surprisingly fun. Even with Her Story basically crushing all competition it was weirdly enjoyable to figure out what category Her Story was going to win next. The audio awards were a little quieter and more niche than the main awards, but also great and worth attending. The audio people are arguably the happiest of everyone at GDC, not sure why but it seemed that way. Also Battlefront won a tonne of awards and I really like the sound of that game so... yeah, I was happy about that.

Awards night

A nice little feature about GDC was that you get scanned as you enter each session, this is then used to immediately email you asking for feedback when the session ends. This resulted in me offering up feedback a lot snappier than I do at most conferences.

Party, party, party

Ah parties, what would a giant conference be without a slew of parties? I think every night I had invites for at least 3 parties. Of course not being a crazy almond I would only go to one per night.

Easily the best party was the Australian party, which I got access to by virtue of being Australian... it was not only full of people that I have actual chances of meeting again outside of GDC but it was in a great place and had a huge bar tab. Also they gave out cool portable batteries as loot. Easily the best loot I have gotten at a conference, cheers Film Victoria!

I am such a great photographer!

I didn't end up going to any of the mega-parties, I really don't see the point behind being in a giant sweaty, hotbox with music too loud to let you think and with too many people to possibly hold a conversation. Give me a few people to chat with any day.

Now it wouldn't be a GDC post-mortem without talking about the Microsoft party. Poor MS, one day I am sure you will get it right... So in case you don't know Microsoft through a party and had go-go dancers. The same Microsoft that earlier that day held a women in gaming event. I am pretty sure that I don't need to say anything about this that hasn't already been said by better writers than I, but for fucks sake MS, don't do this!

Thankfully all the parties I went to didn't have this problem, and as far as I can tell only the Microsoft party did, so yay!

The blob

I was travelling with rather a large number of people, there were eight of us roaming in a giant blob, this had some advantage and some disadvantages.

The big advantage is that it was really easy to vegetate with people you knew if you didn't want to socialise, you never had to worry about travelling alone, or having to make a decision about where to go for lunch. There was also always someone who you could split an uber with.

The big disadvantage is that a giant roaming blob is kinda scary looking from the outside, when all of us were together pretty much no-one we didn't already know was willing to join the group for a chat. To me a very large part of any conference is socialising, so this was certainly not ideal and something I'll have to work on if I ever come back.

Also with 8 of us, splitting an uber meant two cars minimum...

The meh of SF

Ah San Francisco... I don't really like SF much for myriad reasons, but I generally find the city to be a large sprawling, dirty, and very expensive mess. Heaps of the people in SF are vacuous start-up idiots who are annoying and there is a giant homeless problem everyone seems content to just ignore.

With all that said, SF was a lot nicer this time around, it rained a fair amount which I rather enjoy and it washed away the nigh-constant smell of urine and kept the annoying tech-bro's in their startup bunkers away from me. I suppose it isn't really fair to judge the entire city because of the annoying people in it, but SF does have lots of issues and for whatever reason everyone who lives there doesn't seem to want to fix it or even talk about it.

The city does have some really nice bars and restaurants. The coffee isn't too bad, but certainly no Yellow Bernard and once you get over the weirdness that is the city itself it does have a rather lot of charm. While I had been before I did a few of the more touristy things this time around and they were pretty good. The highlight had to be riding the cable car through the city while it was pelting down with rain, to me it felt kinda what Mad Max would be like if it was in the tropics and not a desert.

Toodles

Right, so I think that is everything I can think of about GDC and the trip around it. If you have any interest in game development whatsoever it is an amazing conference and well worth attending. Thanks again to the GDAA for buying me the ticket, you are all the greatest.

I will definitely be trying to come back next year, maybe I should work on a game to help fund the trip...

PhDs are tricky little buggers

So about a fortnight ago I submitted my PhD thesis for assessment, assuming all goes well it should be marked in a few months and if I didn't screw up terribly I should be graduating mid-year. This would make me Dr Tim, a strange concept. I can then fully stop thinking about it and move on to other thoughts, but for now I have to talk a little bit about some of the quirks I encountered while working on it.

This entire post could be summarised as 'despite people telling me it would be like this, I never expected it to be like this' which probably says more about me than the concept of a PhD but I noticed pretty much every other PhD student approach it in a similar manner.

The Writing

I have written lots before, I have written tech books, articles, papers, documentation etc etc etc so I wasn't really worried about the volume of writing I would have to do as I have (or thought I had) done all this before.

I was wrong.

Writing a PhD is very different from writing anything else I have done before, and the reason is because it is something new. Any time I have written something it has always had a structure and information provided elsewhere by someone or something else to help guide it as you write it. A PhD is by its very purpose a new piece of information. You as the author have to create the information you will use, you have to understand and interpret the information you generate, and you have to be the one to weave it together into something that makes sense.

The only guidance you have is past literature and even that you have to play around with and make it into something new to help tell your story.
This was all told to you when you start, but for whatever reason (arrogance in my case) everyone seems to ignore this and dive straight into the writing bog.

The Story

A PhD isn't just a dump of information, you don't just write down what you did and tah-dah you get given the doctorate (I wish they did though). You need to weave all of this into a story, you need to have a buildup and logical progression, the story needs to link backwards into what has been done and forwards into what you are doing.
Eventually you reach the end of the thesis and the reader should feel like you are closing the door you opened way back at the start.

This again was told to me in advance but I didn't really get this until near the end of the whole thing. I was at a stage where I had what I considered a content-complete thesis, essentially all the pieces were there. Reading through this revealed it was a piece of crap, there was no story, no weaving, no flow. It was bloody difficult to read and I spent months moving the different blocks around until I had something that made sense.

The Fraility of Your Squishy Brain

This is pretty much the only one that wasn't told to me in advance but one of the trickiest bits about the PhD was keeping it all in my head at once. When nearing the end and spending all time in writeup you are (or bloody well should be!) the most knowledgeable person in the entire world about your topic. There isn't really anyone you can turn to to ask specific questions because they just don't know the topic like you do. I tried this a few times before giving up simply because explaining the weird minutiae to them got tedious for everyone.

So essentially you need to keep everything to do with your research in your head, you need to remember the literature so you can argue what you are doing is valid and worthwhile and you need to remember your work and how it connects the gaps in the body of human knowledge. You also need to keep the flow in your mind so you don't repeat yourself or explain concepts well before or after they are needed.
I found that tools designed to help with this, generally things like Evernote or combination note and bibliogaphy mangers, were way too slow. At the start they were helpful but near the end switching between them and the document itself required too much context switching. I basically just left little notes and todos littering the document itself for things I couldn't remember.
I don't know if anyone else experienced these sort of issues around their work, but now I really question if any of these notes and memory tools are worth the effort required to use them. Switching back and forth takes too damn long.

The End?

So that is essentially all I wanted to talk about, the weird issues I had when working on my PhD.
Now that it is submitted and I've written this post I can safely stop thinking about it until the assessors come back.
Pass or fail, once the thing is marked, I will write up another post about how I used software dev tools to make working on the thing easier because I am pretty damn certain it did.

Tim goes to TasJam - He likes jam.

Does anyone else find the term jam kinda weird..?

Our valiant organisers. Photo by @parisba

Our valiant organisers. Photo by @parisba

So last weekend I attended the inaugural TasJam game hackathon. I normally consider myself part of the Tasmanian game developers general group but up until this year it hasn't really been anything but a few people who nod at each other when they meet up in the street.

Thankfully this is all starting to change and TasJam was a big part of it. I was pretty confident that TasJam was going to be me, Secret Lab, Giant Margarita, and maybe Kritz. When I got there I was pleasantly surprised to see a whole bunch of people I didn't know or only barely knew. I am glad that the little Tasmanian game group is growing and huge kudos to Jason and Ducky for arranging TasJam to help it continue to grow.

Let the games begin

TasJam was your fairly standard game hackathon, you show up, set up a laptop and make something a bit silly but lot's of fun. We were given a theme of Voices, with a secondary theme of Access.

I joined up with Matt and decided we should make something a bit silly. We are both programmers and were pretty confident we could smash something pretty cool in Unity but we felt like doing something a bit different. We both had the idea we wanted to make a choose your own adventure game.

After a fair amount of debate we settled on a murder mystery but with a twist, first you get to be both the detective and the suspects, and second everyone was going to be a butler so we could make a butler did it joke. Neither of us are writers and we knew what we were doing was a bit gutsy but we thought we'd be able to get plenty done.

Tied up in string

So we pretty quickly decided to use Twine because it seemed to be the go to tool for text adventures. Specifically we used Twine 2.0 because it is the new kid on the block, it looked pretty, so we gave it a shot.

That was when we ran into our first problem, we planned on using git to let us work independently of each other and then merge the changes in. Twine 2.0 does some very weird things, the save file it creates appears to be randomly changed every time you open the file. This makes merging impossible. In the end we had to use separate files and then manually merge them together. Even this was made trickier by Twine not letting you copy and paste individual passages from one story to another.

Our "finished" story in Twine

Our "finished" story in Twine

Ok so we had our little system ready to go and we started writing, the very first thing we learnt is that writing is really hard. We both knew we weren't writers, and we both didn't arrogantly assume every other skill in game development is easy like some programmers, but we totally overestimated how much we could write.

We chose eight characters, Butler, Priest, Femme Fatale, Duchess, Great White Hunter, Professor, Maid, and Doctor. We assumed we'd be able to get 4-6 of those done, we got 2...

I always knew writing was a specific skill, but I have even more respect for it now, still I hope the two characters we finished came out alright.

Finally we had a very clever idea to use a large matrix of possibilities to make the story dynamic, this is where we ran into more Twine issues, Twine 2.0 has terrible documentation. Matt and I are programmers so we are used to terrible documentation and how to get around it but I can't see writers digging into forums to find the answer to their problems. If Twine 2 ever wants to be more they have to fix their documentation.

This was where we also ran into another interesting side effect of Twine, it doesn't handle dynamic linking and stories very well. If your story is just straight forward, Option A, Option B, or Option C, Twine is a great choice, but if you want to dynamically build up the story it gets messy really quickly. As programmers we got frustrated by what were essentially nothing more than loading and retrieving data from arrays.

Which leads into arrays, in Twine they are called arrays, but then don't let you use the standard $varName[index] format to store/retrieve data for fear it will confuse people who aren't programmers. Which is fair enough, but then why call them arrays? List makes far more sense.

With all that said, Twine was a lot of fun, and totally worth everyone having a go at. Assuming they fix the documentation and iron out some of the oddity, it'll be an excellent tool.

Fellow Jamlords

Anyway, enough complaining about Twine, there were heaps of people at TasJam all doing really cool stuff. If you care to take a look, all the submissions are available at itch.io

I was particularly impressed with felius and John Dalton, a team of kids, made a side scrolling game about a music powered machine that saves the world.

There was also another bizarre entry from Kritz that I think was about the danger of mobs, but I am really not sure, it looked cool though.

Secret Lab made some sort of crazy single stick shooter featuring gnome copters and wizard hats, not really sure it fits the theme, but it was fun.

There was also a whole bunch of other games being made but I didn't get a chance to play them.

Bring on the scones

So, that is my terrible and quick blog post about TasJam, I cannot wait for the next one. If for some reason you want to play our game, take a look at ButlerConf

Actually take a look at all the games people made.

Huge thanks to the judges and mentors Kamina, Lauren, and Katie for giving up their weekend and flying down from Melbourne to judge and mentor us puzzled devs.

See you at the next TasJam!

Jumanji - The greatest board game ever?

So it turns out that Netflix has Jumanji on it, which of course makes the service even better than it already is, but it also offers up the opportunity to reexperience a movie I really loved as kid.

This blog post turned out to be a bit longer than I intended, I think I got carried away, but I think it is worth sticking with it and reading through the whole thing. If you want to skip below to the bottom I think the last section mostly makes sense on its own.

My brother was watching Jumanji on Friday when he pointed out that no one in the film actually dies, even if you ignore that the game resets time back to the 60s, no one dies even up until the reset point. Now as a child I don't remember noticing this and I doubt my brother did either when he saw it at the time of release but as an adult I now have the amazing ability to be overly critical, a feature I lacked as a child. So let's look at Jumanji as an overly critical and analytical adult.

Just looking at Jumanji with a bitter angle is hardly in the spirit of being overly analytic, an issue I have with many a critic - they just like to talk about the bad aspects plus I really like the film... So let's take a look at Jumanji, or rather let's look at the board game Jumanji inside of Jumanji itself. 

So I posit that Jumanji is one of the greatest, if not the greatest, game ever created. It creates and presents a brilliant survival adventurous gaming experience in a perfectly safe manner, all without having to go to an actual dangerous jungle.

To get started let's talk a little bit about what a game is so that we are all on the same page. A board game, like almost every game, is trying to create an enjoyable experience for you as the player, to make you feel powerful or wise or brave or any number of different feelings and moments - these are the experience that a game tries to capture and recreate when you play it. Many board games, like almost every computer game, is thematic. Thematic games create a tailored experience around a theme, so the game experience might be to make you feel wise or powerful but it does so by making you a brave warrior, or wizened old man, or a survivor of the apocalypse. These themes are often cliché but they do help a lot at creating an experience.

Jumanji is clearly a thematic game, you play a terrified and helpless person trapped in a wild and dangerous jungle that grows out of your very world. Jumanji wants to turn your dull everyday world into an adrenaline filled jungle survival adventure. You want to feel challenged and scared, but you ultimately want to overcome the challenges and be the one to defeat the dangerous jungle!

Ok so now we know what Jumanji wants let's talk about how the game actually plays, it is a remarkably simple game, a standard path based dice game such as Snakes and Ladders or Monopoly. The rules are:

  • Roll the two dice to see how far along the path you move
  • Rolling doubles gives you an additional turn (no idea if this has a limit)
  • At the end of your turn play moves to the next player
  • After each turn the game creates a special event that affects all players
  • First player to reach the centre and call out Jumanji wins

Now the first thing here is we don't know if the additional turn for rolling a double has a limit, like in monopoly, or can be done infinitely assuming you can keep rolling doubles so some errata here would be nice. The second thing (and the quite literal magic of the game) is in the special events the game creates at the end of each turn. A special event at the end of a turn is a very common pattern in games; Battlestar Galactica and Zombicide are two games that immediately spring to mind that have this pattern. Unlike BSG or Zombicide though which use cards to control the events Jumanji uses a large sphere in the board centre to show the special event and then changes the world to make the event come true. As the special events are what really makes the game, pretty much the rest of the blog is gonna be about them.

The games special events

So with the special events the obvious issue with them is it would make localising the game tricky, they rhyme and use cultural knowledge, so that is a lot of text that would have to be very carefully translated should the game ever hope to have a market outside of the anglosphere. The next issue is that the special events are really dangerous, or at least when I watched the show as a kid years ago they seem very dangerous. Even ignoring the fact that the game can control the world around it, it is the illusion of danger that is the real magic of Jumanji.

The events at first glance seem very dangerous, giant mosquitos, stampedes, monsoons, and psychopathic hunters aren't really high on my List Of Things To Get Killed By but as the game progresses in the film it is made clear that the danger is never real, even if the events themselves are. The game perfectly curates and controls the events to make sure that no one is in any real danger even with a crazy hunter or giant lion running around. Let's take a look at the dangers and events in the film as the game progresses:

So the first roll we get bats, now bats can be dangerous but aren't really a huge threat and all these bats do is chase Sarah away from the game and then do NOTHING ELSE FOR THE NEXT 26 YEARS! Real dangerous there Jumanji!

Second roll we have Alan getting sucked into the game, ok so at first glance this looks pretty bloody dangerous, he's a kid in a dangerous jungle. Maybe I'm wrong.

Third roll we have gigantic mosquitoes, now these put up a bit of a fight but soon leave to cause havoc elsewhere.

Forth roll is monkeys, which promptly leave the players alone to go annoy the general public, yep super scary!

Fifth roll gets Alan out of the jungle, presumably killed by all the horrific things that lurk within, oh no he is physically fine and well fed despite 26 years in a very hostile environment, ok... sure... We also get a large and dangerous lion who is soon locked inside a bedroom behind a door that he can easily bash down as we see him smash part way through it. But that is clearly too much effort, just give up lion and go to sleep.

By now I think you can start to see a pattern emerging, the game creates an illusion of danger with the events it creates but always manages to not quite hurt anyone. Let's keep going though and we'll see if this hold.

So Alan and the kids go outside to find the first player Sarah and we see our first case of actual damage, Mrs Thomas the realtor has been bitten by the mosquito and is in some sort of coma. 15 other people are also affected. Ok so maybe my theory is wrong but when we look closer the realtor seems ok other than the coma and far more importantly none of the players got hurt meaning they can keep playing.

Next roll we get killer bamboo with poison barbs that we don't get to see fire and a giant child eating pod, that shows itself in the room that has a cavalry sword right nearby to chop up the plant, nice spawn protection there Jumanji!

Ok next Alan rolls and out pops Van Pelt. I love Van Pelt because he is such a silly character, he is incredibly over the top with his outfit, his guns and his insults but he doesn't seem to be much of a hunter. His first shot is from a short distance and he misses, we'll forgive him though after all he did just get summoned into existence.

Second shot misses

Third shot misses

Forth shot misses

Fifth shot misses

Sixth shot misses

Seventh shot misses

He then proceeds to shoot at Carls car but doesn't hit him, Van Pelt then takes up aim and has the perfect shot at Alan only to be out of ammo, bugger.

Next roll gets us a Stampede that deals no damage to anyone but does mess up the house quite a bit.

We then have another instance of non-roll danger, Van Pelt has a near perfect shot of Alan when Carl is arresting him but he misses, again!

Next roll has Peter cheating and turned into a monkey thing, which while embarrassing doesn't appear to have caused any real harm.

We then see the monkeys start tearing apart an electronics store, but the owner runs out completely unharmed by the pesky little things.

Out of no where we have a stampeding elephant crush a car with Peter inside it, luckily he is perfectly unhurt.

Now inside a supermarket Van Pelt is firing madly at Sarah, three times he misses!

Van Pelt then manages the amazing, he shoots a tiny chain that was holding a bunch of tires in place, this traps three of the players, so the guy can shoot! He even tells Sarah, Peter, and Judy that he has been missing them simply because it was Alan he needs to kill. So I wonder why he's been missing him so much...

We then get a little more info on the mosquitoes, 98 people have been hospitalised some with violent seizures... ok this doesn't sound great but again no fatalities, it is almost like something is stopping anyone from dying.

Then Carl's car gets crushed by a vine, a vine that fails to grab him letting him escape unharmed.

Back to the rolls we get a monsoon with some crocodiles tossed in for good measure. Alan actually gets taken underwater by one and it starts its death roll, which it then stops and surfaces to let Alan get his breath back before starting this again. Silly crocodile!

After the water is flooded away Alan rolls quicksand which looks like it might kill him by smooshing his head under the floor boards...

Luckily Judy manages to freeze the quicksand with her next roll, which while trapping Alan and Sarah does stop them dying.

Oh no our next roll gets giant killer spiders, a mainstay of fantasy adventure! Killer giant spiders that don't take a bit out of Sarah's leg when they get the chance but instead all charge at Peter swinging an axe at them. The spiders ignore the helpless people stuck in wood to go after the only person with a weapon. Silly spiders!

Judy then gets hit by one of the poison barbs on the vines, Peter drops his axe to go to his sister freeing up the spiders to go for Alan and Sarah, things are looking bleak.

Luckily the next roll is an earthquake, freeing Sarah and Alan from the wood and making the spiders flee, geez that was awfully close.

Final roll

Ok so we are now at what will become the final roll of the game, Alan is so close he only needs a 3 to win!

Oh no!

Van Pelt shows up and is standing close enough that even he couldn't miss!

I actually really like this moment because Jonathan Hyde does a brilliant job of seeming both charming and menacing, and even has a dash of a father figure if he weren't about to shoot someone in the face...

So Van Pelt is standing there ready to shoot, first thing he does is get Alan to drop whatever it is in his hand, starts berating him before congratulating Alan on finally facing his fears while he lines up the perfect shot.

What was it that Alan dropped? Oh only the dice, which rolls enough to end the game, hooray everything is good and to make it better it was Van Pelt which helped to bring about the final roll!

Jumanji undoes everything it did, even rolling back time to the 60's, hooray happy ending!

Jumanji as a brilliant game

Ok so Jumanji is a movie about a game that is either impossible to exist or at the very likely years away from us being able to play it, so why do I think it is such a brilliant game?

Jumanji is a brilliant game because it truly captures the experience of being a survivalist adventurer in an unbelievably dangerous world, but it does this without putting people in danger. It is the ultimate thematic game, it creates the theme for real, there are psychopathic hunters, giant spiders and huge man eating vines, but you as the player are always safe.

The events the game chooses are carefully controlled and selected to make sure that they either prevent you getting hurt by an earlier event, such as the earthquake scaring away the spiders, or will not hurt you, such as Van Pelt missing his first seven shots at Alan or the stampede running past the players and not into them. You are never told this though, the game keeps you safe while making it seem dangerous. Playing Jumanji would be a brilliant adrenaline rush as you are constantly on the edge surrounded by danger yet perfectly safe, when it is doing its job properly you won't even realise it is doing anything at all.

Now you might be saying "woah Tim, people got hurt!" and yes they did, but no one died, the game kept everyone alive because it is a game, people play dangerous games all the time but no one wants to play a game that kills and Jumanji is no different. I would argue Jumanji has been shown to be safer than most contact sports, we saw 26 years of Jumanji and not a single person died because of the game.

I also am certain that Jumanji doesn't kill anyone even though it can reverse time simply because it is a game, as a player of the game you might be ok with the game being dangerous to you, even maybe causing a little bit of havoc outside of the game itself, but I don't think you as a player would be ok with a person dying. Even if Jumanji can undo the death at the end of the game, I cannot imagine how much damage that would do to the players, knowing they were temporarily responsible for people dying, let alone what it would do to you over repeated plays.

Jumanji also does something even more subtle than I think people realise, it ramped up the danger when it knew the game was ending, two rolls shy of the end it poisoned Judy knowing that either Sarah, Alan, or Peter would get to roll and either end the game or it would be able to create an event to get her back - we've already seen it undo moves such as quicksand.

The final tricky thing Jumanji does is optimise the danger away from the players, people did get hurt, damage was dealt, but not to the players, it kept them safe so that they could keep playing the game to eventually finish it. This explains how Alan survived so long in the jungle without help or training. Jumanji has to keep them safe so that they can finish it.

I think I'll end this with one final little note, if the game was actually dangerous why do people not destroy it? I think deep down inside people play it and realise it is safe, why else let someone else be put in its way if not to enjoy the thrill Jumanji has to offer?

Jumanji is a great game because the experience it offers is brilliantly presented, while we cannot play games like Jumanji yet we can try as game developers to make sure the experience of our games are as clearly presented. Jumanji might be impossible, but it makes a nice goal to reach for.

OSCON 2015 - A Tale of Two Tags

OSCON 2015 is over, rest in peace Portland OSCON, long live Austin OSCON.

So this post is my poor attempt at explaining what it was like at OSCON this year for me. For a start, I really like OSCON, pretty much everything about it from the crowds, fellow speakers, the organisers, and even the food and venue is brilliant, so with that in mind let's dump OSCON from my mind into this blog.

Farewell Portland, Hello Austin

Now unless you were at OSCON you might not have noticed that OSCON is moving from Portland (easily my favourite city in the US) to Austin, which I have never visited. Now the general response I got from the crowd was somewhere along the lines of "how horrible" which seems a little weird to me. The general reasoning seemed to boil down to "remember how terrible it was last time OSCON moved?", "Texas is so far away", and "I hate Texas."

All of these seem pretty weak excuses to me, from everything I've heard, when OSCON was in San Jose it was properly terrible, but this does not mean that OSCON has to be in Portland to be good.

The distance argument is really frustrating, I had to travel for ~30 hours (including all time in the airports) to get to OSCON, someone from the Bay Area complaining about 3+ hour travel is pretty hollow and even a little insulting, harden up and sit in a plane, the rest of the world does, time for you to do so.

The hatred of Texas isn't one I can answer, my knowledge of Texas comes from the TV and the few people I know who have visited it, now they have said the state isn't as nice as Oregon and that Austin isn't as nice as Portland. I have no reason to doubt their views, but at the same time they have also said that Austin is a really nice place - so I will reserve judgement until/if I get to go to OSCON next year.

Either way, OSCON is different every year and next year it will be different again and like almost anything that happens at OSCON, r0ml says it better than I ever could:

Every year I hear "#OSCON has changed, I should stop coming."  Every year I realize *I* have changed -- l still love OSCON. - r0ml

Excellent talks, Brilliant Speakers

As with any conference the entire point of attending is for the social interactions and the talks (well that and the expo, but that is a very different kettle of fish), and this year the talks I saw (when I wasn't busy freaking out working on my own slides) were amazing!

If you make time (and I highly suggest you do), you should check out:

As well as the talks I gave/co-presented:

Now probably the primary reason I go to conferences is for the interaction with the other attendees, and this year at OSCON the conversations were brilliant. Seriously you should come to OSCON just to speak to other people there, it is chock full of stupidly clever people!

OSCON vs Gamergate

Ah gamergate... so if you weren't at OSCON you probably didn't even hear but Gamergate got into into their head that OSCON were incorrect for accepting Randi Harper's talk offer. This section isn't going to be a discussion of Gamergate (henceforth referred to as Gobblegoat, because a Gamergate is a type of ant), ggautoblocker, or even free speech, this is a discussion of techniques.

So I don't really know how it all got started, but at some point during the conference Gobblegoat started flooding the oscon hashtag on twitter and mention spamming the twitter account and the  private twitter account of the person who runs it, Josh Simmons.

Now I don't know why it was decided that flooding a twitter hashtag and accounts was the correct way to make O'Reilly and OSCON change their minds about Randi Harper. The correct way is to buy a ticket and talk to the organisers and community there in person.

Now I know a ticket to OSCON is expensive, but there were numerous opportunities to get a free expo ticket which would have given you direct access to everyone who runs OSCON, including Tim O'Reilly and the conference chairs, and all the speakers (who don't really have much power) and attendees. A single person at the conference would have wielded infinitely more influence than every single tweet could have.

But even then, I understand not wanting to attend, but here is the problem, if Gobblegoat doesn't want to attend, then they are not part of the OSCON community and as such their words have no power, as someone at the conference, people were more confused and angry about the flooding. Because of the tweets people went and saw the talk by Randi Harper, people went and installed the blocking tool because they weren't blocking anyone in the community, they were blocking randoms.

People installed the tool and suddenly the noise flooding the conference was gone, how can you expect anything else? Oh as to the OSCON wifi filtering, I was in the speaker lounge when that was discovered and the OSCON staff were as confused as we all were, no one knew the wifi was being filtered.

So, ignoring the general ineffective nature of the spamming to the conference on the whole, the greatest reason the Gobblegoat spamming annoys me is because it was targeted to the wrong person. Josh Simmons did nothing that he wasn't asked to do, people asked for the blocker to be installed so people at the conference could more easily talk to the OSCON twitter account. Josh was just doing his job, and because of it he was attacked.

For everything we do, there is an internal discussion of "do my actions justify my goals" and in the case of targeting Josh, Gobblegoat either misunderstood, or misjudged the actions. As such all they did was harass someone doing his job and annoy a bunch of people at a conference who will now never join the Gobblegoat cause.

The End!

Ah, breathe out, right much more of a larger blog than I intended but it was fun writing it down. OSCON was a tonne of fun (metric not imperial) and I strongly recommend you come to OSCON Europe if you can, or Austin for OSCON 2016.

See you then, Tim.