How to Make Game Subtitles, Better

I couldn’t find any particularly official-looking stats, but it seems from forum polls and asking around, that the vast majority of players play with subtitles turned on. Also in most games that I can recall, subtitles are enabled by default.

There are a huge number of reasons why people play with subtitles enabled:

  • They don’t understand the audio language well enough to be comfortable. Either it’s not their native language, or characters use a dialect they’re not familiar with.
  • They have issues with their hearing, or the mixing is such that voices aren’t clear.
  • They play in a noisy environment.
  • They play in an environment where they can’t turn on the game sound (playing games while others are asleep).
  • They prefer having both sound and text to make sure they don’t miss anything.

So subtitles are a pretty big deal.

How do most games currently do subtitles?

In a word, badly.

Currently most games seem to base subtitles entirely on their corresponding audio cue. A cue is a logical chunk of sound, in our case a line of dialogue read by a voice actor. It could be a short phrase or an entire paragraph, depending on how the voice actor and audio engineer decided to split it up.

Then, in-game, the transcribed dialogue is displayed while the cue is being played.

On-screen, this audio cue could end up as one, two or three lines of text depending on the screen size and font size.

In most cases, the UI programmer or game designer picks a font size that’s kind of legible from a sofa, and calls it a day, focussing on other features instead.

The problem is that there’s a lot more to subtitles than this.

Let’s look at some examples:

▲ Witcher 3 and XCOM 2 are the worst offenders. Their text is tiny, the lines are ridiculously long. We can do a lot better than this.
▲ Dishonored 2 does a much better job, although as we'll see later splitting after an "and" is not ideal.
▲ Metal Gear Solid 5 isn't too bad, the font size and line length are good, but they sometimes have orphans like in the screen above. Their subtitles clash with in-game prompts which is less than ideal.
▲ The best example I've found is Uncharted. Their subtitles look hand-authored, and while they might be longer than what is traditionally used for TV and film, they're very easy to read.

So subtitles are a bigger deal than you might expect. With so many games containing hours of cutscenes and audio dialogue, it would make sense to do subtitles right.

When a beautiful cutscene is being shown on-screen, do we really want players to spend most of their time staring at tiny text at the bottom of the screen?

Are there any other industries we can learn from that might have already solved this problem?

Film and TV

Stating the obvious, but films and TV have dealt with subtitles for decades. Over time some best practices have evolved that all lead to one goal; make subtitles easy-to-read at a glance, so the viewer can concentrate on the scene.

The BBC has made their subtitle guidelines available on-line, and they’re a fantastic resource in general.

A lot of what I describe below is taken directly from the BBC subtitle guidelines. So What Should We Do?

Hopefully by now I’ve convinced you that good subtitles are important, that we can do a lot better, and that there are good examples of what to work towards.

It’s possible to programmatically produce much better subtitles by implementing a system that uses some fairly simple rules.

Note: All the guidelines below are just that &emdash; guidelines. There are always exceptions, and there are many cases where two rules will actively fight against each other. In the case of generating subtitles by hand, it would be up to the judgement of the author. In our case, we have to be at peace with the fact our system will sometimes produce less than ideal results.

Keep Lines Short

Extremely long lines become difficult to read when they stretch on and on without a break.
Extremely long lines become difficult to read
when they stretch on and on without a break.

This is the single most important thing you should care about when setting up your subtitles system. Everything else is just icing on the cake. There are so many studies out there for both print and on-screen media that show people have a hard time reading huge lines of text. In printed text I’ve seen 70-80 characters mentioned, and the BBC has extremely short recommended lines of only 37 characters for historical reasons, but 50-70 characters works well in my experience.

Put Line Breaks In Smart Places

After changing the line length, the next set of rules are all concerned with how split up lines and pages.

The BBC guidelines includes a list of places where you should avoid splitting words:

  • article and noun (e.g. the + table; a + book)
  • preposition and following phrase (e.g. on + the table; in + a way; about + his life)
  • conjunction and following phrase/clause (e.g. and + those books; but + I went there)
  • pronoun and verb (e.g. he + is; they + will come; it + comes)
  • parts of a complex verb (e.g. have + eaten; will + have + been + doing)

It’s important to note that all of the suggestions below are English-specific. The assets you require to implement the rules are also language-specific (a list of conjunctions, pronouns etc).

Let’s explain some of the rules with examples.

Avoid Orphans

Make sure there's not a word all
alone.
Make sure there's not a word
all alone.

Look at that poor word all alone on the last line. In typographical circles this is tastefully called an orphan. It’s more of an aesthetic choice but it does improve readability.

Orphans in typography usually refer to words on their own on a line, as in the example above. However for our purposes we can extend the definition to include single before punctuation, like below.

Even with two sentences there can be
problems. Just like this one.
Even with two sentences there can
be problems. Just like this one.

Avoid Article and Pronoun Splitting

She will travel to Oxford to buy an
automobile for her father.
She will travel to Oxford to buy
an automobile for her father.

Articles in English like a/an/the, pronouns like I/you/he/she, possessive pronouns like my/your/his/her etc. To implement this you will need a language-specific list of pronouns and articles. Using a simple list of pronoun + word will lead to false positives, but the alternative is performing grammatical analysis on the text, which seems like overkill.

We went to Canada to see her
grandmother for Christmas
We went to Canada to see
her grandmother for Christmas

Avoid Adjective Splitting

This requires a huge list of adjectives and adverbs to work in English, so it’s not always achieveable. But the BBC guidelines recommend that adjectives be kept with their subjects. In French this would be even more tricky as adjectives can go before or after the noun.

Today I went shopping and bought a beautiful
parrot from a peculiar boutique.
Today I went shopping and bought a
beautiful parrot from a peculiar boutique.
Today I went shopping and bought
a beautiful parrot from a peculiar boutique.

In the example above not only did we move beautiful onto the next line, but we applied the article-splitting rule to the subsequent result, and moved the article a down too.

Split on Punctuation

The most natural place to split a line or a page is just after punctuation like commas and full stops.

You know I can't go swimming today, it's
much too cold outside.
You know I can't go swimming today,
it's too cold outside.

This often works with the avoid orphans rule. In the example above, we can treat a single word after punctuation on a line as a kind of orphan, and move it down.

Splitting Lines Between Pages

So far we have focussed on splitting lines on a single page. What do we do when the line of dialogue runs over two lines?

Cues that run over two lines will have to be split into multiple pages (paginated). The problem then is you don’t have timing information to know when to switch pages to match the dialogue audio. In my experience, a simple percentage-based guess works surprisingly well. If the page split is at 65% of the dialogue, change pages once 65% of the audio has played. You can change the contribution of different characters to the length, for example increasing full stops and setting spaces to zero length but it doesn’t always help. A more robust alternative that works if you have the time or you haven’t started your audio yet, is to try to keep cues fairly short.

Picking the right place to split pages is almost the same as picking where to split lines, all of the same rules apply.

The BBC guidelines recommend using ellipsis to indicate that a sentence runs between pages.

Language-specific Rules

Sorry 94.5% of the world, I left talking about non-English rules to the end, and glomped it all together. I’m not much of a polyglot, so I can’t give many examples, but you should be aware in your system that some of your assumptions will be broken in other languages.

To give just a few examples that I’ve encountered:

French has spacing between punctuation and the word before. So you can’t use spaces as a foolproof way to split up words, or you’ll end up with punctuation on the next line.

Japanese, Chinese and some other languages don’t use spaces to separate words. There are still the logical concepts of words as grouped characters though. You can sometimes be lax about splitting in the middle of a word over two lines, but not between two pages where the reader would have to wait to see the rest of the word. You should talk to an expert on Japanese but to do word segmentation correctly you could check out tools like ChaSen, Mecab or Juman.

Conclusion

I think that subtitles systems often get implemented quite simply because people think “they’re just subtitles” and if text is displayed on-screen, that’s good enough. I hope these guidelines have shown that something much better is possible.

Further Reading

How to make a scrollable list in Unity's New GUI

This is more for my future reference than anything else.

You can either watch an hour-long tutorial. Or you can look at this list of steps. Because it’s way too complicated for any sane human being to memorize.

Here’s the hour-long tutorial:

The following tutorial will make a scrollable panel with the following features:

  • Vertical scroll
  • Buttons (or any other content) have their preferred sizes
  • The scrollable panel automatically scales to fit the content
  • The content is clipped/masked using an image mask.

Here is your GameObject tree structure with names. I will be referring back to this. Create blank objects with this tree structure.

Panel
    Background
    ScrollView
        ScrollContent
            Button0
            Button1
            (other content)

Details

Here’s what each object does, the components it needs and the settings required:

Panel Object

Panel is just the parent. Stuff it wherever and change its size however you like.

  • Rect Transform Component. No specific settings required.

Background Object

A background graphic for your scrollable area. It’s not required.

  • Rect Transform Component (added automatically). No specific settings required.
  • Canvas Renderer Component (added automatically). No specific settings required.
  • Image Component. No specific settings required.

ScrollView Object

Sets up how the content is viewed.

  • ScrollRect Component. Set the Content to ScrollContent.
  • Mask Component. No specific settings.
  • Canvas Renderer Component (added automatically).
  • Image Component. Set an image to set up a clipping mask for the content in the scroll view.

ScrollContent

Deals with layout.

  • Vertical Layout Group Component. Disable force child expand Height.
  • Content Size Fitter Component. Set vertical fit to “Preferred Size”.

Button0, Button1 etc.

  • Layout Element Component. Set up the Min Height to something sane like 25
  • Any components you want, labels, buttons whatever

And that’s all you need to do to set up a vertical scrollable list!

If you want scrollbars, good luck.

Retour en Europe

Hello everyone!

I will be leaving Japan in August, after 6 years of living here. I’ve made some amazing friends, worked in three different fields and had some wonderful experiences. But now it’s time to start a new chapter.

@petitegeek and I are moving to Paris!

By August I will have finished my current project, a Unity-based rhythm game for PlayStation Vita called IA/VT. I was responsible for UI, shaders and the music video director system (special effects and artist tools).

If you need a generalist programmer with experience shipping a game in Unity on PlayStation Vita, I will be available from August.

For more information my CV/resume is on my website and also I have a LinkedIn thing that’s cool with the kids these days.

Making UIs with Coroutines in Unity

Originally I thought coroutines in Unity were exclusively for performing asynchronous operations, like AssetBundles.LoadAsync(). However when I was planning the UI for my new game, the most wise Eddie and Kalin told me about using coroutines to simulate a state machine. It took me a while to understand how this worked conceptually, and then how to implement it. I couldn’t find any particularly good tutorials either.

This is more of a series of annotated snippets than a full tutorial, but hopefully it will be useful for someone.

The important thing to remember with this is that we’re emulating a finite state machine. If you’re unfamiliar with FSMs, read up on them before starting this and sketch out a few FSMs for games you understand, thinking about what possible transitions there are between states, and what setup/teardown each state does.

With that out of the way, let’s look at some snippets!

Template Snippet

The first snippet is a template of what we’ll be doing with our UI-related coroutines. There’s not much to say about it beyond what’s in the comments.

public class UITest1 : MonoBehaviour {
    // Start can be a coroutine
    IEnumerator Start() {
        // Do initial setup

        while (true) {
            // Check states and jump in if conditions are met

            // Make sure to yield within the while loop
            yield return null;
        }
    }
}

Basic Pause Menu

Our second snippet has some code that actually does something useful.

public class UITest2 : MonoBehaviour {
    // Make private member variable visible in the inspector
    [SerializeField]
    GameObject m_pauseMenu;

    IEnumerator Start() {
        while (true) {
            // Check the condition to enter the state
            if (Input.GetKeyDown(KeyCode.Escape)) {

                // Enter the state
                yield return StartCoroutine(DoShowPause());

            }

            yield return null;
        }
    }


    IEnumerator DoShowPause() {
        // Initial setup on entering the state
        m_pauseMenu.SetActive(true);

        // Need to wait for one frame here as the key press event will still be
        // true during this frame
        yield return null;

        while (!Input.GetKeyDown(KeyCode.Escape)) {
            // Do whatever you want, like listen for menu click events

            // Make sure to yield in your while loops or you'll make Unity freeze
            yield return null;
        }

        // Tidy up on leaving the state
        m_pauseMenu.SetActive(false);
    }

}

The most important line in this snippet is yield return StartCoroutine(DoShowPause()). Here we start the pause coroutine, and importantly pause the Start method at that line until the DoShowPause coroutine finishes. As we are simulating a finite state machine, it will only finish when it exits the state.

One convention I like to use with coroutines is to prefix all coroutine method names with Do. The reason for this is if you call a coroutine without wrapping it in StartCoroutine() the coroutine stops at the first yield, but no warnings or errors are logged. By using a consistent naming convention, it’s easy to notice naked DoSomething() calls.

As in the comments make sure you include yield return null somewhere in your while loops or Unity will appear to freeze as it goes into an uninterrupted loop.

Player/Enemy Turn System with Pause

The third and final snippet for this tutorial is a very simple simulation of a turn-based game. It has alternating Player/Enemy turn states, and a pause state that can be called from either.

This snippet illustrates the possibility of having “global” states that can be called from any state. However you explicitly have to check for the trigger to enter the global state at a safe point in each state.

public class UITest2 : MonoBehaviour {
    enum Turn { Player, Enemy };
    Turn m_turn = Turn.Player;

    [SerializeField]
    GameObject m_enemyOverlay;
    [SerializeField]
    GameObject m_playerOverlay;

    bool IsPauseTriggered { get {
        return Input.GetKeyDown(KeyCode.Escape);
    } }

    IEnumerator Start() {
        while (true) {
            if (m_turn == Turn.Player)
                yield return StartCoroutine(DoPlayerTurn());
            else if (m_turn == Turn.Enemy)
                yield return StartCoroutine(DoEnemyTurn());

            // Make sure to yield within the while loop
            yield return null;
        }
    }

    IEnumerator DoEnemyTurn() {
        // Set up
        m_enemyOverlay.SetActive(true);
        while (m_turn == Turn.Enemy) {
            if (IsPauseTriggered)
                yield return StartCoroutine(DoPause());

            // Process enemy actions

            yield return null;
        }

        // Clean up
        m_enemyOverlay.SetActive(false);
    }

    IEnumerator DoPlayerTurn() {
        // Set up
        m_playerOverlay.SetActive(true);
        while (m_turn == Turn.Player) {
            if (IsPauseTriggered)
                yield return StartCoroutine(DoPause());

            // Wait for player input

            yield return null;
        }

        // Clean up
        m_playerOverlay.SetActive(false);
    }

    IEnumerator DoPause() {
        yield return null; // Wait one frame so Input is not triggered twice
        while (!IsPauseTriggered) {

            // Do nothing
            yield return null;
        }
    }
}

As you can see even with a simple situation, the code can get fairly long. In a more complicated game it would quickly become worthwhile to split up the code for each state into its own class.

So that’s a simple example of how you’d use coroutines to set up a FSM for menus. You can jump in and out of states easily and be sure which state you’re in.

Things I Learned in my First Year of Unity

After a year of using Unity at work, I’ve learned a lot of new techniques, and my coding habits have changed over time.

When I started with Unity I read a bunch of tutorials and recommended practices, but with practical use I learned other techniques.

Some of these techniques may be blindingly obvious to you, others might have slipped you by.

If you have any tips of your own please add them in the comments section!

Make Most of your MonoBehaviour Variables Public

Programming 101 always teaches you to only make member variables public if you really need them to be modified by other classes. Consequently it feels dirty and dangerous to make everything public.

However after a year I think the benefit of being able to view variable values in the inspector outweighs the danger of exposing private variables.

Personally I make the interface of the class with accessors, and make member variables public for the inspector. By prefixing member variables with m_, even if they’re public it’s clear that you’re accessing a member variable.

public float m_width;
public float Width { get { return m_width; } }

Thing.m_width.... // Bad!
Thing.Width  // Good!

Select GameObjects from Debug.Log and Use Color

I didn’t notice there was an optional second parameter for Debug.Log (and LogWarning, LogError). It lets you specify a gameObject that is automatically selected when you click on the log message. Also you can add colors to your log. If you have messages that are specifically for different parts of your team, like artists and designers, it can help to highlight them in specific colours.

Debug.Log("<color=#magenta>♡ ART ♡ - Failed to load model, must be named Character.fbx</color>", modelRoot.gameObject);

Make Everything a Subclass of MonoBehaviour for Visual Debugging

Everything might be too strong a word, but most things that can be made a subclass of MonoBehaviour, should be. Even for classes that you think will have no renderers, making them subclass MonoBehaviour and adding them to a GameObject somewhere means you can visually debug them from the inspector and scene views.

This applies for singletons too. Use a singleton instead of a static class, and make the singleton a subclass of MonoBehaviour means you can have the instance of your singleton be a component on a GameObject in your scene. Beyond the usual ability to use the inspector to view values, you can add OnDrawGizmos to it and visualise its current state better.

public class KittenManager : MonoBehaviour {
  static KittenManager m_instance;
  public static KittenManager Instance { get {
    if (m_instance == null)
      m_instance = Create();
    return m_instance;
  } }

  KittenManager Create() {
    GameObject go = new GameObject("KittenManager");
    return go.AddComponent<KittenManager>();
  }

  void Awake() {
    if (m_instance == null) {
      m_instance = this;
    } else {
      if (m_instance != this) {
        Debug.Log("Instance already exists, destroying this object");
        Destroy(gameObject);
      }
    }
  }
}

Deal with Unity’s 90-degree FBX Rotation

Unity imports most models with a rotation of -90 in X. Most modelling programs use a Z-up coordinate system, whereas Unity uses the Y-up convention. For whatever reason, their attempt to fix this is to give a -90 X rotation to any GameObjects created from model files. With this rotation the models look correct in Unity. However if you child any objects to them, or set rotations based on the assumption that 0,0,0 is “unrotated”, you can get unexpected rotations.

The only way to hack around this is to work with your modelling software and build your model with a baked-in rotation of +90 in X on the root node. That way the model is imported with a rotation of 0,0,0, and looks correct. There are tutorials online that go into more detail about this, but it’s something you need the artists to understand and work with from day 1.

Know About Unity’s Model Animation Quirks

This will only apply to you if you’re using imported models with animation. Unity’s treatment of model animations is most charitably described as “unique”. Animation curves are forced to be regular bezier curves. Meaning that 0-frame instant jump your animator put on the Camera node for animation, does not actually take zero frames In fact there will be at least one frame where the camera is partway between the start and end points of the curve. While Unity’s in-engine animation system supports a variety of animation curve types,

Resources.Load is Slower Than you Think, AssetBundle.AsyncLoad is the Solution

For loading a very small number of very simple objects, Resources.Load() is perfectly fine. However it can get out of hand surprisingly fast.

We had to load around sixteen 256px images and create some complex GameObjects, and the slowest parts of the whole process were but the 16 calls to Resources.Load(). Each one was synchronous and in total they caused the game to freeze for half a second. Changing the process to use a placeholder image, load each image with AsyncLoad(), and replace the image when AsyncLoad completed, stopped any freezing.

Instantiate is not Free, You Need to Work to make it Asnyc

GameObject.Instantiate() is another cuprit for freezing. There’s no async version of it, but the slow part in calling it is not usually the instantiation itself but the code that gets called on instantiation - Awake() and Start() If your Awake(), Start(), OnEnable() or any other functions called on startup are heavy and synchronous, every time you create an instance of that GameObject your game will freeze. Split up the work needed into smaller asynchronous chunks by using coroutines.

Don’t Put Everything in the Resources Directory

As a programmer it’s frustrating to have assets outside of the Resources directory as it means you can’t load them at runtime. You’re stuck with using the inspector to set up links to GameObjects in other parts of the file directory.

Our first reaction to this was to put nearly everything in the Resources directory. The problem with this is if you stop using assets, if they’re in the resources directory they’re still put into every build you make. If they’re in the regular Assets directory, they will only be added to a build if they’re used by something included in the build (another GameObject, Scene).

Pick the Right File Format for your Designers and Artists

This isn’t a Unity-specific tip but I think it’s important nonetheless. Our game required both artists and designers to input a lot of data for setting up the game’s visual content. In the early prototype stage we chose JSON as we had an existing importer and it was what we as programmers were used to. This was not a good idea. The artists and designers put up with it very well but a mixture of picky syntax and verbosity the files quickly became unmanageable. We should have built the tool on their terms. They were most comfortable with Excel, so we should have written an exporter and converter for Excel. It would have taken longer on our part at first, but we would have saved hundreds of hours of debugging in the long-run.

GameObject.Find and Transform.Find should be used Sparingly

When I first started using Unity, the lead programmer and I tried to do as much work as we could through code. We had been burned by SVN conflicts for prefabs and scenes where we had to choose whose work would be preserved, and whose work would be lost. The logic was that if we did everything in code, we could easily merge the results and not lose work. To set up objects, I used Transform.Find("ObjectName") in Awake and Start to find objects higher and lower in the object hierarchy. However some objects were not yet set up in Awake(), so I moved the Find() calls to Start(), but sometimes even then the target object hadn’t been created yet.

Even after untangling that spaghetti, I started noticing how much the calls to Find() were slowing down instantiation.

The solution to this was to get used to Prefabs and Scenes and set up the objects to avoid calls to Find(). There was no golden bullet with SVN conflicts but clearly separating responsibilities and confirming with other team members before making major changes to scenes seemed to work out.

Set up your Directory Structure by Use, Not Filetype

This might be more controversial, and is more of a personal preference than something I can empircally prove is better. One file structure I see recommended in Unity tutorials is based around file format. Separating materials, textures etc. into different subdirectories like below:

/Materials/
/Materials/PetShop/
/Materials/Menus/
/Textures/
/Models/
/Shaders/

The benefit of this is it’s immediately clear where a resource should go. The downside is you often have to jump around the directory tree to find related resources. You find Puppy.png and Kitty.png, but you have to jump to the Materials directory to find the material that might use them.

The alternative looks something like this:

/
/PetShop/
/WorldMap/
/Menus/
/Menus/Options/

Each directory contains a mix of filetypes; materials, textures, models.

We kept shaders and scripts outside of this structure in their own dedicated directories, but maybe mixing them with the assets they use might work.

Conclusion

There’s a lot to learn when starting Unity. Some of the lessons are just things you have to experience yourself, but I hope this post saves you a few hours of pain.

subscribe via RSS