{"action":"created","issue":{"url":"https://api.github.com/repos/thesecretlab/YarnSpinner/issues/39","repository_url":"https://api.github.com/repos/thesecretlab/YarnSpinner","labels_url":"https://api.github.com/repos/thesecretlab/YarnSpinner/issues/39/labels{/name}","comments_url":"https://api.github.com/repos/thesecretlab/YarnSpinner/issues/39/comments","events_url":"https://api.github.com/repos/thesecretlab/YarnSpinner/issues/39/events","html_url":"https://github.com/thesecretlab/YarnSpinner/issues/39","id":147565311,"number":39,"title":"Localisation support","user":{"login":"desplesda","id":901768,"avatar_url":"https://avatars.githubusercontent.com/u/901768?v=3","gravatar_id":"","url":"https://api.github.com/users/desplesda","html_url":"https://github.com/desplesda","followers_url":"https://api.github.com/users/desplesda/followers","following_url":"https://api.github.com/users/desplesda/following{/other_user}","gists_url":"https://api.github.com/users/desplesda/gists{/gist_id}","starred_url":"https://api.github.com/users/desplesda/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/desplesda/subscriptions","organizations_url":"https://api.github.com/users/desplesda/orgs","repos_url":"https://api.github.com/users/desplesda/repos","events_url":"https://api.github.com/users/desplesda/events{/privacy}","received_events_url":"https://api.github.com/users/desplesda/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2016-04-11T21:50:39Z","updated_at":"2016-04-12T21:37:53Z","closed_at":null,"body":"So. I've been thinking about how Yarn can support localisations, and this is the most straightforward that I've come up with so far. Big thanks to Anna Kipnis of [Double Fine](http://doublefine.com) and Ron Gilbert of [Terrible Toybox](https://thimbleweedpark.com) for their advice that helped to figure this out.\r\n\r\nI'm very interested in comments and discussion on this, because this is a large set of changes to Yarn as a language.\r\n\r\n## Executive Summary\r\n\r\nThis issue outlines a system for supporting localisation in Yarn. It includes both a high-level overview of how such a system can work, and the syntax changes needed in the Yarn language.\r\n\r\nI propose that localisation work by providing a tool that moves out all localisable text - both lines, and localisable string values - into a per-language file, and replaces them with unique linecodes. Additional syntax should be added to the Yarn language to support this.\r\n\r\n## Localising lines\r\n\r\nGiven the following script:\r\n\r\n    <<if $knows_plot == true>>\r\n        I know about the villain's plot!\r\n    <<endif>>\r\n\r\nLocalisation would involve converting the script into something like this:\r\n\r\n    <<if $knows_plot == true>>\r\n        {0201}\r\n    <<endif>>\r\n\r\nIn this case, `{0201}` is a reference to a line that's kept in a separate file. The format of the file doesn't really matter, except insofar you could use it and the number \"0201\" and get back the string \"I know about the villain's plot!\". It could be a database of somekind, or an Excel spreadsheet - it doesn't particularly matter. We'll come back to this in a bit.\r\n\r\nWe can very easily create a tool that identifies and replaces lines with unique line numbers, and exports to a file. However, it isn't without its problems.\r\n\r\nThe serious disadvantage of doing this is that it adds a burden to the writer if they need to make changes to the script. With the text of the line separated from the logic of the Yarn program, it adds the hassle of switching between the script and the exported line file. We can mitigate this a little by adding a comment containing the text, like so:\r\n\r\n    <<if $knows_plot == true>>\r\n        {0201} // I know about the villain's plot!\r\n    <<endif>>\r\n\r\nRunning the localisation system would update any comment like this to ensure that the comment attached to the line code correctly represents what's in the line file.\r\n\r\n## Localising lines with embedded values\r\n\r\nSo, that's simple lines. Let's now talk about lines that incorporate other values. Currently, we don't have any syntax for doing this at all; I've added notes in #25 to discuss that. \r\n\r\nUsing the markup proposed in #25, let's say we have a line that looks like this:\r\n\r\n    He's going to [$method_of_destruction] the [$target] on [$day_of_week]!\r\n\r\nPretty much any localisation system that permits embedding values in a line does so in a way that allows for re-ordering the embedded values. This is for sentences that present words in a different order than the original language:\r\n\r\nThis means that localising a line means converting the above example into something like this, for English:\r\n\r\n    He's going to $0 the $1 on $2!\r\n\r\nAnd in this for German:\r\n\r\n    Er wird das $1 am $2 $0!\r\n    \r\n(Please excuse the potentially incorrect translation - I'm not a native German speaker.)\r\n\r\nIn the case of this line, we need to store in the per-language file both the line, and all of the localised values used in the line.\r\n\r\nGiven this, and the fact that the values in this example come from variables at runtime, we need a way to represent this linecode. I propose that it look like this:\r\n\r\n    {0202,$method_of_destruction,$target,$day_of_week} // He's going to {0} the {1} on {2}!\r\n\r\nIn this case, we're specifying line `0202`, and providing a comma-separated list of the variables used to supply. The position of these variables is important, since it's used to specify which variable is used for each of the numbered values. A comment is also added by the localisation tool to indicate both the original line, and the context of the variables used in it.\r\n\r\n## Localising string values in expressions\r\n\r\nWe've just talked about embedding string variables in localised lines. However, the values used in these variables need to be localised as well. \r\n\r\nThe syntax for this could look like this:\r\n\r\n    <<set $target = {\"the stadium\"}>>\r\n    \r\nAfter localisation processing, the string is extracted and replaced with a code. Additionally, a comment is appended to the line, to indicate what string is represented.\r\n\r\n    <<set $target = {0541}>> // \"the stadium\"\r\n\r\nBy using the brace syntax in the same way as is used in lines, we reinforce the idea that numbers in braces represent localised strings, and their value will change based on which language file is being used. \r\n\r\n## Generating line files\r\n\r\nEarlier, I mentioned that the specific format used to store lines doesn't actually matter. There are several formats that exist for storing localised text; these range from simple CSV and Excel spreadsheets to more sophisticated solutions like [XLIFF](https://en.wikipedia.org/wiki/XLIFF). \r\n\r\nIt's my intent that Yarn Spinner try to stay within as limited a domain as it can. Users of Yarn may already be using some kind of localisation system that they're comfortable with, and we don't need to try to invent a new one. \r\n\r\nI *think* that it would be OK for Yarn Spinner to put the burden of loading localised strings given a linecode onto the host application. Much as the `Dialogue` class requires that the host provide functions for implementing variable storage, we could do a similar thing for lines. \r\n\r\nWe could then provide a simple reference implementation that handles common cases like CSV files, for users who aren't currently using a localisation system already (which I expect will be most users), while more sophisticated systems could make use of their own systems or 3rd-party localisation systems.\r\n    \r\n## Disadvantages\r\n\r\nLinecodes can be obtuse. We may want to look at letting users define their own linecodes - perhaps a more human-readable representation.\r\n\r\nThe primary disadvantage of this system is that making changes to localised lines requires updating each of the localised files to make changes, or to blow away the entire line and generate a new entry in each of the localisation files. However, this is a disadvantage that pretty much any localisation system encounters.\r\n\r\n---\r\n\r\nSo, those are my thoughts. What do you all think? Is there a use case that I've missed? Do you have any thoughts on how this could be better?\r\n"},"comment":{"url":"https://api.github.com/repos/thesecretlab/YarnSpinner/issues/comments/209113645","html_url":"https://github.com/thesecretlab/YarnSpinner/issues/39#issuecomment-209113645","issue_url":"https://api.github.com/repos/thesecretlab/YarnSpinner/issues/39","id":209113645,"user":{"login":"desplesda","id":901768,"avatar_url":"https://avatars.githubusercontent.com/u/901768?v=3","gravatar_id":"","url":"https://api.github.com/users/desplesda","html_url":"https://github.com/desplesda","followers_url":"https://api.github.com/users/desplesda/followers","following_url":"https://api.github.com/users/desplesda/following{/other_user}","gists_url":"https://api.github.com/users/desplesda/gists{/gist_id}","starred_url":"https://api.github.com/users/desplesda/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/desplesda/subscriptions","organizations_url":"https://api.github.com/users/desplesda/orgs","repos_url":"https://api.github.com/users/desplesda/repos","events_url":"https://api.github.com/users/desplesda/events{/privacy}","received_events_url":"https://api.github.com/users/desplesda/received_events","type":"User","site_admin":false},"created_at":"2016-04-12T21:37:53Z","updated_at":"2016-04-12T21:37:53Z","body":"Those are some really good points. Thanks!\r\n\r\nAnother problem with embedded values (which isn't directly related to localization) is that embedding values makes it extremely difficult to generate a voiceover script.\r\n\r\nDo you think that it's worth restricting embedded values to those that Yarn Spinner can be aware of, and use that to generate permutations? For example, Yarn Spinner could refuse to localize a line that contains an embedded variable unless that variable is only ever assigned to string constants (and never functions) in the entire program..."}}