Every time i play a modifiable game, i become the curious about the principles behind its design. This curiosity led me to explore the concept of “Data-Driven” techniques, which is an essential programming knowledge in game development.

After conducting thorough research, i decided to write this technical note to delve deeper into the fundamentals of modification theory. Furthermore, i developed a modifiable prototype in Unity using IO, XML and Lua. My intention with this article is to inspire and provide valuable insights to fellow game developers.

Modification

What is Data-Driven

First, let me explain Data-Driven simply. we can define data formats and use integer or string mapping to internal logic, allowing us to achieve complex behavior without hard coding in script.

Note: "Data-Driven" has nothing to do with "Data-Oriented". The latter is used for optimizing game performance.

Once we incorporate Data-Driven techniques into game systems, we can develop games more effectively and indirectly provide players with the means to extend game content. In this section, i will explore three core concepts related to game modification to provide a starting point.

Dynamic Loading

Once the game is developed, the game engine packs and encrypts assets by default, making it difficult to easily change game content. However, in order to provide players with the means to extend the game, developers need to designate specific location where they can make change or add file.

Taking Unity as an example, we can utilize the StreamingAssets folder. This folder is not encrypted in the project build. By using System.IO to load content at runtime, players can edit certain files and add more content after the game is released.

The powerful steam workshop feature is convenient as it provides community content management. However, the game still needs to handle loading and other tasks to make modifications work.

Note: Modes downloaded from Steam are stored in C:\Program Files (x86)\Steam\steamapps\workshop\content\gameID. I referred to many modes during this research.

Define Format

After implementing dynamic loading, modification don’t have any effect on the game unless a specific data format is defined. We can’t randomly put something in and hope it works. Developer needs to define a data format that allows the game to analyze the content accurately. For instance, when defining a monster, we must determine its properties, including text content like name, description, behavior logic, and visual presentation.

Developers can create a template using a lightweight data format such as a .txt file. Based on this format, developers need to program an analysis system that reads the file converts the data into in-game structures like classes or structs.

Here’s an example template:

template.txt

Name:
Describe:

Health:
Speed:
Attack:

Sprite:
Sound:

Players can duplicate the template, edit desired content, and place it in the designated location mentioned earlier. By following this process, a new monster can be added to the game successfully.

Although .txt file can work, i prefer using more readable, modifiable, and analyzable file format like XML or Json to store data. The complexity of the format definition depends on your purpose. You can use a simple format for player modification of parameters or design a more complex format to allow players to create additional logic and behavior.

Here’s an example using JSON:

zombie.json

"Define": 
{
    "Name": "Zombie",
    "Description": "Zombie monster !!",
    
    "Health": 2,
    "Speed": 3,
    "Attack": 1,

    "Sprite": "sprites/zombie.png",
    "Sound": "sounds/errrrr.mp3"
}

Behavior Extend

After implementing the first two functions, modification are work well, and players can duplicate templates and create their own content. However, their permissions are still limited by the defined framework, making it harder to extend more complex systems. If we want players to be more creative, we should allow them to write script themselves and load and execute them at runtime.

For instance, let’s say we want players to define enemy movement, attack, and death behavior. We can define witch function can be overridden by the player and load and execute the corresponding script at specific timings.

monsterAI.lua

function move()
	-- move behavior
end

function attack()
	-- attack behavior
end

function dead()
	-- dead behavior
end

Of course you can allows players extended behaviors to execute at any time, here just shown example simply. However, to make this work, we need to create a loading and analyzing system, as explained earlier.

smartZombie.json

"Define": 
{
    "name": "Smart Zombie",
    "Descripe": "Zombie monster with AI !!!!",
    
    // other perperity ...

    "Bevavior": "scripts/monsterAI.lua",
}

When it comes to programming extensions, there are various resources available, such as the interpreted language Lua, compiled .dll files, or even creating a new programming language and visual machine if desired.

In conclusion, game modification depends on the developer’s intention and skill, excluding special methods like decompilers. In the next section, we will create a simple prototype in Unity, providing an example to demonstrate the implementation of all three techniques.

Prototyping

Although we are prototyping in Unity, the theory discussed here is applicable to any platform or engin. You can use any tool or engine of your choice. In the following articles, we will expand on the key elements involved in prototype, without going into excessive detail about specific tools.

During the prototyping process, we will utilize the following:

• Using System.IO to read files in the StreamingAssets folder for dynamic loading.

• Using System.XML analyze definition files written in XML for format define.

• Incorporating the MoonSharp addon to execute Lua scripts for behavior extension.

The art assets used in the prototype are CC0 pixel art, downloaded from the source 0x72_DungeonTilesetII_v

Entity Define

First, we need to define a game entity, which can represent various elements such as enemies, items, or scenery objects. In this example, we will use a generic entity to demonstrate the process. The entity definition includes an entity ID, a display sprite, and the behavior script it uses.

To define the entity, create a new class and declare variables with the necessary attributes for XML serialization. For example:

cs

[System.Serializable][XmlType("Entity")]
public class EntityDefine
{
    [XmlAttribute]
    public string id;

    [XmlElement("Sprite")]
    public string sprite;

    [XmlElement("Script")]
    public string script;
}

Next, create a new text file and fill it with XML-formatted data based on the defined structure.

enemies.xml

<Entities>
    <Entity id="entityID">
        <Sprite>UseSpriteName.jpg</Sprite>
        <Script>UseScriptName.lua</Script>
    </Entity>
</Entities>

Once the definition file is created, place it in the StreamingAssets folder. Use System.IO to read the content and convert it to an XmlDocument for analyze.

cs

string path = $"{Application.streamingAssetsPath}\\entities.xml";

byte[] entitiesData = File.ReadAllBytes(path);
string dataText = System.Text.Encoding.UTF8.GetString(entitiesData);

XmlDocument dataXML = new XmlDocument();
dataXML.LoadXml(dataText);

Note: The use of slash (/) and backslash (\) in the IO paths may cause issues on different platforms, you can use string replacement to fix.

In Unity’s library, XML does not have an equivalent to JsonUtility. Therefore, a generic serialization function can be used:

cs

public static T ConvertNode<T>(XmlNode node) where T : class
{
    MemoryStream stm = new MemoryStream();

    StreamWriter stw = new StreamWriter(stm);
    stw.Write(node.OuterXml);
    stw.Flush();

    stm.Position = 0;

    XmlSerializer ser = new XmlSerializer(typeof(T));
    T result = (ser.Deserialize(stm) as T);

    return result;
}

Finally, to analyze the data, loopthrough the XML nodes and convert the defined entity into instances.

cs

public List<EntityDefine> defines;

XmlNode root = dataXML.DocumentElement;
for (int i = 0; i < root.ChildNodes.Count; i)
{
    XmlNode node = root.ChildNodes[i];
    Debug.Log(node.Name);

    EntityDefine entity = ConvertNode<EntityDefine>(node);
    defines.Add(entity);
}

Resources Loading

Now, the game system can read the definition data in the folder. However after instancing entities, we also need to load the resources used by them, including sprite and behavior script. To make loading easier and keep the folder organized, developers need to specify the location of resources and define naming rules.

We can create a new folder called “Sprites” in StreamingAssets and use Directory.GetFiles() to get files in it.

cs

string folderName = "Sprites";
string directoryPath = $"{Application.streamingAssetsPath}\\{folderName}";
string[] files = Directory.GetFiles(directoryPath);

Loop through all the resources in the folder and use string.EndsWith() to check if it is an image file. The data read by the IO is in byte array format, so we also need to use ImageConversion to convert the array into a Texture2D for storage.

cs

public Dictionary<string, Texture> textures;

for (int i = 0; i < files.Length; i)
{
    string path = files[i];

    if (path.EndsWith(".png"))
    {
        byte[] data = File.ReadAllBytes(path);

        Texture2D image = new Texture2D(2, 2);
        image.filterMode = FilterMode.Point;
        image.LoadImage(data);

        string name = path.Replace(directoryPath + "\\", "");
        textures.Add(name, image);
    }
}

Loading the behavior scripts follows the same process as loading sprites. Define the folder read the files, check the format, and store them in a dictionary.

cs

string directoryPath = $"{Application.streamingAssetsPath}\\Scripts";
string[] files = Directory.GetFiles(directoryPath);

for (int i = 0; i < files.Length; i)
{
    string path = files[i];
    if (path.EndsWith(".lua"))
    {
        byte[] data = File.ReadAllBytes(path);
        string code = System.Text.Encoding.UTF8.GetString(data);

        Script script = new Script();
        script.DoString(code);

        string name = path.Replace(directoryPath + "\\", "");
        scripts.Add(name, script);
    }
}

The behavior in the Lua script depends on your needs. In this example, we only use the awake function, which is invoked when entity is initialized.

behavior.lua

function awake()
	print("hello from script a");
end

Instance Object

Finally, all the preconditions are finished, we can generate the entities we have defined into the game world. Create a new C# script representing the agent of the entity. It will be instantiated as a GameObject and store all the data that is needed.

cs

public class GameEntity : MonoBehaviour
{
    [SerializeField] string id;
    [SerializeField] Sprite sprite;
    public DynValue function;

    public void SetEntity(string id, Sprite sprite, DynValue awakeFunction)
    {
        this.id = id;
        this.sprite = sprite;
        this.function = awakeFunction;
    }
}

To instance a new entity, we load the resources it used and pass them to the agent object. Reading the Lua function is done using the Globals.Get(); function in MoonSharp. We find the function named “awake” and store it in a DynValue to pass it.

cs

void GenerateEntity(EntityDefine define)
{
    GameEntity entity = Instantiate(entityPrefab);

    Texture2D texture = textures[define.sprite];
    Sprite sprite = Sprite.Create(texture, new Rect(0, 0, texture.width, texture.height), new Vector2(0.5f, 0.5f), 16);

    Script script = scripts[define.script];
    DynValue function = script.Globals.Get("awake");

    entity.SetEntity(define.id, sprite, function);
}

At the end of instantiation, we need to invoke the function loaded from the Lua script. Now the unique entity lives in the game world.

cs

public class GameEntity : MonoBehaviour
{
    // codes...

    void Start()
    {
        Initial();
    }
    void Initial()
    {
        gameObject.name = id;
        GetComponent<SpriteRenderer>().sprite = sprite;

        if (function != null)
        {
            function.Function.Call();
        }
    }
}

Advanced Topic

Congratulations! Now, the game can modify its content using StreamingAssets after build, completing the basic part of game modification.

Note: After build, the folder location will be in Build/Project_Data/StreamingAssets

The core of game modification is simple, but there is still a long way to go beyond the implementation. This prototype ignore many issues in a real project. I cannot explain everything in a single article. so at the end of this journey, let me share some points for reflection on implementing modification.

Hierarchy

In the prototype, we used a simple data structure with only one definition file (entities.xml) and two resources folders (Sprite, Script). However, in a real project, we may have various files that need to be organized.

how to be establish a regulated data hierarchy? Should it be based application in the game or different type of files? So, the first question is about the files layout and naming rules.

Format Define

As an example, we used XML to define an entity with only three properties. In actual implementation, we may have a log of content to use, such as text, parameters, behaviors, visual assets adn audio data.

The second is how to define a format that is easy to read, modify, and reuse content to prevent repetition.

Data Conflict

In the prototype, we loaded modification data and stored it in a dictionary, using string as identifiers. However, if we have multiple content from different mods, how to we isolate the content or allow the mods to interact with each other.

Furthermore, how do we protect the game from crashing if mods cause runtime errors? It would be unfortunate if the community puts a lot of work on creating abundant mods but the mod loaded game will just easily leads to conflicts for players.

Develop Tools

Allow players to make mods does not mean that mods are easy to create. Apart from the modification system itself, developer need to provide some tools to make the mod creation workflow easier.

In addition to documentation and tutorials, some development tools are necessary, such as a resources manager, a data browser, a logger, and a developer mode fore players to test mod content.

Performance

In terms of performance, developers sometimes create specification for resources usage in the game. However, with modification, developers cannot manage all community content. Therefore, how to load and run mod content more effectively another point of reflection.

What is the purpose

The last and most important question is: What is you purpose? Why do you want to make a game that is moddable? Is it for community interaction, to allow players to share their creative works and increase the replayability of the game? Or are you trying to build a game like Rimworld and Minecraft that allows player to change the entire game rules?

Developers need to consider the purpose and how much time and money they are willing to invest in this great endeavor.

Thanks for Reading

Although the article is written for modifications, the development theory is commonly used. Even if you don’t want players to modify game content, data-driven techniques are still essential for making developer’s life easier. It’s a good way to publish hot updates for release DLS.

If you have any feedback or suggestion, please leave comment below :D

> you can find all completed scripts in prototype <

Research References

Game Programming Patterns - Prototype

Game Programming Patterns - Type Object

Game Programming Patterns - Bytecode

Open Library - Game modifications

Techopedia - Modification

Turiyaware - Creating A Moddable Unity Game

Unity Manual - Streaming Assets

Unity Manual - ImageConversion

Microsoft Docs - XmlType

Microsoft Docs - XmlElement

Microsoft Docs - XmlAttribute

Microsoft Docs - .Net Regular Expressions

MoonSharp - Getting Started