Cross Platform Game Engine Series - File IO

Cross Platform File IO

Reading and writing files is an essential part of any application – and games are no exception. So File IO should definitely be a part our ideal game engine, meaning the engine needs a good cross platform way of reading and writing files. Should be simple right? We can just use the C# namespace System.IO right? Unfortunately, no.

What’s wrong with System.IO?

System.IO is great but it has a few problems that prevents it from being the perfect solution for a cross platform application:

1. Windows 8 Store Apps don’t use System.IO

Windows 8 Store Apps use the Windows.Storage namespace instead of the System.IO namespace.

2. Applications, including games, should only have access to a limited number of directories

This is becoming increasingly important as operating systems crack down on incorrectly written code that may endanger the security of the system. For example the iOS development guide has the following quote:

“For security purposes, an iOS app has limited a number of places where it can write its data.”

The list and location of the directories applications can access varies by platform so it can quickly become difficult to write good cross platform File IO code using System.IO.

3. Treating paths as strings is dangerous

As this blog post from Twisted Oak Studios explains – treating paths as strings is dangerous. Since System.IO treats paths as strings it forces end users to write dangerous code.

Due to these problems the engine will provide a cross platform File IO solution. This solution will have asynchronous file IO and cross platform paths – both features are expanded on below.

Asynchronous File IO

The first requirement of the engine’s File IO solution is to have asynchronous File IO. Asynchronous File IO is very important because File IO is a relatively slow task.

As the Microsoft documentation for asynchronous File IO explains:

“Asynchronous operations enable you to perform resource-intensive I/O operations without blocking the main thread. This performance consideration is particularly important in a Windows Store app or desktop app where a time-consuming stream operation can block the UI thread and make your app appear as if it is not working.”

The details of implementing Asynchronous File IO can vary from platform to platform, because not all platforms support the same version of the C# framework.

Cross Platform Paths

The second requirement of the engine’s File IO solution is to have good cross platform paths. This will be accomplished with the engine’s Path object.

As seen above the Path uses a Location enumeration to determine what the root of the path is. This automatically and easily restricts the directories that File IO can occur in without worrying about the specifics of each platform.

Also, the Path object validates each piece of the path that is sent to it – checking to make the user isn’t supplying invalid strings. In addition, it makes sure the user isn’t trying to send the Path object a path as string. For instance, a user might try to use:

new Path(Location.Local, “Path/Subdirectory/Filename”);

Instead of:

new Path(Location.Local, “Path”, “Subdirectory”, “Filename”);

These lines of code look almost identical but only the second line of code is guaranteed to work on every platform. So, the Path object makes sure to just ignore invalid paths – hopefully causing the program to crash informing the programmer of their mistake. Later, we will have a debug system in place that will give us a better solution - but until then this will have to do.

Additionally, the Path object stores the filename separately from the directories – this makes it significantly easily to figure out what part of the path is supposed to represent the filename or if there is a filename at all.

Lastly, the Path object stores directories as a list – this makes adding another directory as simple as adding another string to the list. It also hugely simplifies comparing, combining, and modifying paths.

Side Note: See Does a List guarantee that items will be returned in the order they were added? for discussion on whether List can safely be used when order is important.

Also, note that the Path object can turn the object back into a raw string path – but this method is intended for use inside the engine only. That is what the internal keyword does - it prevents the user created code, which will be in a different assembly than the engine code, from being able to use the GetPlatformPath() method in the Path object.

The Solution

By putting together our asynchronous methods and by using the new engine Path object it is easy to make a new cross platform File IO solution that currently looks as follows:

The conflict options enumeration – is a simple way of informing the File IO solution what to do if a naming conflict was found. For instance, imagine that you wanted to write a file named ‘Test.txt’ but there already was a file named ‘Test.txt’ there. What should the engine do? Should the engine – replace the file or fail? There could be even more possibilities such as renaming the old file, or generating a new file name for the new file. The conflict options enumeration provides the user that choice.

As seen above the File and Directory static classes look somewhat like the System.IO.File and System.IO.Directory static classes but with the engine’s Path object instead of strings.

One of the first things to note, is the weird structure of the public static calls, this is because partial methods in C# cannot have a return value so the code must pass the value it wants back as a reference value and then return that value. This is a minor inconvenience that can be hidden from the end users, while still allowing the platform specific code to be separated into platform specific files.

Another thing to note is that the File static class has no exists method – this is intentional. As pointed out in this StackExchange post – there is no good way of implementing a File.Exists method asynchronously.

Conclusion

Please note that the code in this blog is not complete and is missing some of the methods a complete File IO solution would need – such as getting the file names in a directory, or reading from a file in discrete bursts instead of all in one go, or even adding a new Conflict Option that generates a new name for the file when a conflict is detected. This is due to current time restrains – but expanding the solution to handle those situations is a fairly simple matter.

That said, the File IO solution should be an excellent framework for any File IO the engine needs to do. Which leads us into the topic for next week’s blog: Cross Platform Game Engine Series – The Debug System.

References

1. http://msdn.microsoft.com/en-us/library/System.IO(v=vs.110).aspx
2. http://msdn.microsoft.com/en-us/library/windows/apps/windows.storage.aspx
3. https://developer.apple.com/library/mac/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html
4. http://twistedoakstudios.com/blog/Post4872_dont-treat-paths-like-strings
5. http://msdn.microsoft.com/en-us/library/kztecsys(v=vs.110).aspx
6. http://stackoverflow.com/questions/453006/does-a-listt-guarantee-that-items-will-be-returned-in-the-order-they-were-adde
7. http://msdn.microsoft.com/en-us/library/7c5ka91b.aspx
8. http://msdn.microsoft.com/en-us/library/system.io.file(v=vs.110).aspx
9. http://msdn.microsoft.com/en-us/library/system.io.directory(v=vs.110).aspx
10. http://stackoverflow.com/questions/19076652/check-if-a-file-exists-async/19077180#19077180

Cross Platform Game Engine Series - Basic Objects

Why C#

One of the first decisions before starting to program a game engine is always what programming language and technology to use in the creation of that engine.

An easy way of determining what language to choose is to list the platforms the engine is intended to support and the programming languages that support them. The decision then comes down to which language supports the largest number of target platforms. If there is a tie, the developer might want to consider only going with languages that are supported natively – without the need for 3rd party code.

Native support is important because relying on 3rd party code can put in a bad position if the 3rd party choose to stop supporting the platform.

Our ideal PlayStation Vita engine, as described in the previous post would have a diagram as follows:

	Windows	Mac	Android	iOS	PSVita Mobile
C#	OpenTK1 or SharpDX2	Mono3 or Xamarin.Mac4	Xamarin.Android5	Xamarin.iOS6	Native7
C/C++	Native8	Native9	Android NDK10	Native9	No7
Obj-C	Yes11	Native12	Apportable13	Native12	No7
Java	Yes	Yes	Native14	No	No7

Clearly, C# is the only language that meets our requirements – mostly because the PlayStations Mobile developer program only supports C#. With the language for the engine chosen, let’s continue to one of the most important parts of a game – its game loop.

Game Loop

Most software only responds to user input - meaning that a program simply sits idle until the user provides some input, but games need to run continuously regardless of what the user does.

The first requirement of a game loop, running continuously, is easily accomplished with the following:

  1 while(GameRunning) { /* Do Game Logic */ }

But this loop would quickly crash the system – since it never gives control back to the system. A good example of why passing control back to the system is important can be found in the MSDN documentation15 :

“Unlike MS-DOS-based applications, Windows-based applications are event-driven. They do not make explicit function calls (such as C run-time library calls) to obtain input. Instead, they wait for the system to pass input to them.”

“If a top-level window stops responding to messages for more than several seconds, the system considers the window to be not responding.”

Unfortunately, the code to pass control back to the system is heavily platform dependent. This is where the Platform class comes in. The Platform class, that I will discuss shortly, will allow the engine to pass control back to the system in a platform independent manner.

  1 while(GameRunning) 
  2 { 
  3     Platform.ListenForEvents(); 
  4     // Do Game Logic 
  5 }

So the next step is to actually put some game logic in the loop. Traditionally, this logic is rendering scenes, updating the scene, updating physics, and handling input. So the loop becomes:

  1 while(GameRunning) 
  2 { 
  3     Platform.ListenForEvents(); 
  4     RenderScene(); 
  5     UpdateScene(); 
  6     UpdatePhysics(); 
  7     HandleInput(); 
  8 }

This loop still has problems – as every piece of game logic runs every iteration. This is unnecessary as not every system needs to be updated every frame - a common example may be that the physics are updated 30 times a second, the scene is updated 15 times a second, and input and rendering are updated as fast as the computer can.

A simple solution for this is to use a Regulator which is an object that will determine when specific code can be ran. This changes the loop into a more complicated, but still understandable, form:

  1 Regulator sceneRegulator = new Regulator(15.0); 
  2 Regulator physicsRegulator = new Regulator(30.0); 
  3 while(GameRunning) 
  4 { 
  5     Platform.ListenForEvents(); 
  6     RenderScene(); 
  7     HandleInput(); 
  8     if(sceneRegulator.IsReady()) { UpdateScene(); } 
  9     if(physicsRegulator.IsReady()) { UpdatePhysics(); } 
 10 }

Note: Physics really shouldn't be updated in this matter – but I will address that topic in a separate post later. For a quick discussion on how physics should be implemented view Glenn Fielder’s Fix Your Timestep Article16

Now the game has a main loop that updates different systems at different rates! I will now discuss some of the classes used to make this loop possible.

Platform Class

The platform class is responsible for hiding platform specific code from the rest of the engine’s code base. Its definition is as follows:

As seen above the platform class uses the partial17 and static18 keywords. The static keyword allows the Platform class methods to be used anywhere inside the engine without requiring an instance, and the partial keyword allows the Platform class to split up into multiple files. One downside of the partial method is that, as stated in the partial methods documentation17, they can only be private and cannot return values.

Regulators

The regulators used to control when specific code sections should update are heavily inspired by the Regulators used in Programming Game AI by Example by Matt Buckland on page 329.

As seen above the regulator needs to generate a random number, and also needs to keep track of the time.

Random Number Generation

Generating a random number is a very common method needed in game development so it makes sense to have a single common source of random numbers. The implementation used in this engine is based on a MSDN article¹⁹ ensuring thread safe random numbers.

Time

Getting the time that has passed since the game started is also a very common method required in game development. This is handled by the following static Time class:

In addition the engine has access to the following Timer class that allows monitoring any time frame desired.

Memory Management

As seen above, many of classes derive from an ObjectBase class. This class is there to help with XML serialization and memory management. XML serialization will be discussed later – but memory management is very important to games. In games performance matters a lot and poor memory management can negatively impact that performance.

For example, imagine that you just unloaded a level of the game. C# uses Automatic Memory Management²⁰ so it could be a while until the level is actually unloaded from memory. By using IDisposable the engine can provide a common method to inform the level that it should prepare to be unloaded – the memory still won’t be free but it will signal to the garbage collector that the memory can be freed.

This leads to the ObjectBase implementation inspired by this Stack Exchange post²¹:

Conclusion

This blog series will continue next week with Cross Platform Game Engine Series – File IO.

References

1. http://www.opentk.com/
2. http://sharpdx.org/
3. http://www.mono-project.com/Main_Page
4. https://xamarin.com/mac
5. http://xamarin.com/android
6. http://xamarin.com/ios
7. https://en-support.psm.playstation.net/app/answers/detail/a_id/147
8. http://msdn.microsoft.com/library/windows/apps/hh452744.aspx
9. https://developer.apple.com/library/mac/documentation/General/Conceptual/DevPedia-CocoaCore/ObjectiveC.html
10. https://developer.android.com/tools/sdk/ndk/index.html
11. http://stackoverflow.com/questions/56708/objective-c-for-windows
12. https://developer.apple.com/library/ios/documentation/Cocoa/Conceptual/ProgrammingWithObjectiveC/Introduction/Introduction.html
13. http://www.apportable.com/
14. http://developer.android.com/tools/index.html
15. http://msdn.microsoft.com/en-us/library/windows/desktop/ms644927(v=vs.85).aspx
16. http://gafferongames.com/game-physics/fix-your-timestep
17. http://msdn.microsoft.com/en-us/library/6b0scde8.asp
18. http://msdn.microsoft.com/en-us/library/79b3xss3.aspx
19. http://blogs.msdn.com/b/pfxteam/archive/2009/02/19/9434171.aspx
20. http://msdn.microsoft.com/en-us/library/aa691138(v=vs.71).aspx
21. http://stackoverflow.com/questions/538060/proper-use-of-the-idisposable-interface/538238#538238%20

Cross Platform Game Engine Series - Introduction

Introduction

Video games are very complex projects that combine design, art, and programming that take a lot of work and effort to create. Now, imagine if you had to rewrite a lot of code to render graphics, manage scenes, save and load games, and many other features common to all games every time you wanted to create a new game.

This is why game developers make and use game engines. A game engine handles the vast majority of common things that games need to do so that game developers can focus on making a great game.

But what if a game engine isn't available for the platform the developer wants to work on? Say like the PlayStation Vita? Well, they are going to have to make their own engine. This is what this blog series is about – designing and building an engine for the PlayStation Vita.

Design Goals

 

So what would our ideal PlayStation Vita Engine look like? Well our ideal engine would be:

• Data Driven

A data driven game engine is the theory of separating game logic from engine logic completely. This is often done by having the engine use scripts or data located outside of the engine somewhere – allowing the same engine to be used to make just about any game imaginable.

A more in-depth example along with other resources can be found in this stack exchange answer.

• Cross Platform

Although the target for this blog series is a PlayStation Vita engine – it is easier to design a game engine to be cross platform in the beginning than to try and port the engine to a different platform later.

The platforms our engine will try to target are: Android, iOS, Windows, Mac, Ouya, Windows 8 Metro Apps, and of course the PlayStation Vita. These platforms are all open platforms that have low barriers of entry to developers.

Now that the design goals of the engine are clear there is just one last topic to discuss before getting to work on the engine.

Programming For End Users:

Unlike most software projects – the end users of a game engine will have to be able to see and interact with at least some of the code of the game engine in order to get the full use out of it. This means that more so than in other projects – the code behind a game engine needs to be consistent, clean, clear, and able to guide end-users on the ‘right way’ of doing things.

I will get into more specifics on how to accomplish this in later blog posts – but for right now that means that in general the game engine should:

• Don't Let Code Crash

The end users of all applications have vastly difference experience levels – but when those users will be interacting with your code on some level it can be dangerous. For that reason any code exposed to the end user should, as much as possible, be crash proof.

The end user passed in some nonsense valid to one of your functions? Your code should either – 1) Make sure the code doesn’t compile in the first place OR 2) Fail gracefully and inform the user about the error with as many helpful hints as to how to fix the problem as possible.
 

• Hide Complexity From The User

If the end user doesn’t have any reason for mucking around with a given piece of code, then that piece of code should be effectively invisible to the end-user.

 

• Keep IntelliSense In Mind

Most end users are probably going to want to use code IntelliSense when interacting with the game engine. This means that all engine code needs to be sure to use documentation comments with tags. A good list of the available documentation tags is available from Microsoft.

 

• Keep Documentation In Mind

Good documentation is a must for any game engine – and while IntelliSense is great it doesn’t help users unless they are already digging into code. Fortunately, if the engine has proper documentation comments, tools such as Doxygen can speed up the process of creating documentation.

Conclusion

So far, I have only touched on broad design goals and some general coding principles, but these broad goals and principles will guide how the various systems of the game engine are designed and implemented.

This blog series will continue next week with Cross Platform Game Engine Series – Basic Objects where I will discuss the basic objects that are needed before the game engine can really start to take shape.