Part 1: Porting to a Null Platform

Introduction

This is the first in a series of posts where I document how I ported Quake to OSX. In part 1 I will cover how to get a copy of the source code, and get it to build in XCode without sound or graphics. I will cover getting graphics working in part 2, and sound in part 3.

You check out my progress here: https://github.com/marcomorain/Quake/

The Quake source code was published to github with a GPL license. This does not contain any of the assets. The Quake assets are not released publicly by id Software, so to run Quake you will need to get a copy of them. There are two ways to get a copy: you can get the shareware version of Quake online, or if you have a registered copy on CD you can use that. See the readme.txt in the root of the source code for more information.

The first thing to is fork the id-Software/Quake repository on github into your own account, and then clone this repository to your local machine. When I did this I did not know how the code was structured, what was included in the repository, or what platforms were supported.

The readme says that the package contains the “complete source code for glwinquake, glquake, quakeworld, and glquakeworld”. If you open the source folder, you will see 3 folders at the root level: WinQuake, QW, and qw-qc. WinQuake contains the source needed to build the single player version of the game (WinQuake and glquake), the QW folder contains the source to build quakeworld and glquakeworld, the online version of the game. The contents of these folders are very similar. I’m not sure why the source is duplicated like this, I assume its because the code is from two branches of version control, or the code was copied to implement quake world after WinQuake was complete.

Creating the XCode Project

I decided to port WinQuake to OSX first, and leave quakeworld until later. The first thing to do is to create a new git branch for your work.

git clone git@github.com:<you>/Quake.git
git co -b part-1

Then open Xcode and choose new project and then choose OSX > Application > Command Line Tool. Just enter Quake for product name, organisation name and company identifier, and finally project type C. Finally, choose the Quake/WinQuake directory as the location for the project.

Create An Empty Project

This will give you an empty project with a little main.c file that prints hello world. Build and run this project to make sure your everything is running as expected.

Run the empty project

commit

Compiling Quake’s Main Function

Now we are ready to start building the Quake code. The first thing to do is find which file contains the main function, and to compile that. Initially I added all of the .c and .h files to my project to see what would happen. As I will show you, there are lots of files included in the repository that we don’t need. These files contain platform specific code for Windows, Linux, Solaris, etc,.

In Xcode choose File > Add Files to Quake, and then navigate to the WinQuake directory. For now just include sys_null.c, and then remove the autogenerated main.c from your project. sys_null.c includes a main function, so we have all we need to run a C app. The code will compile OK, but with a bunch of warnings and linker errors:

Warning and Errors

The warnings are all benign. They are warnings about conversions from long and size_t to int. Quake was written for 32-bit architectures, where long and size_t were 32-bits, the same as int. On 64-bit machines this is not true – a variable of type size_t will be 8 bytes long on a 64-bit machine (for example, the result of strlen), an integer (int) will be 4 bytes long. The biggest number that an int can hold is about 2 billion, the largest size_t is 9 quintillion. The compiler is warning that if you have a size_t that represents a number larger than 2 billion (maybe the length of a very long string), and you assign the value to an int, you will lose data.

Let’s ignore the warnings for now. The source code is full of them, and we can deal with them later.

commit

Our main function defined at the bottom of sys_null.c looks like this:

void main (int argc, char **argv)
{
    static quakeparms_t    parms;

    parms.memsize = 8*1024*1024;
    parms.membase = malloc (parms.memsize);
    parms.basedir = ".";

    COM_InitArgv (argc, argv);

    parms.argc = com_argc;
    parms.argv = com_argv;

    printf ("Host_Init\n");
    Host_Init (&parms);
    while (1)
    {
        Host_Frame (0.1);
    }
}

The errors that we get when we compile are linker errors. This means that our code is calling functions or accessing variables that have not been included in our build (The linker refers to functions and variables collectively as symbols). The symbols that we are missing are COM_InitArgv, Host_Frame, Host_Init, com_argc and com_argv. These sybols mut be defined in the other source files that we have not yet included in the build.

You can use the find command to find out how many C files are included in the WinQuake folder.

~/dev/Quake/WinQuake $ find . -name "*.c" | wc -l
     129

So there are 129 different C files in the package, but many of these are not needed. I think that the best thing to do is to add the files one by one until the program builds. This will ensure that we are building the minimum number of files to get Quake running. Programs written in C are structured in a sort of tree: the root of the tree is the main function, which branches into the functions that main calls, which branch into the functions that are called from those functions, and so on. To build Quake we want to build sys_null.c (which includes our main function), and all of the files that include all of the functions that are called in the tree.

Now, to find COM_InitArgv you can use git grep to search for a string within the current git repository.

~/dev/Quake/WinQuake $ git grep COM_InitArgv
Binary file WinQuake.ncb matches
common.c:COM_InitArgv
common.c:void COM_InitArgv (int argc, char **argv)
common.h:void COM_InitArgv (int argc, char **argv);
sys_dos.c:      COM_InitArgv (c, v);
sys_linux.c:    COM_InitArgv(c, v);
sys_null.c:     COM_InitArgv (argc, argv);
sys_sun.c:    COM_InitArgv (argc, argv);
sys_win.c:      COM_InitArgv (parms.argc, parms.argv);
sys_wind.c:     COM_InitArgv (argc, argv);
sys_wind.c:             COM_InitArgv (argc, argv);

Take a close look at this output. These few lines will show you a lot about how the source code of Quake is structured, and how the modules and namespaces work.

You can see that the string COM_InitArgv appears in common.c and common.h, and bunch of sys_ files. If you are familiar with C, you will quickly see that since COM_InitArgv occurs in both common.h and .c that this is being declared in the header and implemented in the source file (.c). The fact that there is no ; immediately after the occurance in common.c is another indicator that that is where the function is defined.

As you have probably noticed a function called COM_ is defined in a file called common.h. The COM_ prefix here refers to the common library. Open up common.h and have a quick scan through it (press ⇧ + ⌘ + O to open the quick-open menu in XCode and type common.h). You will see many functions that start with COM_: COM_CheckParm, COM_Init, COM_InitArgv, COM_SkipPath, COM_StripExtension, COM_FileBase, COM_DefaultExtension, etc. This is how libraries are implemented in Quake (and many other C programs). The COM_ prefix creates a namespace for library, in the same way gl_ is the namespace for OpenGL functions.

Take another look at the other references to COM_InitArgv, and notice how all of the files share a sys_ namespace: sys_dos.c, sys_linux.c, sys_null.c, sys_sun.c, sys_win.c, sys_wind.c and sys_wind.c. The Quake source code supports many platforms, and this is how the source code is aranged per platform. The sys_ in the filename denotes that this is the system library, and the suffix represents the platform (DOS, Linux, Sun, Windows…).

The null platform (sys_null, vid_null) is a special empty platform that makes porting the code much easier. The developers have included a empty implementation for each cross-platform module.

Let’s add common.c to the project. Press ⌘+1 to select the project navigator, and choose File > Add Files to Quake..., and then open the WinQuake folder and add common.c. Build the project and the last linker errors have been resolved, but they have been replaced with 18 new ones.

New Linker Errors

commit

The next step is to repeat the process of finding where these symbols are defined, and adding them to the project, and building, until there are no more errors. It’s not that simple though, you will notice a few things:

You will find that one function - draw() is defined in two places: draw.c and gl_draw.c. There are 2 renderers in the source code. A software render, and an OpenGL renderer. You should include gl_draw.c rather than draw.c, since we want to build the OpenGL version of Quake. The same issue occurs with screen.c and gl_screen.c both implementing SCR_EndLoadingPlaque() and other SCR_ functions.

Once you try to build gl_draw.c you will get a syntax error:

OpenGL Error

/Users/marcomorain/dev/Quake/WinQuake/gl_draw.c:54:22: error: use of undeclared identifier 'GL_LINEAR_MIPMAP_NEAREST'
int             gl_filter_min = GL_LINEAR_MIPMAP_NEAREST;

This is odd, since GL_LINEAR_MIPMAP_NEAREST is an OpenGL constant. It should be defined in an OpenGL system header. Let’s see if we can find out where.

~/dev/Quake/WinQuake/Quake $ mdfind GL_LINEAR_MIPMAP_NEAREST
/Users/marcomorain/Library/Developer/Xcode/DerivedData/Quake-ddleqlitpiyrivelzscqqesfmljn/Build/Intermediates/Quake.build/Debug/Quake.build/build-state.dat
/System/Library/Frameworks/OpenGL.framework/Versions/A/Headers/gl3.h
/System/Library/Frameworks/OpenGL.framework/Versions/A/Headers/gl.h

This indicates that the constant should be in gl.h. Let’s check:

$ grep GL_LINEAR_MIPMAP_NEAREST     /System/Library/Frameworks/OpenGL.framework/Versions/A/Headers/gl.h
#define GL_LINEAR_MIPMAP_NEAREST          0x2701

Yup, there is it. If that constant can’t be found, then it’s likely that gl.h is not being #included correctly. Let’s search the code for gl.h:

~/dev/Quake/WinQuake $ git grep --fixed-strings "gl.h"
Binary file WinQuake.ncb matches
glquake.h:#include <GL/gl.h>
glquake2.h:#include <gl\gl.h>

Note we have to pass --fixed-strings to git-grep to disable regex matching, since the dot in gl.h will match any character otherwise. Let’s open up glquake.h to see why gl.h is not being included. Opening glquake.h we see:

#include <GL/gl.h>
#include <GL/glu.h>

So gl.h is being included – maybe glquake.h isn’t being included?

~/dev/Quake/WinQuake $ git grep --fixed-string "glquake.h"
WinQuake.dsp:SOURCE=.\glquake.h
Binary file WinQuake.mdp matches
Binary file WinQuake.ncb matches
Binary file WinQuake.opt matches
quakedef.h:#include "glquake.h"

So let’s look in quakedef.h:

#ifdef GLQUAKE
#include "glquake.h"
#endif

Aha! So we need to define GLQUAKE. The quick way to do this would be just #define GLQUAKE in the top of quakedef.h. This might not work in all cases - what if a file conditionally compiles something based on GLQUAKE, but that file doesn’t include quakedef.h? It’s best to define something like this in the XCode project instead. We can do that in the project settings as shown:

define GLQUAKE

If we build now we get another OpenGL related problem:

GL Location

On OSX, the OpenGL headers are located at a different path to on Windows. Let’s replace the #include with the following:

#ifdef __APPLE__
#include <OpenGL/gl.h>
#include <OpenGL/glu.h>
#else
#include <GL/gl.h>
#include <GL/glu.h>
#endif

Finally, we want to add the OpenGL framework to the project, so that the OpenGL libraries are linked in to the app. You do this in the build settings for the Quake target, as shown.

Target

Frameworks

Interestingly, this is the first modification to the Quake source code that we have had to make. We are about to have to make another change. Look at the following error in gl_draw.c:

No return

The error is warning that the compile thinks that Scrap_AllocBlock has does not return anything if the control flow gets to the bottom at the call to Sys_Error. Sys_Error never returns though, since it calls exit(1). exit is a function that quits the app – you can read about it by running man 3 exit in the terminal. The compiler does now know that Sys_Error never returns when it is compiling gl_draw.c, since the compile looks only at the header files, not the implemention of other functions. There is nothing in the sys.h header file to indicate that Sys_Error never returns.

void Sys_Error (char *error, ...)
{
    va_list         argptr;

    printf ("Sys_Error: ");   
    va_start (argptr,error);
    vprintf (error,argptr);
    va_end (argptr);
    printf ("\n");

    exit (1);
}

The solution here is to let the compiler know that Sys_Error never returns. We can do this using a GCC attribute, that clang also supports. In sys.h we can make the following change:

void Sys_Error (char *error, ...)

to:

void Sys_Error (char *error, ...) __attribute__((__noreturn__));

When you get to the sound functions like S_StartSound, we want to include snd_null rather than any of the other snd_ implementations (like snd_linux.c or snd_dos.c). We want to implement the null sound system for now. We will do that in part 3.

For building the network system, you should add net_bsd.c to the project, along with net_udp.c and net_dgrm.c. You will then get a syntax error about conflicting types for gethostname.The definition in net_udp.c is like this:

/Users/marcomorain/dev/Quake/WinQuake/net_udp.c:41:12: error: conflicting types for 'gethostname'
extern int gethostname (char *, int);
           ^
In module 'Darwin' imported from     /Users/marcomorain/dev/Quake/WinQuake/quakedef.h:41:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform    /Developer/SDKs/MacOSX10.9.sdk/usr/include/unistd.h:620:6: note:     previous declaration is here
int      gethostname(char *, size_t);

The first thing to do to diagnose the problem is to find out what gethostname is. If you run man gethostname from the terminal you can see that it is a function from the C standard library. The Quake source code is forward declaring this function, but the signature does not match the signature in the standard library. Here is the Quake declaration:

extern int gethostname (char *, int);

but the platform header unistd.h defines it as

int  gethostname(char *, size_t);

As we noted about, int and size_t were the same size when Quake was written, so this would not have been a problem then. We can easily fix this by changing net_udp.c to match the platform definition:

extern int gethostname (char *, size_t);

We also need to include inet.h to the prevent the warning about the implicit definition of inet_addr.

#include <arpa/inet.h>

Here is a link to the source changes:

https://github.com/marcomorain/Quake/commit/8940b5da911b9fa661eb62e3ef20fcffffdefd44

Some symbols are defined in the Video system, such as the symbols GL_BeginRendering and GL_EndRendering. There is no null video sysmtem, so we will have to build one. Create alongside gl_vidlinux.c and gl_vidnt.c let’s create gl_vidnull.c, by copying gl_vidlinux.c.

~/dev/Quake/WinQuake $ cp gl_vidlinux.c gl_vidnull.c

I’ll then remove the body of all of the functions, to strip this back to an empty implementation. You can see my implementation here: gl_vidnull.c

The last linker error is this one:

Undefined symbols for architecture x86_64:
  "_isDedicated", referenced from:
      _Host_WriteConfiguration in host.o

A quick search for isDedicated through the codebase finds the following results:

~/dev/Quake/WinQuake $ git grep isDedicated
Binary file WinQuake.ncb matches
host.c: if (host_initialized & !isDedicated)
quakedef.h:extern qboolean              isDedicated;
sys_dos.c:static qboolean               isDedicated;
sys_dos.c:      if (!isDedicated)
sys_dos.c:      if (!isDedicated)
sys_dos.c:      isDedicated = (COM_CheckParm ("-dedicated") != 0);
sys_dos.c:      if (!isDedicated)
sys_linux.c:qboolean                    isDedicated;
sys_sun.c:qboolean                      isDedicated;
sys_win.c:qboolean                      isDedicated;
sys_win.c:      if (isDedicated)
sys_win.c:      if (isDedicated)
sys_win.c:      if (isDedicated)
sys_win.c:      if (!isDedicated)
sys_win.c:      isDedicated = (COM_CheckParm ("-dedicated") != 0);
sys_win.c:      if (!isDedicated)
sys_win.c:      if (isDedicated)
sys_win.c:              if (isDedicated)

You can see that sys_win.c, sys_dos.c and the other sys_ platforms declare qboolean isDedicated, but sys_null.c does not. We can add a defintion to sys_null.c and initialise it in the same manner as in the other files. This is as simple as adding a declaration at the top of the file, and setting the initial value in the same was on Windows (sys_win.c).

Add isDedicated

The program will now build and run:

Host_Init
FindFile: can't find gfx/pop.lmp
Playing shareware version.
FindFile: can't find gfx.wad
Sys_Error: W_LoadWadFile: couldn't load gfx.wad
Program ended with exit code: 1

The error is due to the fact that the game content can’t be found. You just need to edit the working directory to the folder where you have the Quake assets. Do this in Xcode in Product > Scheme > Edit Scheme.

Working directory

The game will now run! We hit a crash immediately on launch though, with the callstack as shown. The crash in Q_strncasecmp:

Crash

Thread 1, Queue : com.apple.main-thread
#0  0x0000000100035d31 in Q_strncasecmp at /Users/marcomorain/dev/Quake/WinQuake/common.c:263
#1  0x0000000100031f48 in Draw_Init at /Users/marcomorain/dev/Quake/WinQuake/gl_draw.c:386
#2  0x00000001000432f1 in Host_Init at /Users/marcomorain/dev/Quake/WinQuake/host.c:890
#3  0x000000010003dc49 in main at /Users/marcomorain/dev/Quake/WinQuake/sys_null.c:228

The code that is calling Q_strncasecmp is the following:

// 3dfx can only handle 256 wide textures
if (!Q_strncasecmp ((char *)gl_renderer, "3dfx",4) ||
    strstr((char *)gl_renderer, "Glide"))
    Cvar_Set ("gl_max_size", "256");

The string gl_renderer is being initialised to null in this debug build of the game:

const char *gl_vendor;
const char *gl_renderer;
const char *gl_version;
const char *gl_extensions;

Let’s set some default values:

const char *gl_vendor = "null_vendor";
const char *gl_renderer = "null_renderer";
const char *gl_version = "null_version";
const char *gl_extensions = "null_extensions";

If we save and re-run the code will get a little further before crashing again:

Thread 1, Queue : com.apple.main-thread
#0  0x00007fff8c302142 in glBindTexture ()
#1  0x0000000100031346 in GL_Bind at /Users/marcomorain/dev/Quake/WinQuake/gl_draw.c:83
#2  0x0000000100032436 in GL_LoadTexture at /Users/marcomorain/dev/Quake/WinQuake/gl_draw.c:1264
#3  0x0000000100031ff5 in Draw_Init at /Users/marcomorain/dev/Quake/WinQuake/gl_draw.c:402
#4  0x00000001000432b1 in Host_Init at /Users/marcomorain/dev/Quake/WinQuake/host.c:890
#5  0x000000010003dc09 in main at /Users/marcomorain/dev/Quake/WinQuake/sys_null.c:228

It’s clear that we need to set up an OpenGL context before we can get any further.

That’s it for this part 1! In part 2 I will use GLFW to set up a window with an OpenGL context for rendering.