How to develop an OpenXDK application

Developing your application using OpenXDK

OK, so you have (hopefully) read the overview, the XBOX details and are still keen to develop XBOX applications. You need to have installed the OpenXDK headers and libraries as per the instructions in the installation guide. Before you progress to actually developing application, there are a few more housekeeping items that probably need to be described.

For simplicities sake, I will assume that you have installed the libraries into /usr/local/openxdk. It doesn’t really matter where you put it… I just need somewhere so I can describe relative paths of various files. In addition, although you don’t actually need the source to the OpenXDK libraries to compile your own application, I will assume you extracted the OpenXDK source into /home/craig/openxdk.

Development process

The general process for developing an OpenXDK application is as follows:

  1. Write your application code using whatever editor/development environment you like. The entry point for your application will be a function with the following signature:
    void XBOXStartup()
  2. Compile your object files (making sure you use the compiler settings described below)
  3. Link your object files with the provided OpenXDK libraries (making sure you use the link settings described below)
  4. Post-process the executable with CXBE to produce the final executable
  5. Transfer the executable onto your XBOX and run!

CXBE

As mentioned previously, there are quite a few similarities between PE files and XBE files. However, I am not yet smart enough to figure out how to create a cross-compiled version of gcc to produce xbe files directly (maybe one day!). So, how do you convert .EXE files to .XBE files? Fortunately, Caustik has written a little utility (CXBE) that does this for us. So, once we have compiled our XBOX application to a .EXE form using gcc, we have to post process it with CXBE. CXBE does the following key things (among others):

  • Creates the XBE-specific header based on values in the PE file
  • Sets the base address to 0x00010000 and performs the relocations for the internal relative offsets. This relocation is a key piece in ensuring that the .XBE is setup properly to be run at location 0x00010000.
  • Determines the entry point for the application based on the first thunk address of the PE file

The syntax for CXBE is:

cxbe [options] [exefile]

The options that are available for CXBE are:

Option Description
-OUT:filename The output filename for the XBE file
-DUMPINFO:filename The filename for some debug information about what was converted
-TITLE:title The title for the application. This is what will appear by default in your dashboard application
-MODE:{debug|retail} Whether the XBE is in debug or retail mode. You will nearly always want retail (debug isn’t what you think it is).

An example invocation might look like:

cxbe -OUT:default.xbe -DUMPINFO:cxbe.txt -TITLE:CoolApp CoolApp.exe

cxbe is built as part of the installation process and can be found in the /usr/local/openxdk/bin directory.

libc

In a normal gcc development environment you get a system library called libc that gives you such features as malloc, fopen, etc. Now, this library is very operating system specific (think about it – it allocates memory, access files, etc) and is normally provided by your Cygwin installation. However, the Cygwin libc obviously doesn’t know how to access the XBOX file system, or allocate memory, and so on, so we had to provide our own implementation. The implementation that I chose to use was newlib. It is designed to be portable with new implementations filling out a bunch of predefined stubs. If you are curious to see how that works, have a look at:

/home/craig/openxdk/newlib-1.12.0/newlib/libc/sys/xbox/syscalls.c

Compiler Options

In order to get gcc to compile code so that it will run successfully on the XBOX, we need to provide some non-typical gcc flags. The XBOX is a bit finicky when it comes to some of these things, so you may find that you can omit some of them and it will still work, but these are the combinations that I have found to be best.

Flag Rationale
-std=gnu99 I am actually not sure whether we necessarily need this flag. I think I probably put it in when I first started to try and ensure I wasn’t writing any funky C code that might cause problems, but in retrospect, it may not actually be required.
-ffreestanding Assumes that the standard libraries may not exist. I believe that this is a hint to the linker that we aren’t necessarily linking using typical libraries.
-nostdlib Only use library directories specified on the command line. We need to have this so that we don’t accidentally pick up the Cygwin libc library (or any other libraries that may happen to be in the path)
-fno-builtin Don’t recognise any internal built-in libraries. Cygwin has a couple of internal functions that it uses to handle things like alloca that we need to disable because they don’t have any meaning on the XBOX
-fno-exceptions Disable exceptions. This is mainly because my crt0 implementation (in newlib) doesn support exception handling. If I ever get around to fixing that, we may be able to remove this flag.
-mno-cygwin Disable Cygwin specific extensions. There are a bunch of Cygwin specific #ifdefs in the newlib headers/code that get included if you are compiling using Cygwin. However, they assume that you are compiling with a target of Windows in mind – which is not what we want.
-march=i386 This sets the architecture to i386, but I can’t remember exactly why I had to set this, but I think that it had something to do with compiling newlib properly
-I/usr/local/openxdk/i386-pc-xbox/include
-I/usr/local/openxdk/include
This is where we are going to look for any include files. Note that the installation steps will have copied all of the pre-requisite OpenXDK libraries and header files into these directories.

Linker Options

Compiling is only half the battle. The next challenge is linking correctly so that CXBE stands a fighting chance of being able to convert the resultant EXE file. In particular, we need to make sure that there are no other DLLs that we inadvertantly link to. The table below lists the options that I use to get ld to behave correctly.

Flag Rationale
-nostdlib Only use library directories specified on the command line. We need to have this so that we don’t accidentally pick up the Cygwin libc library (or any other libraries that may happen to be in the path)
-Wl,–file-alignment,0x20
-Wl,–section-alignment,0x20
These flags are very important as they setup the file and section alignments. These are the only combinations that I could find that worked
-shared This flags actually forces the linker to create a DLL. We need this to force the linker to add the relocation section (so CXBE can weave its magic when relocating to base address 0x00010000). Interestingly, the only difference in a PE file between a DLL and an EXE is one single bit (and CXBE ignores that bit when converting to a XBE)
-Wl,–entry,_WinMainCRTStartup Explicitly sets the entry point of the application so that OpenXDK has a consistent entry point. OpenXDK does some initialization in this function, and then calls XBOXStartup() – which is your application’s entry point.
-Wl,–strip-all Strips the debugging symbols. We don’t necessarily need this flag, but it reduces the size of the resulting EXE
-L/usr/local/openxdk/i386-pc-xbox/lib
-L/usr/local/openxdk/lib
Because we set the -nostdlib flag, we need to tell the linker where to get its libraries from. newlib is installed in the first directory, and the rest of the OpenXDK libraries go into the second.
-lSDL
-lopenxdk
-lhal
-lc
-lhal
-lc
-lusb
-lxboxkrnl
In general, linking using the GNU toolset is a funny business. The basic logic is that the linker goes through all of the object files looking for unresolved symbols, which it holds in a list. It then traverses the list of names of libraries provided by the -l flag (in the order that they are listed on the command line.) For each library, it finds which symbols are resolved by that library and fills in the gaps from the object files. However, the libraries themselves may have unresolved symbols, which is where it becomes a bit tricky. An example probably explains it the best.Consider the case where you have code that calls a function called a1. You also have two libraries: A and B. A has a function a1 that depends on a function b that is inside library B. However, b depends on a function a2 that is inside library A. If you use -lA -lB, the linker knows it needs to resolve a1 and it finds it in library A. However, in doing that, it adds another unresolved symbol, b, to the list. It can’t find that in A, so it looks in B. Yep, no dramas. However, b depends on a2. Because the linker didn’t keep track of the fact that A contained a2, the link process fails because it couldn’t find a2. The way you get around it is to use -lA -lB -lA, so that after it it looks at library B, it then has one more look through library A. Now, I hear you ask, why don’t you just put B on the list first like -lB -lA? This doesn’t work, because when it looks through B, it is only looking for a1. It won’t find it in B, so it skips to library A. It will find a1 in A, but remember that a1 references b (which won’t be found unless there is another -lB) so it will bail out with an unresolved symbol. Weird, huh? Get used to it though! That is just how it works.

Anyway, the point of all this ramble is to explain why I have such a convoluted collection of libraries being linked (with some of them being included several times). For example, newlib (-lc) depends on the hal (-lhal), which depends on some common OpenXDK routines (-lopenxdk) that depend on both hal and newlib. Yikes!

Example makefile

An example of a makefile that I normally use is:

#
# update this variable to wherever you installed the OpenXDK libraries
#
PREFIX = /usr/local/openxdk

CC = i386-pc-xbox-gcc
CXBE = cxbe

SDLFLAGS = -DENABLE_XBOX -DDISABLE_CDROM 
CC_FLAGS = -c -g -std=gnu99 -ffreestanding -nostdlib -fno-builtin -fno-exceptions -mno-cygwin -march=i386 $(SDLFLAGS)
INCLUDE  = -I$(PREFIX)/i386-pc-xbox/include -I$(PREFIX)/include -I$(PREFIX)/include/SDL

CLINK = -nostdlib
ALIGN = -Wl,--file-alignment,0x20 -Wl,--section-alignment,0x20 
SHARED = -shared
ENTRYPOINT = -Wl,--entry,_WinMainCRTStartup 
STRIP = -Wl,--strip-all
LD_FLAGS = $(CLINK) $(ALIGN) $(SHARED) $(ENTRYPOINT) $(STRIP)
LD_DIRS = -L$(PREFIX)/i386-pc-xbox/lib -L$(PREFIX)/lib 
LD_LIBS  = $(LD_DIRS) -lSDL -lopenxdk -lhal -lc -lhal -lc -lusb -lxboxkrnl

all: blah.exe

.c.o:
	$(CC) -c $< $(CC_FLAGS) $(INCLUDE)

blah.exe: blah.o 
	$(CC) -o $@ $< $(LD_LIBS) $(LD_FLAGS)
	$(CXBE) -TITLE:"$@" -DUMPINFO:"blah.cxbe" -OUT:"blah.xbe" $@ > /dev/null

clean: 
	rm -f *.o *.exe *.dll *.xbe *.cxbe

Available APIs

So, now that you have enough background to actually do some, what sort of APIs are available?

API Description
libc The libc implementation is provided by newlib and includes the normal C library functions. There are many more than is listed below, but it should give you a good idea of the sorts of functions:

  • malloc
  • memcpy
  • memset
  • printf
  • sprintf
  • fopen
  • fread
  • fwrite
  • fclose
  • open
  • stat
  • close
  • time
  • strcpy
  • strchr
  • and so on

For more information on the functions that are available on newlib, check out the newlib web site.

hal The Hardware Abstraction Layer (HAL) is pretty much where all the action happens! If you want to manipulate files using an XBOX specific API (very similar to Win32), read data from the XBOX contollers, reboot the XBOX, etc, this is how you would do so. For more details, click here.
xboxkrnl This library is the mechanism through which you would invoke the raw XBOX kernel APIs. Not for the faint of heart, but available if there is no other API exposed through the hal or libc.
openxdk The openxdk API is not really an API per se. It is a collection of functions that are used internally within OpenXDK. The main function that comes with the openxdk library is the implementation of the WinMainCRTStartup() function. This is where the OpenXDK gains control and it is this function that will eventually call your XBOXStartup() function.
usb This library was 100% written by Ben Kenwright. I don’t know too much about how it works yet… I just know that it does. Once I gain a better understanding, I will try to document the available functions and how you call them.
SDL The Simple DirectMedia Layer (SDL) is an awesome collection of cross platform multimedia libraries. I have ported the video and joystick components to run under OpenXDK. There are plenty of books, articles and tutorials on how to use SDL avaialable at the libsdl website. Hopefully as time progresses, more SDL functionality will be ported over to run in OpenXDK (audio are fairly high on my priority list)

In general, you will find that there are several APIs available that do the same thing. A good example of this is opening a file. You can do this in one of three ways. The table below shows the different ways you might choose to do it:

API Code
libc FILE *fp = fopen("c:/tmp/blah.txt", "r");
hal int handle;
XCreateFile(&handle, "c:/tmp/blah.txt", GENERIC_READ, FILE_SHARE_READ, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL);
xboxkrnl int handle;
String file = "\\Device\\Harddisk0\\Partition2\\blah.txt"
ANSI_STRING tmp;
OBJECT_ATTRIBUTES objectAttributes;
IO_STATUS_BLOCK ioStatusBlock;
NTSTATUS status;
RtlInitAnsiString(&file, tmp);
objectAttributes.RootDirectory = NULL;
objectAttributes.Attributes = OBJ_CASE_INSENSITIVE;
objectAttributes.ObjectName = tmp;
status = NTCreateFile(
(PHANDLE)handle,
GENERIC_READ,
&objectAttributes,
&ioStatusBlock,
NULL,
0,
0,
FILE_OPEN,
FILE_RANDOM_ACCESS);

As you can see, using the xboxkrnl APIs make for much more complicated code. The hal and libc libraries hide much of the complexity for you. Typically, you would be using either hal or libc for day-to-day coding, and using the SDL capabilities for your multimedia.