Pool of Radiance on the Tandy Color Computer 3

Pool of Radiance never came out for any 6809 machine. Here is how its original logic, decompiled and translated, came to run on a real 1986 Tandy Color Computer 3, and how 16 recorded play sessions prove it plays like the PC original.

32 min read Pool of RadianceCoCo 36809porting

A real CoCo 3 with a Hitachi 6309, booting the port from a SuperIDE card

This is the Tandy Color Computer 3, from 1986. It shipped with a Motorola 6809 at less than two megahertz. This one has a Hitachi 6309 instead, and 512 KB of memory.

I type LOADM"POOL/BIN:200":EXEC, a splash with a progress bar fills up, and then SSI's title screen comes up. That is Pool of Radiance, the first of the Gold Box games, the one that kicked off a whole era of computer role playing. It came out on the PC, the Commodore 64, the Apple II and more, but it never came out on the CoCo. In fact, no Gold Box game was ever released on any 6809 machine.1

This isn't a remake, and it's not an emulator. It's the original game's logic, translated line for line, running on a machine it was never written for. And over every play session we recorded, we can prove it plays exactly like the PC version. This article is how.

There's a video that covers the same ground; this written version has the code, the tables of measured numbers and the links into the repository. If you just want to play it, the builds (floppies, CoCo SDC, SuperIDE card, XRoar and MAME) are on the downloads page.

Every episode in the playlist

The machine

The CoCo 3 on the desk, with the SuperIDE cartridge in its slot

If you've never crossed paths with a CoCo 3, it's a 6809 based 8-bit machine with a few 16-bit tendencies, running at about 1.79 MHz. It has a 64 KB address space, which means it can only see 64 KB of memory at any given time.

It also has a chip called the GIME, which handles video and memory management. The GIME manages physical memory in 8 KB blocks, so a 512K machine has 64 of them. Its memory management unit, the MMU, picks which 8 of those blocks the processor sees at any moment. Change a register, and a different block appears in the window. This is called banking, and this whole port relies on it.

The CPU's 64 KB as eight windows over 64 blocks of 8 KB

Pool of Radiance, on the other hand, is a Turbo Pascal program for 16-bit x86, written for a PC with up to 640 KB of memory. It reaches all of it through far pointers, addresses made of two parts, a segment and an offset, and swaps its 35 overlay units of code in from disk as needed. So we have a game built for half a megabyte or more, on a machine that can look at 64 KB at a time. The table has the rest. The difference that bites quietly is byte order: the PC is little endian and the 6809 is big endian, so every 16-bit number in every data file is stored backwards for the CoCo.2

PC original CoCo 3 port
CPU 8088/x86, 16-bit, segments 6809 at 1.79 MHz, or a Hitachi 6309
Address space 640 KB, far pointers 64 KB at a time; 512 KB in 64 blocks of 8 KB through the MMU
Code 197,641 bytes of x86 in 35 overlay units 258,179 bytes of 6809 code for all 46 units
Data 113 archives, 1.27 MB packed, 2.6 MB unpacked the same files, repacked into a sector image
Video EGA, 320 by 200, 16 colours, planar GIME, 320 by 200, 16 colours, 4 bits a pixel
Sound PC speaker, or the Tandy 1000's three voice chip a 6-bit digital to analogue converter (DAC) at $FF20
Storage hard disk or floppies 35-track floppies, CoCo SDC, SuperIDE CompactFlash
Byte order little endian big endian

One thing up front: this port needs a 512K CoCo 3. At boot it points the MMU at four blocks spread across memory, and checks that they hold different bytes. A 128K machine fails that test, and the game politely sends you back to BASIC.3

An emulated 128K CoCo 3, sent back to BASIC

Three layers

This port rests on two layers of earlier work, and that's why it was possible at all.

The first layer is the decompilation. We rewrote Pool of Radiance in Turbo Pascal 6.0, the compiler SSI used, until that compiler turned our Pascal back into the exact original machine code, routine by routine, byte for byte. All 804 routines match, and the full game linked from that source is identical to the original. That story, and the team of AI agents behind it, is the first article in this series. What matters here is that we had the game's logic as readable source code, proven to be the original.

The second layer is a translation into C. A Python translator of about six thousand lines converts the Pascal into portable C, and the game logic is generated, never edited by hand.4 Under it sits a hand written runtime that does what Turbo Pascal's runtime did, plus a model of the EGA card. That C version runs on a modern PC, verified by replaying recorded play sessions against the original game.

The third layer is the CoCo 3 port. The same translator, with a CoCo profile, generates the game logic for the 6809. The engine underneath is written by hand: about 2,700 lines of 6809 assembly and about 2,900 lines of C, for graphics, sound, disk, keyboard and memory.5

Here's the game's dice roll in all three forms:

Pascal
{ src/OVL24.PAS }
function RollDice(Count, Sides: Byte): Byte;
var
  I: Byte;
  Total: Byte;
begin
  Total := 0;
  if Count > 0 then
    for I := 1 to Count do
      Inc(Total, Random(Sides) + 1);
  RollDice := Total;
end;
C
/* c/game/ovl24.c, the C build for the PC */
Total += (u16)(rtl_Random(Sides) + 1);

/* the same line from the CoCo profile */
Total += (u16)((u16)rtl_Random(Sides) + 1);

That extra cast matters; the compiler section explains why.

One routine three ways, and who writes what

Ten translated routines have also been rewritten by hand, for speed, and every one of them still has to pass the same play sessions against the original. Take ScanTargets, which is how a monster picks who to attack. In the City Hall fight, with 30 members, it runs 560 times. Each call looks at every combatant, and walks a line for every pair of cells in its arc. The AI took 40 percent of the time the game spent working. The patch skips any line that can't succeed: a line of N steps can't when N is past the scan's range, and once a line to a combatant is found, any pair that can't give a shorter one is skipped too. The result is the original's. Here is the heart of it, as first written:6

C
/* ports/coco3/patches/ovl30_scantargets.c, step 8.9e */
N = (u16)(DxA > DyA ? DxA : DyA);
if (N > MaxSteps || (Found && Best <= (u16)(N * 2)))
  continue;
if (InDirArc(CandX[J], CandY[J], TargX[K], TargY[K], Dir)) {
  Ex = TargX[K];
  Ey = TargY[K];
  StepCount = MaxSteps;
  if (WalkLine(CandX[J], CandY[J], &Ex, &Ey, &StepCount, Map)) {
    Found = 1;
    if (StepCount < Best) {
      Best = StepCount;
      ...

With the line walk patch under it and one smaller fix, that step took the fight from 579.5 seconds of work to 454.7. The file in the repository has a sharper bound today, found later, but the idea is the same.

Every successful full game port to the CoCo 3 we studied followed one rule: keep the logic, replace the hardware. The decompilation let us take that literally. The logic wasn't an approximation from watching the game. It was the game.

And then there was a gift. On the PC the game supports several video cards, and one of them is the Tandy 1000. The Tandy path packs two pixels into each byte, four bits each, with a base address for every row:

Pascal
{ src/RES07.PAS, the Tandy 1000 path }
AdapterTandy:
  begin
    ...
    for Row := FirstRow to LastRow do
    begin
      Move(PPicture(Buf)^.Data[SourceOfs], Mem[TandyScanlineSegments[Row]:FirstCol], RowBytes);
      Inc(SourceOfs, RowBytes);
    end;
  end;

That's exactly how the CoCo 3's graphics memory is laid out. So the port keeps the game's picture bytes exactly as they are; only where each row lands has to be worked out for the CoCo. Without knowing it, SSI had written a code path for our screen, decades before anyone thought of this port.

The Tandy path and the CoCo 3's screen: the same layout

Measure first

Before writing the engine, we measured. Step zero was a hello world with banking: a core, two code banks, and calls between them. A call across banks needs a trampoline, a little piece of code in the middle that saves the current bank, maps the new one, jumps, and undoes all of that on return. A far call measured 287 cycles, against 111 for a direct call. The estimate had been 60 to 90 extra cycles; it was 176, two to three times worse.7 That became a theme: every estimate got measured, and a lot of them turned out to be wrong.

Where do the 176 cycles go? (The listing above is today's trampoline, which has gained a bank check and a least recently used stamp since that measurement.) Every bank starts with a table of three byte jumps, so a small stub in the core can reach any entry. The trampoline moves the caller's return address onto a separate bank stack, so the routine in the bank sees exactly the stack of a direct call, then maps both halves of the 16 KB window and jumps:

Output
* ports/coco3/src/engine/farcall.asm
__farcall
        stx     <FC_X                   target entry
        sty     <FC_Y                   caller's Y
        stu     <FC_U                   caller's U
        puls    x                       X = caller's return address
        ldu     <FC_BSPTR
        pshu    x                       bank stack: return address
        ldx     <FC_Y
        pshu    x                       caller's Y
        lda     <FC_CURBANK
        stb     <FC_CURBANK
        pshu    a                       the bank that was mapped
        stu     <FC_BSPTR
        ldu     <FC_U                   caller's U back before the call
@retry  ldx     #_banktab-1
        lda     b,x
        beq     @missing                the bank is not in memory: the loader has to bring it
        bsr     fcstamp                 used now: the loader evicts the bank used longest ago
        sta     MMU_SLOT4               map the callee's bank
        sta     MMU_SH
        inca
        sta     MMU_SLOT5
        sta     MMU_SH+1
        jsr     [FC_X]

One known cost: U, Y and X, three of the 6809's registers, all carry live values, so the bookkeeping goes through scratch words in memory (FC_X, FC_Y, FC_U; remember FC_X, it comes back later). But this is a turn based game, and a key press with a hundred far calls spends about ten milliseconds in trampolines.

Step one was to compile every unit of the game with CMOC, a C compiler for the 6809. Everything compiled, and at first the overlay code came out only about six percent bigger than the original x86. At that point only one unit was bigger than 16 KB, at 22.7 KB, and it splits cleanly in two, but the biggest single routine, the one that creates a character, is 9,172 bytes. So 8 KB code windows were out, and 16 KB windows work. Code wasn't the problem.8

Data was. The game logic wants its globals and its heap in reach of a plain pointer at any moment: 45 to 58 KB. Next to a 16 KB code window, the engine and the stack, that doesn't fit in 64.

So we measured harder: we counted every data reference the C build made over seven recorded sessions, 23.1 million of them. 98.9 percent came from the picture, blit and text code, exactly the code the new engine replaces. The game logic made a median of 135 per key press.9

23.1 million data references, and who made them

Almost all the data traffic was drawing, which was moving into the engine anyway, so most of the logic's data could live in banks, as long as what it uses most stayed within reach. That shaped the design. The engine gets its own MMU task, which is its own private map of memory, and it owns the pictures and the screen. The game's hot data, about 22 KB, stays mapped where the code can reach it directly. Cold data lives in banks, behind small accessor routines.

The hybrid data model: the game's map, and the engine's

A compiler that counts in bytes

Here's the first real trap. In standard C, when you multiply two bytes, the maths happens in int, which is 16 bits here. CMOC does byte arithmetic in bytes:10

C, with bytes Standard C CMOC
row * 40, with row = 10 400 144
a + b, bytes 200 and 100 300 44
a - b, bytes 7 and 10, stored in an int -3 253

CMOC's warning for this misses every case that involves a constant. The fix couldn't be a hunt through tens of thousands of lines of generated C; it had to be in the translator. The CoCo profile puts an explicit width cast on every operation (the extra (u16) in RollDice above), so the 6809 computes exactly what Turbo Pascal computed. The code grew from 245,986 to 258,179 bytes. Correct beats small.

CMOC had real bugs too. Two of them live on as probes in the repository (the comments here are ours):11

C
/* ports/coco3/probes/asm/cmoc-bug-decb.c */
enum { Step = 5 };
#define Morale (*(unsigned char *)0x6825)

void lower(void)
{
    Morale -= Step;     /* CMOC 0.1.100: DECB #$05, not a 6809 instruction */
}

/* ports/coco3/probes/asm/cmoc-bug-negbyte.c */
#define Roll (*(signed char *)0x6822)

void miss(void)
{
    Roll = -1;          /* CMOC 0.1.100: LDB #$FFFF, rejected by the assembler */
}

The second is sixteen bits in an 8-bit load. Both only happen at a literal address, and that's how the generated C writes every global in the game, so the translator never writes those two shapes.

Then came the empty battlefield. The first time combat ran, every cell of the combat map was the same tile, tile zero.

Every cell tile zero: a reconstruction from the fix's own notes

CMOC had folded a comparison of minus two against two as unsigned, and decided a loop never ran. So the routine that builds the battle map compiled into a single memory fill. The translator now works out constant loop bounds itself, and the battlefield came back.12

You may be wondering why we stuck with CMOC, given all these code generation problems. We did look at the main alternative, gcc6809, a port of the GNU C compiler to the 6809, and on paper it looked better:13

CMOC gcc6809
One published benchmark of small game routines about 2 times the cycles, 1.28 times the code the baseline
Byte arithmetic in bytes (the trap above) standard C rules
Far calls our own trampoline built in
Maintenance maintained (0.1.100), already in our toolchain container based on a GCC from 2008, described as hard to build; its native compiler reported to crash on Apple Silicon
CoCo support a CoCo target and runtime; real CoCo 3 games ship with it no start-up code or runtime for the CoCo
The game's long integers supported (32-bit long, through slow library calls) the prebuilt runtime lacked the 32-bit multiply

But that benchmark measured small routines, not code like ours. Every pixel loop in this port is assembly whatever the compiler, and the game is turn based. So we stuck with CMOC for now, and kept the generated C neutral enough to try another compiler later. The real test, compiling three of our own units with both compilers and comparing size, speed and correctness, hasn't been done yet. Trying a better compiler is something I'd love to do in the future.

These are the bugs this port is made of: not the game's bugs, the toolchain's. And the only way to find them is to compare against something you trust.

How we know it is right

So how do we know it's right? We trust the original game. We recorded seventeen play sessions on the PC original, running in our own emulator harness, which we call the oracle: creating a character, exploring Phlan, camping and saving, shopping, fighting at City Hall, casting spells. Sixteen of them run on the CoCo.

The CoCo side runs in MAME, the multi-machine emulator, headless, with no window at all. A Lua script inside it types each key of a recorded session, one key every 0.4 seconds, because Pool of Radiance polls for keys in between and misses letters if you type too fast. The script keeps every screen that stays still for a fifth of a second and compares it with the original's. It also dumps memory and compares the game's global data place by place, then the party records, and every file the game writes.14

The oracle records, MAME replays, and everything is compared

Even the copy protection has to match. The game asks for a word from the code wheel, chosen by a random roll out of 13. The harness reads which question the game chose and answers it, then resets the random number generator to the seed the original had at that moment. From then on, every die rolled on the CoCo is the die rolled on the PC.

So when 16 of 16 sessions match, every checkpoint screen is equal, there are no unexplained differences in the game's data, and the party records and saved files match. We re-run all of it after every optimisation.

It catches things that would be hard to catch any other way. In one of the early fights, a kobold moved three half squares instead of six, and the screens drifted apart. The first difference in the dice logs of the CoCo and of the C build on the PC was a Random(100) where it should have been a Random(1). Digging deeper, the kobold's carried weight read 11,776 instead of 46: one 16-bit word hadn't been byte swapped when the monster was loaded (46 is $002E, and $2E00 is 11,776). The game thought the kobold was carrying a small house.15

The harness had its own bugs too: for most of the combat work it sorted screen names as text, so screen 100 came before screen 32, and every long session looked like a failure. And when we added ten new sessions to test more of the game, we found that no spell had ever worked, on the CoCo or in the C build on the PC. The line that calls the spell handler had been translated into an expression that does nothing. On top of that, every shop price was 256 times too high: a hand axe cost 256 instead of 1.16

Coverage is now 52.7 percent of the lines of the translated game, up from 39.5. Not everything, but everything those sessions touch is checked against the original after every change.

Graphics

The graphics started out easy. The GIME has a 320 by 200 mode with 16 colours, 4 bits a pixel and 160 bytes a row: the Tandy layout the game already knew. The palette maps too. EGA and the GIME both use 6-bit colour, and one line of bit shuffling turns an EGA colour into the exact GIME colour on an RGB monitor:17

C
/* ports/coco3/src/engine/gfx.c, from its header:
   GIME bits R1 G1 B1 R0 G0 B0 = EGA bits 2 1 0 5 4 3, that is ((e & 7) << 3) | (e >> 3) */

Text was harder. The GIME's text mode has its own fixed font, so the game's font, 177 glyphs of 8 by 8 pixels, is drawn in software. One glyph cost 8,360 cycles at first, and 2,840 after the first rewrite.

That matters because of how the game draws. A window frame is about a hundred separate symbol calls, and a menu screen is three thousand tiles and eight thousand glyphs; on the CoCo each one was a far call into the engine. So the cure wasn't just faster drawing, it was fewer, bigger calls: a whole run of tiles, or a whole line of text, in one go. With both, a key press in the shop went from 1.26 seconds to 0.66.18

The inner loops are 6809 assembly, in a file of kernels: copy, mirror, mask, fill. The best trick is an old one: the 6809 has a second stack pointer, the user stack, and it can push several registers to memory with one instruction. Point the user stack at the screen, load the registers with the fill colour, and each push writes six bytes. A fill writes 24 bytes in 52 cycles:

Output
* ports/coco3/src/engine/kernels.asm, k_fill (comments ours)
@quad   lda     9,s                     the fill byte
        tfr     a,b                     D = two of them
        tfr     d,x
        tfr     d,y
@l24    cmpu    <FC_D
        blo     @l4
        pshu    d,x,y                   six bytes, one instruction
        pshu    d,x,y
        pshu    d,x,y
        pshu    d,x,y
        bra     @l24

The 3D view is an 11 by 11 grid of 8 by 8 tiles, 88 pixels square, with walls painted back to front. Before optimisation, one step forward drew 198 tiles through about 200 separate engine calls, and took 2.34 seconds. We profiled it, and one of the biggest costs wasn't graphics at all. Every step runs the area's script, and its decoder built strings one character at a time. In Pascal, s := s + Chr(c) looks innocent; in our runtime it copied the whole 256 byte string, twice, per character, half a second of every step:19

Change A step forward
Before 2.34 s
Append to a string in place 1.81 s
Copy 8 by 8 tiles straight from the picture's block 1.33 s
Row address and wall piece lookups 1.27 s
A brief sound waits only as long as it sounds 1.19 s
Draw a whole wall piece in one call 1.13 s
Look a picture up once a call 1.07 s
Window registers 1.06 s
32-bit compares made 16-bit 1.04 s

One more that every 6809 programmer has hit: LEAU B,U, which adds an 8-bit register to a pointer, treats the register as signed. Rows are 160 bytes, more than 127, so drawing wrapped backwards, and the credits screen showed it. ABX, which adds unsigned, fixed it.

Sound

The PC version of Pool of Radiance beeps, through a sound driver, linked in as assembly, that plays tunes on the PC speaker or on the Tandy 1000's three voice chip. The game plays eleven of them, all sound effects; there's no music.20

The driver is basically a small interpreter of tune streams. A timer interrupt runs it 236.7 times a second, and on each tick it steps up to four voices. A voice has a pitch that can slide, a level that can rise or fall, an envelope and a vibrato table, and its stream tells it when to change. The PC speaker can only play one square wave, so the driver sends it the first voice that has something to play, as a divisor for the timer chip.21

The original driver: a timer tick, four voices, a tune stream, one square wave out

The CoCo has neither a PC speaker nor the Tandy's chip, but it does have that 6-bit DAC, so the port makes the square wave by hand. The driver itself is the same; only its last step changes. The divisor becomes a half period for the GIME's timer, which can trigger a fast interrupt (FIRQ), and every half period the handler flips the DAC between two levels, $40 and $C0.

A low note needs a half period longer than the timer can count, so the interrupt counts several of them. A new note starts at the timer's next expiry, never by restarting the timer, so no part of a wave gets lost. The handler is about 71 cycles. It's copied at boot into the constant page near the top of memory ($FE00), a page that stays put whatever the MMU is showing. The driver's own clock, 236.7 ticks a second, is just a count of those interrupts.22

A square wave by hand

The driver isn't hand written. It's generated from the C port of SSI's driver, which was proven first: it produces the original's 7,045 speaker events, event for event.

Six effects: each original, from the oracle's record, above the CoCo's DAC in MAME

For ten of the eleven effects the game uses, the number of DAC flips is within five percent of what our model of the driver predicts, and that model makes the original's speaker events one for one. The eleventh is the step, a thud only five flips long, and the CoCo made four.

DAC flips against the model, per effect

There are limits. Pitches above 3.6 kHz are capped there, but only tunes the game never plays go that high. The sound is only as steady as the game's polling, and a disk read masks the interrupt, so the wave stops for its length.

When we first wired it all together, it was quiet: every tune would load, but none would play. The graphics setup was writing to a GIME register, $FF90, that also holds the enable bit for the fast interrupt, and it switched the interrupt off. The engine's own sound test had passed, because it started interrupts last.

With sound on, all 16 sessions still match.

Memory: no ceilings

Here's where memory gets tricky. After the ten new sessions, three of them stopped with a fatal error: the mapped heap had run out of space.

The mapped heap is part of the 22 KB of hot data that stay within reach. It's where the game keeps the records it creates as it plays (characters, monsters, items and spell effects), and it had about eight and a half kilobytes. A big fight can fill it.

The City Hall fight on the CoCo, a crowd of monsters

Every cheaper option we looked at had a ceiling of about 33 combatants, but the game's scripts can put up to 63 members into a single encounter. I said no ceilings. This is a full game port: if a script in some corner of the map builds a 63 member fight, the CoCo has to be able to run it.

So characters, combatants, effects and items moved into a 61 KB banked heap, reached through an assembly accessor that maps the right block in about 65 cycles. The mapped heap's peak use dropped from 8.5 to 3.3 KB, and a stress build with 63 members fills 46 of the 61 KB without a fatal error.23

Then there's the code. At that point, the game's code was 48 banks of 16 KB, and only a few could be in memory at once, in a pool of slots. In one fight, the game made 23,741 calls across banks in 520 seconds, and three routines made more than half of them (52 percent).24

48 code banks, a pool of slots, 23,741 far calls in one fight

The pool had already bitten us once. Two builds a few bytes apart stopped the same fight in the same place: in a memory dump, the bank stack held eight nested banks, every slot the pool had then, and the game wanted a ninth. As far as we can tell, whether it happened depended on where a big unit was split in two.

The bank stack from the dump, and a ninth bank asked for

The fix is in the trampoline: if a caller's bank was evicted while the callee ran, the return path loads it back.

Output
* ports/coco3/src/engine/farcall.asm, the way back
@remap  ldx     #_banktab-1
        lda     a,x
        beq     @reload                 the caller's bank was taken for another while the callee ran: bring it back

Now the long sessions pass even with a pool of two banks.

Back to all those calls. First, bank groups: banks that are used together get merged into one. Second, a smarter pool: the trampoline stamps every bank it enters (that's the bsr fcstamp in the listing above), and when a slot is needed, the least recently used bank goes. Then we found two more blocks that the disk routines in ROM leave alone, and turned them into a seventh slot.25

Bank loads over fifteen sessions
Before 20,123
Bank groups and the least recently used pool 6,972
A seventh slot 2,760

Down 86 percent. It felt like we had won.

Storage: the disk that lied

It turns out we hadn't. During development the CoCo's "disk" was a Lua script in the emulator that returned every sector instantly, which was great for testing, and hid the real cost of a bank load for ten whole steps of the project.

Once we built the floppy disks and measured them in the emulators, the picture changed. In a five minute session, 88 percent of the bytes read from disk were code banks being reloaded: 2.3 MB of 2.65. At floppy speed that's about eleven minutes of waiting in a five minute session. For comparison, the PC original reads 384 KB of overlays in a whole combat session. The first player report, from someone running the floppy set in the XRoar emulator, said it was bootable, but "excessively slow".26

The floppy disks, measured in the emulators

So we made storage a rule. There's now an I/O budget,27 and a change that reads more than the budget doesn't get merged.

The fix used the original game one more time. The oracle can trace which routines the PC game runs, and in what order, in every session. A tool takes those traces and packs routines from any unit into banks, by what actually runs together. It uses simulated annealing: a search that keeps trying small swaps, and slowly gets pickier about which ones it keeps. A pool simulator, which reproduces real runs to within one load (498 loads against 499 measured), scores each layout.28

The first layouts had one more surprise: a routine nobody saw. A fight used seven banks, the whole pool, and then BriefWait made it eight. PlaySound calls it, but BriefWait is ours, a helper the port added so a short sound only waits as long as it plays (the "brief sound" line in the 3D step table). The original doesn't have it, so the oracle's traces never showed it. It sat in a cold bank, and every call knocked a hot one out: 43 extra loads in one session. Now the tool knows that PlaySound calls BriefWait, and every layout leaves one slot spare, for the next routine nobody saw:

Python
# ports/coco3/tools/rebank.py (comment shortened)
# Routines the port runs that the oracle never calls (helpers the port added, such as BriefWait, which mkgame.py adds to PlaySound in step
# 8.9d): they run right after the routine that calls them.
FOLLOWERS = {"PlaySound": ["BriefWait"]}
BriefWait, the routine nobody saw
Code read in a mean session
Before 2,360 KB
Banks packed from the oracle's traces 236 KB

A factor of ten.

Along the way we had to fix the tracer itself: some original routines exit by unwinding the stack, the tracer never noticed, and one combat profile ran for five hours without getting halfway. Fixed, it takes five to fifteen minutes.

Four ways to read a sector

So far I've said the game reads from disk. But which disk? The game never touches a drive itself: everything goes through one engine call, sec_io(cmd, lsn, count, block, offset), read or write this many sectors, starting at this sector number, into this block of memory. Behind it are four backends: the standard engine picks one of three at boot by looking at drive zero's directory, and the SuperIDE build has an engine of its own, which looks for the card.29

One call, four backends

The first backend is the one that lied. In development, sec_io rings a doorbell, and the Lua script in the emulator copies the sectors straight into memory.

The floppy build is a set of twelve 35-track disks. It reads each sector through DSKCON, the disk routine in Disk BASIC's ROM, one call per sector. When a sector is on a disk that isn't in a drive, it asks for that disk on the CoCo's own text screen.

The floppy build asks for a disk: INSERT DISK n IN THE SECOND DRIVE, THEN PRESS ENTER

That prompt had a bug of its own: after a saved game loaded, the game ran its area script forever. The prompt cleared the screen with the fill kernel from the graphics section, which borrows a scratch word as it goes. That word is FC_X, where the trampoline keeps its target across a bank load, so the next load jumped to $A004. Every read returned the same data as in the development run; only the control flow was different, and comparing the engine call logs of the two runs is how we found it.30

Another annealer chose which files go on which disk, to cut the swaps. Over the sixteen sessions, with two drives, the simulated swaps went from 2,513 in the PC's file order to 1,044, and with the new code banks from the last section, to 274.

Swaps aside, floppies are slow for a simpler reason. The ROM routine reads one sector per call, and while the call returns the next sector passes under the head, so most of the time goes to waiting for the next sector it wants to come under the head. We measured about four kilobytes a second in XRoar. A driver of our own that reads a whole track, eighteen sectors, in one turn would be five to six times faster, but it isn't written yet.

Why the floppies are slow: one sector per call, then waiting for the next one to come under the head

The CoCo SDC is an SD card interface. Its build is one file, a small bootable disk that Disk BASIC can see followed by the whole game image, so there are no disk swaps at all. The engine skips Disk BASIC and talks to the SDC's command mode directly: it writes a sector number and a read command to the SDC's registers, waits until it's ready, and takes the sector as 128 transfers of two bytes. So far, though, it's only been tested against a model of the card's registers.

The SuperIDE is an IDE interface that takes a CompactFlash card. Its driver also talks to the interface directly, and only borrows HDB-DOS, the card's own disk system, to load the program. HDB-DOS puts one 256 byte sector in each 512 byte card sector and wastes the other half; the game packs two in each, except for its saves, and reads up to 16 KB, a whole bank, with one command.31

The SuperIDE card: six virtual drives, two game sectors in each card sector

In its fast mode the card hands over one byte per read. A 6809 reads two bytes and stores them with one instruction, about nine cycles a byte. A 6309, a faster compatible chip some owners fit, moves 256 bytes with a single block move instruction, TFM, at three cycles a byte:

Output
* ports/coco3/src/engine/ide_xfer.asm, 256 bytes in 8 bit mode
gq8     tst     >_cpu6309
        bne     gq8t
        ldy     #16                     bytes on a 6809: two reads of the same register, one 16 bit store
gq8l    lda     >D_DATA
        ldb     >D_DATA
        std     ,x++
        ...                             (eight times in all)
        leay    -1,y
        bne     gq8l
        puls    cc,pc
gq8t    ldy     #D_DATA                 a 6309: LDW #256, TFM Y,X+ (the source is fixed, the destination counts up)
        fcb     $10,$86,$01,$00
        fcb     $11,$3B,$21
        puls    cc,pc

The 6309 instructions are written out as bytes, because the assembler runs in 6809 mode.

Medium Speed
Floppy, through the ROM's DSKCON about 4 KB/s measured in XRoar
Floppy, a whole track at a time about 22 KB/s estimated, driver not written
SuperIDE, 6809 about 150 KB/s estimated, not yet measured on a card
SuperIDE, 6309 with TFM about 500 KB/s estimated, not yet measured on a card

If no card answers, the engine doesn't hang: it prints an error screen with the status bytes and a message such as NO IDE DEVICE ANSWERED. And that block move instruction is one reason the 6309 is worth a closer look.

The 6309

Some CoCo 3 owners, the enthusiasts, swap their 6809 for a Hitachi 6309: a compatible chip with more registers and a faster native mode. The port detects it at boot with one instruction that behaves differently on each chip, and switches to native mode, so the same image runs on both:32

Output
* ports/coco3/src/engine/boot.asm
        ldy     #0
        fcb     $1F,$62                 TFR W,Y
        cmpy    #$FFFF
        beq     @is6809
        fcb     $11,$3D,$01             LDMD #1: native mode (NM = 1; FM stays 0)
        lda     #1
        sta     >_cpu6309
@is6809

On a 6309, TFR W,Y copies the extra W register into Y; on a 6809, which has no W, Y comes back as $FFFF.

Native mode runs the same instructions in fewer cycles: a simple register instruction drops from two cycles to one, a subroutine call from eight to seven, and a load or store from memory takes one cycle less. Instructions that reach memory through a pointer (indexed addressing, such as LDD ,Y++) don't change, though, and our copy loops are made of those. The translated game code doesn't change either, because CMOC can't generate 6309 code. So native mode speeds up everything a little, with no change to the code.

We measured it over the same play sessions on MAME's 6309 machine. Busy time is the time the machine spends working instead of waiting for a key:33

Measured, 6809 against 6309 native 6809 6309
A step in the 3D view 1.05 s 0.91 s -13 %
The aim cursor test, 65 key presses on the combat map 11.3 s 9.7 s -14 %
The 30-member fight at City Hall 455.9 s 413.1 s -9 %
All sixteen sessions, busy time 1,848 s 1,692 s -8.4 %

The 8.4 percent understates it a little, because busy time includes the game's own timed pauses, and a faster CPU doesn't shorten those.

Then TFM, three cycles a byte against eight or nine for our unrolled copy loops. We put it in the three kernels that copy big areas: changed rows to the screen, opaque picture rows, and the combat map's ground and icons, each checking once per call which CPU it's on. Where the screen is redrawn in big pieces, it shows (measured on a later build, so the native numbers differ a little from the table above):34

Measured on the later build Native Native and TFM
The aim cursor test 9.8 s 8.6 s -12 %
Casting a spell at camp, with its big portrait 77.9 s 73.5 s -5.6 %
A step in the 3D view 0.92 s 0.91 s -1 %
All sixteen sessions, busy time 1,704 s 1,679 s -1.5 %

The fills stayed as they were, because the stack push trick from the graphics section already writes a byte in about two cycles (1.8), faster than TFM's three.

Native mode also crashed, inside the ROM. Disk BASIC's interrupt handler throws away a 12 byte stack frame, but in native mode the 6309 pushes 14. The play sessions didn't catch it, because they serve the disk without the ROM; the engine tests did. Now the disk driver leaves native mode around every call into the disk ROM.

The last bytes

A few small things. The engine's shared code was first linked at $0400, which is BASIC's text screen. So while the game loaded, you watched seventeen seconds of machine code drawn as letters. We moved it (to $1D80), and turned that screen into a splash with a progress bar.35

The old LOADM, machine code drawn as letters (reconstructed), and the splash that replaced it

Then quality of life. The CoCo build drops the code wheel, adds a credits line, and tells you to return to BASIC instead of DOS. A build switch puts the faithful game back, and every test runs on the faithful game.36

The credits on the quality of life build

After the 6309 work, the engine ended with seven bytes left before its limit. We could have made room by giving up picture slots, but bigger fights need them. No ceilings, again.

Real hardware day

Up to this point, everything ran in emulation: MAME for testing, and XRoar, a second CoCo emulator, for a second opinion. On day 18 of the whole project, counting from its very first commit, I put the SuperIDE build on a CompactFlash card and into a real CoCo 3.

And it worked, the first time. Boot to the title screen in under twenty seconds. A saved game to town in about twenty more. A battle starts in about ten.37

The first run, on a stock 6809

The card held the SuperIDE build on HDB-DOS virtual drives 200 to 205 of a 256 MB card; before it writes anything, the card tool reads the card and refuses unless those six drives are free. The first real run never needed the error screen. And one thing surprised me: my CoCo turned out to be a stock 6809, not a 6309 as I had thought, so that first run took the plain 6809 path, which is what most CoCos have.

Since then, I've replaced the 6809 in this same CoCo with a Hitachi 6309, and the photo at the top of this article is the port running on it. The splash detects it and goes into native mode. The card accepts its 8 bit transfers, so every sector the game reads comes in through the TFM path of the ide_xfer.asm listing above. From EXEC to the title screen takes about eleven seconds. Loading a saved game to the 3D view takes about fifteen. After choosing to fight, the battlefield is on screen in about six seconds. In the fight against a group of orcs, Arcadia casts Sleep, an orc falls asleep, and the party wins.

The 6309 run: Sleep cast on a group of orcs, on the real machine

This is a later build than the first run, so the times aren't a clean comparison between the two chips.

What is still open

It's worth being clear about what isn't proven yet:38

Status
16 of 16 sessions, on an emulated 6809 and an emulated 6309 proven in emulation
All 16 sessions on the floppy set, and on models of the SDC and the IDE card proven in emulation
SuperIDE on a stock 6809: boot, a saved game, a fight proven on my CoCo 3
SuperIDE on a 6309 in native mode: boot, a saved game, two fights proven on my CoCo 3
A game saved to the card survives a power cycle checked by hand on my CoCo 3
The PC speaker effects, ported to the DAC, on the real machine checked by hand on my CoCo 3
The CoCo SDC driver tested against a model only
Real floppy drives not yet
How fast the card actually reads not yet measured
The wilderness, the ending, most spells no recorded session yet

But I think we've proven the heart of it. SSI's PC game, its logic intact, is running on a 1986 machine with 64 KB of address space, and in every session we recorded, it's screen for screen the same as the original.

If you have a CoCo 3 with 512 KB, or an emulator, the builds are on the downloads page, and reports from real hardware are very welcome. Everything described here is in the GitHub repository, with a doc for every step of the port:

Where What
ports/coco3/ The port: its README, Makefile and tests
ports/coco3/docs/ One doc per step, from 8.0 to the last, with every measurement
ports/coco3/src/engine/ The hand written engine, assembly and C
ports/coco3/patches/ The routines rewritten by hand for speed
docs/coco/ The research and the porting guide written before step 8.0
c/, tools/c/ The C translation and its translator
src/ The matched Pascal

What you won't find there is the game itself: to build any of it from source, you need your own copy.

Next in this series, we leave the hardware behind and look inside the game itself: one evening in Phlan, traced through every part of the engine.

Notes and sources

  1. docs/coco/research/06-prior-art-games.md, from Wikipedia's Gold Box and SSI pages. That it was the first Gold Box game is general history, not something we proved. ↩︎

  2. docs/coco/PORTING-GUIDE.md, section 2. The code size for the port is from PHASE-8.2.md. ↩︎

  3. ports/coco3/docs/PHASE-8.9i.md, section 2: the test maps blocks $00, $10, $20 and $30 in turn. ↩︎

  4. tools/c/pas2c.py and its companions, 6,199 lines of Python; how the C build is made and proven is in c/README.md. ↩︎

  5. 2,736 lines of 6809 assembly and 2,914 lines of C (headers included) in ports/coco3/src/engine/. The profile is described in PHASE-8.2.md. ↩︎

  6. PHASE-8.9e.md, sections 1 to 3; the listing is the step 8.9e version (ovl30_scantargets.c at that commit); the file today is ports/coco3/patches/ovl30_scantargets.c, whose comment explains both bounds, with PHASE-8.9k.md. ↩︎

  7. PHASE-8.0.md, section 5. ↩︎

  8. PHASE-8.1.md. The six percent was measured before the width casts of the next section; with them, the overlays are 1.11 times the x86. ↩︎

  9. PHASE-8.1c.md. ↩︎

  10. PHASE-8.0.md, 3.1, and PHASE-8.2.md, section 1. ↩︎

  11. ports/coco3/probes/asm/. Found while compiling the game in PHASE-8.1.md, 2.2. ↩︎

  12. PHASE-8.7.md, finding 1. ↩︎

  13. docs/coco/PORTING-GUIDE.md, 6.4 and decision D11, and docs/coco/research/04-cmoc-lwtools-toolchain.md. The benchmark is of small Vectrex routines, with CMOC 0.1.98. ↩︎

  14. ports/coco3/lua/watch.lua and PHASE-8.5.md, section 1; what "match" means is in PHASE-8.7c.md, section 0. ↩︎

  15. PHASE-8.7.md, section 1.2 and finding 6; the sort is finding 2. ↩︎

  16. PHASE-8.7b.md, section 0, with the coverage figures. ↩︎

  17. The comment at the top of ports/coco3/src/engine/gfx.c. The glyph costs are from PHASE-8.3.md, section 3. ↩︎

  18. PHASE-8.9f.md, table 1. ↩︎

  19. PHASE-8.9d.md, sections 2, 3 and 6. The signed offset is in PHASE-8.9e.md, step 4. ↩︎

  20. c/README.md, "Sound": tunes 12 to 20 are in the driver, but nothing plays them. ↩︎

  21. The header of src/RES03.ASM and of its C port, c/driver/res03_driver.c. ↩︎

  22. PHASE-8.8.md, sections 0 to 3. The flip counts were measured again for the video with make snd-wav. ↩︎

  23. HEAP-STUDY.md and PHASE-8.7c.md. ↩︎

  24. PHASE-8.9a.md, section 2. ↩︎

  25. PHASE-8.9a.md, section 0, and PHASE-8.9b.md, section 0. ↩︎

  26. PHASE-8.10.md, section 8, and PHASE-8.11.md, sections 1 and 2. ↩︎

  27. ports/coco3/data/io_budget.json. ↩︎

  28. PHASE-8.11.md, sections 3 to 5; the tool is ports/coco3/tools/rebank.py. ↩︎

  29. PHASE-8.10.md, sections 1 to 4, and PHASE-8.12.md for the SuperIDE. ↩︎

  30. PHASE-8.10.md, 2.3. The swap counts are from its section 3 and from PHASE-8.11.md, 5.1. ↩︎

  31. PHASE-8.12.md, sections 1 to 3, and ports/coco3/src/engine/ide_xfer.asm. ↩︎

  32. PHASE-8.9h.md, section 1, and ports/coco3/src/engine/boot.asm. ↩︎

  33. PHASE-8.9h.md, section 2, for all sessions; the per flow numbers come from the research runs on MAME's coco3 and coco3h machines. ↩︎

  34. PHASE-8.9j.md, sections 1 and 2. ↩︎

  35. PHASE-8.9i.md. ↩︎

  36. PHASE-8.13.md. The engine's last bytes are in PHASE-8.9j.md, section 4. ↩︎

  37. PHASE-8.12.md, 4b, and ports/coco3/tools/ide_card.py. ↩︎

  38. PHASE-8.9j.md, section 3; PHASE-8.10.md, sections 4, 6 and 7; PHASE-8.12.md, 4b; RISKS.md. ↩︎

Related