Main Page
Blog
Downloads
Credits
FAQ
Forum Thread
 

rpix86 - Blog

Feb 8th, 2019 - Status update

Wow, it has been a while since I last posted a blog post here! Regarding my rpix86 project, sadly there has not been any progress since 2015. I have been busy with my other projects. In case you are interested in learning what I am currently working on, I decided to announce my current project in this blog as well.

I am working on porting my old LineWars II game into mobile virtual reality! The game is called LineWars VR, and it will be released within this year (2019) on Oculus Go and Samsung Gear VR. You may be aware that rpix86 is actually based on my earlier dsx86 project for Nintendo DS. It was actually the launcher program LW2.COM for my LineWars II game that I used as a test software when originally developing DSx86, so actually my rpix86 project is in a round about way related to my LineWars games.

You can follow my progress with LineWars VR from my dedicated web pages for that project at http://linewars.patrickaalto.com/LWVR.html. I just released an introductory trailer video for LineWars VR on YouTube. Thanks again for your interest in my projects!

June 28th, 2015 - rpix86 0.19 released!

Okay, here is a new version of rpix86 again. For the past month or so I have been working on the problems in TradeWars 2002 and Tyrian 2000. Both of the games had problems that were very difficult and time consuming to track down. In this version I have managed to fix most of the problems, but it looks like Tyrian 2000 is still having some issues which I will continue to look into. TradeWars 2002 seems to be running fine now.

TradeWars 2002 compatibility improvements

As I mentioned in the previous blog post, I still had some FPU opcodes unimplemented in the previous 0.18 version of rpix86, which caused TradeWars 2002 not to run. In this version I implemented those last (as far as I know) missing FPU opcodes. The missing opcodes were the 16-bit integer operations (that is, loading and storing a floating point register value as a 16-bit short integer, and arithmetic operations with a floating point register and a 16-bit integer value), and the FBLD and FBSTP opcodes. The latter are BCD (binary coded decimal) floating point load and store operations. Those were a bit tricky to implement (which is why I had not implemented them originally), and they are probably not quite as efficient as they could be, but since TradeWars 2002 was the first game I have encountered that uses them, they are probably not used very much.

After implementing those missing FPU opcodes, TradeWars 2002 began crashing with a runtime error: 

    TW2002 has ended in error. The errorcode is 205
    EC=205 LF=StrTable.D8A L1= L2=ReadAt_STRING_TABLE Addr=11F6:7E33
It took me quite a long time and a lot of debugging of the game code to find the problem. While I was debugging this, I decided to improve my FPU logging, and doing this I ran into a really strange problem, where even a simple 
    printf("%f", 1.0);
resulted in 0.0, and printing the rpix86 FPU register values sometimes printed some really nonsensical values. I was completely at loss as to what is going on, and thus started a thread at the Raspberry Pi forum asking for help and ideas to help me debug the problem. It turned out that I had completely forgotten the fact that the stack needs to be 8-byte aligned when calling C routines from ASM! Thanks again to the forum member Paeryn who pretty quickly figured out what my error was.

After I had proper logging I was able to compare the FPU opcode results from rpix86 with the corresponding results in DOSBox. This showed me that I had nothing obviously wrong in my FPU opcodes, so I had to simply trace the game code after the last succesful FPU opcode until the error happened. This again took a while, but finally I found the code that generates the 205 runtime error: 

    1E6E:0BE4    sub    sp,000A         Make room for 10 bytes on the stack
                 mov    bx,sp
                 fwait
                 fstpt  ss:[bx]         Store the floating point value to stack as an 80-bit (10-byte) value
                 nop
                 fwait
                 add    sp,0002
                 pop    cx
                 pop    bx
                 pop    dx
                 pop    ax
                 mov    di,ax
                 and    ax,7FFF         AX = exponent part of the stored floating point value
                 sub    ax,3F7E
                 jbe    0C1A
                 or     ah,ah
                 jne    0C25            Jump taken if need to exit with ErrCode 205
                 ..
    1E6E:0C25    mov    ax,00CD         0x00CD == 205 == error code
                 jmp    010F            Go to runtime error handler
This looks like a sanity check code that makes sure the exponent is within a sensible range. What was strange was that my FPU log showed the FPU value being exactly 0.0! However, the AX value that the code tested was 0x4940, which fell outside of the acceptable range. I checked my FSTPT opcode implementation, and sure enough, when the floating point value was zero, I had forgotten to clear the exponent part when saving the value to memory! Fixing this got finally rid of the errorcode 205 problem in TradeWars 2002.

The next problem was that the TW2002 configuration program TEDIT crashed with an unsupported JPE opcode. Since rpix86 has no native parity flag emulation, I need to implement support (or a "hack") for every game that uses JPE or JPO opcodes. The implementations themselves are usually quite easy and simple to do, though. After fixing the couple of places where I found the JPE or JPO opcodes in the TEDIT.EXE, it also began to run fine, as far as I was able to test it.

Tyrian 2000 debugging

Tyrian 2000 had a strange problem where it seemed to run fine with SoundBlaster digital audio enabled on my old Raspberry Pi Model B, but it crashed immediately with a Runtime Error 100 followed by an Unhandled Exception 000B when running on my new Raspberry Pi 2 device. I suspected some kind of a timing issue, or thread synchronization issue at first.

This problem also required quite a few days of debugging. I started by looking into that Runtime Error 100 problem, as I suspected it was the root problem which then later on caused the Unhandle Exception to occur. Debugging this was quite difficult, as adding some debugging code made the game run slower, and then it suddenly began to work without problems (as in my slower Raspberry Pi). So, I needed to adjust my debugging code all the time to even get the problem to occur!

Finally I was able to determine that the problem occurs when the game is attempting to read in data from a file (which makes sense, as the Borland standard Runtime Error 100 means Disk Read Error). The game requested 4 bytes, but my INT 21 handler returned 0 bytes read. For a while I looked at a possible problem in my file reading code, until I realized that the real-mode INT21 handler actually gets a request for reading 0 bytes as input! I was able to determine that the protected mode call gives the CX register a value of 4, but my real mode DOS handler gets CX register with a value of 0. So something goes wrong within the protected mode to real mode data transfer.

After a lot of further debugging I found out that the game code reads a value from protected mode address 00BF:0010 (which is in a real-mode address 9B60:0010) for some internal checking, and that value was 9 in all the prior disk read operations, but was 0x8080 when the problem read occurred. Next I used my memory watch feature to track when that value (which looked suspiciously like the Sound Blaster sample 8-bit silence value 0x80) gets written at. I found this code, which looks like the SoundBlaster digital audio sample copying routine: 

    0257:2010   mov     cx,017F
                sub     cx,ax
                ...
    0257:2036   repe    movsb
After that operation AX register had a value of 0x180, so it looked like the CX register (that the copy operation got as input) was 0xFFFF, meaning that the sample copy copied 65535 bytes! However, the DS and ES segment selectors were both 02DF, which had the base at 0x8BC00 and a size of only 0x61F (1567) bytes! For performance reasons I do not check for protected mode segment overruns (this is mostly unnecessary as the 32-bit protected mode programs most likely use a flat 32-bit memory addresses, and when they do not use that, they will not try to write beyond their allowed ranges). So, it looked like the AX value was too large, it should be at most 0x17F and never 0x180 or more. With the full 65535 bytes copied, it overwrote the values at the real-mode address 9B60:0010 (which it a base address 0x9B610, 64016 bytes after the 02DF segment base address).

I then checked where the AX register gets it's value, and found out that it checks the DMA registers to determine how far the SoundBlaster digital audio playing has progressed. Checking the DMA register values I realized I had a bug in my DMA register code, as the game set the DMA length to 0x17F but my DMA current address values went 1 byte over the end! I decreased the DMA current register value by one, and that fixed this immediate crash in Tyrian 2000.

There are still some issues with Tyrian 2000, it looks like it still crashes with a similar Runtime Error 100 occasionally when playing the demo games for a long time. I will still continue looking into that problem, it may be caused by something similar.

Anyways, I hope you find the latest version of rpix86 useful, and thank you again for your interest in my emulators!

June 14th, 2015 - Received my Raspberry Pi 2!

Setting up a Raspberry Pi for compiling rpix86

I had ordered a Raspberry Pi 2 about a week before, and on Friday the 5th of June it arrived! I spent that weekend setting it up, with a goal of switching my rpix86 programming work over to the newer and faster Pi. Almost since my original Raspberry Pi was new, I have been having some SD card related problems with it. It runs for a couple of hours, then it suddenly starts failing with "Journal has aborted" and "journal commit I/O error" messages, after which I can only turn it off and back on. After the restart it again works for a couple of hours. I have been worrying that it will eventually corrupt my SD card, and I would have a hard time restoring my rpix86 build system back to working order.

So, when I got my new RPi 2, I copied all my rpix86 sources over, and then just began trying to compile it. For future reference I'll log here all the compile and linking errors I encountered and how I solved them. This should help me in the future if I need to reinstall everything at some point.

  1. The first problem was 

        /usr/include/interface/vmcs_host/vcgencmd.h:33:27: fatal error: vchost_config.h: No such file or directory
    I googled for solutions to this problem, and according to this forum post one (if perhaps not the best) way to solve this is to simply edit the vcgencmd.h file to have the correct path to that include file. I did that, which fixed this problem.

  2. Next I encountered 

        fatal error: X11/Xlib.h: No such file or directory
    This got solved by installing the X11 development environment: 
        sudo apt-get install libx11-dev

  3. The next step was to build the libilclient.a library, which was missing. This was done by running make in the /opt/vc/src/hello_pi/libs/ilclient directory.

  4. The next problem was that the curses libraries were missing. I use the ncurses library when rpix86 is launched from the terminal window. To fix this I just needed to install the ncurses development package. 

    sudo apt-get install libncurses5-dev

  5. Somewhat strangely, next I needed to add -ldl -lm to my Makefile. I did not need to have these libraries specifically included in my original Raspberry Pi Makefile, for some reason.

  6. Finally, I wanted to access my Raspberry Pi 2 rpix86 folder from my Windows 8.1 PC, similarly to how I had shared the corresponding folder on my original Raspberry Pi. I used the instructions in this post for the basic Samba configuration, and then just copied the relevant /etc/samba/smb.conf sections from my old Pi over to the new Pi 2: 

    [rpix86]
comment = rpix86 share
path = /home/pi/rpix86
valid users = @users
force group = users
create mask = 0666
directory mask = 0777
read only = no

[ipc$]
valid users = nobody @users
    I remember having some problems when I tried to configure samba on my original Pi, but this time it began working immediately. I think it was that last [ipc$] section that was originally missing, and it seems to be required for Windows access to Raspberry Pi.

Work on supporting TradeWars 2002 continues

Since my last blog post I received some additional information about how to properly install and set up TradeWars 2002 (thanks for that!), and found out that I still had some FPU opcodes unimplemented. I implemented the missing FPU opcodes, which made the game progress further. I am currently looking into the problem where the game exits back to DOS with the following error message: 

Opening Planet File

TW2002 has ended in error. The errorcode is 205
EC=205 LF=StrTable.D8A L1= L2=ReadAt_STRING_TABLE Addr=11F6:7E33
I have managed to use the built-in debugger to run up to the top level routine (that calls some subroutines where this error happens), and am currently in the process of digging deeper into those routines to determine the actual cause.

Tyrian SoundBlaster audio fails on Raspberry Pi 2

I was also told that the SB digitized audio crashes Tyrian 2000 in rpix86 running on a Raspberry Pi 2, but on my old original Raspberry Pi it works fine. I am looking also into this problem now that I have my own Raspberry Pi 2. This looks like some timing or thread synchronization problem. The original Pi only has a single CPU core, so it does not need as much thread synchronization as a proper multi-core CPU that the newer Raspberry Pi 2 has. It seems that the problem is a non-present SB buffer table segment, but determining why the buffer is not present when the first SB IRQ occurs still needs further digging into.

Thanks again for your interest in my work on rpix86, I plan to release a new version after I get at least these two games working properly in rpix86.

May 31st, 2015 - rpix86 0.18 released!

Here finally is the latest version of rpix86! Some pretty big internal changes in this version, so I hope I did not break any games that used to run in the previous version of rpix86.

1. Implemented FPU support in real mode (TradeWars 2002)

I already mentioned this change on my previous blog post. At that point the TradeWars 2002 game did not start up, as it complained that SHARE.EXE was not loaded or was unresponsive. I was able to use DOSBox to determine that the same problem happens there as well, and it is caused by an unsupported INT 21 AH=5C: FLOCK - RECORD LOCKING DOS call. I changed my code to not report an error (but it still does not actually perform any record locking) and that allowed TradeWars 2002 to progress further. However my version seems to miss a file (or I have not set it up correctly) so I don't know if the game actually runs in rpix86 yet.

But in any case, if there are other real-mode programs that need FPU support to run, those should now run properly in this version of rpix86.

2. Implemented support for Borland RTM (Jazz Jackrabbit, Tyrian 2000)

I described this change pretty thoroughly in my previous blog post. After that post I spent some time debugging a timer I/O port problem I had, which caused Tyrian 2000 to hang intermittently. The timer input port 0x40 MSB/LSB toggle got out of sync, and the game got larger values than it expected and went into a never-ending loop. The problem turned out to be my using the same temp variable both for port 0x40 input and output. I changed the temp variable to be different, and this problem seemed to go away. Tyrian 2000 still seems to hang at launch occasionally, but I haven't been able to determine what causes this. If it gets past the launch time, it seems to run fine (but pretty slow).

 

3. Implemented new -noems command line parameter (Ultima VII)

After those fixes I still had some time to implement further features, so I decide to add the -noems command line switch to disable EMS memory (and emulation of EMM386 memory manager features) in rpix86. This seemed to fix Ultima VII, as it checks whether EMM386 is loaded at startup, and fails to start if it is loaded.

4. Changed the GPIO RS-232 port handling to emulate a null modem cable connection

I have not heard from rpix86 users having success using the GPIO serial pins to connect serial devices to their Raspberry Pi and controlling them from rpix86. As this should work but I have not been able to test this myself, I thought that maybe my emulation is not quite as good as it should be. In this version I decided to experiment adding loop-back handshaking to my GPIO serial port emulation. For USB-to-RS232 device this is not needed, as the device should take care of the handshaking. However, since there are no RS-232 handshake pins in the GPIO connector, I thought that it might be a good idea to fake the handshake signals in this situation. At least my Telix communications program now thinks that my GPIO serial port is already connected when I start it up. Let me know if you can test this out and see any improvements in the behavior or compatibility. Or if it now works worse than before.

That's it for the changes in this version! I hope you find this new version useful, and again let me know of any bugs you encounter. Thanks again for your interest in rpix86!

May 24th, 2015 - rpix86 work continues!

Okay, after a break of about a year, I am back working on my emulators! For the past several weeks I have been working either on my Windows Phone version of pax86 or on rpix86. I returned to working on my emulators in the beginning of April, when I got an email describing problems running TradeWars 2002 on Raspberry Pi. I have received various bug reports during the past year, and have been adding them to my todo-list, but this bug report was slightly different as it looked like the game should actually already work on rpix86. So, I thought I would spend some time looking into what is going on with this game.

Work on supporting TradeWars 2002

TradeWars 2002 has two versions, one running under DPMI DOS extender and one running without it. The problem was that the DPMI one exited with "Cannot enable A20" and the non-DPMI version with "Numeric co-processor required". What bothered me was that rpix86 should already support enabling/disbling the A20 line, and it also has support for the numeric co-processor. So why does the game not detect them?

I started with the numeric co-processor problem, as there are several ways the game could attempt to enable the A20 line, and I thought it might be difficult to determine which method the game attempts to use. After some looking into the rpix86 code, I remembered that I had only coded complete FPU support for 32-bit protected mode, as all the games I have encountered so far have only needed FPU support in that mode. The real-mode FPU code simply silently skipped all operations. Perhaps the game tried to use FPU opcodes in real mode, and when the results were not correct, it exited?

I disabled the real mode FPU opcode skipping code, and indeed, the game encountered an FPU opcode in real mode: 

-----------------------[TW2002]------------------------
TW2002: Unsupported opcode! INTVectors=0x42788000
CPU: REAL, 16, 0000FFFF
EAX=00000000 EBX=00003D38 ECX=00007FBD EDX=00001E6E
ESP=00009FFA EBP=00000000 ESI=00000264 EDI=00000042
ES=02D3 CS=1E6E SS=2494 DS=20BF FS=0000 GS=0000
NV UP EI PL ZR NA PO NC VM=0 IOPL=3
1E6E:09EB D93F            fstcw [bx]
That looks like the game attempts to store the FPU control word into memory in real mode. I checked the game code further, and noticed that it does indeed do some floating point calculations in real mode. I decided to spend my Easter vacation working on implementing the real-mode FPU opcodes, as now I had a game to use for testing them. This work mainly consisted of adding the real-mode opcode tables, as the main FPU operations are the same between the real and protected modes. It took several days, though.

After I had implemented the FPU opcodes, the game progressed further, but then exited with the following message: 

File Error (1) - TWNODE.DAT : Invalid Function
05/24/27 00:00:08 AM Game Halted:
SHARE.EXE not loaded or unresponsive.

c:\games\2002v309>
I have spent several days trying to debug the game and see how it determines that SHARE.EXE is not loaded (well, it isn't loaded, but I would like to fool the game into thinking it is), but have not yet been able to determine this. I recently tested the game in DOSBox, and it gets the exact same error, so I can not even use DOSBox to debug the game behaviour. Perhaps I will look into FreeDOS implementation of SHARE.EXE to determine the possible detection routines that the game uses. However, I got a bit bored with working on this game so I switched to working on my Windows Phone version of pax86 instead. I wrote a blog post about my work there a couple of weeks back.

Implementing support for Borland RTM DOS Extender

I recently got an email with a debug log from Tyrian 2000. The game crashed in the IRET opcode, which in principle should be supported in both real and protected modes. So, I decided to switch back to working on rpix86 for a while. 

-----------------------[RTM]------------------------
RTM: Unsupported opcode! INTVectors=0x7387c000
CPU: PROT, 16, 0000FFFF
EAX=000002A8 EBX=00000FFC ECX=000000B7 EDX=00003000
ESP=00000F94 EBP=00000F8C ESI=00000F9E EDI=00001000
ES=005B CS=0020 SS=0018 DS=0018 FS=0000 GS=0000
NV UP DI PL NZ NA PO CY VM=0 IOPL=3
0020:000015EC CF              iret
Since the game runs in protected mode, there are several possible reasons why rpix86 would exit at that point. I improved my error logging so that instead of the generic "Unsupported opcode" rpix86 would report the unsupported situation with IRET that it encountered. Running with the enhanced logging resulted in this log: 
-----------------------[RTM]------------------------
RTM: Unsupported IRET to outer level!
CPU: PROT, 16, 0000FFFF
EAX=000002A8 EBX=00000FFC ECX=000000B7 EDX=00003000
ESP=00000F94 EBP=00000F8C ESI=00000F9E EDI=00001000
ES=005B CS=0020 SS=0018 DS=0018 FS=0000 GS=0000
NV UP DI PL NZ NA PO CY VM=0 IOPL=3
0020:000015EC CF              iret
Okay, now that is pretty clear, as I had not yet added support for interrupt return to outer (less privileged) protection level. The fact that the running program is called RTM made me remember that I actually worked on similar issues when I added support for Jazz Jackrabbit into my zerox86 emulator. Since zerox86 runs on MIPS architecture instead of ARM as in Raspberry Pi, the assembler code is very different and thus I had not added similar support to my ARM emulators at that time. However, I had written several blog posts about supporting Jazz Jackrabbit in both my DS2x86 and zerox86 blog posts, so I thought that using those blog posts and taking the ideas and algorithms from my MIPS emulator code would make it reasonably straight forward to implement the same support to my ARM emulators.

  1. Interestingly, when I started up Jazz Jackrabbit, it exited with "Cannot Enable A20". The same problem that the DPMI -version of TradeWars 2002 had. At this point I thought it would help me quite a bit if I got my debug-version of DOSBox running again. I had switched to a new hard drive and installed Windows 8.1 since I last ran Visual C++ 2008 Express for DOSBox, but luckily I still had the install program for that old Visual C++ on my old hard drive. It installed fine, and pretty soon I was able to run Jazz in DOSBox. I noticed that it uses the keyboard controller method of setting the A20 line, and somewhat to my surprise I noticed that I did not have a proper handler for that yet in rpix86. I implemented that, and then got Jazz go drop into debugger in rpix86 at the same point as where Tyrian 2000 drops. Thus, I switched to working on Jazz Jackrabbit, so I could follow my old blog posts step by step.

  2. The first step was adding that IRET return to outer priority level case. This was not terribly difficult, as I was able to simply look at my implementation for zerox86, mentally convert the MIPS asm to ARM asm code line by code line. I had to be careful to use the correct CPU registers in my new ASM code, by comparing this code to the normal IRET code I already had implemented, though.

  3. After I had implemented the IRET opcode, the game not surprisingly encountered the priority level jump to the opposite direction. This is done using an INT opcode. 

    -----------------------[RTM]------------------------
RTM: Unsupported opcode! INTVectors=0x42840000
CPU: PROT, 16, 0000FFFF
EAX=00000300 EBX=00000021 ECX=00000000 EDX=000002A8
ESP=00000F9E EBP=00000FD4 ESI=00000F9E EDI=00000FA2
ES=005B CS=00B7 SS=005B DS=0000 FS=0000 GS=0000
NV UP DI PL NZ NA PO CY VM=0 IOPL=3
00B7:000001F7 CD31            int  31
    Very similar approach worked here, I implemented the new code based on my zerox86 code.

  4. Next, the game encountered a HLT (halt) opcode in protected mode. This opcode is not allowed in any other priority level except 0 (the most privileged), otherwise it will cause a General Protection Fault Exception. 

    -----------------------[RTM]------------------------
RTM: Unsupported opcode! INTVectors=0x428b0000
CPU: PROT, 16, 0000FFFF
EAX=00000303 EBX=00000735 ECX=00000000 EDX=000000F4
ESP=00000FCE EBP=00000FFA ESI=00002258 EDI=00002195
ES=0000 CS=0113 SS=005B DS=00B7 FS=0000 GS=0000
NV UP DI PL NZ NA PO CY VM=0 IOPL=3
0113:0000000C F4              hlt
    The game was running in priority level 3 at that point, so it was actually supposed to cause a GPF. However, I had not implemented any exception handling in rpix86 yet, so that was the next step in my work on supporting the RTM DOS Extender.

  5. At this point I had to spend some time hunting down a bug I had typed into the new INT handling code. Such bugs are pretty difficult to hunt down, even when I know the bug is very probably in the code I just added. When staring at the code you have written, you always see it as you meant it, not as you accidentally typed it in. :-) Anyways, after fixing that bug I was back on track (as I kept following my prior blog posts for those MIPS emulators), the next problem was the LES opcode that encountered a segment that was not present. 

    -----------------------[RTM]------------------------
RTM: Unsupported opcode! INTVectors=0x42830000
CPU: PROT, 16, 0000FFFF
EAX=00000100 EBX=000003F4 ECX=000004AA EDX=00002000
ESP=000003D2 EBP=000003D4 ESI=0000396C EDI=00000000
ES=009F CS=00DF SS=009F DS=00E7 FS=0000 GS=0000
NV UP DI PL ZR NA PO CY VM=0 IOPL=3
00DF:0000C391 C47E06          les  di,[bp+06]
    Here I added the Page Fault handling using my new exception handling mechanism, and again was able to progress further.

  6. There had been several additional steps at this point that I had had to do for my MIPS emulators. However, since those enhancements were done into the C code, I had already implemented them also for my ARM emulators. The MIPS and ARM emulators share most of the C code, but the ASM code is obviously separate. Thus I was able to skip any new changes for reporting no extended memory for the Borland RTM, adding the "last fit" memory allocation strategy, and returning the proper return value from DOS call INT 21 AH=4D.

  7. The next step was to handle opcode 0x8E, which loads a segment selector register. This also needed page fault handling, much like the LES opcode above. As you may notice from the debug log, at this point the actual game [JAZZ] is already running, while in the previous log entries rpix86 was still running the [RTM] loader program. 

    -----------------------[JAZZ]------------------------
JAZZ: Unsupported opcode! INTVectors=0x427d8000
CPU: PROT, 16, 0000FFFF
EAX=00000000 EBX=000003EE ECX=00000540 EDX=00000000
ESP=000003EE EBP=000003F0 ESI=00002386 EDI=000023C7
ES=009F CS=00E7 SS=009F DS=00EF FS=0000 GS=0000
NV UP DI PL ZR NA PO CY VM=0 IOPL=3
00E7:00000593 8E46FE          mov  es,[bp+FE]

  8. Then I had to implementation the Page Fault handling also to the RETF opcode handler... 

    -----------------------[JAZZ]------------------------
JAZZ: Unsupported opcode! INTVectors=0x427f0000
CPU: PROT, 16, 0000FFFF
EAX=0000FB43 EBX=00000000 ECX=000001C7 EDX=00000000
ESP=00001FFC EBP=00000000 ESI=00000282 EDI=000023C6
ES=01AF CS=00E7 SS=028F DS=01AF FS=0000 GS=0000
NV UP EI PL ZR NA PO CY VM=0 IOPL=3
00E7:000003A6 CB              retf

  9. ... and next the corresponding Page Fault in the CALL opcode. 

    -----------------------[JAZZ]------------------------
JAZZ: Unsupported opcode! INTVectors=0x42810000
CPU: PROT, 16, 0000FFFF
EAX=00000000 EBX=00000277 ECX=0000078A EDX=00000277
ESP=00002000 EBP=00000000 ESI=0000028B EDI=0000EA32
ES=0287 CS=01D7 SS=028F DS=0287 FS=0000 GS=0000
NV UP EI PL ZR NA PO CY VM=0 IOPL=3
01D7:0000000C 9AA4395702      call 0257:39A4

  10. At this point the game started up, but then dropped to the debugger with an "Unsupported SB DSP command FF!". As I mentioned in the DS2x86 blog post, the game seems to have a bug where it attempts to send the DMA length as parameters to SB DSP command 0x1C (start 8-bit autoinit DMA). Since this command does not actually take any parameters, those extra bytes are interpreted as additional (undocumented) commands. 

    0417:25DE	cmp	word [22ED],0200	
		jbe	25F8			Jump if SoundBlaster DSP version <= 2.00
		mov	al,1C
		call	22F5			Send DSP command 0x1C (start 8-bit autoinit DMA)
		mov	ax,[22F1]		ax = 0x0800
		dec	ax			ax = 0x07FF
		call	22F5			Send DSP command 0xFF (undocumented!)
		mov	al,ah
		call	22F5			Send DSP command 0x07 (undocumented!)
0417:25F7	ret
    I had hacked these unsupported commands 0xFF and 0x07 to be silently ignored in my MIPS code, so I added a similar hack to the ARM code.

  11. At this point the game started into graphics mode, but it kept crashing almost immediately with "Unhandled exception 0008 at 3042 0020" followed by differing ErrCodes. At times this happened even before the game went into graphics mode, but usually during the intro graphics. The game immediately dropped back to DOS at that point, and usually the whole rpix86 went unresponsive after a few key presses. To be sure I also tested Tyrian 2000, and it crashed similarly, so this was obviously caused by some bug in my code. This never happened in my MIPS emulators, so it looked like this was something specific to my ARM implementation.

    I googled for that Exception 0008, and it seems to mean "Double fault", which occurs when the processor detects an exception while trying to invoke the handler for a prior exception. I never generate that exception in my code, so it looks like the Borland RTM generates that by itself when it encounters that situation. But how do I find out where and why that happens?

    I assumed that RTM considers exceptions and hardware interrupts to be equivalent in that situation, so the most likely cause was that my hardware IRQ emulation interrupts an exception handler. I began looking at how I disable hardware interrupts when an exception occurs. The code seemed fine, the interrupts are disabled as soon as the exception handler starts. I added logging for every situation where the Disable Interrupts flag changes value, but that did not seem to show anything out of the ordinary. Finally I added also logging for whenever an interrupt or exception handler starts, and that produced this interesting code snippet: 

    0277:000001C0 EAX=00003F60 EBX=05AF0005 ECX=00000060 EDX=00000000 ESP=000016DA      Enable interrupts
              EBP=000016E4 ESI=00001A0B EDI=00001B3E DS=05FF ES=028F SS=028F 1      1 = interrupts are now enabled
              FB9C sti
0020:00000C7A EAX=00000006 EBX=05AF05FF ECX=00000060 EDX=00000000 ESP=00000FEC      GPF Exception Handler starts
              EBP=000016AA ESI=00001A0B EDI=00001B3E DS=00BF ES=028F SS=0018 0      0 = interrupts are now disabled
              558B push bp
0020:00000B52 EAX=00000006 EBX=05AF05FF ECX=00000060 EDX=00000000 ESP=00000FE6      Timer IRQ Handler starts
              EBP=000016AA ESI=00001A0B EDI=00001B3E DS=00BF ES=028F SS=0018 0      0 = interrupts are now disabled
              558B push bp
    It looked like the GPF Exception handler had just started and turned the interrupts off, but then immediately the timer interrupt handler started, even though the interrupts should have been disabled!

    I looked into my code that disables the interrupts, and found a potential race condition. If an interrupt happened while the exception was being generated, it was possible for the interrupt code to mark that the next opcode to run needs to be the interrupt handler. The exception handler did not reset that mark when it turned the interrupts off, only the hardware interrupt handler handled this situation correctly. I made the exception handler behave similarly to how the hardware interrupt handler behaves, and after that change the game no longer encountered that Exception 0008 problem.

  12. The next step was to add code to read data from file directly to Mode-X VRAM memory. This reading from file directly to graphics RAM is only done in two games that I have encountered, Heimdall which uses this in the EGA graphics mode, and Jazz Jackrabbit in Mode-X. The C call for this routine already existed but was commented out, as I had not implemented the actuall handler in the ASM code yet. Without this code the Jazz character in the 3D intro game did not have any textures and was black, as in this screen copy:

    Adding the code was pretty simple, as I had commented the MIPS code pretty well in my zerox86 source code. It took only a few minutes to add the same code to the ARM Mode-X ASM code.

  13. My zerox86 blog post was also very useful in implementing the enhancements for the few Mode-X opcode handlers that Jazz Jackrabbit uses in the game menu selector drawing. These images from my zerox86 blog show the problem on the left and the fixed code on the right:

     

  14. I had spent considerable time debugging the Jazz audio problems in zerox86, so I added the same code enhancements to my ARM SoundBlaster emulation code. Thus Jazz Jackrabbit sounded pretty good even the first time I ran it.

Okay, now Jazz Jackrabbit was running fine, so I switched back to checking Tyrian 2000. It too seemed to run otherwise fine, but it did not play any audio. I will need to look into this a bit further.

I have not yet had time to test the new rpix86 properly, so I did not want to release the new version today. However, if nothing major happens, I should be able to release version 0.18 next Sunday! I am still checking my todo-list for additional improvements I might have time to implement, but at least Jazz Jackrabbit will run in the next release. Thanks again for your interest in my emulators!

Apr 6th, 2014 - rpix86 version 0.17 released!

Here is the latest version of rpix86! Not a lot of changes, as I still have not had all that much time to work on rpix86, but there are a couple of changes that you might find useful.

 

Sorry I have not had time to work on more enhancements. I hope you enjoy this version, and let me know of any bugs you encounter!

Mar 9th, 2014 - rpix86 version 0.16 released!

Okay, after a bit of a hiatus on the rpix86 front, here finally is a new version of rpix86! I have ported a few features over from my other project, and also managed to spend a couple of days fixing some minor bugs for this version. Here are the most notable changes:

1. Implemented new experimental CD-ROM support

You can now give a new -c (as in CD-ROM) command line parameter to rpix86. This parameter gives the directory on your Raspberry Pi that should be used as the root directory of D: drive. This D: drive is emulated as a CD-ROM drive (when a DOS program queries the type of the drive). For example, if your game is located in /home/pi/rpix86/Games/MYGAME and you have a subdirectory CDROOT under that directory that contains all the files from the CD of the game, you can give a parameter -c/home/pi/rpix86/Games/MYGAME/CDROOT to have rpix86 use the contents of that directory as a D: drive.

Note that ISO images (or any other similar CD images) are not supported, you need to extract all the files from the image to a directory on your Raspberry Pi. I am looking into a possibility of adding ISO image support in the future.

2. Fixed the rpix86 screen copy feature

I had broken the screen copy feature in rpix86 several versions ago, but I had not had much need for that feature myself so I had not bothered to fix it. Now I finally decided to look into this problem. It turned out that I had accidentally moved the screen copy routine to a location in the code after the screen was cleared, which is why all the screen copies had just a black screen. In this version you can again take a screen shot by pressing the PrintScreen button on your keyboard. The screen copies are written into SCR??.BMP files, with the ?? being an incremental number.

3. Ignore the "drop-to-debugger" key press if debugger is not available

I added code to detect whether the current screen size allows running the inbuilt debugger, and if the debugger is not activated, the "drop-to-debugger" key (right Windows key) will not do anything. The built in debugger is meant for my own debugging use, so you can freely ignore this information completely. :-)

4. Improved previous fix for writing to unmapped EMS page

The fix I made in version 0.15 for the Aces Over Europe bug was not a very good one, so I made a better fix in this version. Now writing to an unmapped EMS page writes to normal low memory, just like it works on a real PC.

5. Improved mouse initialization (fixes crash in Fragile Alliance)

There was a minor problem in the mouse initialization routine which left some mouse scaling variables uninitialized. This could cause a division by zero problem in some games that tried to change the mouse scaling immediately after initializing the mouse. This problem is now fixed.

6. Added support for HLT opcode in protected mode (Fragile Allegiance)

Another minor problem was that the game "Fragile Allegiance" used the HLT (Halt) opcode from protected mode, and I had not yet implemented that yet. Implementing it was quite easy, and that fix allowed Fragile Allegiance to run.

7. Implemented a missing EGA opcode (Spacewrecked)

The game "Spacewrecked" failed to run because I had not implemented a rare EGA mode opcode it used. After implementing that the game began to run fine.

Thanks for your continued interest in rpix86, and let me know of any bugs you find in this version! My time to work on a new rpix86 version continues to be somewhat limited, but I will try to work on it at least every now and then.

Feb 23rd, 2014 - Lack of rpix86 progress

First off, sorry for my lack of blog updates for the recent weeks. I have not had time to work on rpix86 for a few weeks now, as I have been busy working on stuff relating to my license agreement with Retro Infinity. I am currently porting my x86 emulation core to a new platform, but this work falls under a NDA so I can not tell you more about it. This work has a deadline, and I prefer to get all stuff that has a deadline done sooner rather than later. Thus, rpix86 is on hold until this work is done.

I can however tell you about one enhancement that will be in the next version of rpix86. I will add a simple CD-ROM support to the next version. This means that you can give a second file folder parameter to rpix86, and this will be used as a D: drive and will show up as a CD-ROM to the DOS programs. This will allow you to run some programs that expect the CD to be present, and will also help installing some DOS programs distributed on a CD. This is also a small step towards ISO image support, which I hope to eventually have in rpix86.

In any case, thanks again for your continued interest in rpix86, and I hope to be able to get back to working on it within a few weeks or so.

Jan 5th, 2014 - rpix86 version 0.15 released!

Happy New Year! I decided to start a new blog page now that the year has changed, so that the single blog page does not grow too big. You can find the previous blog entries using the link at the bottom of this page.

This version has mostly game-specific fixes, but also a new somewhat experimental -f2 filtering option which may improve performance in some games. Here are the details of the fixes:

1. Fixed Crystal Caves hang on title screen

I got a test report asking if the Crystal Caves hang could be fixed, and I was somewhat surprised to hear that rpix86 had such a problem. This game has been working in DSx86 for ages, so it should work fine in rpix86 as well. I began to debug the problem, and noticed that indeed it hangs completely on the title screen. After some digging I found out that the reason it hangs is that the Interrupt Enable CPU flag is not set while the game waits for either timer interrupt or keyboard interrupt. So it never gets either of those interrupts and thus hangs.

The question was then to find out what code clears the Interrupt Enable flag. I added some debug logging to all the code that clears the flag, but that produced tons of output as every interrupt clears the flag for the duration of handling the interrupt. In the end I almost accidentally noticed that the game uses INT 03 (opcode 0xCC) which is a debugger breakpoint interrupt. This has some special handling in my code, and when looking at that code (and comparing it with older code from DSx86) I finally found the problem. In my rpix86 code I pushed the CPU flags with the interrupt flag already cleared, while I should have pushed the flags BEFORE clearing the interrupt flag. Fixing that bug allowed Crystal Caves to run normally. It uses similar code to Commander Keen 1 for horizontal smooth scrolling, though, so the scrolling may be somewhat jittery.

2. Removed unnecessary delay loop from default BIOS keyboard IRQ handler

While debugging Crystal Caves, I noticed that it overrides the BIOS INT 9 keyboard interrupt and checks for the pressed key using input from port 0x60 (the Keyboard Data Port). Input from that port clears the Output Buffer Full bit in the Keyboard Status Port 0x64. However, the game then calls the original BIOS INT 9 handler, which first loops up to 65536 times waiting for the Keyboard Status Port to report that the keyboard buffer is full, before proceding to read the key from port 0x60. This code is based on the actual BIOS of a PC clone, and I hadn't bothered to change that behaviour. (As a curious side note, one of my first PC clones (from around 1985 or so) contained the full printed BIOS source code as an appendix of its manual. That was quite helpful when I coded my own emulated BIOS routines originally for DSx86.)

This waiting for the keyboard status meant that when playing Crystal Caves (and possibly other similar games) there was an unnecessary delay loop of 65536 cycles that happened after every key press. Since I know that the keyboard interrupt in rpix86 only happens after a key is pressed, I removed that delay loop from my emulated BIOS routines.

3. Added experimental -f2 filtering option, for hardware dispmanx scaling

Next I looked into a strange problem in Lemmings where the game runs at more or less normal speed when I run it in a 640x480 window (which I usually do when debugging games in rpix86), but has bad stuttering when it is run full-screen (on my full-HD display). Since nothing in my code except the OpenGL ES drawing routines behave in any way differently depending on the screen size, this seemed quite strange. The OpenGL ES routines should handle all scaling in hardware (or so I thought), but just to be sure I added some timing code around the glTexSubImage2D() call I use to copy the emulated graphics memory to an OpenGL texture. This was the result:

Screen size and filteringTimeMax theoretical framerate
640x400 -f01.5 ms667 fps
640x400 -f11.5 ms667 fps
1920x1080 -f015 ms66 fps
1920x1080 -f125 ms40 fps

To my big surprise, this single call took a lot more time when the screen size increased! With texture filtering on (-f1 parameter) it caused rpix86 to not even reach 60fps emulation speed, and even with no texture filtering (-f0) it only left very little time for the actual CPU emulation!

So, it occurred to me that perhaps I should check what happens if I do the size scaling using the dispmanx call instead of OpenGL ES, so that OpenGL can always draw into 640x480 (the maximum VGA resolution that rpix86 supports) screen size. This produced the following timings, so I decided to include this feature as a -f2 filtering mode. Feel free to experiment with this parameter, it may help performance in some situations. The scaling is not as smooth as with the -f1 parameter, though.

Screen size and filteringTimeMax theoretical framerate
640x400 -f01.5 ms667 fps
640x400 -f11.5 ms667 fps
640x400 -f21.6 ms625 fps
1920x1080 -f015 ms66 fps
1920x1080 -f125 ms40 fps
1920x1080 -f21.6 ms625 fps

4. Implemented missing CD-ROM call INT2F AH=1501 (X-Wing INSTALL.EXE)

Next I checked why the INSTALL.EXE of X-Wing caused a soft crash in rpix86. I noticed that the game uses a CD-ROM detection method that I had not implemented yet, so I added that code (as a dummy routine, so that the game sees there is no CD-ROM device). This allowed the INSTALL.EXE to show the main selection screen.

5. Fixed DOS Get STDIN Status call to handle enhanced keyboard correctly

However, after getting the INSTALL.EXE main selection screen to show, I could not actually change the selection, as the cursor keys did not seem to work! This also needed some more debugging, until I found out that the game uses the DOS INT21 AH=0B GET STDIN STATUS call to check if a key is available. I had support for that call, but looking at that description I realized that I did not handle the enhanced keys properly. I originally emulated the old 83-key cursor keys in DSx86, and only later switched to enhanced 101/102-key DOS keyboard emulation. At that point I did not fix that DOS call to also handle enhanced key codes correctly. The fix was quite minor, and it made the INSTALL.EXE work properly.

6. Implemented several missing Mode-X opcodes (Aces Over Europe)

Next I tested Aces Over Europe, which had a soft crash when the actual flying should start. There were several VGA Mode-X opcodes that this game uses that have not been used by any other game. It looks like the game actually stores data (like pointers and such) into the VGA graphics memory, and then uses those later when drawing the screen. This is probably some sort of a speed trick, although it actually only slows things down in rpix86 as the graphics memory is slower to handle than normal RAM. In any case, after adding several such new Mode-X opcode handlers the game progressed to the actual flying quite fine.

7. Fixed writing to non-mapped EMS page (bug in Aces Over Europe)

However, after exiting the game back to 4DOS prompt, giving any command in 4DOS caused a soft crash. Obviously something in Aces Over Europe had corrupted memory that belonged to 4DOS.

I used my inbuilt memory watch routine to detect when the 4DOS memory gets corrupt, and noticed that the memory is already corrupt when 4DOS loads the code back from EMS memory. So, I changed the watch address to be inside the physical memory that 4DOS maps into EMS, and then caught the code in Aces Over Europe that writes to this memory. I confirmed in DOSBox that this seems to be a bug in the game, it writes into unmapped EMS memory page. I had been lazy in rpix86 and left unmapped EMS pages to point to the beginning of my EMS memory area, and in this case it meant that the game wrote to memory that belonged to 4DOS.

I did a quick fix to this problem by reserving one EMS page to be the "unmapped" page, so that any game that tries to write to an unmapped page writes safely to this reserved area. This fixed the crash in 4DOS after exiting Aces Over Europe.

Thanks again for your interest in rpix86, let me know of any new bugs you find in this version!

Previous blog entries