Remote debugging of real mode code with gdb

08/2011

For my work, I recently had to debug an old MS-DOS application that were running under QEmu. QEmu exhibited strange graphical behaviours with this particular application: only the upper half of the screen seemed to be correctly displayed, the other half left blank. The same program works perfectly well on a "physical" old machine. Other facts: it was exactly the same with bochs, but worked good with dosbox. But for various reasons, the program had to work with QEmu, I could not rely on dosbox.

The other difficulty was that I did not had access to the program's sources. It has apparently been built with a DOS-based ancestor of Windev, called "Hyper Screen".

I have been using two complementing approaches to find and fix the problem: debugging and disassembling. The first one allows to focus on a short portion of code where the problem might be located. Once located, the use of a disassembler becomes helpful.

Special mode for GDB

About the debugging part, and because I run the program through QEmu, I had access to the debugging infrastructure offered by Qemu. In particular, it implements a gdb server, meaning a gdb client can connect to it and interact with the emulated CPU.

For that purpose, Qemu may be run with the following flags -S -s: it will freeze the cpu and wait for a gdb client to connect on localhost:1234. (use the target remote localhost:1234 command inside gdb)

The problem is that we want to debug real mode code. This a very annoying mode of the Intel processor, where registers and data are 16 bits wide and 20 bits (well ... 21 sometimes) of address are available. In this mode, every reference to a physical address in memory is made by the use of two pointers: a segment and an offset inside this segment. Each of them is 16 bits wide, and the resulting physical address is obtained by segment * 16 + offset.

GDB and the gdb remote server implementations of Qemu (and bochs) has poor support for this mode. For instance, it does not know anything about segmented memory access and only considers linear memory addresses. That is why you always have to switch between these two representations of memory.

Fortunately, GDB can be scripted quite easily. That is what I did to feel more confortable with this old piece of code. It has been largely inspired by this. The result is very close to what the author displays in one of its post here (in french).

For gdb, the current instruction pointer is given by the register eip. However, in real mode, you also have to consider the code segment register cs. The current instruction in real mode is located in memory at an address pointed to by cs:ip.

When you want to add a breakpoint at a given address inside the code, gdb (and the qemu/bochs remote server) will break when the value of eip reaches this particular address. There are then two problems:

For example, suppose the current values of cs and ip are respectively 0xF000 and 0xE384 and that you have asked the disassembling of cs * 16 + ip , gdb will display something like :

0xfe384       xor ax,ax
0xfe385       call 0xfe395
0xfe388       ...
...
0xfe395

Suppose now that you want to add a breakpoint just after the return of the function, you will have to set a breakpoint at 0xe388 and not 0xfe388. But if you are inspecting a code in memory, you cannot be sure of the offset part of the address. For instance, here you would have the very same display with cs = 0xF100 and ip = 0xD384.

Hereafter is an extract of a debugging session using this special gdb mode.

---------------------------[ STACK ]---
0000 0000 0000 0000 0000 0000 0000 0000 
0000 0000 0000 0000 0000 0000 0000 0000 
---------------------------[ DS:SI ]---
00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
---------------------------[ ES:DI ]---
00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
----------------------------[ CPU ]----
AX: 0000 BX: 0000 CX: 0000 DX: 0633
SI: 0000 DI: 0000 SP: 0000 BP: 0000
CS: F000 DS: 0000 ES: 0000 SS: 0000

IP: C41F EIP:0000C41F
CS:IP: F000:C41F (0xFC41F)
SS:SP: 0000:0000 (0x00000)
SS:BP: 0000:0000 (0x00000)
OF <0>  DF <0>  IF <0>  TF <0>  SF <0>  ZF <0>  AF <0>  PF <0>  CF <0>
ID <0>  VIP <0> VIF <0> AC <0>  VM <0>  RF <0>  NT <0>  IOPL <0>
---------------------------[ CODE ]----
   0xfc41f:	mov    eax,cr0
   0xfc422:	and    eax,0x9fffffff
   0xfc428:	mov    cr0,eax
   0xfc42b:	cli    
   0xfc42c:	cld     
   0xfc42d:	mov    eax,0x8f
   0xfc433:	out    0x70,al
   0xfc435:	in     al,0x71
   0xfc437:	cmp    al,0x0
   0xfc439:	jne    0xfc44e

The secret lies inside the stop-hook() function of the script: it will be executed each time gdb gets awaken.

You can then play with the classical nexti function that will step to the next instruction.

Some helpful macros have been added:

Ideas for a better debugging experience

Considering that gdb server implementations still lack the support of interesting features like the hardware watchpoints and do not have a true support for the real mode (always switching from segmented representation to linear representation is really exhausting), why not use a dedicated client that speaks the gdb "protocol" ?

It could not find anything fancy in this direction, except proprietary solutions, like IDAPro. If someone knows better, I would be grateful.

You can find the gdb script here. Copy it to ''~/.gdbinit'' and let the magic happens.

Tools for disassembling

Once I knew an approximate location of the incriminated code, I switched to a disassembler to figure out the overall logic of this particular portion of code.

Here again, I did not find something that helps the analysis of asm code as mush as IDAPro can do. I recently stumbled upon metasm, but did not had time to deeply test.

And the problem was ...

That the VGABios implementation used by Qemu and bochs was missing some obscure functions used in a very uncommon screen mode (640x350): the computation of the number of lines available to text was not good if you selected a non-standard font first ...

It has now been fixed.