This project is read-only.
1
Vote

from_address crashes when accessing structure members

description

Hi,

First I'm starting a kernel-mode debugging and entering a process context, while the ntdll.dll is loaded at an address 00007fff`aa2b0000, which is shown below. The dd command verifies that this is indeed a PE file.
0: kd> lmi
00007fff`aa2b0000 00007fff`aa481000   ntdll      (pdb symbols)          ntdll.dll

0: kd> dd 00007fff`aa2b0000 
00007fff`aa2b0000  00905a4d 00000003 00000004 0000ffff
00007fff`aa2b0010  000000b8 00000000 00000040 00000000
00007fff`aa2b0020  00000000 00000000 00000000 00000000
00007fff`aa2b0030  00000000 00000000 00000000 000000d8
00007fff`aa2b0040  0eba1f0e cd09b400 4c01b821 685421cd
00007fff`aa2b0050  70207369 72676f72 63206d61 6f6e6e61
00007fff`aa2b0060  65622074 6e757220 206e6920 20534f44
00007fff`aa2b0070  65646f6d 0a0d0d2e 00000024 00000000
The current installed version of Python after loading pykd extension:
0: kd> .load pykd
0: kd> !info

pykd bootstrapper version: 2.0.0.5

Installed python:

Version:        Status:     Image:
------------------------------------------------------------------------------
* 2.7 x86-64    Loaded      C:\Windows\system32\python27.dll
  3.5 x86-64    Loaded      C:\Program Files\Python 3.5\python35.dll
Then I'm using ctypes by first creating the IMAGE_DOS_HEADER structure, then casting an address (where ntdll.dll is loaded) to an actual object, which works fine. However, when accessing the structure member the pykd crashes as presented below.
>>> import ctypes
>>> class IMAGE_DOS_HEADER(ctypes.Structure):
...     _fields_ = [
...         ("e_magic",    ctypes.c_ushort),
...         ("e_cblp",     ctypes.c_ushort),
...         ("e_cp",       ctypes.c_ushort),
...         ("e_crlc",     ctypes.c_ushort),
...         ("e_cparhdr",  ctypes.c_ushort),
...         ("e_minalloc", ctypes.c_ushort),
...         ("e_maxalloc", ctypes.c_ushort),
...         ("e_ss",       ctypes.c_ushort),
...         ("e_sp",       ctypes.c_ushort),
...         ("e_csum",     ctypes.c_ushort),
...         ("e_ip",       ctypes.c_ushort),
...         ("e_cs",       ctypes.c_ushort),
...         ("e_lfarlc",   ctypes.c_ushort),
...         ("e_ovno",     ctypes.c_ushort),
...         ("e_res",      ctypes.c_ushort * 4),
...         ("e_oemid",    ctypes.c_ushort),
...         ("e_oeminfo",  ctypes.c_ushort),
...         ("e_res2",     ctypes.c_ushort * 10),
...         ("e_lfanew",   ctypes.c_ushort),
...     ]
...     
... 
>>> dos = IMAGE_DOS_HEADER.from_address(0x00007fffaa2b0000)
>>> dos
<__main__.IMAGE_DOS_HEADER object at 0x00000000078B6B48>
>>> dos.e_lfanew
c0000005 Exception in C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\pykd.py debugger extension.
      PC: 00000000`1d1ac910  VA: 00007fff`aa2b003c  R/W: 0  Parameter: 00000000`00000000
Does anybody know if there are any fixes regarding this issue, are you aware of this issue?

comments

kernelnet wrote Jul 17 at 9:54 AM

You can not use ctypes in this way.

pykd has special class for typed access to the target memory:
dos = typedVar('ntdll!_IMAGE_DOS_HEADER', 0x76990000)
print dos
print dos.e_lfanew
If you'd like to use ctypes anyway, you need manually read the target memory:
dataSize = sizeof(IMAGE_DOS_HEADER)
data = loadBytes(  0x00007fffaa2b0000,  dataSize )
dos = IMAGE_DOS_HEADER.from_buffer(data)

evelyette wrote Jul 17 at 12:42 PM

Let me present how from_address can be used just fine. First start python.exe, then attach WinDbg to the process as presented below.

Image

Then issue lmi command to get base addresses:
0:001> lmi
start end module name
000000001d000000 000000001d00a000 python (deferred)
000000001e000000 000000001e264000 python27 (deferred)
0000000056bb0000 0000000056c53000 MSVCR90 (deferred)
0000000073720000 0000000073728000 wow64cpu (deferred)
0000000073730000 000000007378c000 wow64win (deferred)
0000000073790000 00000000737cf000 wow64 (deferred)
0000000074b50000 0000000074b5c000 CRYPTBASE (deferred)
0000000074b60000 0000000074bc0000 SspiCli (deferred)
0000000074bc0000 0000000074c61000 ADVAPI32 (deferred)
0000000074ca0000 0000000074cb9000 SECHOST (deferred)
0000000075260000 00000000752c0000 IMM32 (deferred)
00000000755a0000 00000000756a0000 USER32 (deferred)
00000000756a0000 000000007576c000 MSCTF (deferred)
00000000757b0000 00000000758a0000 RPCRT4 (deferred)
0000000075a50000 0000000075a97000 KERNELBASE (deferred)
0000000075b40000 0000000075b4a000 LPK (deferred)
0000000075b90000 0000000075be7000 SHLWAPI (deferred)
0000000075c10000 0000000075d20000 KERNEL32 (deferred)
0000000076030000 00000000760c0000 GDI32 (deferred)
00000000760d0000 000000007616d000 USP10 (deferred)
0000000076170000 0000000076dbb000 SHELL32 (deferred)
0000000076dc0000 0000000076e6c000 msvcrt (deferred)
0000000077090000 000000007723a000 ntdll (pdb symbols) C:\Windows\SYSTEM32\ntdll.dll
0000000077270000 00000000773f0000 ntdll_77270000 (deferred)
Then continue the process with g in order to let python.exe continue and import ctypes, define IMAGE_DOS_HEADER and form an object from the address where ntdll.dll was loaded.
>>> import ctypes
>>> class IMAGE_DOS_HEADER(ctypes.Structure):
...     _fields_ = [
...             ("e_magic",    ctypes.c_ushort),
...             ("e_cblp",     ctypes.c_ushort),
...             ("e_cp",       ctypes.c_ushort),
...             ("e_crlc",     ctypes.c_ushort),
...             ("e_cparhdr",  ctypes.c_ushort),
...             ("e_minalloc", ctypes.c_ushort),
...             ("e_maxalloc", ctypes.c_ushort),
...             ("e_ss",       ctypes.c_ushort),
...             ("e_sp",       ctypes.c_ushort),
...             ("e_csum",     ctypes.c_ushort),
...             ("e_ip",       ctypes.c_ushort),
...             ("e_cs",       ctypes.c_ushort),
...             ("e_lfarlc",   ctypes.c_ushort),
...             ("e_ovno",     ctypes.c_ushort),
...             ("e_res",      ctypes.c_ushort * 4),
...             ("e_oemid",    ctypes.c_ushort),
...             ("e_oeminfo",  ctypes.c_ushort),
...             ("e_res2",     ctypes.c_ushort * 10),
...             ("e_lfanew",   ctypes.c_ushort),
...     ]
...
...
>>> dos = IMAGE_DOS_HEADER.from_address(0x77090000)
>>> dos
<__main__.IMAGE_DOS_HEADER object at 0x02155080>
>>> dos.e_lfanew
224
>>>
As you can see, the e_lfanew was printed as expected, so no issues there. The reason why I do not want to use typedVar is the fact that it relies on the current symbols managed by WinDbg. If the symbols are not present, WinDbg will download them at runtime, import them into WinDbg from where pykd can use them. However, if WinDbg doesn't have access to the PDB symbols due to symbols not being present (PDBs are sometimes not published immediately by Microsoft, which means this won't work), or if the WinDbg's sympath is configured incorrectly (I know, the user is to blame here, but bare with me).

Other than that, from_buffer should work the same as from_address, since ctypes provides the following functions to get object instances. Although there is some confusion about the actual usecase, since the from_buffer/from_buffer_copy return a ctypes instance, while from_address returns a ctypes type instance.
  • from_buffer: This method returns a ctypes instance that shares the buffer of the source object.
  • from_buffer_copy: This method creates a ctypes instance, copying the buffer from the source object buffer which must be readable.
  • from_address: This method returns a ctypes type instance using the memory specified by address which must be an integer.
Can you elaborate what is the exact difference between the ctypes instance and__ctypes type instance__, which seems to be important here. Also, why does the ctypes example I present even work then, if from_address is not supposed to be used like that?

kernelnet wrote Jul 17 at 1:50 PM

You must understand, there are two process: 1) The target application 2) Debugger process ( windbg ).
ctypes is loaded into the debugger process and is working with its address space. But you are trying to use address from the target application's address space. This address is not valid for debugger process.

If you want to work with pykd without symbols, you can define it:
IMAGE_DOS_HEADER = createStruct ("IMAGE_DOS_HEADER")

IMAGE_DOS_HEADER.append(' e_magic', UInt2B)
...
or compile from C code

src = '

typedef unsigned shot   WORD;

typedef struct _IMAGE_DOS_HEADER {      // DOS .EXE header
    WORD   e_magic;                     // Magic number
    WORD   e_cblp;                      // Bytes on last page of file
    WORD   e_cp;                        // Pages in file
    WORD   e_crlc;                      // Relocations
    WORD   e_cparhdr;                   // Size of header in paragraphs
    WORD   e_minalloc;                  // Minimum extra paragraphs needed
    WORD   e_maxalloc;                  // Maximum extra paragraphs needed
    WORD   e_ss;                        // Initial (relative) SS value
    WORD   e_sp;                        // Initial SP value
    WORD   e_csum;                      // Checksum
    WORD   e_ip;                        // Initial IP value
    WORD   e_cs;                        // Initial (relative) CS value
    WORD   e_lfarlc;                    // File address of relocation table
    WORD   e_ovno;                      // Overlay number
    WORD   e_res[4];                    // Reserved words
    WORD   e_oemid;                     // OEM identifier (for e_oeminfo)
    WORD   e_oeminfo;                   // OEM information; e_oemid specific
    WORD   e_res2[10];                  // Reserved words
    LONG   e_lfanew;                    // File address of new exe header
  } IMAGE_DOS_HEADER, *PIMAGE_DOS_HEADER;
'

IMAGE_DOS_HEADER = getTypeFromSource( src, '_IMAGE_DOS_HEADER' )

dos = typedVar( IMAGE_DOS_HEADER, address )

evelyette wrote Jul 18 at 8:45 PM

Thank you for your response, it clears everything! I have one additional minor thing I wanted to ask.

The following example produces the "expected a readable buffer object" exception, because the data is apparently not readable buffer object.
data = pykd.loadBytes(myaddr, ctypes.sizeof(IMAGE_DOS_HEADER))
dos = IMAGE_DOS_HEADER.from_buffer_copy(data)
print(dos.e_lfanew)
However, if I copy the contents of a buffer by using a bytearray as follows, then everything works ok.
data = pykd.loadBytes(myaddr, ctypes.sizeof(IMAGE_DOS_HEADER))
dos = IMAGE_DOS_HEADER.from_buffer_copy(bytearray(data))
print(dos.e_lfanew)
This is all fine, but I would like to avoid the need to copy the memory contents if possible, which is why I choose from_buffer_copy instead of from_buffer function (the from_buffer produced the error "expected a writeable buffer object", which is evident from the documentation). More about this has been discussed at issue11427, but the from_buffer_copy was supposed to solve the issue.

I'm interested why the problem occurs and whether it's necessary to copy the buffer again instead of using the copy that is already provided by from_buffer_copy.