pykd in multiprocessing

Apr 19, 2014 at 4:57 AM
Edited Apr 19, 2014 at 9:08 AM
pykd version 0.2.0.29

I want to trace multi processes, so I have to use multiprocessing
import multiprocessing
import pykd
def attach(pid):
    pykd.attachProcess(pid)

proc=multiprocessing.Process(target=attach,args=(pid,)) 
proc.start()
and I get following error:
BaseException: IDebugClient::AttachProcess failed

the environment is python2.7 x86 in win7 64bit, and target process is 32bit.

I tested python2.7 x64 in win7 64bit, no problem for both 32 and 64 target process

help, Thanks
Coordinator
Apr 20, 2014 at 7:45 AM
1) I tried ti reproduce issue with multiprocessing and I've failed
2) Your code snippet has one serious error: when your code will be forked by multiprocessing, python run all global statements and the process will run recursively. So, I rewrite it for my test:
import multiprocessing
import pykd
import sys
def attach(pid):
    try:
           pykd.attachProcess(pid)
           print "ok"
    except pykd.BaseException:
        print "failed"      

# This prevents from recursively run: 
if __name__ == "__main__":  
    proc=multiprocessing.Process(target=attach,args=(sys.args[1],)) 
    proc.start()
3) While forked process will stopped, attached process will be terminated.
4) You can work with different process from one python program - pykd supports mutitarget ( but I'd recommend for mutlitarget scripts use 0.3 alpha version )

Please, report more detail to I could reproduce issue
Apr 20, 2014 at 11:41 AM
Edited Apr 20, 2014 at 11:42 AM
ok, I reinstalled python2.7 x86 in win7 64, and now it works.

1.
I downloaded "pykd-0.3.0.4-x64-python-2.7-setup.exe" and installed
but I don't know why pykd 0.3 always gives me following error:
>>> import pykd
>>> print pykd
<module 'pykd' from 'C:\Program Files\Windows Kits\8.0\Debuggers\x64\winext\pykd.pyd'>
>>> pykd.startProcess("test.exe")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
__main__.DbgException: pykd is not initialized
0.2.0.29 is ok.

2.
how does pykd trace different process from one python program?
I want to record exceptions of 1.exe/2.exe/3.exe/...

Thanks
Coordinator
Apr 20, 2014 at 2:06 PM
1) For pykd 0.3.0.4 you need call pykd.initialize() from thread where you suppose to control debugging process. 0.2.x always call it implicitly from the first thread where 'import pykd' is called. So, there are some problem with multithread scripts on 0.2.x version. So we added explicit initialization. It is a bit annoying, but you can make mutithread application.

2) startProcess return unique id. And you should everywhere getting current process id by call getCurrentProcessId(). Unfortunately, we have forgotten to export this function. It's bug and it will be fixed next pykd build. You can make a workarround:
def getCurrentProcessId():
    return getProcessIdBySystemID( getProcessSystemID() )
Coordinator
Apr 20, 2014 at 3:25 PM
Apr 22, 2014 at 7:59 AM
Hi, I did some tests for pykd 0.3.0.4.

this is my test code and it works:
#!/usr/bin/env python
# -*- coding: utf-8 -*
# author: SAI
import os,sys,time,traceback
import pykd

def getCurrentProcessId():
    return pykd.getProcessSystemID()#return target processid in my machine
    return pykd.getProcessIdBySystemID(pykd.getProcessSystemID()) #return unique id

class ExceptionHandler(pykd.eventHandler):
    def onException(self, exp):
        print exp,getCurrentProcessId()
        print '-'*10

g_allid={}
def monitor():
    handler=ExceptionHandler()
    try:
        while 1:
            pykd.go()
    except Exception, err:
        print traceback.format_exc()
        print err,'over' #program over but no exception

def test_thread():
    pykd.initialize() 
    for name in ['test.exe','test.exe']:
        id=pykd.startProcess(name) #pykd.attachProcess(xxxpid)
        g_allid[id]=name
    monitor()

import threading
def main():
    th=threading.Thread(target=test_thread)
    th.start()

 if __name__=='__main__': 
    main()
but how can I launch(or attach) another process after main function?
I always need to trace many created dynamically processes.


Thanks.
Coordinator
Apr 22, 2014 at 9:38 AM
I don't understand the last question.
You want to debug children processes? Then you should start process with optional parameter debugChildren: pykd.startProcess( commandLine, debugChildren = True )

If you want to debug all process in your system, you should use some additional tool for process monitoring.

Thank you very much for 0.3.x using. It is in alpha testing state now so please report about any problem you found, I'll try to fix it fast.
Apr 22, 2014 at 6:20 PM
Edited Apr 22, 2014 at 6:20 PM
I mean:
def test_thread():
    ...
    while 1:
            pykd.go()
    ...

def main():
    th=threading.Thread(target=test_thread)
    th.start()
    ....
    #after a while,  then I want to  launch(or attach) and debug new processes.
    #but I can not use pykd.startProcess directly at this time, because the "pykd.go()" is still working  in test_thread function, and there are not the same thread
Coordinator
Apr 23, 2014 at 6:58 AM
You can break debugging by call breakin(). Of course you should do it from another thread. After breakin is called pykd,go() would return "executionStatus.DebugStatusBreak". Then you can start new process and continue debugging with pykd.go()
Apr 24, 2014 at 6:12 PM
ok, maybe breakin() can work (I don't test), but it will cause me to change my code framework,
so, I still decided to use multiprocessing


Thanks very much!