Compiling Python extensions with distutils and MinGWJune 28th, 2008 at 8:32 am
Yesterday I wrote about compiling Python 2.5 extensions using MS Visual Studio. However, I also wanted to make it work using
distutils (as described here), which appeared to be the simplest method.
Trying distutils with Visual Studio 2005 Express
I have the free Visual Studio 2005 Express installed. I used it to successfully compile Python from sources, and wanted to tie it to
distutils to compile the extension. Unfortunately,
distutils is very strict about the MSVC version you compile extensions with, and complained:
running build consulted the Python IRC channel, and got a warm recommendation to use MinGW anyway, because many people has tested its compatibility with the Python distribution.
distutils with MinGW, first attempt
I’ve downloaded the latest version of MinGW, and tried to call
distutils with the appropriate flag:
python setup build -c mingw32 build\temp.win32-2.5\Release\hellomodule.o:hellomodule.c:(.text+0x3e): undefined reference to `_imp___Py_NoneStruct' build\temp.win32-2.5\Release\hellomodule.o:hellomodule.c:(.text+0x46): undefined reference to `_imp___Py_NoneStruct' collect2: ld returned 1 exit status error: command 'gcc' failed with exit status 1
This got me stumped, because I could find nothing useful about this error in Google. However, I did get to this page, where the solution was found in the comments.
The solution
Apparently, the Python export lib (
libs/python25.lib in the Python distribution) is not good enough for MinGW, and it wants a Unix-y
.a file to list the exports of Python’s DLL (the one where the interpreter itself lives).
So you have to create
libpython25.a file for the Python library. If you want, download the one I’ve created and skip all these steps. It should work with Python 2.5
To create
libpython25.a on your own (taken from here with minor modifications):
- Find
python25.dll(it will probably be in
windows/system32)
- Now you have to list the exports from this DLL. Several tools can do this (including
dumpbin), but
pexportsis the most convenient for this task. Get it from here or here (I’ll keep this link alive for a long time, promise).
- Run
pexports.exe python25.dll > python25.def
- Now, if you’ve installed MinGW and added its
bindirectory to your path, you should be able to just call
dlltool(a utility that comes with MinGW):
dlltool --dllname python25.dll --def python 25.def --output-lib libpython25.a
- This has created
libpython25.a
Now just place
libpython25.a in the
libs directory of your Python distribution, and run
distutils again2:
Big success ! You can now use the extension from your Python code.
Updates
- It has been mentioned in a couple of places that with the latest MinGW the
.afile is no longer needed and it’s enough to place the Python DLL in
libs. I’m using the latest MinGW, ActiveState Python and
distutils, and it didn’t work for me that way.
1 I’m using a trivial sample extension called ‘hello’. Here’s its C file:
#include); }
And here is the
distutils setup file:
from distutils.core import setup, Extension module1 = Extension('hello', sources = ['hellomodule.c']) setup (name = 'PackageName', version = '1.0', description = 'This is a demo package', ext_modules = [module1])
2 Note that you can run either
setup.py build -c mingw32 or
setup.py build --compiler=mingw32 - they are suggested in different places, but actually mean the same.
Related posts:
July 25th, 2008 at 07:31
This is a very cool and useful example, thanks for posting it!
I followed your tutorial and it worked perfectly (only had to clean up the ampersand HTML from your C code).
I will just mention that the binary win32 distribution of Python 2.5 ships with the ‘libpython25.a’ file, so users such as myself can skip the pexports step.
The reason I am here is that I am looking for a solution to building the mcrypt library (libmcrypt) on win32 so that I can then build its extension, python-mcrypt. I am stuck in several ways:
1. Problem compiling libmcrypt in VS 2003 because the __declspec(dllexport) issue causes a bunch of syntax errors during compile.
2. Using distutils with “-c mingw32″, I get the error “initializer element is not constant”
3. Using Cygwin to build libmcrypt works perfectly in Cygwin; unfortunately, I cannot for the life of me figure out how to carry over whatever is built there to the Windows side so I can built python-mcrypt.
Do you have any clues?
Thanks,
Basil
July 25th, 2008 at 07:53
Hi Basil,
Could it be a problem with VS’s C/C++ standard compatibility ? Although that probably doesn’t explain the problem with mingw32. However, ‘initializer element is not constant’ should be simple enough to diagnose.
I’m not really an expert on these matters, so you might consider asking in the distutils mailing list.
Eli
July 25th, 2008 at 15:15
Hi Eli,
It is possibly one of the few million compiler/linker/project options in VS, but I am not an expert in that either.. or quite possibly my best guess at the project’s composition of files is wrong.
Both the VS problem and the mingw32 problem have answers floating around the net, though nothing definitive; but I’ll keep looking. After all, I only stumbled on your page after two days of shooting in the Google dark
The other excellent resource is frank’s “Building Python extensions for Windows with only free tools” that you link to. I also have Cygwin. The libmcrypt library compiles successfully only with default (i.e., Cygwin-dependent) settings during ./configure. I had no success in using frank’s recipe with the “-mno-cygwin” switches; but then again, I know very very little about the *POSIX environment.
I’ll keep looking.
Keep up the good work!
Basil
July 26th, 2008 at 19:44
I figured it out! If interested, please see this page.
July 27th, 2008 at 21:52
Basil,
I’m glad you’ve found the solution. I’ve actually linked to this page in the post itself.
August 5th, 2008 at 09:29
Hi,,thank’s for lot’s of information
but i’m having one problem again..i have created .pyd file using gcc from mingw for one sample example but when called from python as
>>import sample
>>print sample.cal(2,3)
it’s not showing anyvalue or error, I used SWIG for creating wrap and .o file.
March 26th, 2009 at 17:03
Thanks for the guide! Helped alot.