USING NUMPY C-API
7.2 Using Python as glue
7.2.4 Cython
def configuration(parent_package='', top_path=None) from numpy.distutils.misc_util import Configuration
config = Configuration('f2py_examples',parent_package, top_path) config.add_extension('add', sources=['add.pyf','add.f'])
return config
if __name__ == '__main__':
from numpy.distutils.core import setup setup(**configuration(top_path='').todict()) Installation of the new package is easy using:
python setup.py install
assuming you have the proper permissions to write to the main site- packages directory for the version of Python you are using. For the resulting package to work, you need to create a file named__init__.py(in the same directory as add.pyf). Notice the extension module is defined entirely in terms of the add.pyfandadd.ffiles. The conversion of the .pyf file to a .c file is handled bynumpy.disutils.
Conclusion
The interface definition file (.pyf) is how you can fine-tune the interface between Python and Fortran. There is decent documentation for f2py found in the numpy/f2py/docs directory where-ever NumPy is installed on your system (usu- ally under site-packages). There is also more information on using f2py (including how to use it to wrap C codes) at https://scipy-cookbook.readthedocs.iounder the “Interfacing With Other Languages” heading.
The f2py method of linking compiled code is currently the most sophisticated and integrated approach. It allows clean separation of Python with compiled code while still allowing for separate distribution of the extension module. The only draw-back is that it requires the existence of a Fortran compiler in order for a user to install the code. However, with the existence of the free-compilers g77, gfortran, and g95, as well as high-quality commercial compilers, this restriction is not particularly onerous. In my opinion, Fortran is still the easiest way to write fast and clear code for scientific computing. It handles complex numbers, and multi-dimensional indexing in the most straightforward way.
Be aware, however, that some Fortran compilers will not be able to optimize code as well as good hand- written C-code.
(continued from previous page) ext_modules=[Extension('filter', ['filter.pyx'],
include_dirs=[numpy.get_include()])], cmdclass = {'build_ext':build_ext})
Adding the NumPy include directory is, of course, only necessary if you are using NumPy arrays in the extension module (which is what we assume you are using Cython for). The distutils extensions in NumPy also include support for automatically producing the extension-module and linking it from a.pyxfile. It works so that if the user does not have Cython installed, then it looks for a file with the same file-name but a.cextension which it then uses instead of trying to produce the.cfile again.
If you just use Cython to compile a standard Python module, then you will get a C extension module that typically runs a bit faster than the equivalent Python module. Further speed increases can be gained by using thecdefkeyword to statically define C variables.
Let’s look at two examples we’ve seen before to see how they might be implemented using Cython. These examples were compiled into extension modules using Cython 0.21.1.
Complex addition in Cython
Here is part of a Cython module namedadd.pyxwhich implements the complex addition functions we previously implemented using f2py:
cimport cython cimport numpy as np import numpy as np
# We need to initialize NumPy.
np.import_array()
#@cython.boundscheck(False) def zadd(in1, in2):
cdef double complex[:] a = in1.ravel() cdef double complex[:] b = in2.ravel() out = np.empty(a.shape[0], np.complex64) cdef double complex[:] c = out.ravel() for i in range(c.shape[0]):
c[i].real = a[i].real + b[i].real c[i].imag = a[i].imag + b[i].imag return out
This module shows use of thecimportstatement to load the definitions from thenumpy.pxdheader that ships with Cython. It looks like NumPy is imported twice;cimportonly makes the NumPy C-API available, while the regularimportcauses a Python-style import at runtime and makes it possible to call into the familiar NumPy Python API.
The example also demonstrates Cython’s “typed memoryviews”, which are like NumPy arrays at the C level, in the sense that they are shaped and strided arrays that know their own extent (unlike a C array addressed through a bare pointer). The syntaxdouble complex[:] denotes a one-dimensional array (vector) of doubles, with arbitrary strides. A contiguous array of ints would beint[::1], while a matrix of floats would befloat[:, :].
Shown commented is thecython.boundscheckdecorator, which turns bounds-checking for memory view ac- cesses on or off on a per-function basis. We can use this to further speed up our code, at the expense of safety (or a manual check prior to entering the loop).
7.2. Using Python as glue 115
Other than the view syntax, the function is immediately readable to a Python programmer. Static typing of the variable iis implicit. Instead of the view syntax, we could also have used Cython’s special NumPy array syntax, but the view syntax is preferred.
Image filter in Cython
The two-dimensional example we created using Fortran is just as easy to write in Cython:
cimport numpy as np import numpy as np np.import_array() def filter(img):
cdef double[:, :] a = np.asarray(img, dtype=np.double) out = np.zeros(img.shape, dtype=np.double)
cdef double[:, ::1] b = out cdef np.npy_intp i, j
for i in range(1, a.shape[0] - 1):
for j in range(1, a.shape[1] - 1):
b[i, j] = (a[i, j]
+ .5 * ( a[i-1, j] + a[i+1, j]
+ a[i, j-1] + a[i, j+1]) + .25 * ( a[i-1, j-1] + a[i-1, j+1]
+ a[i+1, j-1] + a[i+1, j+1])) return out
This 2-d averaging filter runs quickly because the loop is in C and the pointer computations are done only as needed.
If the code above is compiled as a moduleimage, then a 2-d image,img, can be filtered using this code very quickly using:
import image
out = image.filter(img)
Regarding the code, two things are of note: firstly, it is impossible to return a memory view to Python. Instead, a NumPy arrayoutis first created, and then a viewbonto this array is used for the computation. Secondly, the viewb is typeddouble[:, ::1]. This means 2-d array with contiguous rows, i.e., C matrix order. Specifying the order explicitly can speed up some algorithms since they can skip stride computations.
Conclusion
Cython is the extension mechanism of choice for several scientific Python libraries, including Scipy, Pandas, SAGE, scikit-image and scikit-learn, as well as the XML processing library LXML. The language and compiler are well- maintained.
There are several disadvantages of using Cython:
1. When coding custom algorithms, and sometimes when wrapping existing C libraries, some familiarity with C is required. In particular, when using C memory management (malloc and friends), it’s easy to introduce memory leaks. However, just compiling a Python module renamed to.pyxcan already speed it up, and adding a few type declarations can give dramatic speedups in some code.
2. It is easy to lose a clean separation between Python and C which makes re-using your C-code for other non- Python-related projects more difficult.
3. The C-code generated by Cython is hard to read and modify (and typically compiles with annoying but harmless warnings).
One big advantage of Cython-generated extension modules is that they are easy to distribute. In summary, Cython is a very capable tool for either gluing C code or generating an extension module quickly and should not be over-looked.
It is especially useful for people that can’t or won’t write C or Fortran code.