Cython: counter-tutorial behavior

Question

Tutorial says .pyx and .pxd files should not have the same name unless .pyx is the realization of the definitions from .pxd file.

Note that the name of the .pyx file must be different from the cqueue.pxd file with declarations from the C library, as both do not describe the same code. A .pxd file next to a .pyx file with the same name defines exported declarations for code in the .pyx file. As the cqueue.pxd file contains declarations of a regular C library, there must not be a .pyx file with the same name that Cython associates with it.

Yet I ran into a situation when it works properly only when the same name is given to those two files even though .pxd is cdef extern cpp declaration unrelated to .pyx code.

py_test.pyx:

# distutils: language = c++

from py_test cimport Test

def f():
    Test[double](2.) + 3.

zzz.pyx:

# distutils: language = c++

from py_test cimport Test

def f():
    Test[double](2.) + 3.

py_test.pxd:

cdef extern from "cpp_test.h": 
    cdef cppclass Test[T]:
        Test()
        Test(T value)
        T value
    cdef Test[T] operator+[T](Test[T]&, T)

cpp_test.h:

template<typename T>
class Test { 
public: 
    T value; 
    Test():value(0){}
    Test(T value):value(value){}
    ~Test(){}
};

template<typename T>
Test<T> operator+(const Test<T>& x, T y) {
    return Test<T>(x.value + y);
}

setup.py:

from distutils.core import setup
from Cython.Build import cythonize

setup(
    name = "demo",  # unused
#   ext_modules = cythonize('py_test.pyx'), # ok
    ext_modules = cythonize('zzz.pyx'), # Invalid operand types for '+' (Test[double]; double) 
)

score 2 · Accepted Answer · edited May 23 '17 at 12:02

The reason the tutorial says that the .pyx and .pxd files should not have the same name is that a .pyx file will automatically do the equivalent of from pxd_file_with_the_same_name cimport * if it finds a file with the same name.

So the reason to avoid having unrelated files with the same name is just to avoid confusing behaviour where something gets cimported that you don't expect.

This specific case has to do with how Cython imports non-member C++ operators which is frankly a bit buggy. See this previous question.

The non-member operator will only be available to Cython if it's cimported into the current module scope (i.e. just importing the associated class won't do it). Unfortunately there's no way to cimport it directly that I know of, but if you change zzz.pyx to start with

from py_test cimport *

then the * does managed to cimport the operator and it should work. Because py_test.pyx does this implicitly then it works without further changes.

(With respect to your previous question about complex which motivated this I tried messing around with this kind of idea and couldn't immediately get it to work, but perhaps you'll have more luck than me)

Thanks, that's exactly what I needed! I know another way of cimporting `operator+` though (see my answer below). — axil, Mar 27 '17 at 10:26

score 1 · Answer 2 · answered Mar 27 '17 at 10:25

Another way of cimporting operator+ is like this:

zzz.pyx

# distutils: language = c++

from py_test cimport Test, add

def f():
    add(Test[double](2.), 3.)

pytest.pxd:

cdef extern from "cpp_test.h": 
    cdef cppclass Test[T]:
        Test()
        Test(T value)
        T value
    cdef Test[T] add "operator+"[T](Test[T]&, T)

In certain cases it might be preferable over cimport *, yet it is not ideal as it alters the semantics of the operator invocation.

Cython: counter-tutorial behavior

2 Answers2

Linked