Call a method through Interop which returns an instance of a struct

Question

I'm new to Interop and need to call a managed C++ method from C# which returns an instance of the following struct:

typedef struct DataBlock_ {
  unsigned char data[10240];
  unsigned int numberOfBytes;
  unsigned long int startAddr;    
} DataBlock;

The C++ method which returns the instance is declared as follows:

__declspec(dllexport) DataBlock getDefaultPass( void ) 
{
    DataBlock default_pass = {
        {
            (char)0xFF,(char)0xFF,(char)0xFF,(char)0xFF,
            (char)0xFF,(char)0xFF,(char)0xFF,(char)0xFF,
            (char)0xFF,(char)0xFF,(char)0xFF,(char)0xFF,
            (char)0xFF,(char)0xFF,(char)0xFF,(char)0xFF,
            (char)0xFF,(char)0xFF,(char)0xFF,(char)0xFF,
            (char)0xFF,(char)0xFF,(char)0xFF,(char)0xFF,
            (char)0xFF,(char)0xFF,(char)0xFF,(char)0xFF,
            (char)0xFF,(char)0xFF,(char)0xFF,(char)0xFF
        },
        32,
        0xFFE0
    };
    return default_pass;
}

I've declared the struct and method in C# as follows:

public static partial class My
{
    [StructLayout(LayoutKind.Sequential)]
    public struct DataBlock
    {
        [MarshalAs(UnmanagedType.ByValArray, SizeConst = 10240)]
        public byte[] data;
        //public fixed byte data[10240]; <-- this requires 'unsafe' but still doesn't work      
        public UInt32 numberOfBytes;
        public UInt32 startAddr;
    }

    [DllImport("my.dll")]
    static public extern DataBlock getDefaultPass( );

    [DllImport("my.dll")]
    static public extern byte sendPassword(DataBlock data);
}

I call the method from C# as follows:

var defaultPassword = My.getDefaultPass();
var response = My.sendPassword(defaultPassword);

but the call to getDefaultPass() throws

An unhandled exception of type 'System.Runtime.InteropServices.MarshalDirectiveException' occurred in ConsoleApplication1.exe

Additional information: Method's type signature is not PInvoke compatible.

Based on this question, I tried changing the declaration of data to public fixed byte data[10240] and marked the struct as unsafe, but then the method returns an instance with numberOfBytes and startAddr set to 0, and the subsequent call to sendPassword() fails (note that in this answer the subsequent calls use a pointer to the struct as opposed to the instance itself, as is my case). How then should I be calling the method(s) from C#?

The project targets .NET 3.5 and x86.

Thanks in advance for any help.

@MatthewWatson No, the struct is actually blittable. While returning large structures like this is tricky, blittability is not the problem. — Luaan, Feb 03 '16 at 13:12
Are you using GCC to compile the native code? If so, it's very likely that it's emitting the function to actually take a `ref` argument rather than returning the struct. The real signature would then be something like `static public extern void getDefaultPass(out DataBlock data)`. — Luaan, Feb 03 '16 at 13:13
@MatthewWatson See https://msdn.microsoft.com/en-us/library/75dwhxf7(v=vs.110).aspx. `byte` is blittable. Fixed-size arrays of blittable types are themselves blittable. Thus, `byte[]` (as defined by the OP) is blittable. — Luaan, Feb 03 '16 at 13:16
@Luaan I'm being told it's compiled in Visual Studio as well (I don't maintain it, nor do I think I could change it). Does that help or would you need more specifics about build/link configuration? — user5877732, Feb 03 '16 at 13:23
@MatthewWatson `char` != `byte` in the C# world. `byte` is blittable. `char` is not. In this case, the OP wants `byte`, not `char`, which is fine. — Luaan, Feb 03 '16 at 13:28
@Luaan Ahh I misread the array as being chars (since I was looking at the C++ struct definition, which uses `unsigned char` instead of `byte`). — Matthew Watson, Feb 03 '16 at 13:30
Well, first you should make sure the calling conventions are in order, as explained in Hans' comment - the name may be mangled, and the C++ code might be using a different calling convention than the default StdCall; the C++ guys should be able to answer this for you easily. If that doesn't help, try the `out` trick. — Luaan, Feb 03 '16 at 13:33
@Luaan Brilliant! The `out` trick worked! The calling convention is `__cdecl (/Gd)`. Cheers — user5877732, Feb 03 '16 at 13:37
@Luaan PS: feel free to include that as an answer, unless it's already been linked and I missed it (?) — user5877732, Feb 03 '16 at 13:43
Yeah, you really want to use `[DllImport("my.dll", CallingConvention = CallingConvention.Cdecl)]` as well if you're using CDecl. — Luaan, Feb 03 '16 at 13:43
For the sake of completeness, adding the `CallingConvention` to the method's declaration in C# still failed. Only changing the method to take an `out` parameter worked — user5877732, Feb 05 '16 at 10:15

score 1 · Accepted Answer · answered Feb 03 '16 at 13:53

The struct is fine - it fulfills all the rules to be used as a return value in P/Invoke.

You need to use the proper calling convention (in your case, CallingConvention.Cdecl).

There's also an extra optimization that some compilers use, where a big structure (such as yours) is passed by reference, rather than returned. You can replicate this in C# like so:

static public extern void getDefaultPass(out DataBlock data);

score 0 · Answer 2 · answered Feb 05 '16 at 10:08

For the sake of completeness and to complement Luaan's answer, since the C++ method in the question has no parameters, I wanted to cover the case where a method did have parameters, specifically as it relates to the position of the out parameter when the method takes 2 or more arguments.

Consider the C++ method

__declspec(dllexport) DataBlock readText(char * dataArray , int bytesToRead)

It's not immediately obvious whether the out parameter should be first or last in the C# method. Contrary to the framework's convention of placing the out parameter as the last parameter (e.g. TryParse), here it must be the first parameter, otherwise the call will fail:

[DllImport("my.dll", CallingConvention = CallingConvention.Cdecl)]
static public extern void readText(out DataBlock dataBlock, string dataArray, int bytesToRead);

Call a method through Interop which returns an instance of a struct

2 Answers2