0

Assume this code snippet:

int test(int input){
  // using asm() manually populate the `return register` with `input` and return.
}

int main(){
 return test(-1);
}

Assume we write our own assembly code to populate the return register under test. The 64 bit return register is set to -1 sign extended to 64 bits which is 0xFFFFFFFFFFFFFFFF but the return type is a 32 bit int (i.e the expected value in register should be 0xFFFFFFFF).

My question is, according to C, is it the responsibility of test() to return the correct value in the register OR is it the responsibility of callee (main in this case) to cast the returned value correctly and truncate it if needed?

Or in other words, is test supposed to make sure the register value is set to 0xFFFFFFFF and not 0xFFFFFFFFFFFFFFFF before returning?

Dan
  • 2,694
  • 1
  • 6
  • 19
  • 4
    This isn't standardized. There's two common calling conversions for PC, "cdecl" and "stdcall". See this: https://stackoverflow.com/questions/3404372/stdcall-and-cdecl/3404412. For other systems, it is wildly implementation-defined. In embedded systems it can vary from compiler to compiler even. – Lundin Jun 02 '21 at 14:06
  • My two cents: If you define `test` as returning an `int`, then its _implementation defined_ as the C standard only requires `int` to be at least 16 bits, therefore you should set the return register **accordingly to the implementation**. If you've specified `int32_t` _(stdint.h)_ as a return type, writing the whole 64 bits return register would have been wrong as it will be unexpected by the compiler (until you 've told it you're doing that). Here, i only see an `int` at each level, so there should never be any cast required. – Zilog80 Jun 02 '21 at 14:52
  • Yes, you have to look to your platform's calling conventions / ABI (you haven't said what your platform is). It will give the precise rules for returning values of every possible type. For `int`, it will state which register the value should be in, and if that register is larger than the size of an `int`, it will specify whether sign or zero extension is required. – Nate Eldredge Jun 02 '21 at 15:52
  • Note that many common 64-bit machines actually provide 32-bit architectural registers as the low half of a 64-bit register. For instance, on x86-64 under most common conventions, you return an `int` in the 32-bit register `eax`, which is the low half of the 64-bit register `rax`. The upper half of `rax` is to be ignored by the caller, so it is not necessary to sign extend (but it is harmless if you do). – Nate Eldredge Jun 02 '21 at 15:54

0 Answers0