Calling Conventions Explained

What Are Calling Conventions?

Calling conventions define how functions receive parameters and return values. They are essential to ensure correct execution when calling functions in assembly, C, or any other language.

Key Questions and Answers

How do we get the CPU back to the correct state?
The return address is stored on the stack, and the ret instruction ensures execution resumes at the caller.
How are local variables restored?
Local variables are stored on the stack and accessed relative to the base pointer. The function restores them before returning.
How do we pass parameters?
In x86-64, the first six integer arguments are passed in registers (RDI, RSI, RDX, RCX, R8, R9), while additional ones are passed on the stack.
How is the stack frame cleaned up?
The stack frame is cleaned by restoring rbp and adjusting rsp. Some conventions require the caller to clean up the stack.
What changes between 32-bit and 64-bit programs?
In 32-bit, arguments are passed on the stack, while in 64-bit, registers are used more frequently for arguments, improving efficiency.
Why do we need different calling conventions?
Different architectures and programming languages have different ways of handling function calls, requiring different conventions.
What happens if calling conventions are not followed?
The function may not receive the correct parameters, leading to crashes or incorrect behavior.

Example: X86 Function Call

section .text  ; Code section starts
            global _start  ; Entry point for the program
        
        _start:
            call my_function  ; Call the function my_function
            mov eax, 60  ; Load exit system call number (60) into eax
            xor edi, edi  ; Set exit status to 0 (edi = 0)
            syscall  ; Invoke system call to exit the program
        
        my_function:
            push rbp  ; Save the old base pointer
            mov rbp, rsp  ; Set new base pointer to the current stack pointer
            ; Function body here (can include logic)
            pop rbp  ; Restore the old base pointer
            ret  ; Return to the caller

Quick Quiz

What is the purpose of the ret instruction in assembly?

Which register is used to store the return address?

Where are function parameters stored in x86-64?

Which register is used to pass the first integer argument in x86-64?

What is the primary purpose of the stack frame?

Which calling convention is used in Linux 64-bit systems?

What does the mov rbp, rsp instruction do?

Which instruction is responsible for calling a function in assembly?