Return-oriented programming

Content sourced from Wikipedia, licensed under CC BY-SA 3.0.

Return-oriented programming (ROP) is a security technique attackers use to run arbitrary code on a computer, even when protections are in place that try to stop them from injecting new code.

How ROP works in simple terms
- When a program has a bug, like a buffer overrun, an attacker can take control of the program’s flow by changing the return address on the stack.
- Instead of writing new code to the stack, the attacker looks for tiny chunks of code already in the program or its libraries. Each chunk ends with a return instruction and is called a gadget.
- By carefully chaining many gadgets together, the attacker makes the computer perform complex operations as if it were running a real program, all using existing code in memory.
- This lets the attacker bypass defenses that block launching new or injected code.

Why ROP appeared after early protections
- Early attacks relied on placing and executing their own code on the stack. Many systems later started marking data areas as non-executable, so that trick stopped working.
- With data execution prevention (DEP) in place, attackers switched to using only the already-executable code that was present in memory. ROP takes advantage of this by stitching together gadgets instead of injecting new instructions.

A quick history of the idea
- Return-into-libc: One of the first ideas was to hijack control flow to call existing library functions (like parts of the C library) instead of injecting payloads.
- 64-bit and calling conventions: As processors evolved, how functions receive arguments changed. Attackers adapted by using not just whole library calls but pieces of code that set up the right arguments under new rules.
- From library chunks to a full language: Researchers showed that enough gadgets could simulate programming constructs such as loops and branches, effectively creating a tiny, low-level language inside the program. This is why ROP is described as Turing-complete enough to perform many tasks.

Gadgets and tooling
- Gadgets are short sequences of instructions found in the program or its libraries that end with a return. An attacker links these gadgets by overwriting the return addresses on the stack.
- There are automated tools that search binaries for useful gadgets and help assemble an attack, such as ones that try to spawn a shell or perform specific actions.

Defense and how attackers try to beat it
- Address space layout randomization (ASLR) tries to hide where code and libraries sit in memory, making gadgets harder to find.
- However, ASLR is not perfect: on 32-bit systems, only a portion of the address space may be randomized, and leaking memory contents can reveal locations. 64-bit systems offer more room for randomization but can still be attacked with information leaks.
- Some attacks look for gadgets that don’t start at predictable points or use non-return tricks, which complicates defenses that only look for return instructions.
- New ideas aim to make gadgets harder to use or reduce where they can appear:
- G-Free removes unaligned return-like instructions inside a binary, so attackers can’t easily jump to gadget chains.
- Pointer authentication and branch-target checks add cryptographic checks to ensure a branch goes to a valid target, making gadget chaining much harder.
- BTI (branch target identification) marks valid indirect-branch destinations so landing in an unexpected place triggers a fault.
- ARM and other architectures have hardware and compiler-based defenses to sign pointers and validate branch targets, increasing the cost of successful ROP chains.
- Other approaches try to prevent gadget formation altogether, such as changing how code is compiled or introducing runtime variations in software to break predictable gadget layouts. Some ideas trade performance for stronger protection.

Broader context and ongoing work
- ROP is part of the broader effort to make security defenses robust against a wide range of control-flow attacks, including those that don’t rely on a simple return instruction.
- Researchers continue to study and develop methods to identify and neutralize gadget chains, as well as practical defense mechanisms that work efficiently on real-world software and hardware.

Bottom line
- Return-oriented programming shows how attackers can turn existing, legitimate code into a stealthy attack engine.
- Defenses focus on making memory unsafe for code execution, scrambling where code appears, and verifying where branches and calls can legally go.
- As hardware and software evolve, new protections continue to emerge to keep gadget-based attacks from succeeding.