Reversing a tiny obfuscated bytecode interpreter layered beneath control flow distractions.
The challenge ships a stripped x86-64 PIE ELF binary (crack_me_vault.bdsec) that prints an ASCII-art banner, asks for a flag, and either grants or denies access. The interesting part is that the "checker" isn't a straightforward comparison — the binary implements a tiny bytecode interpreter. A handful of opcodes (themselves obfuscated with a rolling XOR key) drive a sequence of operations: validate length, transform the input, and compare the transformed input against a hidden encrypted target. This indirection is exactly what the flag itself is hinting at.
Decompiling FUN_001010c0 (Ghidra, base address 0x100000) gives the core logic. Cleaned up, the structure is:
fgets(input, 0x80, stdin);
len = strcspn(input, "\r\n");
input[len] = 0;
byte *bytecode = &DAT_001022f2; // opcode stream
byte key = 0xa5; // opcode XOR key, += 0x11 per step
bool ok = true;
byte hash[50] = {0};
do {
byte op = *bytecode ^ key;
if (op == 0x6b) {
// VERIFY opcode
...
} else if (op > 0x6b) {
if (op == 0xe0 && ok) { puts("Access granted"); return 0; }
break; // anything else > 0x6b denies
} else if (op == 0x11) {
// LENGTH CHECK opcode
ok &= (len == 0x32);
} else if (op == 0x37 && len == 0x32) {
// TRANSFORM opcode
...
} else {
break; // unknown opcode denies
}
key += 0x11;
bytecode++;
} while (key != 0xe9);
puts("Access denied");
So the "VM" is just 4 opcodes long, each byte in &DAT_001022f2 XORed against an incrementing key (0xa5, 0xb6, 0xc7, 0xd8) to reveal the real opcode. Everything about control flow (the if/else chain, the early breaks) is a red herring layered on top of a fixed, tiny program — hence the flag's theme.
Extracted from .rodata at file offset 0x22f2 (i.e. Ghidra address 0x1022f2 minus the 0x100000 image base):
raw bytes: b4 81 ac 38
keys: a5 b6 c7 d8
| i | raw | key | opcode | meaning |
|---|-----|-----|--------|---------|
| 0 | 0xb4 | 0xa5 | 0x11 | LEN_CHECK — require len == 50 |
| 1 | 0x81 | 0xb6 | 0x37 | TRANSFORM — scramble input into hash[50] |
| 2 | 0xac | 0xc7 | 0x6b | VERIFY — compare hash[50] against decrypted target |
| 3 | 0x38 | 0xd8 | 0xe0 | GRANT — success, if ok is still true |
This confirms the intended program order: check length → transform → verify → grant.
0x37)byte roll_key = 0x41; // += 0x1d per iteration
byte add_key = 0x00; // += 0x0b per iteration
int idx = 0; // += 0x11 (mod 50) per iteration
for (i = 0; i < 50; i++) {
byte c = input[i] ^ roll_key;
byte rot = (i % 7) + 1; // rotate amount, 1..7
byte r = rotl8(c, rot); // 8-bit rotate-left
hash[idx % 50] = (add_key ^ 0x17) + r; // byte addition
roll_key += 0x1d;
add_key += 0x0b;
idx += 0x11;
}
Simplifying the index math: idx % 50 after i steps is (17*i) mod 50. Since gcd(17,50)=1, this is a bijective permutation of 0..49 — every input character ends up scattered to a unique slot in hash[], not in original order. That's the "confusion" layer, on top of the per-character rotate+XOR+add "diffusion" layer.
0x6b)byte *enc = &DAT_001022c0; // 50-byte encrypted target
byte key = 0x44; // += 0x0d per iteration
int idx = 0; // += 0x11 (mod 50) per iteration
for (i = 0; i < 50; i++) {
byte expected = enc[i] ^ key;
ok &= (expected == hash[idx % 50]);
key += 0x0d;
idx += 0x11;
}
Same (17*i) mod 50 permutation is used to index into hash[], and a second independent rolling XOR key (0x44, +0x0d) decrypts the 50 bytes stored at &DAT_001022c0. Once decrypted (and un-permuted), this gives us the exact 50 target bytes that hash[] must equal.
Extracted from .rodata at file offset 0x22c0:
59 d5 1c 78 61 0f 09 4d c0 bd 4c 57 e7 67 0a 13
3a 46 07 d2 70 eb e7 16 90 46 19 f6 6f 11 a3 12
2c 03 94 e6 a7 6b b7 41 92 18 4c 59 69 e5 c5 8a
9c 5d b2
Both the transform and the verification decryption are simple invertible bijections, so the whole thing can be run backwards:
0x44, +0x0d), and un-permute them using idx = (17*i) mod 50 to recover expected_hash[0..49] — the target hash[] array in its natural index order.i from 0 to 49, invert the per-character transform:
val = expected_hash[(17*i) mod 50]
rotated = (val - ((11*i mod 256) ^ 0x17)) mod 256 # undo the addition
xored = rotr8(rotated, (i % 7) + 1) # undo the rotate
input[i] = xored ^ ((0x41 + 29*i) mod 256) # undo the XOR
#!/usr/bin/env python3
data = open('crack_me_vault.bdsec', 'rb').read()
# .rodata VMA == file offset (0x2000), Ghidra base = 0x100000
bytecode = data[0x22f2:0x22f2 + 8]
target = data[0x22c0:0x22c0 + 50]
def rotl8(v, n):
n %= 8
return ((v << n) | (v >> (8 - n))) & 0xff
def rotr8(v, n):
n %= 8
return ((v >> n) | (v << (8 - n))) & 0xff
# --- decode the opcode stream (sanity check) ---
key, i = 0xa5, 0
while key != 0xe9:
print(hex(key), hex(bytecode[i]), hex(bytecode[i] ^ key))
key = (key + 0x11) & 0xff
i += 1
# --- decrypt + un-permute the 50-byte target into expected_hash[] ---
expected = [0] * 50
key = 0x44
for i in range(50):
b = target[i] ^ key
idx = (17 * i) % 50
expected[idx] = b
key = (key + 0x0d) & 0xff
# --- invert the transform, character by character ---
flag = [0] * 50
for i in range(50):
idx = (17 * i) % 50
val = expected[idx]
keyi = (0x41 + 29 * i) & 0xff
roti = (i % 7) + 1
rotated = (val - ((11 * i & 0xff) ^ 0x17)) & 0xff
xored = rotr8(rotated, roti)
flag[i] = xored ^ keyi
print(bytes(flag).decode())
0xa5 0xb4 0x11
0xb6 0x81 0x37
0xc7 0xac 0x6b
0xd8 0x38 0xe0
BDSEC{c0nTr0L_fl0w_1s_4_l13_bUt_bYt3c0d3_d03s_n0t}
if/else maze in FUN_001010c0 is only 4 meaningful branches deep; the rest is opcode dispatch dressed up to look like a bigger state machine.key += constant each iteration) — no brute force needed, just replay the same progression in Python.(17*i) mod 50) is a coprime bijection, easily inverted by just running every i and recording where it wrote; you never need to solve for the inverse permutation explicitly.