vvm — HackTheBox

Description

A new startup claims to have developed an unbreakable client-side password validation system. VCs have invested millions, but I'm a bit skeptical of their claim. Can you prove them wrong?

A password-protected zip (password: hackthebox) contains an ELF named vvm. It prints a banner vvm v0.0.3, asks What is the password:, and validates the input. The goal is to recover the accepted password, which is the flag.

Analysis

Recon

rev_vvm/vvm is an ELF 64-bit LSB PIE executable, x86-64, dynamically linked, stripped. Notable imports:

• ptrace — anti-debug
• getline — reads the password
• malloc, memcpy, __printf_chk

The name "vvm" is the hint: it is a custom virtual machine. On Apple Silicon (macOS/ARM) the x86-64 ELF cannot run natively, so analysis was done inside a Docker --platform linux/amd64 (ubuntu:22.04) container, with qemu-x86_64 -g <port> + gdb for dynamic analysis against the qemu gdbstub.

VM architecture (proven by static + dynamic analysis)

main flow:

1. Banner setup.
2. A setup routine that decrypts ~28 opcode handlers on-demand into RWX mmap pages (lazy-decrypted). The dispatch table in .bss at virtual address 0x74e0 is populated as opcodes are first used.
3. A dispatch loop.

Dispatch loop (.text 0x2870, call site 0x28d8): reads the current opcode dword from the bytecode stream, indexes table[opcode], calls the handler. The VM bytecode/program lives in .data at virtual address 0x5540 (IP starts at 0x5544). The HALT opcode is 28 (0x1c).

Handler calling convention:

reg	meaning
rdi	bytecode base
rsi	&IP
rdx	stack array (vaddr 0x6220)
rcx	&sp counter (vaddr 0x6200)
r8	handler table

Anti-debug: opcode 7 = ptrace(PTRACE_TRACEME). Under qemu it returns -1 and the program exits. Bypassed by NOP-ing the conditional jump in the binary (producing vvm_patched) for clean tracing; the final flag was verified on the original unpatched binary (which prints Correct!).

qemu-user note: deterministic load base = 0x4000000000, which makes gdb software breakpoints reliable. The initial gdb stop is in ld.so (~0x4001829290), not the program entry — you must use the fixed base 0x4000000000 for vvm addresses.

Recovered opcode set (stack VM)

top = stack[sp-1].

op	semantics
0	strlen(top_string)+1
1	SHL (32-bit)
2	MOD
3	READ INPUT via getline → push char*
4	DIV (signed, truncate toward zero)
5	ADD
6	MUL
7	ptrace anti-debug
8	OR
9	top = (top+17) % 3
10	printf("%s", top); free(top) (PRINT)
11	JMP relative (IP = IP_at_operand + operand, dword units)
12	top = 6*top - 12
13	strip trailing newline in string
14	pop/clear
15	PACK N: build a string from low byte of top N stack entries; push char*
16	COPY/pick: operand idx; push stack[sp-1-idx]
17	CALL subroutine: operands (target, nargs); runs dispatch at bytecode_base + target*4 until HALT on the SAME stack; result (top) moved down by nargs slots (sp -= nargs)
18	INDEX: operand idx; top = (signed char)((char*)top)[idx]
19	EQ
20	NEQ
21	top = 8*top + 24
22	JNZ: operand off; pop cond; if cond != 0 then IP += off (dword units)
23	DUP
24	SUB
25	PUSH IMM (next dword)
26	SHR
27	JMP backward
28	HALT

Validation logic

The full decrypted bytecode is 808 dwords. An initial gdb dump of only 1984 bytes / 496 words truncated it; re-dumping the whole region was essential — the real check lived past the first dump boundary.

1. Build & print the What is the password: prompt (PACK 22, PRINT).

2. READ INPUT, strip newline.

3. Length gate: strlen+1 is run through an affine transform; the computed value must equal 76. Brute-forcing input length 5..59 shows only length 32 makes the equality hold (EQ → 1, JNZ taken over the failure branch). So the password is exactly 32 characters (8 packed words × 4 bytes).

4. 8 × 32-bit words: VM subroutine at offset 282 = PACK4 takes 4 bytes (b0,b1,b2,b3) and returns b0 | b1<<8 | b2<<16 | b3<<24. Eight words are built from a permutation of all 32 input byte indices (0..31, each used exactly once). The permutation per word (each list = the 4 input indices feeding b0,b1,b2,b3):

   word	indices [b0,b1,b2,b3]
   W0	[24,14,27,15]
   W1	[11,7,12,4]
   W2	[6,9,2,18]
   W3	[19,13,20,26]
   W4	[28,16,23,8]
   W5	[3,30,21,22]
   W6	[25,5,10,31]
   W7	[29,1,0,17]

5. Rotate + permute: VM subroutine at offset 456 = ROTL(value, n) = (value<<n)|(value>>(32-n)) (32-bit). The 8 packed words are each rotated by a fixed amount and reordered, then each is subtracted from a magic 32-bit constant; all 8 differences must be zero:

   check	computed = rotl(packed, rot)
   c0	rotl(W7, 15)
   c1	rotl(W6, 19)
   c2	rotl(W5, 7)
   c3	rotl(W4, 18)
   c4	rotl(W3, 12)
   c5	rotl(W2, 20)
   c6	rotl(W1, 14)
   c7	rotl(W0, 7)

   Constants (signed int32, in check order): [706975780, -1972114847, -1170424423, -574761715, -213630203, 1193747491, 353885596, -1237732311].

Solution

For each check i: packedWord = rotr(const_i, rot_i); unpack into 4 bytes [b0,b1,b2,b3] (low→high); place those bytes at the corresponding permutation indices. Reassembling all 32 positions yields the 32-character flag.

#!/usr/bin/env python3
# Invert the vvm password check to recover the flag.

MASK = 0xFFFFFFFF

def rotr(v, n):
    v &= MASK
    return ((v >> n) | (v << (32 - n))) & MASK

# Permutation per packed word: indices feeding [b0, b1, b2, b3].
perm = {
    0: [24, 14, 27, 15],
    1: [11,  7, 12,  4],
    2: [ 6,  9,  2, 18],
    3: [19, 13, 20, 26],
    4: [28, 16, 23,  8],
    5: [ 3, 30, 21, 22],
    6: [25,  5, 10, 31],
    7: [29,  1,  0, 17],
}

# Check order maps computed_i = rotl(W[word_i], rot_i) == const_i.
# (word index, rotation amount) in check order c0..c7:
checks = [
    (7, 15),
    (6, 19),
    (5,  7),
    (4, 18),
    (3, 12),
    (2, 20),
    (1, 14),
    (0,  7),
]

consts_signed = [706975780, -1972114847, -1170424423, -574761715,
                 -213630203, 1193747491, 353885596, -1237732311]
consts = [c & MASK for c in consts_signed]

flag_bytes = [0] * 32
for i, (word, rot) in enumerate(checks):
    packed = rotr(consts[i], rot)        # invert ROTL
    b = [(packed >> (8 * k)) & 0xFF for k in range(4)]  # unpack low->high
    for k, idx in enumerate(perm[word]):
        flag_bytes[idx] = b[k]

flag = bytes(flag_bytes).decode()
print(flag)  # HTB{v1rTu4L_p4sSw0rD_t3ChN0loGy}
assert len(flag) == 32

Running the recovered password against the original (unpatched) binary prints Correct!, confirming the flag.