oracle — UMDCTF

Description

No separate organizer description was present in the provided workspace files.

English summary: the challenge ships a stripped ELF named oracle plus an encrypted feed.bin. The binary behaves like a market-settlement verifier: it reads a base64 ticket from ticket.txt, validates the decoded 32-byte body with a custom VM, derives 32 bytes of key material on success, and uses that material to decrypt the final archive record containing the flag.

Challenge overview

At a high level, the solve path was:

1. Decrypt the hidden .xtext code that main unseals at runtime.
2. Decode the runtime strings to understand the file format and execution flow.
3. Reverse feed.bin, which needs two outer decrypt passes before an ARC\x01 archive decode with a rolling XOR stage.
4. Reconstruct the VM semantics from record b and fix the critical misunderstanding about where the 32-byte ticket body is actually read from.
5. Solve the eight 32-bit constraints with Z3.
6. Use the VM output as AES-128 key || IV to decrypt record g.

The useful local artifacts created during solving were:

• decrypt_xtext.py
• patch_oracle.py
• decode_strings.py
• decode_feed.py
• emulate_crypto.py
• run_vm.py
• solve_ticket.py

Recon and hidden code decryption

The ELF is a stripped PIE x86-64 binary with a hidden encrypted section named .xtext. The visible main function is mostly a loader:

• it hashes the chk section with seed 0xc0def1ab
• derives four TEA key words from .rodata
• builds an IV from .rodata
• decrypts .xtext
• jumps into the now-decrypted logic

The important detail is that the hidden section is not plain AES or TEA-CBC. It is a TEA-based CFB-like stream mode:

• initial state = IV
• keystream block = TEA_encrypt(state, key)
• plaintext = ciphertext XOR keystream
• next state = previous ciphertext block

For a normal non-debugged run, the recovered values were:

• hash: 0x72196a90
• key: [0x1a2b3c4d, 0x5e6f7081, 0x92a3b4c5, 0xd6e7f809]
• IV: 00 11 22 33 44 55 66 77

That logic was reimplemented in decrypt_xtext.py, which produced xtext.dec for static analysis.

Decoding runtime strings

After opening the decrypted code, the next blocker was that several user-facing strings were still stored in an obfuscated form and decoded through helper routines at 0x1510 and 0x1540.

Using decode_strings.py, the important runtime strings became:

• banner: oracle v1 - market settlement verifier
• ticket file: ticket.txt
• feed file: feed.bin
• output prefix: resolution:

Those strings immediately clarified the intended flow:

1. load ticket.txt
2. base64-decode it
3. parse an internal ticket container
4. load and decode feed.bin
5. validate the ticket against data from the feed
6. print a resolution: ... string on success

Reversing feed.bin and the custom archive format

feed.bin turned out to have two different protection layers.

Outer decryption

The outer wrapper is not a standard archive. It must be decrypted twice with the same custom pass. Each pass:

• keeps a 0x40-byte header
• uses per-block indices derived from header bytes
• derives three 16-byte vectors from the main ELF's .rodata
• computes key = AES-ECB-decrypt(block_b, key_seed)
• decrypts each 0x40-byte chunk with AES-CBC(key, iv)

In code this was captured by:

def decode_outer_feed(data: bytes) -> bytes:
    out = one_pass(one_pass(data))
    if out[:4] != b"ARC\x01":
        raise ValueError("bad inner header")
    x = struct.unpack("<I", out[4:8])[0]
    body = bytearray(out)
    for j in range(8, len(body)):
        x = (((x & 0xFFFF) * 0x4650) + (x >> 16)) & 0xFFFFFFFF
        body[j] ^= x & 0xFF
    body[4:len(body) - 4] = body[8:]
    return bytes(body[:-4])

Inner archive

After the double outer decryption, the blob starts with ARC\x01. The archive body is additionally protected by a rolling XOR stream seeded from offset 4, then parsed as a tiny container:

• byte 4: record count
• for each record:
  • 1 byte name length
  • name bytes
  • 4-byte little-endian size
  • payload

The parsed records were:

• a: 14 bytes
• b: 848 bytes
• g: 96 bytes

b contains the VM bytecode and constants, and one embedded string inside it is the salt:

umdctf-v2026-unseal-salt

Ticket format and parser behavior

The decrypted logic base64-decodes ticket.txt, then parses a strict binary ticket structure.

The accepted raw ticket format is:

raw_ticket = b"TKT\x01" + struct.pack("<I", 0x20) + body32

So the file on disk must contain the base64 encoding of those 40 raw bytes.

The final accepted ticket.txt content is:

VEtUASAAAAApD5RkMvZB09nhNlWQ8FQK7DNS3ozstXxn7UgRITVa1g==

Decoding that base64 gives:

• magic: TKT\x01
• body length: little-endian 0x20
• body: 32 bytes

Reversing the VM and correcting the input semantics misunderstanding

The hardest part of the challenge was the custom VM in record b.

My initial misunderstanding was assuming that the VM consumed the ticket body from the object passed in rsi to fcn.000041f0. That was wrong.

The critical reversing insight is:

• fcn.000041f0 does not read the input ticket body from the output object in rsi
• instead it reads from the root state object:
  • state + 0x10 = body pointer
  • state + 0x18 = body length
• rsi is still used as the output/result object

That bug in my mental model explained why early VM emulation attempts looked inconsistent. Once run_vm.py wrote the 32-byte body into the root state, the VM behavior became coherent.

High-level VM behavior

The VM first loads eight little-endian dwords from the 32-byte ticket body into registers r0..r7, then runs an ARX/mix routine parameterized by constants from segment 0 of record b.

The recovered semantics were:

• some steps XOR with constants
• some steps ADD constants or another register
• rotates are by [3, 10, 17, 24, 31, 6, 13, 20]
• every other block multiplies one register by 0x5ABC7F01
• the tail is a compare-chain against segment 1 constants

The reconstructed transform was:

def transform(words, seg0_words):
    r = list(words)
    C = 0x5ABC7F01
    r[1] ^= seg0_words[0]
    r[4] += seg0_words[1]
    r[7] = rol(r[7], 3)
    r[2] ^= r[5]
    r[0] += r[1]
    r[3] *= C

    r[6] ^= seg0_words[2]
    r[1] += seg0_words[3]
    r[4] = rol(r[4], 10)
    r[7] ^= r[2]
    r[5] += r[6]
    r[0] *= C

    r[3] ^= seg0_words[4]
    r[6] += seg0_words[5]
    r[1] = rol(r[1], 17)
    r[4] ^= r[7]
    r[2] += r[3]
    r[5] *= C

    r[0] ^= seg0_words[6]
    r[3] += seg0_words[7]
    r[6] = rol(r[6], 24)
    r[1] ^= r[4]
    r[7] += r[0]
    r[2] *= C

    r[5] ^= seg0_words[8]
    r[0] += seg0_words[9]
    r[3] = rol(r[3], 31)
    r[6] ^= r[1]
    r[4] += r[5]
    r[7] *= C

    r[2] ^= seg0_words[10]
    r[5] += seg0_words[11]
    r[0] = rol(r[0], 6)
    r[3] ^= r[6]
    r[1] += r[2]
    r[4] *= C

    r[7] ^= seg0_words[12]
    r[2] += seg0_words[13]
    r[5] = rol(r[5], 13)
    r[0] ^= r[3]
    r[6] += r[7]
    r[1] *= C

    r[4] ^= seg0_words[14]
    r[7] += seg0_words[15]
    r[2] = rol(r[2], 20)
    r[5] ^= r[0]
    r[3] += r[4]
    r[6] *= C
    return r

Solving the ticket with Z3

Once the VM semantics were correct, the tail compare-chain reduced the problem to eight 32-bit equations:

for yi, target in zip(transform(x, seg0), seg1):
    s.add(yi == target)

Z3 solved the eight dword constraints directly.

The valid 32-byte ticket body is:

290f946432f641d3d9e1365590f0540aec3352de8cecb57c67ed481121355ad6

Split as little-endian dwords, that is the body that the VM accepts when it is placed at state + 0x10 with length 0x20 at state + 0x18.

The accepted base64 ticket file is therefore:

VEtUASAAAAApD5RkMvZB09nhNlWQ8FQK7DNS3ozstXxn7UgRITVa1g==

Decrypting the final record

On success, the VM returns 32 bytes of derived material:

e46caa0c59a1da91689f629dc1664c6523c2e633d9331d5c5ca40c40f51626b0

Those 32 bytes are used as:

• first 16 bytes = AES-128 key
• second 16 bytes = IV

Decrypting record g with AES-CBC and removing PKCS#7 padding yields the final plaintext and the flag.

Full solve script

This compact script reproduces the last stage from the already recovered helpers:

#!/usr/bin/env python3
from pathlib import Path
from base64 import b64encode
from Crypto.Cipher import AES

from decode_feed import decode_outer_feed, parse_arc
from run_vm import run_ticket


BODY = bytes.fromhex(
    "290f946432f641d3d9e1365590f0540aec3352de8cecb57c67ed481121355ad6"
)


def unpad_pkcs7(data: bytes) -> bytes:
    pad = data[-1]
    if pad == 0 or pad > 16 or data[-pad:] != bytes([pad]) * pad:
        raise ValueError("bad padding")
    return data[:-pad]


def main():
    raw_ticket = b"TKT\x01" + (0x20).to_bytes(4, "little") + BODY
    print("ticket.txt =", b64encode(raw_ticket).decode())

    items = parse_arc(decode_outer_feed(Path("feed.bin").read_bytes()))
    vm = run_ticket(items["b"], items["a"], BODY)
    material = vm["data"]
    print("derived =", material.hex())

    key, iv = material[:16], material[16:32]
    pt = AES.new(key, AES.MODE_CBC, iv).decrypt(items["g"])
    print(unpad_pkcs7(pt).decode())


if __name__ == "__main__":
    main()

Expected output includes:

ticket.txt = VEtUASAAAAApD5RkMvZB09nhNlWQ8FQK7DNS3ozstXxn7UgRITVa1g==
derived = e46caa0c59a1da91689f629dc1664c6523c2e633d9331d5c5ca40c40f51626b0
UMDCTF{oh_no_my_prediction_market_feed_has_been_compromised_what_ever_will_i_do}