Partial Encryption — HackTheBox

Description

Static-Analysis on this program didn't reveal much. There must be a better way to approach this...

PE32+ executable (console) x86-64 Windows binary partialencryption.exe (13312 bytes). Static analysis reveals neither the flag nor meaningful strings — only Windows API imports. The task uses runtime code decryption via hardware AES-NI instructions.

Analysis

Initial Reconnaissance

$ file partialencryption.exe
PE32+ executable (console) x86-64, for MS Windows

$ strings partialencryption.exe
# Only Windows API imports:
# VirtualAlloc, VirtualProtect, VirtualFree, putchar
# Standard CRT functions
# No flag, no "correct/wrong" messages

The imports VirtualAlloc / VirtualProtect / VirtualFree are a classic sign of self-modifying code: the program allocates memory, decrypts code, makes it executable, calls it, and frees the memory.

Architecture — Self-Modifying Code with AES Decryption

The binary uses the following scheme:

.data section (VA 0x140004000–0x140004830)
    │
    │  Encrypted code blobs
    │
    ▼
Decryption function (0x140001000)
    │  AES-NI: AESKEYGENASSIST + AESDECLAST
    │
    ▼
VirtualAlloc → copy decrypted code
    │
    ▼
VirtualProtect(PAGE_EXECUTE) → code becomes executable
    │
    ▼
CALL → execute decrypted code
    │
    ▼
VirtualFree(MEM_RELEASE) → free memory

Decryption Algorithm (function 0x140001000)

For each 16-byte block i of encrypted data:

1. Key = byte i, broadcast to all 16 positions: [i, i, i, ..., i]
2. kg0 = AESKEYGENASSIST(key, 0x00) — subkey generation with rcon=0x00
3. kg1 = AESKEYGENASSIST(key, 0x10) — subkey generation with rcon=0x10
4. xmm2 = data_block XOR kg1 — XOR data block with subkey
5. result = AESDECLAST(xmm2, kg0) — final AES decryption round:
   • InvSubBytes(InvShiftRows(xmm2)) XOR kg0

Key observation: the key for each block is simply the block index, which makes the scheme weak and allows offline decryption of the blobs.

Decrypted Code Blobs

Blob	VA in .data	Size	Purpose
blob1	0x140004000	0x70	Prints ./chal <flag>\n via putchar (usage)
blob2	0x140004070	0x40	Prints Nope (wrong flag)
blob3	0x1400040b0	0x30	Prints No
blob4	0x1400040e0	0x30	Prints Yes (correct flag)
blob5	0x140004110	0x30	Calls exit(1)
blob6	0x140004140	0x1a0	Checks argv[1][0..3] == HTB{ and argv[1][21] == }
blob7	0x1400042e0	0x1e0	Checks argv[1][4..9] == W3iRd_
blob8	0x1400044c0	0x270	Checks argv[1][10..17] == RUnT1m3_
blob9	0x140004730	0x100	Checks argv[1][18..20] == DEC

Main Logic (0x140001580)

result = 0

# Each checker: decrypt → execute → OR result
result |= blob6(argv[1])   # HTB{...}  (wrapper)
result |= blob7(argv[1])   # W3iRd_    (chars 4-9)
result |= blob8(argv[1])   # RUnT1m3_  (chars 10-17)
result |= blob9(argv[1])   # DEC       (chars 18-20)

if result == 0:
    blob4()  # "Yes" — all checks passed
else:
    blob2()  # "Nope" — at least one check failed

Each checker returns 1 if at least one character doesn't match, and 0 if everything is correct. Results are OR'd into an accumulator — if any checker returns 1, the result is non-zero → Nope.

Solution

Approach 1: Offline Blob Decryption

Knowing the AES decryption algorithm, we can extract the encrypted blobs from the .data section and decrypt them manually:

#!/usr/bin/env python3
"""
Offline decryption of code blobs from partialencryption.exe.
Requires: pycryptodome or manual implementation of AES-NI operations.

Algorithm for each 16-byte block i:
  key = bytes([i] * 16)
  kg0 = aeskeygenassist(key, rcon=0x00)
  kg1 = aeskeygenassist(key, rcon=0x10)
  xmm2 = block ^ kg1
  result = aesdeclast(xmm2, kg0)
         = InvSubBytes(InvShiftRows(xmm2)) ^ kg0
"""

# AES S-Box inverse (for InvSubBytes)
INV_SBOX = [
    0x52, 0x09, 0x6A, 0xD5, 0x30, 0x36, 0xA5, 0x38, 0xBF, 0x40, 0xA3, 0x9E, 0x81, 0xF3, 0xD7, 0xFB,
    0x7C, 0xE3, 0x39, 0x82, 0x9B, 0x2F, 0xFF, 0x87, 0x34, 0x8E, 0x43, 0x44, 0xC4, 0xDE, 0xE9, 0xCB,
    0x54, 0x7B, 0x94, 0x32, 0xA6, 0xC2, 0x23, 0x3D, 0xEE, 0x4C, 0x95, 0x0B, 0x42, 0xFA, 0xC3, 0x4E,
    0x08, 0x2E, 0xA1, 0x66, 0x28, 0xD9, 0x24, 0xB2, 0x76, 0x5B, 0xA2, 0x49, 0x6D, 0x8B, 0xD1, 0x25,
    0x72, 0xF8, 0xF6, 0x64, 0x86, 0x68, 0x98, 0x16, 0xD4, 0xA4, 0x5C, 0xCC, 0x5D, 0x65, 0xB6, 0x92,
    0x6C, 0x70, 0x48, 0x50, 0xFD, 0xED, 0xB9, 0xDA, 0x5E, 0x15, 0x46, 0x57, 0xA7, 0x8D, 0x9D, 0x84,
    0x90, 0xD8, 0xAB, 0x00, 0x8C, 0xBC, 0xD3, 0x0A, 0xF7, 0xE4, 0x58, 0x05, 0xB8, 0xB3, 0x45, 0x06,
    0xD0, 0x2C, 0x1E, 0x8F, 0xCA, 0x3F, 0x0F, 0x02, 0xC1, 0xAF, 0xBD, 0x03, 0x01, 0x13, 0x8A, 0x6B,
    0x3A, 0x91, 0x11, 0x41, 0x4F, 0x67, 0xDC, 0xEA, 0x97, 0xF2, 0xCF, 0xCE, 0xF0, 0xB4, 0xE6, 0x73,
    0x96, 0xAC, 0x74, 0x22, 0xE7, 0xAD, 0x35, 0x85, 0xE2, 0xF9, 0x37, 0xE8, 0x1C, 0x75, 0xDF, 0x6E,
    0x47, 0xF1, 0x1A, 0x71, 0x1D, 0x29, 0xC5, 0x89, 0x6F, 0xB7, 0x62, 0x0E, 0xAA, 0x18, 0xBE, 0x1B,
    0xFC, 0x56, 0x3E, 0x4B, 0xC6, 0xD2, 0x79, 0x20, 0x9A, 0xDB, 0xC0, 0xFE, 0x78, 0xCD, 0x5A, 0xF4,
    0x1F, 0xDD, 0xA8, 0x33, 0x88, 0x07, 0xC7, 0x31, 0xB1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xEC, 0x5F,
    0x60, 0x51, 0x7F, 0xA9, 0x19, 0xB5, 0x4A, 0x0D, 0x2D, 0xE5, 0x7A, 0x9F, 0x93, 0xC9, 0x9C, 0xEF,
    0xA0, 0xE0, 0x3B, 0x4D, 0xAE, 0x2A, 0xF5, 0xB0, 0xC8, 0xEB, 0xBB, 0x3C, 0x83, 0x53, 0x99, 0x61,
    0x17, 0x2B, 0x04, 0x7E, 0xBA, 0x77, 0xD6, 0x26, 0xE1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0C, 0x7D,
]

# AES S-Box forward (for AESKEYGENASSIST SubBytes)
SBOX = [
    0x63, 0x7C, 0x77, 0x7B, 0xF2, 0x6B, 0x6F, 0xC5, 0x30, 0x01, 0x67, 0x2B, 0xFE, 0xD7, 0xAB, 0x76,
    0xCA, 0x82, 0xC9, 0x7D, 0xFA, 0x59, 0x47, 0xF0, 0xAD, 0xD4, 0xA2, 0xAF, 0x9C, 0xA4, 0x72, 0xC0,
    0xB7, 0xFD, 0x93, 0x26, 0x36, 0x3F, 0xF7, 0xCC, 0x34, 0xA5, 0xE5, 0xF1, 0x71, 0xD8, 0x31, 0x15,
    0x04, 0xC7, 0x23, 0xC3, 0x18, 0x96, 0x05, 0x9A, 0x07, 0x12, 0x80, 0xE2, 0xEB, 0x27, 0xB2, 0x75,
    0x09, 0x83, 0x2C, 0x1A, 0x1B, 0x6E, 0x5A, 0xA0, 0x52, 0x3B, 0xD6, 0xB3, 0x29, 0xE3, 0x2F, 0x84,
    0x53, 0xD1, 0x00, 0xED, 0x20, 0xFC, 0xB1, 0x5B, 0x6A, 0xCB, 0xBE, 0x39, 0x4A, 0x4C, 0x58, 0xCF,
    0xD0, 0xEF, 0xAA, 0xFB, 0x43, 0x4D, 0x33, 0x85, 0x45, 0xF9, 0x02, 0x7F, 0x50, 0x3C, 0x9F, 0xA8,
    0x51, 0xA3, 0x40, 0x8F, 0x92, 0x9D, 0x38, 0xF5, 0xBC, 0xB6, 0xDA, 0x21, 0x10, 0xFF, 0xF3, 0xD2,
    0xCD, 0x0C, 0x13, 0xEC, 0x5F, 0x97, 0x44, 0x17, 0xC4, 0xA7, 0x7E, 0x3D, 0x64, 0x5D, 0x19, 0x73,
    0x60, 0x81, 0x4F, 0xDC, 0x22, 0x2A, 0x90, 0x88, 0x46, 0xEE, 0xB8, 0x14, 0xDE, 0x5E, 0x0B, 0xDB,
    0xE0, 0x32, 0x3A, 0x0A, 0x49, 0x06, 0x24, 0x5C, 0xC2, 0xD3, 0xAC, 0x62, 0x91, 0x95, 0xE4, 0x79,
    0xE7, 0xC8, 0x37, 0x6D, 0x8D, 0xD5, 0x4E, 0xA9, 0x6C, 0x56, 0xF4, 0xEA, 0x65, 0x7A, 0xAE, 0x08,
    0xBA, 0x78, 0x25, 0x2E, 0x1C, 0xA6, 0xB4, 0xC6, 0xE8, 0xDD, 0x74, 0x1F, 0x4B, 0xBD, 0x8B, 0x8A,
    0x70, 0x3E, 0xB5, 0x66, 0x48, 0x03, 0xF6, 0x0E, 0x61, 0x35, 0x57, 0xB9, 0x86, 0xC1, 0x1D, 0x9E,
    0xE1, 0xF8, 0x98, 0x11, 0x69, 0xD9, 0x8E, 0x94, 0x9B, 0x1E, 0x87, 0xE9, 0xCE, 0x55, 0x28, 0xDF,
    0x8C, 0xA1, 0x89, 0x0D, 0xBF, 0xE6, 0x42, 0x68, 0x41, 0x99, 0x2D, 0x0F, 0xB0, 0x54, 0xBB, 0x16,
]

def aeskeygenassist(key_bytes, rcon):
    """
    Emulation of AESKEYGENASSIST xmm, xmm, imm8.
    Input:  16 bytes (xmm register)
    Output: 16 bytes
    
    Takes dwords X1 (bytes 4-7) and X3 (bytes 12-15) from input:
      result[0:4]  = SubWord(X1)
      result[4:8]  = RotWord(SubWord(X1)) ^ RCON
      result[8:12] = SubWord(X3)
      result[12:16]= RotWord(SubWord(X3)) ^ RCON
    """
    def sub_word(w):
        return bytes([SBOX[b] for b in w])
    
    def rot_word(w):
        return bytes([w[1], w[2], w[3], w[0]])
    
    X1 = key_bytes[4:8]
    X3 = key_bytes[12:16]
    
    sw1 = sub_word(X1)
    rw1 = rot_word(sw1)
    rw1_xor = bytes([rw1[0] ^ rcon, rw1[1], rw1[2], rw1[3]])
    
    sw3 = sub_word(X3)
    rw3 = rot_word(sw3)
    rw3_xor = bytes([rw3[0] ^ rcon, rw3[1], rw3[2], rw3[3]])
    
    return sw1 + rw1_xor + sw3 + rw3_xor

def inv_shift_rows(state):
    """InvShiftRows: inverse row shift in AES state matrix."""
    # State as 4x4 matrix (column-major)
    m = [list(state[i::4]) for i in range(4)]
    # Row 0: no shift
    # Row 1: shift right by 1
    m[1] = m[1][-1:] + m[1][:-1]
    # Row 2: shift right by 2
    m[2] = m[2][-2:] + m[2][:-2]
    # Row 3: shift right by 3
    m[3] = m[3][-3:] + m[3][:-3]
    result = []
    for col in range(4):
        for row in range(4):
            result.append(m[row][col])
    return bytes(result)

def inv_sub_bytes(state):
    """InvSubBytes: inverse S-Box substitution."""
    return bytes([INV_SBOX[b] for b in state])

def aesdeclast(data, round_key):
    """
    Emulation of AESDECLAST xmm1, xmm2.
    result = InvSubBytes(InvShiftRows(data)) XOR round_key
    """
    shifted = inv_shift_rows(data)
    subbed = inv_sub_bytes(shifted)
    return bytes([a ^ b for a, b in zip(subbed, round_key)])

def decrypt_blob(encrypted_data):
    """Decrypt a single blob (set of 16-byte blocks)."""
    result = bytearray()
    num_blocks = len(encrypted_data) // 16
    
    for i in range(num_blocks):
        block = encrypted_data[i*16 : (i+1)*16]
        key = bytes([i] * 16)
        
        kg0 = aeskeygenassist(key, 0x00)
        kg1 = aeskeygenassist(key, 0x10)
        
        xmm2 = bytes([a ^ b for a, b in zip(block, kg1)])
        decrypted = aesdeclast(xmm2, kg0)
        result.extend(decrypted)
    
    return bytes(result)

# Usage:
# with open("partialencryption.exe", "rb") as f:
#     data = f.read()
# # Extract .data section and decrypt each blob
# # Then disassemble the decrypted code

Approach 2: Dynamic Analysis

Run the binary under a debugger (x64dbg, WinDbg), set a breakpoint on VirtualProtect and dump the decrypted code from memory after each call.

Flag Assembly

Analysis of the decrypted checkers (blob6–blob9) reveals character-by-character comparisons:

blob6: argv[1][0..3]  == "HTB{"   and  argv[1][21] == "}"
blob7: argv[1][4..9]  == "W3iRd_"
blob8: argv[1][10..17] == "RUnT1m3_"
blob9: argv[1][18..20] == "DEC"

Assembling: HTB{ + W3iRd_ + RUnT1m3_ + DEC + } = HTB{REDACTED}