The Fundamentals of Unpacking
Before going into any detail I want to clarify that all this information is just a small summary made by me and that all the information comes from the book “The art of Unpacking”!!!.
+ I will most likely keep this topic/blog updated, I will continue to add and explain in summary form the other methods and techniques found in the book.
1. Introduction
Hello, so today we´ll be looking at two of the most basic anti-reversing methods used by executable packers and protectors, while also examining two fundamental techniques that can be employed to bypass/neutralize these protections.
I hope that you (the reader) found this material useful and able to apply the techniques presented. 
2. Techniques: Debugger detection
This section list the techniques used by packers to determine if the process is being debugged, or if a debugger is running in the system.
2.1. PEB.BeingDebugged Flag: IsDebuggerPresent()
This is the most basic debugger detection technique since the flag itself is a redflag. This technique involves checking the BeingDebugged flag in the PEB (Process Environment Block). The kernel32!IsDebuggerPresent() API normally check the value of this flag to see if the process is being debugged by a UM (user-mode) debugger.
In this code i will the show the actual implementation of the IsDebuggerPresent() API. Its basically accesses the TEB (Thread Environment Block) of the PEB and then checks the BeingDebugged flag at the offset 0x02 of the PEB
mov eax,large fs:18h
mov eax, [eax+30h]
movzx eax, byte ptr [eax+2]
retn
Instead of just calling IsDebuggerPresent, some packers manually check the PEB for the BeingDebugged flag, this is primarily in case a reverser sets a breakpoint or patches the said API
Heres an example code to identify if a debugger is using the IsDebuggerPresent() API and the PEB.BeingDebugged flag
; call kernel32!IsDebuggerPresent()
call [IsDebuggerPresent]
test eax,eax
jnz .debugger_found
; check PEB.BeingDebugged directly
mov eax,dword [fs:0x30] ;EAX = TEB.ProcessEnviromentBlock
movzx eax,byte [eax+0x02] ;AL = PEB.BeingDebugged
text eax,eax
jnz .debugger_found
Solutions
This technique can be easily neutralized by manually patching the PEB.BeingDebugged flag with the byte value of 0x00
2.2. PEB.NtGlobalFlag, Heap Flags
Its important to know that the PEB has another field called NtGlobalFlag (specifically its offset is 0x68), the said field is normally used by packer to detect if a program had been loaded by debugger (an example of this is attaching an application to x64dbg and pressing the run button in the debugger). Normally when a proess is not being debugged, the NtGlobalFlag contains the value 0x0, however if the process is being debugged, the said fiel will normally contain the value 0x70, this value means that the following flags are set:
- FLG_HEAP_ENABLE_TAIL_CHECK (0x10)
- FLG_HEAP_ENABLE_FREE_CHECK (0x20)
- FLG_HEAP_VALIDATE_PARAMETER (0x40)
These flags are set inside the ntdll!LdrpInitializeExecutionOptions(). Its useful to know that the default value of PEB.NtGlobalFlag can be overridden using the gflags.exe tool.
Due to the flags set in NtGlobalFlag, heaps that are created will have several flags turned on, this behavior can be seen inside ntdll!RtlCreateHeap(). Typically, the initial heap created for the process (PEB.ProcessHeap) will have its Flags and ForceFlags fields set to 0x02 (HEAP_GROWABLE) and 0x0 respectively. Instead, if the process is being debugged the flags are normally set to 0x50000062 depending on the NtGlobalFlag and 0x40000060
Additionally, the following heap flags are set when a heap is created on a debugged process:
- HEAP_TAIL_CHECKING_ENABLED (0x20)
- HEAP_FREE_CHECKING_ENABLED (0x40)
Solution
The solution here is to patch PEB.NtGlobalFlag and PEB.HeapProcess flags to their corresponding values as if the process is not being debugged.
