Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information in this book is sold without warranty either expressed or implied.
Practical Reverse Engineering of a Windows Executable 321
Reverse engineering is a tool used for analyzing software to exploit its weaknesses and strengthen its defenses.
Who this book is for
What this book covers
Chapter 8, The Sandbox: Virtualization as a Component for RE, shows how to use emulation to inform reverse engineering and overcome barriers to running on hardware other than target binary support. Chapter 12, Practical Reverse Engineering of a Windows Executable, covers the practical use of the tools we know at this point.
To get the most out of this book
Chapter 11, Tricks Against Analysis, shows how to identify and deal with tricks against reversal and debugging. Chapter 13, Turning Different File Types, covers analyzing different file types using the latest tools.
Download the example code files
The code bundle for the book is also hosted on GitHub at https://github.com/. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/.
Download the color images
Conventions used
Get in touch
If you find an error in this book, we would be grateful if you could let us know. Piracy: If you come across illegal copies of our works in any form on the Internet, we would be grateful if you could provide us with the location address or website name.
Reviews
In this first chapter, we will introduce reverse engineering and explain what it is for. In this chapter, we will briefly introduce the process and types of tools used in software reverse engineering.
Reverse engineering
We will start by discussing some insights already applied in different aspects that will help the reader understand what reverse engineering is. Based on this information, a reverse engineer could determine exactly where in the software to inspect.
Technical requirements
Reverse engineering as a process
Seeking approval
Static analysis
Dynamic analysis
Low-level analysis
Reporting
Tools
Binary analysis tools
Why were specific codes used in the program where it was meant to work on. The following screenshot shows information indicating that the program was most likely built using Window Forms:.
Disassemblers
Debuggers
Monitoring tools
Decompilers
Malware handling
Basic analysis lab setup
It should also be noted that there is software that can detect it being run, and doesn't like being run in a virtualized environment. Disk management software that can store images or reimage disks would be the best solution for us here.
Our setup
Fill in the Snapshot Name and Snapshot Description fields and then click the OK button. Remember that any network communication in the virtual machine uses the host computer's IP address.
Samples
This means that any network resources required by the virtual machine will use the host computer's IP address. The IP address of the virtual machine is taken from the VirtualBox virtual DHCP service.
Summary
In this chapter, we'll point out common techniques used by malware to become persistent. Important behaviors of malware and some tools capable of identifying how the malware installs itself in the system will also be shown.
The operating system environment
In addition, download and install this software: SysInternals suite (https://docs.microsoft.com/en-us/sysinternals/ . downloads/sysinternals-suite) .
The filesystem
A file has two forks stored in the file system, the data fork and the resource fork. During the transfer, only the file is transferred, but the resource fork is stripped in the process.
Memory
The registry system
In the left pane, the registry values are found under the Name column, as can be seen in the following screenshot:.
Typical malware behavior
Persistence
Run keys
Load and Run values
Here ServiceName is the service name ImagePath = [sys/dll file path]. The entry in the registry value (Default) executes the files described by [File type or extension name].
Startup values
1 refers to the executable being executed as is, while %* refers to the command line arguments.
The Image File Execution Options key
In addition to registry entries, an executable file can also be triggered through a schedule using the task scheduler or cron jobs.
Malware delivery
Instant messenger
The computer network
Media storage
The autorun.inf file at the root of the drive contains information about which file to run. The Autorun.inf for the previously shown VirtualBox autoplay setup dialog contains the text as shown in the previous screenshot.
Exploits and compromised websites
Together with the deleted copy, it generates an autorun.inf file that points to the 2. Once the user clicks the link, the site is typically redirected to another compromised site, and another, until the exploit kit landing page is reaches.
Software piracy
Usually, compromised sites are manually hacked and injected with a hidden script that redirects to another website. From the user's web browser, the exploit kit gateway collects information about the machine, such as software versions, and then determines whether the software is known to be vulnerable or not.
Malware file properties
In a malware campaign, social engineering is used to trick users into visiting links that are actually compromised. It also uses misleading file names, such as the words INVOICE, New File, Scandal, Expose, Pamela, Confidential, etc.
Payload – the evil within
Since our goal for this chapter is to extract components, we will discuss tools that can help us find suspect files. However, there are also third-party tools that we can use that can help us identify suspicious files.
Autoruns
The Process explorer
We discussed the three main areas of the Windows operating system environment: memory, disk, and the registry. To do that, we listed common boot areas in the system that we can search.
Further reading
The source code must be compiled into its assembly language code form so that the machine can understand it. In a way, what we're building here is a way to reverse engineer an assembly language code to source code.
Binary numbers
It also includes tutorials that can help you develop programs using assembly language. It is preferred, but not required, that the reader has some basic knowledge of any programming language.
Bases
In base-2 numbers, the value of a place is twice the value of the place to its right. In base 16 numbers, the value of a place is 16 times the value of the place to its right.
Converting between bases
Converting from hexadecimal to binary simply requires knowing how many binary digits are in a hexadecimal digit. So why the use of base-2 and base-16 in computers, rather than our everyday base-10 use.
Binary arithmetic
To maximize the information that can be stored in 4 bits, we can represent numbers from 0 to 15 instead. Lists of printable characters that can be typed on the keyboard and some of the extended keyboard format characters can be found at https:///.
Signed numbers
Convert the result to decimal and put the – sign at the beginning, as this is a 3.
Registers
This flag indicates whether the number of set bits is odd or even since the last instruction operation. This flag is set when the result of the last instruction operation is a negative number.
Memory addressing
Endianness
Basic instructions
Opcode bytes
Copying data
MOV and LEA
The for-loop statement takes the value of y and the pointer address of y in a struct test element. Knowing that a DWORD has 4 bytes is equivalent to value = test[i].y; in assembly language will look like mov edx, [ebx+ecx*8+4].
Arithmetic operations
Let's say the base of the test array is in EBX, the for-loop counter, i, is in ECX, the integers are DWORD values, and so struct Test will contain two DWORD values.
Addition and subtraction
Increment and decrement instructions
Multiplication and division instructions
Other signed operations
Bitwise algebra
Control flow
JC/JB/JNAE CF = 1 Jump if carrying/Jump if below/Jump if not above or equal JNC/JNB/JAECF = 0 Jump if not carrying/Jump if not below/Jump if above or equal. YES/JNBE CF = 0 and ZF = 0 Jump if above/Jump if not below or equal JNA/JBE CF = 1 or ZF = 1 Jump if not above/Jump if below or equal JG/JNLE ZF = 0 and SF = OR Jump if greater/Jump if not less than or equal JNG/JLE ZF = 1 or SF != OR Jump if not greater/Jump if less than or equal JL/JNGE SF != OR Jump if less/Jump if not greater than or equal to JNL / JGE SF = OR Jump if not less/Jump if greater or equal JP/JPE PF = 1 Jump if parity/Jump if parity is even JNP/JPO PF = 0 Jump if not parity/Jump if parity is odd.
Stack manipulation
If the top of the stack, stored in ESP, is at address 002FFFFCh, then ESP becomes 002FFFF8h and stores 1 in the new ESP address. POP takes the value from the top of the stack (ESP) and stores it in the register or memory location indicated in the operand.
Tools – builder and debugger
If the address of the top of the stack, stored in ESP, is at address 002FFFF8h, and the DWORD value stored at the top of the stack is 0xDEADBEEF, then 0xDEADBEEF will be stored in EAX, while ESP becomes 002FFFFCh. PUSH/PUSHAD both push all general-purpose registers onto the stack in this order (for 32-bit builds): EAX, ECX, EDX, EBX, EBP, ESP, EBP, ESI, and EDI.
Popular assemblers
MASM
The MASM package comes with a text editor that has a menu containing a compiler and linker for building the source as an executable. This is very convenient as there is no need to go to the command line to run the compiler and linker to build the executable.
NASM
FASM
WinDbg
WinDbg has a graphical user interface and by default displays a command window where you can type and enter commands. Windbg can be downloaded from https:///docs.microsoft.com/en-us/windows-hardware/drivers/debugger/..
Ollydebug
Hello World
Installation of FASM
It works!
Dealing with common errors when building
Dissecting the program
It is in the .code section where the start: tag is and where our code is. The printf is actually a label that is the local address in the program declared in the .idata section.
After Hello
Calling APIs
Common Windows API libraries
WS2_32, WININET, URLMON, and NETAPI32 are libraries that contain functions related to networking and Internet communication.
Short list of common API functions
Debugging
This causes the debugger to pause in ntdll.dll before we even get to our helloworld code. By the time you finish this book, assembly language will feel like a walk in the park.
Assessment and static analysis
We begin static analysis by determining the file type, and then move on to understanding the file format.
File types and header analysis
Extracting useful information from file
PEid and TrID
MASTIFF
Other information
PE executables
For example, consider that if the PE file imports the KERNEL32.DLL library, we should expect the file to contain core APIs that can access files, processes, and threads, or dynamically load other libraries and import functions. KERNEL32.DLL and libraries such as USER32.DLL, WININET.DLL, and ADVAPI32.DLL have functions that are used to pass information to the native functions to perform actual system-level operations.
Deadlisting
The statically imported modules, libraries, and functions are hints about what we can expect the program to access. USER32.DLL : This contains window control functions related to anything displayed on the monitor, such as dialog boxes, show message boxes, and position window boxes where they should be.
IDA (Interactive Disassembler)
Some high-level programs are compiled using p-code, such as C# and Visual Basic (p-code version). A high-level syntax usually has an equivalent block of p-code that can be identified by the decompiler.
ILSpy – C# Decompiler
But since it is still a high-level language, some blocks of code can be traced back to their C syntax. The changes may include changes made by the operating system, and it is up to the analyst to identify which changes were caused by the program.
Memory regions and the mapping of a process
Sandboxie (freemium) - This tool is used in the environment where the program will run. Each VAS has a kernel space, which is listed in the page tables as a space with exclusive rights.
Process and thread monitoring
Network traffic
Monitoring system changes
Post-execution differences
Now we have the option to validate the path the program will take when debugging.
Try it yourself
The creation date is when the file was written or copied to the directory where it is now. To statically analyze the behavior of the file, we will use a disassembly tool known as IDA Pro.
For example, if the static analysis identified the file as a Win32 PE file executable, then tools for analyzing PE files will need to be prepared. We also introduced some tools that we can use to engage with both statically and dynamically.
Analysis environments
For example, if a file is identified as an ELF file type, we would need to use tools to parse a Linux executable file. There are many tools available out there - don't limit yourself to the tools discussed in this book.
Virtual machines
Windows
For the second option in the previous list, after installing the appliance, it is best to take an initial snapshot before running the virtual machine. Returning to this initial snapshot should reset the expiration to the time the appliance was installed.
Linux
Information gathering tools
File type information
Hash identifying
HashTab: This tool runs in Windows and can be embedded as a tab in the property information of a file. 7-zip: This tool is actually a file archiver, but it has an extension tool that can be activated to calculate the hash of a file in MD5, SHA1, SHA256 and so on.
Strings
CaptureBAT: In addition to what Process Monitor can do, this command-line tool is also capable of monitoring network traffic.
Default command-line tools
Information on how to install the rails can be found on its website, available at https://rada.re. Source and precompiled binaries can be found at https://github.com/icsharpcode/ILSpy.
Network tools
Editing tools
Attack tools
Automation tools
Software forensic tools
Automated dynamic analysis
In malware analysis, volatility is one of the most popular pieces of open source software used. The difference in the photos can be compared, showing what changes have been made to the system.
Online service sites
In the next chapter, we'll learn more tools as we deal with reverse engineering on Linux platforms. In the previous chapter, we introduced a number of Linux command-line tools that are built-in by default.
Setup
We will conclude this chapter by setting up a Windows sandbox client, running a program in it, and monitoring the network traffic coming from the sandbox. As a reverse engineer, no technology should be an obstacle, and it is never too late to learn this technology.
Linux executable – hello world
You can go to https:///www.ubuntu.com/ to get an ISO for the Ubuntu installer.
The rest are put there by the compiler itself, as part of its code, which prepares and finishes the graceful execution of our code. If you notice, the disassembly syntax is different from the Intel assembly language format we learned.
What have we gathered so far?
Going further with debugging
Now that we're in the main function, we can execute any step-in instruction. stepi command) and step over (the nexti command). Here is a table showing the changes in the values of the esp, ebp and ecx registers after the instruction at the given address.
A better debugger
As of this writing, it is suggested that Radare2 be installed by running sys/install.sh or.
Hello World in Radare2
This is not the entry point of the hello program, but rather an address in the dynamic loader. You will need to refresh radare2's visual mode by opening it again to see the changes.
What is the password?
From the looks of this graph, there is a loop, and to exit the loop, it would require the value in local_414h to be greater than or equal to the value in local_410h. The entered password must have a length of 7 characters and the sum of all characters in the password must be equal to 734.
Network traffic analysis
We may not come across much analysis of Linux files in the industry, but knowing how to do it will certainly come in handy when the task calls for it. Since Windows is one of the most popular operating systems in the world, most software in the cyber world has been written for it.
Learning about the APIs
