That .data file can be opened and viewed
Cheers.
I know what you mean there. Also....
PE files contain sections.Common sections are:
.text (AKA .code) which contains the compiled
ML.data which contains variables and strings etc., that the program can use
and modify as it runs
.data? is the same as .data, except that it initially contains no values (uninitialised data). The .data? section is a virtual section that has 0 size in an .exe file but is created when the image is mapped to a process' memory by the system loader. It's a way of pre-defining memory to be allocated, with the bonuse of that memory being inside the module's image.
.rsrc "resource" usually contains things like bitmaps, constant strings (that are not modified during execution), dialog templates etc...
Not all files have all sections, and indeed the names, despite being almost always as listed above, are just a convention. The actual function of a given section is defined by flags set in the
Characteristics member of the IMAGE_SECTION_HEADER structure.
Everything you never wanted to know about windows .exe files is
here. (You want the file called "pecoff.docx")
'COFF' stands for Common Object File Format. Executable files started out as a sub-set of COFF files, they share many internal structures. Things like .obj "object" files are COFF files - .obj files are produced by compilers which are then fed to linkers which create the finished .exe file.
.... anyway so what I mean by this:
....buffers have overflowed and corrupted .data or .data? section values...
is that when the image is mapped to memory certain areas will have been allowed for certain data, that has an expected maximum size, so there
could, for instance be something like this:
.data?
mapArray db 4000h dup (?) ;-- some map array or other
hGLthing dd ? ;-- handle to some important GL structure
otherStuff dd ?,?,?,? ;-- all sorts of other important dataas the max map size is 128x128 = 16384 (0x4000) and the map size is checked when it is loaded, whatever routine that puts data from the map or whatever into that array doesn't bother doing any checking, so when we gymp the 128x128 map size checks, the program just goes right ahead and writes data that is supposed to all fit in that array straight over the top of all the variables that are stored after it.
This is where image data sections can be tricky, because all the references to them are hard coded into the ML instructions. From one angle this makes them very easy to debug because you can always immediately see what is being accessed by a given instruction.
When memory is
dynamically allocated, its trickier to debug, but much easier to resize. the same thing could be achieved with:
.data?
ptrMapArray dd ? ;-- *POINTER* to some map array or other
hGLthing dd ? ;-- handle to some important GL structure
otherStuff dd ?,?,?,? ;-- all sorts of other important datathen somewhere in the initialization code there is:
.code
push 4000h
call GlobalAlloc
mov ptrMapArray,eax ; <-- functions return values in the eax register
so this just asks the OS to allocate space for the array then just stores a pointer to the allocated memory in the data section. Later when the program wants to access this array you might see something like:
mov edi,ptrMapArray
; ...... then further on
mov [edi+ebx],dl ; ... or whatever
in this case edi woud contain the pointer to the base address of the array, ebx would contain an offset into the array, and the low byte of edx would contain the value it is writing there.
At least that is what the source could look like. Once it is compiled then decompiled, you dont have the benefit of the meaningful variable names that the guy coding it in the first place used, so your:
mov edi,ptrMapArray has become maybe:
mov edi,[4A789Ch]... where 0x4A789C is the actual address of that dword in the .data? section (this is what the linker does)
things like:
call GlobalAllocwill still be there because GlobalAlloc is exported by name from kernel32.dll and the disassembler will extract the proc name from the IMAGE_EXPORT_DIRECTORY structure in kernel32.
... so if our array is dynamically allocated, and we want to change the max map size from 128x128 to 256x256 all we need to do to resize this particular buffer is change:
push 4000h ; (128x128)
call GlobalAlloc
to:
push 10000h ; (256x256)
call GlobalAlloc
BUT....
.... if this buffer (and not a pointer to it) was itself allocated in the .data? section, to resize it we have to create more space in that section of the PE image.
This in itself is trivial - you just adjust the VirtualSize member of the IMAGE_SECTION_HEADER structure for the .data? section and the system loader will create a larger image - however this doesn't change the ML in the .text section, and as I mentioned the addresses of the variables, and importantly for this case, those
after the array (i.e. 'hGLthing') have been hard-coded into the ML instructions by the linker, so the code is still going to read and write the value of that handle to the same location, and the code writing to the array is still going to corrupt it. In this case all we have done is allocate extra memory at the end of the .data? section that isn't used anyway.
To resize this kind of array you would have to increase the size of the .data? section by N bytes, then find every reference to anything in the .data? section that is
above the base address of that array (there could be hundreds of addressed data locations with multiple reference to each) and increase all of these addresses, which are hard-coded into the instructions, by N bytes,
and to make matters worse there could be pointer tables to some of these locations stored literally
anywhere that would all need to be found and adjusted.
With the original source code, you just change your:
mapArray db 4000h dup (?) to:
mapArray db 10000h dup (?) then recompile and its done (approx. 5 seconds total)
WHen reverse engineering a complex program this is can be almost impossible, and WC2 was pushing the limits of CPU and RAM capacities back in 1996, so to get the maximum possible performance they used lots of little tricks involving pointer tables to reference things and also to control program flow. It really is a cool chunk of code (Lamby gets all dreamy-eyed and cracks a nerd-boner) but it can be hard to get your head around.
....anyway if faced with this situation I would opt for injecting a small amount of code somewhere that dynamically allocated a larger buffer, and just leave the buffer in the .data? section as a blank space. Then every instruction that accessed the original buffer would have to be located and modded to point to the dynamically allocated one, which could be a bit painful, but nowhere near as painful as doing this for every one of the possibly hundreds of memory locations in the data section that are above the address of the original buffer....
and a nice trick for this type of thing is using a debugger to put a memory-watch on the original buffer then pause execution and report the value of the EIP register whenever it is accessed 
And for specifically Blizz related allocation, much of the WC2 stuff is, as you probably know, stored using Mike O'Brians Mo'PaQ format (.mpq files on disk). These are loaded into dynamically allocate memory by various functions exported by ordinal from storm.dll, so if any of these need to be resized it would be done by adjusting the size of the relevent resource in the .mpq archive.
Some notes on the code bits here:It's all ASM code because that's how I think when I'm doing this stuff, and it really is the only way to think when you're reverse-engineering anything.
DATA DECLARATIONS
.data?
hGLthing dd ?
this just allocates 4 bytes in the .data? section and names that location "hGLthing "
'dd' is 'declare double' it allocates a DWORD
'dw' allocates a WORD (2 bytes)
'db' allocates a BYTE
(also 'dq' allocates a QWORD, but you won't find any of those in WC2)
with this:
mapArray db 4000h dup (?)'dup' means duplicate. Its declaring bytes (db) and its declaring 0x4000 of them.
In all of these the '?' is the data value the location is initialised with.
Because these are in the .data? section which holds only uninitialised data they can't have any defined initial value do they must all be '?'.
In the .data section you can have initialised data so you might see:
.data
dwThisVar dd 0FE339D2h
myByte db 20h
thisString db "Hello World",0 ; <-- an initialised zero terminated string
myWords dw 3,3,3,3 ; <-- 4 words all set to 3 (8 bytes all up)
ymWords dw 4 dup (3) ; <-- exactly the same ( 4 words all set to 3 )when compiled this .data would look like:
D2:39:E3:0F:
20:
48:65:6C:6C:6F:20:57:6F:72:6C:64:00
03:00:03:00:03:00:03:00:
03:00:03:00:03:00:03:00:(off the to of my head - dont crucify me if I messed up, but you get the idea..)
WC2, was of course written in C, but it all compiles to ML anyway. Apart from ASM (which is for the most part 1:1 ML instructions) C is the lowest level language around and therefore closest related to ASM and the compiled .exe, also most C compilers support in-line ASM code, so performance critical sections of code can be written in ASM, within the C source and then all compiled at once.
Data containers declared in C end up roughly the same as they would have if declare in ASM, i.e. a C pointer variable '*ptrArray' or whatever, if declared globally would end up as a 'ptrArray dd ?' in the data section, whereas if it was declared locally in a procedure it would be created as a stack variable. Which is temporary use of the 'stack' to store values. The stack is a special area of memory that is heavily used by the CPU to shunt numbers around... the 'push' and 'pop' instructions move values onto and off the stack. It acts like a big pile of numbers (FILO) but the current stack location is held in the ESP register ... the
stack pointer.
.. and OMG don't look now, but here here comes a bit of compilers, procs and stack frames 
This leads to the actual compiler that was used to compile the source, as they have diferent methods for implimentng procedure calls and allocating local storage. You can allocate 4 bytes of stack space simply with:
sub esp,4but it is vital to ensure that the stack is 'balanced' - returned to its correct state before executing the next 'ret' -
return instruction. In this case you could
add esp,4or just
pop eaxwhich would get the value off the stack and put it in the eax register. (
push and
pop automatically adjust the esp register). The other way to do this is to save the value of esp when you enter a procedure.
The Microsoft compiler does this by repurposing ebp - the
base pointer to save the value in esp when entering a procedure, so it might produce something like:
push ebp ; save the value in esp
mov ebp,esp ; use ebp to store the current stack pointer
sub esp,8 ; in this case 8 bytes of local storage has been allocated for the proc (most likely 2 DWORDS)
At this point the arguments that were passed to the procedure are also sitting on the stack behind the return address that was left there by the
call instruction that called the proc and the original value of ebp, which we have just stored there. ebp holds the value of the stack pointer after this proc entry, so if it is not changed the passed arguments can be accessed directly using ebp provided we allow for the 8 bytes that these 2 DWORD values use, so the first argument can be accessed at [ebp+8] and assuming its a DWORD (which they almost always are) the second argument is at [ebp+0Ch] etc. At the same time the locals that were allocated with the
sub esp,8can be accessed at [ebp-4] and [ebp-8]. This area of the stack, from the start of the passed arguments to the end of any local allocations, for the life of the proc is often referred to as the
stack frame for that procedure.
Meanwhile you can merrily
push and
pop your heart out or call other procs etc. (sending the stack pointer all over the shop) because the position of the stack pointer at the proc entry is stored in ebp, and the entry value of ebp is itself stored on the stack. So when it comes time to exit the proc they just do the following:
mov esp,ebp ; <-- restore the stack pointer from the base pointer
pop ebp ; <-- pop the base pointer value off the stack
ret ; <-- pop the return address off the stack and jmp to that locationIn actual fact, Mr Gates made such extensive use of this method that his mates at Intel eventually designed their CPUs with special instructions to acommodate it. The
leave instruction does the combination of
mov esp,ebp and
pop ebp in one hit.
Anybody who has not nodded off by now may have noticed the other issue. Despite having balanced the stack from procedure entry to exit (thus taking care of any locals allocated), whatever arguments were passed to the proc when it was call are still sitting on the stack, as these were pushed there by the caller prior to calling the proc, so esp still has to be increased by an amount equal to the total size of the passed args to put things to rights, but this has to happen
after the return value is removed from the stack. Alternate versions of the
ret instruction (often denoted as
retn) that take a numeric value are used for this, so at the end of a proc that is passed 2 DWORD args, you might see:
mov esp,ebp ; <-- restore the stack pointer from the base pointer
pop ebp ; <-- pop the base pointer value off the stack
retn 8 ; <-- pop the return address off the stack, add 8 to esp, then jmp to that addressOR...
leave <-- do the esp/ebp square dance
retn 8 ; <-- pop the return address off the stack, add 8 to esp, then jmp to that address...sadly, however. Blizzard didn't use MASM to compile WC2. By the looks of it, it's Borland code,....
<EDIT>
...Actually, on further investigation, it looks like they used M$ VisualC 2.0 Presumably there's compiler options for proc layouts/calling conventions etc. Interesting, but changes nothing anyway - it is what it is
</EDIT>.... not that there's anything wrong with that as far as executing it goes, but when decompiled, the stack frame addressing isn't anywhere near as clean to read.
For example, where the MASM code references arg1 by [ebp+8], and anywhere in that proc that you see [ebp+8] you know that it is addressing arg1, Borland doesn't bother with using ebp and just adresses arg1 as [esp+4]
..... until the first time a
push instruction is used then 4 is added to esp, so now arg1 is at [esp+8] ...
push again its at [esp+0Ch]... then
pop something and its back to [esp+8] etc.
So the location of the first argument to a proc is esp + (number of
pushes - number of
pops +1) x 4 from the proc's entry point.
EDIT: The exception is, of course, values that are pushed onto the stack as
arguments to another proc that is being called from within the proc being
examined, these will be balanced at the exit of the proc being called.Modern deompilers like IDA (
btw use that link to the free version 5 - they don't advertise that, and all the other versions cost mega$) take a fair bit of the hard work out of this for you by getting creative about how they present their disassembly, but due to the way WC2 uses embedded proc call addresses for optimisation you will still see red flags all over the place warning about stack imbalances because there's simply no way for the decompiler to decipher where many of the procs are being called from, so no way for them to be able to interpret what the state of the stack is before the call.
OK .... lol
I know this is a shit-ton of geekage, so I don't expect you to absorb it all at once, however I wouldn't have wasted my time if I didn't think you (and a couple of others who I expect are following...) were up to it. So come back to this post and re-read it occasionally because
DAMN I wish somebody had sat me down and explained even half of this stuff to me 20 years ago..... I love you assholes, really I do.
