What is a Buffer Overflow

A buffer overflow is a common problem for today's users
of computer systems.

It occurs if memory allocated by an application program
does not provide enough storage to serve a current
request.

If, for example, an application has an interface providing
some simple input controls without field length validation
it will be possible for the user to cause such an overflow,
if the amount of data entered exceeds the range of
memory allocated by the application. As a result memory
would get corrupted. It would effectively destroy data
stored in the location following the buffer that got
overwritten.

This is just an example, because only few users would
attempt to hack their own systems. 

You might ask yourself, how can this be used to introduce
malicious code into a system ?

If we look at the most common form of overflows, the
stack based buffer overflow, the answer lies in the way
the processor stores internal, control flow relevant data,
together with application data on the stack. Local
variables declared in a program's function are usually
allocated on the stack. The processor itself uses the
stack area during function calls to store the return
address, in order to be capable of continuing the control
flow, upon function completion, where the function has
been called from. Because this return address is stored
following a function's local variables, it can be potentially
overwritten by a buffer overflow. This allows an attacker
to decide where the CPU should continue the control
flow upon the function's completion. If the attacker
correctly chooses the data he or she uses to overflow
the buffer, it will be possible to let the CPU continue
execution within some part of the data used to overflow
the buffer, thus allowing the introduction and execution
of malicious code.

A heap based overflow works differently. Applications
use the heap area of memory to store data available
during the entire lifetime of the process. There are two
different kinds of heaps on Windows systems. The
single instance of the application default heap is al-
located by the system during process creation. Dynamic
heap is allocated by an application. To increase per-
formance, multithreaded applications tend to allocate
an additional heap for each running thread. The reason
for this is that the single instance of the default heap
requires synchronization mechanisms to prevent two
threads from accessing the same address at one time.
Lets imagine an application using some function pointers
stored on the heap, to allow some kind of dynamic data
processing. It would be possible to overflow another data
buffer located on the heap as well, to gain control of those
function pointers. An attacker could potentially let those
pointers point to some malicious code, contained in the
data used to overflow the data buffer.

Another potential method is overwriting global variables
or static variables defined within the scope of a function.
Theese are stored within the data segment of an app-
lication. As a result static variables are not being re-
moved from the stack upon function completion, op-
posed to stack based variables. A very common
scenario for this are C++ class methods, because
they are similar to globally available variables (struct)
and they use function pointers to implement a classes
methods causing them to end up in the data segment.

The described methods can be used by an attacker
in many ways. Some potentially scenarios are
described below:

• Getting a network service to execute code upon
  receiving maliciously crafted requests. This
  recently happened to the Microsoft RPC Service
  in the form of the LoveSan virus. Other potential
  targets for this kind of attack are Web Servers,
  Mail Servers and others.
    

• Getting an eMail client to execute code upon
  receiving mail traffic by using a maliciously
  crafted email message.
   

• Getting a Web Browser to execute upon visiting
  a maliciously crafted web page.