4. By using various forms static/dynamic analysis (taint analysis, fault propagation, fault injection).
5. By fuzz-testing the software.
Taint analysis is looking at which registers/memory locations you have access to. You inject data into a process and then check where in memory that data shows up, which execution branches are taken, which registers are used with the data, what the stack does, etc.
Fault propagation looks at where errors and faults are handled and how they are handled. The idea is to get a good idea of how a process manages errors and then find an error that is not handled (correctly).
Fault injection is injecting errors into a process and then watch the fireworks. Think of this as the brute-force approach to fault propagation analysis.
Fuzz testing is bombarding the process with semi-valid/semi-random input and watching what happens. There are many tools with this. Example output for browser-fuzzing could look like this:
<html>>style="\>>overflow:none"></body><html>
Obviously none of today's browsers would be exploited by that, but with mutli-threaded rendering, memory management, javascript reading/writing html and all that stuff going on at the same time it's not surprising that fuzz testing can turn up lots of errors (though not all exploitable).
4. By using various forms static/dynamic analysis (taint analysis, fault propagation, fault injection).
5. By fuzz-testing the software.
Taint analysis is looking at which registers/memory locations you have access to. You inject data into a process and then check where in memory that data shows up, which execution branches are taken, which registers are used with the data, what the stack does, etc.
Fault propagation looks at where errors and faults are handled and how they are handled. The idea is to get a good idea of how a process manages errors and then find an error that is not handled (correctly).
Fault injection is injecting errors into a process and then watch the fireworks. Think of this as the brute-force approach to fault propagation analysis.
Fuzz testing is bombarding the process with semi-valid/semi-random input and watching what happens. There are many tools with this. Example output for browser-fuzzing could look like this:
<html>>style="\>>overflow:none"></body><html>
Obviously none of today's browsers would be exploited by that, but with mutli-threaded rendering, memory management, javascript reading/writing html and all that stuff going on at the same time it's not surprising that fuzz testing can turn up lots of errors (though not all exploitable).