[ACCEPTED]-How to make my program DEP-compatible?-winforms

DEP runs in one of two modes:

1) Hardware 37 DEP is for CPUs that can mark memory pages 36 as non-executable. This helps to prevent 35 certain exploits such as buffer overflows.

2) Software 34 DEP is for CPUs that do not have hardware 33 DEP support. It doesn't prevent execution 32 of code in data pages, but instead stops 31 SEH overwrite (another type of attack).

On 30 Windows XP with CPUs that support it, hardware 29 DEP is enabled by default only for certain 28 Windows system binaries, and also for programs 27 that choose to "opt-in".

On Vista with CPUs 26 that support it, hardware DEP is enabled 25 by default for nearly all processes. This 24 can occasionally be problematic, usually 23 for older programs and drivers, and for 22 ISVs that haven't done any Vista testing.

So 21 I suspect that the first step is to discover 20 whether you're dealing with software or 19 hardware DEP. Also, are you using C#/VB 18 or Managed C++? And are you using any native 17 code or components? If your application 16 uses a native component or an ActiveX control 15 that was built using the old ATL framework, then 14 it's quite possible that your application 13 will fail with hardware DEP.

Since .NET Framework 12 2.0 SP1, I believe that the C# compiler 11 emits managed code which is DEP-compatible. But 10 if your application is generating DEP exceptions, then 9 you can try clearing the IMAGE_DLLCHARACTERISTICS_NX_COMPAT 8 flag for your executable. To do this you 7 use EDITBIN.EXE from the VC toolset like 6 so:

editbin.exe /NXCOMPAT:NO <your binary>

If you're using Visual Studio, you can 5 add a post-build step to your executable's 4 project. You'll need to setup the environment 3 so that EDITBIN's dependencies can be resolved. When 2 I'm using native code as part of my app, the 1 post-build step looks like this:

call $(DevEnvDir)..\tools\vsvars32.bat
editbin.exe /NXCOMPAT:NO $(TargetPath)  

Older versions of ATL are not DEP aware, so 7 if you use any ActiveX controls that are 6 built using ATL, and were built on that 5 version of ATL (version 7.1 and below, I 4 think), you'll get DEP errors.

As a last 3 resort, you can in fact disable DEP for 2 the process by calling an API function: SetProcessDEPPolicy.

More 1 information on SetProcessDEPPolicy

The compilers that shipped with .NET 2.0 4 SP1 turn on the NXCOMPAT flag in the executable 3 file header. You can turn that flag off 2 in a Post Build step by running EditBin.exe 1 with the /NXCOMPAT:NO option.

FWIW, it's worth explicitly mentioning that 13 in many cases, applications aren't "incompatible 12 with DEP" but rather were about to crash 11 anyway and DEP "dove in to save the day." Very 10 often, once you disable DEP, you'll find 9 that you're hitting an "ordinary" AV.

If 8 your project is written solely in .NET 3.0, this 7 is almost certainly the case, because .NET 6 doesn't do any of the "crazy" things that 5 trigger DEP (e.g. function thunking, etc).

To 4 debug, install a debugger or enable Watson 3 to generate a .DMP file, then take that 2 .DMP file to the developer's machine and 1 figure out what went wrong.

Start by trying to figure out where and 6 how your program is failing. Can you replicate 5 the issue on your system? With enabling 4 DEP for the application on your system? When 3 you can replicate the issue and get the 2 error (access violation), you can look to 1 fixing your program.

See the MSDN article for information on DEP.