Vista + DEP = Crash

[JohnnyMalaria]

Hello,

This may benefit anyone having problems with software crashing under Vista.

A user of our Enosoft DV Processor contacted me the other day to report that as soon as it is launched under Vista Home Premium, it crashes. The extended error info was:

Problem signature:
  Problem Event Name:	APPCRASH
  Application Name:	EnoDVProcessor.exe
  Application Version:	1.0.1.0
  Application Timestamp:	45df7636
  Fault Module Name:	StackHash_9acc
  Fault Module Version:	0.0.0.0
  Fault Module Timestamp:	00000000
  Exception Code:	c0000096
  Exception Offset:	00203ab3
  OS Version:	6.0.6000.2.0.0.768.3
  Locale ID:	2057
  Additional Information 1:	9acc
  Additional Information 2:	6a8744423ea921d631d3f444d596bb77
  Additional Information 3:	3b92
  Additional Information 4:	29e79221a1a86157a6a4ef17ce48c750

The odd thing about this is the faulting module name - StackHash_9acc. A Google search for StackHash revealed very few hits. For the most part, the hits were from other newsgroups complaining of programs crashing under Vista. However, one hit provided a useful lead relating to Data Execution Prevention (DEP).

DEP helps protect against damage from viruses and other security threats. Modern CPUs implement it in hardware. By default, Vista uses DEP for "essential Windows programs and services" but not user-installed programs. However, you can set Vista to use DEP for "all programs and services except those I select". When I changed from the default to the latter, our program would crash on launch.

If you are having problems getting software to launch without crashing under Vista, try the following:

1. Goto Control Panel\System
2. Click on the 'Advanced' tab
3. Under 'Performance', click the Settings... button
4. A new window will open - 'Performance Options'
5. Click the 'Data Execution Prevention' tab
6. You will see two options:
( ) Turn on DEP for essential Windows programs and services only
(.) Turn on DEP for all programs and services except those I select: <------ make sure this one is selected
7. Click the Add... button and look for the Enosoft DV Processor main executable (Program Files\Enosoft\Enosoft DV Processor - Unlicensed\EnoDVProcessor.exe)
8. Click OK. You may or may not have to reboot - Windows will tell you.
9. Close any other dialogs.

(Obviously, change step 7 for a different application.)

The reason DEP kicks in for our software is probably because it is wrapped in a third-party program that manages licensing, on-line purchasing etc. It also protects the main program against reverse engineering. One of the things it does is modify itself when it launches - DEP sees that as a security risk, but it isn't.

[SIRCOOKS]

More way$ for micro$oft to make money indirectly!!!!!!!

[JohnnyMalaria]

More way$ for micro$oft to make money indirectly!!!!!!!

How so? It doesn't cost anything to turn off DEP. Plus, it isn't Microsoft - it's Intel and other CPU manufacturers that build it into their chips.

[SIRCOOKS]

Dont you see how this causes some folks to only buy microsoft/microsoft sponsored software?

[Xylob the Destroyer]

I think it's time for you to make a new tin foil hat.

[JohnnyMalaria]

Dont you see how this causes some folks to only buy microsoft/microsoft sponsored software?

By default, DEP is only active for Windows apps and services. i.e., third-party apps that "break the rules" won't trigger a DEP exception. Since my development machine uses a default install of Vista Ultimate, DEP issues won't occur. However, because one of our users had set the DEP rules to the second option (see original post), it did cause a DEP exception.

Coincidentally, the day I found the root cause of the problem, I got an email from the vendor of the wrapper software (causing the DEP trigger) stating that they have a new version that is DEP-compatible.

[TwistedLincoln]

Dont you see how this causes some folks to only buy microsoft/microsoft sponsored software?

By default, DEP is only active for Windows apps and services. i.e., third-party apps that "break the rules" won't trigger a DEP exception.

I think that's the point. If at any time Microsoft decides to change the "rules" regarding which apps trigger a DEP exception, it could easily make it very difficult to install non-Microsoft programs.

Imagine this -- your non-technical friend buys an app. and installs it on Vista. It generates a DEP exception, and your friend, not knowing anything about the technical issue itself, thinks the software is malicious, or poorly designed, or whatever, and ends up using a Microsoft product instead.

Even more insidious is the possibility that Microsoft could implement DEP to refuse any non-approved software from running on the system. They'd do such a thing under the guise of "enhanced security," but in reality it would have the side effect of requiring all software to be approved by Microsoft before it would work...

[JohnnyMalaria]

DEP isn't a Microsoft thing. It's part of the architecture of the CPU (NX or XD bit).

Typically, you can disable it in the BIOS.

Support for DEP exists in XP SP2, too.

See: http://support.microsoft.com/default.aspx/kb/912923

Moreover, the following OSes also support the technology:

FreeBSD
OS X
Linux
NetBSD
OpenBSD

http://en.wikipedia.org/wiki/NX_bit