Vista and DEP

[torrid]

On Vista, the installer installs fine but when a user with DEP "on" runs our application it throws and error and kills it.

Of course the USER blames US. Is there a way to disable DEP during install?

Or, do we have to add a screen to install telling them to disable it for our app?

If that's the case is there a way to detect if DEP is enabled or not?

ANy help and advice in this area is much appreciated...

[linder]

Unfortunately, the user is completely right. It's your fault. Sorry for the bad news

DEP detected a buffer overrun or something like that in your application. Disabling DEP does not help in any way to fix this serious problem. You have to debug your application and find out what causes the "buffer overrun".

BTW, it's not possible to disable DEP programmatically. DEP is an important security feature.

Does this help?

--
Friedrich Linder
Lindersoft
www.lindersoft.com
+1.954.252.3910

SetupBuilder "point. click. ship"
Create Windows Vista ready installations in minutes

-- Official Comodo Code Signing and SSL Certificate Partner

[torrid]

No does not really help... how do I detect a "buffer overrun"? how do we detect it/find it? any tips or tools?

They said it occurred on start-up. So it's not like they were using program and maybe adding hundreds of entries to our grid and then it had a memory issue.

We've had only one report of this... but obviously others could have it and not report it.

[linder]

Hello,

You have to find out where this happens in your application (e.g. at program start-up or if a specific function is executed) and then review that part of your source code. It's quite possible that not all of your customers see it -- and please note that not all processors support hardware based DEP.

In "most" cases, it's caused by a "buffer overrun". For example, you have a string defined as 260 bytes and your program writes to, say, byte 270. One of your 3-rd party producs can also cause this!

But if it is reported, then it is a very serious issue and has to be fixed in your application. You should not ask your customer to add your program to the exclusion list.

Friedrich

[torrid]

My problem is that I cannot get the VB6 IDE running in Vista... read all sorts of posts on it online but don't have the 2000 version of the original setup files for the VB6 IDE in order to install and debug on Vista.

[linder]

Hello,

If you can't debug your (Visual Basic 6?) application on Vista (and other UAC-aware systems) then the only possible solution to fix the bug in your program is to review the complete source codes and try to find the culprit.

But if there is really no other way to fix the bug, you can try the following. But it is NOT RECOMMENDED to do this! And no guarantee that it works!

You can try to add your application to the DEP exclusion list via a "secret" registry setting.

HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers\%_SB_INSTA LLDIR%\<YourAppHere>.exe, DisableNXShowUI

Set the FULL path of the program and a string value of the name of the compatibility mode being applied. A reboot is required.

"NX" stands for "No execute". You have to switch the installer into 64-bit mode before you write this registry key on an x64 operating system!

Again, the above is really NOT RECOMMENDED and SHOULD NOT be used at all!

Friedrich