This morning I had alerts that all clients failed to update. When I checked the ERA’s Tools > Server Options > Updates tab, I see 4522 is 100% complete. Clicking on Update Now only changes the time; the version is still 4522.

The clients, however are all at 4520 and failing. I opened the client that is installed on the server and tried the manual update twice, once with diagnostic logging. During the first manual update attempt, I saw it download about 20MB of update files. The second manual update, with diagnostic logging enabled, did not download files and failed very quickly. Now in the client logs, I see:

5:34am:  Update, Event:  [no message] (Most automatic updates failed a without a message.)

9:05am:  Update, Event:  Updater: compile error 0.

9:05am:  Update, Event:  Updater: retval = 0×1105, failures: 7

Solution

A Wilders Security Forum thread gave me the solution:

1. On the server, delete the contents of this folder:
   C:\Documents and Settings\All Users\Application Data\ESET\ESET Remote Administrator\Server\mirror
2. Start ESET Remote Administrator. Go to Tools > Server Options > Updates tab and click on Update Now. This will re-load the update from the ESET server.
3. Start the ESET NOD32 client and update again. This time the update completes successfully.
4. Use the ERA to force an update on the remaining clients.

Apparently one of the files in the original download was corrupted. Re-downloading solved the problem and allowed the clients to update successfully.

Zenith Infotech monitoring sometimes downloads files that trip NOD32’s HTTP filter (e.g. SpyBot and BitDefender executables). Zenith recommends excluding “update.itsupport247.net” from antivirus scanning. It’s not hard to do in NOD32, but it is hard to find someone who knows how to do it!

Here’s the basic procedure:

1. Open the NOD32 client interface and press F5 to load the Advanced dialog. Navigate to Antivirus and antispyware > Web access protection > HTTP > Excluded addresses.
2. Add “update.itsupport247.net/*” to the list.

Note:  do not put “http://” in front of the address, but do put “/* ” after it! If you forget the “/*”, the exclusion won’t work:  NOD32 will block Zenith from downloading files.

Deploy Using ESET Remote Administrator Console

Want to deploy that from a configuration file? Here is where things really get strange.

First, even though you don’t own the firewall product, to set up an HTTP exclusion, open the ESET Configuration Editor and navigate to ESET Smart Security, ESET NOD32 Antivirus > Personal firewall > Setup > List of URL addresses excluded from filtering. Click on Edit to add “update.itsupport247.net/*”.

The problem is, when you deploy that, the exclusion will not show up in the client’s user interface if that client has never had an HTTP exclusion. However, if you first set up any HTTP exclusion on the client, then deploy the correct exclusion from the ESET Remote Administrator Console, the correct exclusion will appear on the client.

I have not been able to figure out why the exclusions don’t appear until one is first added to the client. But hopefully, with that annoying manual effort, the exclusion will at least work and allow the clients to download the Zenith updates as necessary.

I’m running running ESET Remote Administrator 2.0.56. After upgrading from the trial to the purchased version, I started receiving the event below in my Application Event
Log.

Source:  ERA
Event ID:  502
Type:  Error
Description: The description for Event ID ( 502 ) in Source ( ERA ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: Error: Update failed, code: 4, additional code: 8193.

Solution 

Here’s the solution provided by ESET support:

1. Open the Remote Administrator Console (RAC).
2. Go to Menu >> Tools >> Server options…
   >> Updates tab.
3. Make sure you have the correct user name and
   password.
4. Click on Update Now button.

Sure enough, although I had updated the username and password in the 2.7 mirror setup, I had forgotten to update them in the RAC as well (where the 3.0 mirror is configured). Once I made that change, the errors stopped appearing in my event log.

KeyFinder Problems

I've been building a UBCD4WIN version 3.12 ISO file as my test case. UBCD4WIN includes a
large number of plug-ins. One of them is keyfinder.exe, the Magical Jelly Bean
Keyfinder version 1.51. This handy program can display and update Windows and
Office installation keys.

I installed the Trend 3.5 agent on my workstation and tried the UBCD4WIN
build. The build failed because the keyfinder.exe file was missing. By
uninstalling and re-installing UBCD4WIN, and temporarily disabling the Trend
agent, I confirmed that Trend is deleting this file without logging it as a
virus or spyware and without sending it to quarantine. Trend is supposed to
encrypt and save suspicious files on the client in C:\Program Files\Trend
Micro\Client Server Security Agent\Suspect, but that folder is empty. I finally
got Trend to leave the file alone by adding keyfinder.exe to Trend's exclusion
list.
 
Is this a bug? I guess I'd better try the latest version, Trend 3.6 with Patch 1.
I downloaded the 336MB installation file, upgraded the server, and let it push
out the 3.6.1095 client. No reboot was requested.
 
After removing the file exclusion from the Trend configuration, I opened Windows
Explorer and highlighted keyfinder.exe (but did not execute it). The Trend icon
in the system tray indicated activity, then I got a message from Windows
indicating that my system may be vulnerable because Trend was not running. The
Trend system tray icon disappeared when I put the mouse over it. So scanning
keyfinder.exe caused the Trend Real-Time scanner to crash.

I rebooted the client and did the same test, highlighting the file in Windows Explorer. This time keyfinder.exe was
not deleted and the Trend real-time agent did not crash. However, the
UBCD4WIN build process, which actually copies keyfinder.exe, failed again
because access was denied on that file. When I went back to look at
keyfinder.exe in Windows Explorer, it was deleted before my eyes. The Trend
Client/Server Security Agent real-time scan window still tells me that there are
0 infected files; “Last virus/malware found” is blank. So Trend is again
deleting it without any warning or logging. I had to add the file back to the
exclusion list so I could complete the UBCD4WIN build.

The Numbers

Once I got the UBCD4WIN build to complete, I tested it with various levels of
extension exclusions as I had NOD32. The results, along with the NOD32 2.7
results from the previous post:

Some other numbers that are interesting from a system administration point of view are
installation size and memory overhead. The table below summarizes these numbers
for both server and workstation installations.

The server install of Trend Client-Server includes its web-based management
console. The server install of NOD32 includes the
ESET Remote Administrator Server and Console 2.0.56.

Conclusions

ESET NOD32 has a reputation for being lean and fast. Compared to Trend Client-Server 3.6, NOD32
definitely looks “lean” on disk space and memory footprint. However, Trend
allowed the UBCD4WIN build to proceed a little faster than NOD32.

My greatest concerns with Trend come from areas other than performance. One
was the experience back in February 2007 of Trend passing on a “possible worm”
to the client desktop, which would have allowed users to run the worm. I was amazed that there
was no way to configure Trend to not pass possible malware to end users. The
other experience is the one described above:  deleting a file without
warning and without logging. I wonder if I have lost other files that way and
will never know.

Clearly there is no perfect anti-virus solution. Both NOD32 and Trend CS(M)
have their own configuration hassles and “gotchas.” I do
appreciate the reduced memory footprint of NOD32, and the fact that my SBS
server no longer sends me a daily alert that it is running out of allocated
memory. We'll see how well it performs in the long term.

As I mentioned in the updates to my previous blog post, after encountering some issues with NOD32 version 3.0, and on the advice of experienced NOD32 system admins, I downgraded to version 2.7. 

What I didn’t realize is that by doing so, I would be giving up the ability to exclude files and folders from my scheduled and on-demand virus scans. Why is that a problem? Some years ago, I wrote and sold a set of Microsoft Word macros. NOD32 calls these a “possible unknown macro virus.” Fair enough, but I know what they are so I want to exclude them from scanning. In NOD32 version 3.0, I can exclude them from both real-time and on-demand scans. In NOD32 version 2.7, I can only exclude them from the real-time scanner. With 2.7, if I want the virus alerts to stop, I have to submit the macros to ESET for evaluation. I’m not too keen on sending my intellectual property to a software vendor for review.

Performance Comparison

The main reason given by other system admins for using NOD32 version 2.7 is that it performs much better than version 3.0. After seeing version 2.7 using 50% of the CPU during an in-depth server scan, I started wondering about this. I wondered more when I noticed that my desktop occasionally seemed to “stall out” for a few seconds running 2.7–an issue that I had not had with 3.0.

The final impetus for doing a real comparison was the sluggishness of the desktop when building a UBCD4WIN ISO file. I decided this might be a good test, since it involves copying a large number of files from both a CD and from disk.

About Extensions

One thing that 2.7 and 3.0 have in common is that by default, they both scan all files, regardless of extension. This turns out to make a big difference in performance. For example, during the UBCD4WIN build process, there is a step that appends data to a 450KB file called txtsetup.sif. From observing the process in FileMon, I see that this process accesses the file thousands of time, apparently working with file offsets to copy one line at a time. When NOD32 (either version) is set to its default to scan all extensions, it takes six minutes to append to this file. However, if I uncheck the “all extensions” box in the NOD32 setup dialog and allow it to scan only its pre-defined list of extensions, UBCD4WIN blows by this append step in about five seconds.

The Numbers

My comparison between the versions was not terribly rigorous. I just ran the UBCD4WIN build process multiple times, noting the start and stop times. Since I didn’t note seconds, there’s at least a ±1 minute margin of error. I ran the tests on an old Dell Dimension 2400 (Pentium 4 2.66GHz) with 1GB of RAM. I compared NOD32 2.70.39 to 3.0.642, both with current virus signatures.

The times to build the UBCD4WIN ISO file were as follows:

No antivirus software installed:  9 minutes. 

NOD32 version 2.7, real-time scanning disabled:  9 minutes.
NOD32 version 2.7, scanning only specific extensions:  14 minutes.
NOD32 version 2.7, scanning all extensions:  20 minutes.

NOD32 version 3.0, real-time scanning disabled:  10 minutes.
NOD32 version 3.0, scanning only specific extensions:  14 minutes.
NOD32 version 3.0, scanning all extensions:  24 minutes.

With both versions, watching Performance Monitor, I saw the antivirus process frequently exceeding 90% CPU usage, especially during the six-minute scan of txtsetup.sif.

Conclusions

Perhaps the most obvious conclusion is that, regardless of NOD32 version, it’s hard to justify accepting the default to scan all extensions, since it adds 40% or more to file access times. When set to only scan extensions that ESET identifies as vulnerable to viruses, there is no significant performance difference between the versions.

If performance is about the same, what other factors might influence one’s choice between 2.7 and 3.0?

• Both versions have quirks that make it difficult or impossible to deploy some configuration settings using ESET’s .xml configuration files.
• Version 3.0 has the important ability to exclude files and folders from on-demand and scheduled scans. It may be possible to work around this in 2.7 using Task Scheduler and NOD32’s command-line scanner.
• In Version 3.0, the correlation between the client and the configuration editor (used for mass deployments) is less clear than in 2.7.

As the newer product, version 3.0 may still have issues that make 2.7 the safer bet for the short term. However, it’s encouraging that version 3.0’s real-time performance is on par with 2.7, at least with the specific-extensions list. I’d be interested to hear if others get similar results.

The time had come to renew my antivirus program. I decided instead to try one that I’ve been reading good things about on some Yahoo forums:  ESET’s NOD32. The main attraction is the promise of a fast, lightweight, and accurate scan engine.

As warned, the setup of the server components is not exactly intuitive. It’s made more complex by the fact that ESET is currently transitioning from version 2.7 to 3.0, a change that includes some fundamental reconfiguration of how modules are named. Not all documentation has been updated yet, and so far there is no Exchange component for 3.0.

I will say, though, that once I got the hang of creating XML configuration files using the ESET Configuration Editor, creating install packages, then pushing those files to server and client computers, it’s been a pretty straightforward process.

Update 3/1/2008:  a couple of weeks into this experiment with NOD32,  my reservations are increasing, at least regarding version 3.0. I’ve had two complete server lockups so far, and I’m not alone. ESET Support has suggested some changes to the server configuration, but it has been maddening trying to figure out how to implement those changes in the ESET Configuration Editor. There is a complete lack of continuity between the client UI and the ESET Configuration Editor, and it seems that some things simply do not appear in the Editor at all (like disabling email and web protection, as support recommended for the server). I have had some success by directly editing the XML files, but that is not fun. Many admins are reverting to version 2.7. Since I’m just starting with NOD32, I’d rather not have to learn an old version, but I may be forced to if version 3.0 doesn’t stabilize.

Update 3/6/2008:  the 3.0 configuration changes seem to have helped–no crashes for six days. However, after checking with other NOD32 users, I decided to downgrade to 2.7 for now. I was able to continue with the 3.0 Remote Administrator and downgrade only the client software. All the exclusions etc. have to be set up again, but in the case of 2.7, there is a better correspondence between the Configuration Editor and the client interface, so this seemed to go a little more smoothly. I did get a new error on a previously unexcluded file (C:\WINNT\security\tmp.edb), so I extended the list below to include a few files from that folder.

File Exclusions

The main thing I wanted to focus on here is the file exclusions. NOD32 scans all files by default. While I wasn’t experiencing problems, that can theoretically cause lots of grief especially on Small Business Server, which runs a domain controller, DNS server, Exchange server, and SQL/MSDE servers.

It took a call to ESET support to clarify that if I add file and folder exclusions to ESET Kernel > Setup, that will cover all accesses by the program; I don’t need to additionally add specific exclusions of various extensions to all the individual program modules. 

With the help of a this thread in the official ESET forum, I was able to come up with a list of files to exclude on my SBS 2003 server and its clients. I also found a workaround for a bug that was preventing the exclusion lists from pushing to the server and clients. I won’t duplicate all the explanations of what each file and folder is, but the procedure is as follows:

1. Create a text file listing the files to exclude. Paste the following into Notepad and save as a text file , e.g. “SBS exclusions.txt”:

D:\Mail Server\Exchsrvr\MDBDATA\*.*
C:\Program Files\Exchsrvr\Mtadata\*.*
C:\Program Files\Exchsrvr\MCB03.log\*.*
C:\Program Files\Exchsrvr\Mailroot\*.*
C:\Program Files\Exchsrvr\Mdbdata\*.*
C:\Program Files\Exchsrvr\Conndata\*.*
C:\Program Files\Exchsrvr\srsdata\*.*
C:\WINNT\system32\inetsrv\*.*
C:\WINNT\IIS Temporary Compressed Files\*.*
C:\WINNT\NTDS\*.*
C:\WINNT\sysvol\*.*
C:\WINNT\ntfrs\*.*
C:\WINNT\security\edb*.log
C:\WINNT\security\tmp.edb
C:\WINNT\Security\Database\secedit.sdb
C:\WINNT\system32\CertLog\*.* - added 3/13/2008 (Certificate Authority files)
C:\WINNT\system32\dhcp\*.*
C:\WINNT\system32\wins\*.*
C:\Program Files\Microsoft SQL Server\MSSQL$BKUPEXEC\Data\*.*
C:\Program Files\Microsoft SQL Server\MSSQL$SBSMONITORING\Data\*.*
C:\Program Files\Microsoft SQL Server\MSSQL$SHAREPOINT\Data\*.*
F:\MSSQL2000\MSSQL\Data\*.*
C:\WINNT\System32\ntmsdata\*.*
C:\Program Files\Microsoft Windows Small Business Server\Networking\POP3\Failed Mail\*.*
C:\Program Files\Microsoft Windows Small Business Server\Networking\POP3\Incoming Mail\*.*
C:\WINNT\SoftwareDistribution\DataStore\*.*
C:\pagefile.sys
C:\WINNT\system32\licstr.cpa
C:\WINNT\system32\lls\*.* - corrected 3/10/2008
G:\*.*
H:\*.*

Note that your drives and paths may vary. The last two entries exclude drives that store disk-based backup files.

2. In the ESET Configuration Editor, open the following node:

ESET Smart Security, ESET NOD32 Antivirus > ESET Kernel > Setup > Exclusions > Exclusions

and click on the Edit button.

3. Click on the +List button and import the text file from step 1. Click on OK.

4. Save the configuration file as a separate file, e.g. “SBS exclusions.xml”.

5. Close the ESET Configuraiton Editor and open the “SBS exclusions.xml” file in an XML editor. (I used FrontPage 2003.) Replace all occurrences of

<NODE NAME=”Exclusion” DELETE=”0″>

with

<NODE NAME=”Exclusion” TYPE=”SUBNODE” DELETE=”0″>

This works around a bug and will allow the configuration file to update exclusions when it is pushed to the server. 

6. Use the ESET Remote Administrator Console to push the configuration file to the server in the usual manner. (In the Clients tab, right-click on the server name and select New Task > Configuration Task.)

7. Once the configuration task completes (see the Tasks tab), open the NOD32 client on the server, press F5 to open the Advanced Setup window, then check the Antivirus and antispyware > Exclusions node to make sure that the exclusions were imported correctly.

Exclusions for Client Computers 

I followed a similar procedure to add following exclusions to the configuration file that I push to client computers:

C:\WINDOWS\SoftwareDistribution\DataStore
C:\Program Files\Microsoft Windows Small Business Server\Clients\SBSClientApps.log

Note that the exclusion of *.mdf and *.ldf as suggested in the original version of this article do not work as expected:  exluding them at the root of C:\ does not exclude them in all subfolders. They can either be excluded using the extension exclusions in the various scanning components, or those extension settings can be reversed so that instead of scanning all extensions (the default), only a long list of specific file extensions are included. If you choose the latter route, be sure to remove the “*.MD?” extension that is included in the default list of extensions to be scanned.