zenoss 4.2.0 - zenwinperf keeps dieing

[mshundal]

mshundal [http://community.zenoss.org/people/mshundal] created the discussion

"zenoss 4.2.0 - zenwinperf keeps dieing"

To view the discussion, visit: http://community.zenoss.org/message/68532#68532

--------------------------------------------------------------
It just dies randomly and I can't keep it running or more than a few hours. Here a tail log...


2012-09-07 18:07:08,544 WARNING zen.winperf.PerfRpc: Bad counter for device 192.168.4.36: \LogicalDisk(\\?\Volume{6cc36847-a83d-11e1-b0f1-806e6f6e6963})\Disk Write Bytes/sec
2012-09-07 18:07:08,545 WARNING zen.winperf.PerfRpc: Bad counter for device 192.168.4.36: \LogicalDisk(\\?\Volume{6cc36847-a83d-11e1-b0f1-806e6f6e6963})\% Disk Write Time
2012-09-07 18:07:08,545 WARNING zen.winperf.PerfRpc: Bad counter for device 192.168.4.36: \LogicalDisk(\\?\Volume{6cc36847-a83d-11e1-b0f1-806e6f6e6963})\Free Megabytes
2012-09-07 18:07:23,540 WARNING zen.winperf.PerfRpc: Bad counter for device 192.168.4.31: \LogicalDisk(\\?\Volume{78347767-1160-11e1-ba7a-806e6f6e6963})\Disk Read Bytes/sec
2012-09-07 18:07:23,541 WARNING zen.winperf.PerfRpc: Bad counter for device 192.168.4.31: \LogicalDisk(\\?\Volume{78347767-1160-11e1-ba7a-806e6f6e6963})\% Disk Read Time
2012-09-07 18:07:23,541 WARNING zen.winperf.PerfRpc: Bad counter for device 192.168.4.31: \LogicalDisk(\\?\Volume{78347767-1160-11e1-ba7a-806e6f6e6963})\Disk Write Bytes/sec
2012-09-07 18:07:23,542 WARNING zen.winperf.PerfRpc: Bad counter for device 192.168.4.31: \LogicalDisk(\\?\Volume{78347767-1160-11e1-ba7a-806e6f6e6963})\% Disk Write Time
2012-09-07 18:07:23,542 WARNING zen.winperf.PerfRpc: Bad counter for device 192.168.4.31: \LogicalDisk(\\?\Volume{78347767-1160-11e1-ba7a-806e6f6e6963})\Free Megabytes
2012-09-07 18:07:49,485 ERROR zen.CollectorCmdBase: Unable to scan device 192.168.1.30: NT_STATUS_NET_WRITE_FAULT
2012-09-07 18:08:40,497 ERROR zen.CollectorCmdBase: Unable to scan device MOONDB6: NT_STATUS_IO_TIMEOUT



Even tried using a cron script below to revive it but that doesnt seem to work (i used something similar on debian and it worked like a charm however i've had to use this disrto just to be able to use zenoss)


[***@zenoss mhundal]$ uname -a
Linux zenoss.puretracks.com 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
[***@zenoss mhundal]$ zenoss status
Daemon: zeneventserver program running; pid=26132
Daemon: zopectl program running; pid=26248
Daemon: zenhub program running; pid=26302
Daemon: zenjobs program running; pid=26381
Daemon: zeneventd program running; pid=26449
Daemon: zenping program running; pid=26489
Daemon: zensyslog program running; pid=26592
Daemon: zenstatus program running; pid=26567
Daemon: zenactiond program running; pid=26607
Daemon: zentrap program running; pid=26691
Daemon: zenmodeler program running; pid=26692
Daemon: zenperfsnmp program running; pid=26724
Daemon: zencommand program running; pid=26755
Daemon: zenprocess program running; pid=26790
Daemon: zenrrdcached program running; pid=26795
Daemon: zenjmx program running; pid=26839
Daemon: zenwin program running; pid=26915
Daemon: zeneventlog program running; pid=26951
Daemon: zenwinperf not running
[***@zenoss mhundal]$ cat ~/scripts/chk-zenwinperf
#!/bin/bash
#
# This script, invoked by crontab, checks for status of ZENWINPERF daemon and restarts if needed
#                                                                                 - M S Hundal (Sept. 4, 2012)
#


if [ -z "$ZWPTST1" ]
then
   ZWPTST1="0"
fi


ZWPTST1=`ps ax | grep zenwinperf | grep -v grep | wc -l`
ZWPTST2=$(($ZWPTST1 - 2));


if [ $ZWPTST2 == 1 ]
then
    echo "ZWPTST2 script ran at `date` and all is well" >> /opt/zenoss/log/script.log
else
    /opt/zenoss/bin/zenwinperf start
    echo "Zenwinperf daemon testing script had to intervene at `date`" >> /opt/zenoss//log/script.log
fi
[***@zenoss mhundal]$ crontab -l
*/10 * * * *          /home/zenoss/scripts/chk-zenwinperf
[***@zenoss mhundal]$



I know the script runs cuz it appends to the log file in either case however zenwinperf isn't started for some reason. If i run the script manually, it works.

If anyone has seen this before, I'd really apericiate a hint to resolve it.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68532#68532]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Rob Eagle]

Rob Eagle [http://community.zenoss.org/people/reagle] created the discussion

"Re: zenoss 4.2.0 - zenwinperf keeps dieing"

To view the discussion, visit: http://community.zenoss.org/message/68533#68533

--------------------------------------------------------------
Do you know how many devices you are collecting for?  (ie total vs wmi only)  Are you also getting sporadic collection of statistics for devices even when daemon is running and seeing a lot of the 'unable to scan devices'.  You might be collecting too much data for the daemon to handle?  I had a similar issue on enterprise and lowering the maxperallel from default of 500 to something like a 100 in the zenwinperf.conf seemed to help.
--Rob
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68533#68533]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[mshundal]

mshundal [http://community.zenoss.org/people/mshundal] created the discussion

"Re: zenoss 4.2.0 - zenwinperf keeps dieing"

To view the discussion, visit: http://community.zenoss.org/message/68534#68534

--------------------------------------------------------------
Thanks, its worth a shot. I have 96 devices out of which approx. 80 are Windows servers - didn't think that would be too much for Zenoss.

I've lowered it to 300 and see if that makes a diff - if not I will lower it to 100 and test again.

I will post back if it works and what value it liked.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68534#68534]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[dhopp]

dhopp [http://community.zenoss.org/people/dhopp] created the discussion

"Re: zenoss 4.2.0 - zenwinperf keeps dieing"

To view the discussion, visit: http://community.zenoss.org/message/68524#68524

--------------------------------------------------------------
I'm not sure why the daemon is dieing.  But as far as the script goes I had similar trouble getting things to run in cron for zenoss and I resolved them by adding:

. ~zenoss/.bashrc

To the top of the script (just below the #!/bin/bash line)

--Dennis
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68524#68524]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[OneLoveAmaru]

OneLoveAmaru [http://community.zenoss.org/people/OneLoveAmaru] created the discussion

"Re: zenoss 4.2.0 - zenwinperf keeps dieing"

To view the discussion, visit: http://community.zenoss.org/message/68627#68627

--------------------------------------------------------------
I was having the same exact problem. To band aid the issue, you can't run scripts as zenoss as there are LOTS of environment variables. I was running in cron, as root /sbin/service zenoss start as it'll skip any services that are already running. I knew to do this because I found it in another post about a guy having problems with zenhub.

Anyways, I also changed my zenwinperf.conf to 100, so we'll see if that fixes it. I disabled my cron job so we'll kow soon.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68627#68627]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[mshundal]

mshundal [http://community.zenoss.org/people/mshundal] created the discussion

"Re: zenoss 4.2.0 - zenwinperf keeps dieing"

To view the discussion, visit: http://community.zenoss.org/message/68618#68618

--------------------------------------------------------------
After trying a bunch of things, the following works on CentOS.

[code]
[***@zenoss ~]$ crontab -l
*/2 * * * *          /home/zenoss/scripts/chk-zenwinperf
[***@zenoss ~]$ cat scripts/chk-zenwinperf
#!/bin/bash
#. ~zenoss/.bashrc
# This script, invoked by crontab, checks for status of ZENWINPERF daemon and restarts if needed
#                                                                                 - M S Hundal (Sept. 4, 2012)
#


if [ -f /etc/bashrc ]; then
        . /etc/bashrc
fi


# User specific aliases and functions
export ZENHOME="/opt/zenoss"
export INSTANCE_HOME="/opt/zenoss"
export PATH="${ZENHOME}/bin:${PATH}"
export PYTHONPATH="/opt/zenoss/lib/python"


if [ "${USE_ZENDS}" = "1" ];then
    export LD_LIBRARY_PATH="${ZENDSHOME}/lib:${ZENHOME}/lib"
    export PATH="${ZENDSHOME}/bin:${PATH}"
else
    export LD_LIBRARY_PATH="${ZENHOME}/lib"
fi


export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION="cpp"


if [ -z "$ZWPTST1" ]
then
   ZWPTST1="0"
fi


ZWPTST1=`ps ax | grep zenwinperf | grep -v grep | wc -l`
ZWPTST2=$(($ZWPTST1 - 2));


if [ $ZWPTST2 == 1 ]
then
    echo "ZWPTST2 script ran at `date` and all is well" >> /opt/zenoss/log/script.log
else
    /opt/zenoss/bin/zenwinperf start
    echo "Zenwinperf daemon testing script had to intervene at `date`" >> /opt/zenoss//log/script.log
fi
[***@zenoss ~]$
[/code]
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68618#68618]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[rebelinux]

rebelinux [http://community.zenoss.org/people/rebelinux] created the discussion

"Re: zenoss 4.2.0 - zenwinperf keeps dieing"

To view the discussion, visit: http://community.zenoss.org/message/68636#68636

--------------------------------------------------------------
In this case i use watchdog check zenwinperf daemon configuration from gui it resolve the problem for my enviroment.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68636#68636]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[mshundal]

mshundal [http://community.zenoss.org/people/mshundal] created the discussion

"Re: zenoss 4.2.0 - zenwinperf keeps dieing"

To view the discussion, visit: http://community.zenoss.org/message/68637#68637

--------------------------------------------------------------
if i wanted gui i would have used window server 2008.

the entire point of this was those of us forced to use this pathetic distro of an excuse for *nix, ie centos be able to restart the daemon...

harsh? yes i hope so.. rhel/centos is the dane cook of linux - forcing the entire community to use this crap just so we can keep up with zenoss if just as harsh - so, good luck with your gui and restart issues if you dont know how to use vi/nano
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68637#68637]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[rebelinux]

rebelinux [http://community.zenoss.org/people/rebelinux] created the discussion

"Re: zenoss 4.2.0 - zenwinperf keeps dieing"

To view the discussion, visit: http://community.zenoss.org/message/68652#68652

--------------------------------------------------------------
mshundal don’t be a troll,

zenoss has a built-in feature that can restart died daemons it call watchdog. this can be activated from the daemons config file:


[***@utg-zenoss-01v]$ cat /opt/zenoss/etc/zenwinperf.conf | grep watchdog
*watchdog True*
[***@utg-zenoss-01v]$

If you correctly configure it you will have the following entries in your zenwinperf logs:

[***@utg-zenoss-01v]$ cat /opt/zenoss/log/zenwinperf.log | grep watchdog
*2012-09-18 01:03:58,940 ERROR watchdog: Child died: Killed with signal 6*
*2012-09-18 01:04:23,225 ERROR watchdog: Restarting zenwinperf*
[***@utg-zenoss-01v]$ zenwinperf restart
stopping...
starting...
[***@utg-zenoss-01v]$

The watchdog utility is faster than a cron job.

http://jira.zenoss.com/jira/browse/ZEN-3197 http://jira.zenoss.com/jira/browse/ZEN-3197
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68652#68652]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[mshundal]

mshundal [http://community.zenoss.org/people/mshundal] created the discussion

"Re: zenoss 4.2.0 - zenwinperf keeps dieing"

To view the discussion, visit: http://community.zenoss.org/message/68640#68640

--------------------------------------------------------------
good solution

as per centos issue - nop it will be a while before this forced switch stops being annoying
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68640#68640]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Lily Regland]

Lily Regland [http://community.zenoss.org/people/authenticlv] created the discussion

"Re: zenoss 4.2.0 - zenwinperf keeps dieing"

To view the discussion, visit: http://community.zenoss.org/message/68629#68629

--------------------------------------------------------------
All of us know http://www.authenticlouisvuitton-mall.com/ Authentic Louis Vuiton Handbags are very expensive. But most women are men are mad about http://www.authenticlouisvuitton-mall.com/ Louis Vuiton Handbags because they can bring them the great feeling of high-ranking. Some people spend more than 1000USD to purchase http://www.authenticlouisvuitton-store.com/louis-vuitton-monogram-canvas-handbags-c-19_25.html Louis Vuitton Monogram Canvas Handbags. I think this is a waste of money. And many people are not rich enough to offer such expensive http://www.authenticlouisvuitton-mall.com/ Louis Vuiton. 
Here I would like to introduce ourselves about cheap authentic Louis Vuitton. We are authorized Louis  Vuitton outlet seller. Our site is http://www.authenticlouisvuitton-mall.com/ www.authenticlouisvuitton-mall.com. All Louis Vuitton items are available on this site at very reasonable price, which makes all of us can offer them, such as the hot sale items http://www.authenticlouisvuitton-mall.com/louis-vuitton-artsy-c-19_64.html Louis Vuitton Artsy, Louis Vuitton Neverfull, http://www.authenticlouisvuitton-mall.com/louis-vuitton-speedy-c-19_71.html Louis Vuitton Speedy and http://www.authenticlouisvuitton-mall.com/louis-vuitton-alma-c-19_60.html Louis Vuitton Alma. 
Our company is now very popular among people in America and Europe. Many people go to our site to purchase  Christmas gifts now. We sell thousands of http://www.authenticlouisvuitton-mall.com/louis-vuitton-monogram-canvas-handbags-c-19_25.html Louis Vuitton Monogram Canvas Handbags, Louis Vuitton  Epi Leather Handbags, http://www.authenticlouisvuitton-mall.com/louis-vuitton-damier-ebene-canvas-handbags-c-19_38.html Louis Vuitton Damier Ebene Canvas Handbags and so on. 
Ladies and gentlemen, take your time to choose your favorite Christmas gifts from our company. We promise 100% satisfaction!
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68629#68629]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[C James]

This post might be inappropriate. Click to display it.