Gaps in all performance graphs?

Discussion:

Paul Warren

2011-12-16 01:52:06 UTC

Paul Warren [http://community.zenoss.org/people/pawarren] created the discussion

"Gaps in all performance graphs?"

To view the discussion, visit: http://community.zenoss.org/message/63320#63320

--------------------------------------------------------------
Hi folks,

We've been using zenoss for a while now, monitoring various websites and server installs, and recently, we've found a problem with the graphs.Â With alarming regularity, there are gaps of 25 minutes, every 2.5 hours appearing in all graphs, across all devices, looks like this:

Loading Image...

Loading Image...

In the RRD files, the corresponding times have values of NaN.

I'm running Zenoss V3.1.0 as a VM appliance (that I converted to run on VirtualBox). The VM is able to ping the internet during these graph outages, and the zenoss logs don't appear to be producing anything interesting, which ones should I be looking at?

We have around 15 devices we're monitoring. and the localhost load is 0.2 or so, with little CPU usage generally.

I'm a little bit lost as to how this is happening, anyone come across it before, or have suggestions for me to try?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/63320#63320]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

mwcotton

2011-12-16 01:56:20 UTC

Permalink

mwcotton [http://community.zenoss.org/people/mwcotton] created the discussion

"Re: Gaps in all performance graphs?"

To view the discussion, visit: http://community.zenoss.org/message/63298#63298

--------------------------------------------------------------
Usually this is because of the zenperfsnmp cycle time being exceeded.
rrds require new data every 5 minutes, if they dont get it they consider the data invalid.
Causes of zenperfsnmp cycle time being exceeded:
1. Many devices not responding to snmp.
2. The zenoss server doesnt have enough memory and maybe some swapping is happening.
3. The zenoss server is experiancing very high I/O.

( check for zenperfsnmp heatbeat errors on the event list , this usually indicates slow cycle times )
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/63298#63298]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

Paul Warren

2011-12-16 04:29:24 UTC

Permalink

Paul Warren [http://community.zenoss.org/people/pawarren] created the discussion

"Re: Gaps in all performance graphs?"

To view the discussion, visit: http://community.zenoss.org/message/63321#63321

--------------------------------------------------------------
Hrm, there is quite a bit of swap usage, looks like the VM was given only 512MB of RAM, will that be causing problems?

I'm not seeing any zenperfsnmp heartbeat errors coming up in the event list, the devices all seem to respond appropriately when I do a 'zencommand runÂ -d10Â etc...' And if that were the case, would we seem it happen to every single graph across all devices at the same time?

Most of our monitoring is via a plugin script we wrote, that returns a nagios string of metrics we care about. I'm told that it's done this way because SNMP tends not to work well across many routers and firewalls, but I'm not totally familiar with how it all hangs together yet.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/63321#63321]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

mwcotton

2011-12-16 14:20:10 UTC

Permalink

mwcotton [http://community.zenoss.org/people/mwcotton] created the discussion

"Re: Gaps in all performance graphs?"

To view the discussion, visit: http://community.zenoss.org/message/63304#63304

--------------------------------------------------------------
The swap usage will slow everything down, you should work to make that go away as your first step.

If you are collecting your snmp values via a command script, then I belive any error messages would be in the zencommand log. I know the zenperfsnmp daemon is specifically designed to collect tons of snmp values and I bet the zencommand daemon is not. Probably the reason you are seeing the gaps is your custom script is not returning the values fast enough.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/63304#63304]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

Paul Warren

2011-12-19 01:30:31 UTC

Permalink