Simon Jackson
2013-06-05 11:17:49 UTC
Simon Jackson [http://community.zenoss.org/people/simon_goodier] created the discussion
"Zenoss No-longer Recording Event Data"
To view the discussion, visit: http://community.zenoss.org/message/73472#73472
--------------------------------------------------------------
Hello,
I have had a rather successful installation of Zenoss 4.2.3 Core; collecting device information for all of our Production estate (switches, routers, access points, printers, VMware hosts + VMs, SANs and LTO tape loaders).. all impressive stuff when we bundled a few ZenPacks..
I have 101 devices currently monitored.. and all seemed well until:
I cleared ALL events out; and added two additional IPs in to the discovery pool. One of those two IPs is not a real IP (the IP was changed on the device by someone else.. so it remained un-contactable). Zenoss never recovered from these instructions (aka Jobs)... It stayed for 6+ days showing '2 jobs remaining'.
I cleared all jobs and stopped and started every service/deamon.. much to no avail.
I've even looked around through the log files - what I can see the ZenJob and ZenCtrl logs are quite chatty.. the zenoss tool is definately still communicating with all my network nodes. It's just not storing the events in the database for me to poll through.
I'm thinking it's something to do with RabbitMQ..
Here is some (hopefully) useful information:
Software Component Versions
| Zenoss (http://www.zenoss.com/) | Zenoss 4.2.3 |
| OS (http://www.tldp.org/) | Linux (x86_64) 2.6.18 (Linux srvmanzen01.oakland.local 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009 x86_64) |
| Zope (http://www.zope.org/) | Zope 2.13.13 |
| Python (http://www.python.org/) | Python 2.7.2 |
| Database (http://www.mysql.com/) | MySQL 5.5.13 (5.5.13) |
| RRD (http://oss.oetiker.ch/rrdtool) | RRDtool 1.4.7 |
| Twisted (http://twistedmatrix.com/trac) | Twisted 11.0.0 |
| RabbitMQ (http://www.rabbitmq.com/) | Unavailable |
| Erlang (http://www.erlang.org/) | Erlang 5.6.5 |
| NetSnmp (http://net-snmp.sourceforge.net/) | NetSnmp 5.3.2 |
| PyNetSnmp (http://www.zenoss.com/) | PyNetSnmp 0.30.6 |
| WMI (http://www.zenoss.com/) | Wmi 1.3.15 |
Loaded ZenPacks
||
|| Pack || Package || Author || Version || Egg ||
| | ZenPacks.TwoNMS.PrinterMIB (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.TwoNMS.PrinterMIB) | TwoNMS | 2NMS - Maarten Wallraf / Modified for Zenoss 4 by Jane Curry | 1.1 | Yes |
| | ZenPacks.zenoss.ActiveDirectory (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.ActiveDirectory) | zenoss | Zenoss | 2.1.0 | Yes |
| | ZenPacks.zenoss.ApacheMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.ApacheMonitor) | zenoss | Zenoss | 2.1.3 | Yes |
| | ZenPacks.zenoss.DellMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.DellMonitor) | zenoss | Zenoss | 2.2.0 | Yes |
| | ZenPacks.zenoss.DeviceSearch (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.DeviceSearch) | zenoss | Zenoss | 1.2.0 | Yes |
| | ZenPacks.zenoss.DigMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.DigMonitor) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.DnsMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.DnsMonitor) | zenoss | Zenoss | 2.1.0 | Yes |
| | ZenPacks.zenoss.EsxTop (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.EsxTop) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.FtpMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.FtpMonitor) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.HPMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.HPMonitor) | zenoss | Zenoss | 2.1.0 | Yes |
| | ZenPacks.zenoss.IISMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.IISMonitor) | zenoss | Zenoss | 2.0.2 | Yes |
| | ZenPacks.zenoss.IRCDMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.IRCDMonitor) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.JabberMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.JabberMonitor) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.LDAPMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.LDAPMonitor) | zenoss | Zenoss | 1.4.0 | Yes |
| | ZenPacks.zenoss.LinuxMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.LinuxMonitor) | zenoss | Zenoss | 1.2.0 | Yes |
| | ZenPacks.zenoss.MSExchange (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.MSExchange) | zenoss | Zenoss | 2.0.4 | Yes |
| | ZenPacks.zenoss.MSMQMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.MSMQMonitor) | zenoss | Zenoss | 1.2.1 | Yes |
| | ZenPacks.zenoss.MSSQLServer (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.MSSQLServer) | zenoss | Zenoss | 2.0.3 | Yes |
| | ZenPacks.zenoss.MySqlMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.MySqlMonitor) | zenoss | Zenoss | 2.2.0 | Yes |
| | ZenPacks.zenoss.NNTPMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.NNTPMonitor) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.NtpMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.NtpMonitor) | zenoss | Zenoss | 2.2.0 | Yes |
| | ZenPacks.zenoss.PySamba (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.PySamba) | zenoss | Zenoss | 1.0.0 | Yes |
| | ZenPacks.zenoss.WindowsMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.WindowsMonitor) | zenoss | Zenoss | 1.0.5 | Yes |
| | ZenPacks.zenoss.XenMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.XenMonitor) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.ZenAWS (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.ZenAWS) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.ZenJMX (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.ZenJMX) | zenoss | Zenoss | 3.9.3 | Yes |
| | ZenPacks.zenoss.ZenossVirtualHostMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.ZenossVirtualHostMonitor) | zenoss | Zenoss | 2.4.0 | Yes |
zeneventserver Log File
2013-06-05T07:52:01.168 [ZEP_HEARTBEAT_PROCESSOR] WARN org.zenoss.zep.impl.Application - Failed to process heartbeat events java.lang.NullPointerException: null at org.zenoss.amqp.AmqpConnectionManager.openChannel(AmqpConnectionManager.java:90) ~[amqp-api-1.2-SNAPSHOT.jar:na] at org.zenoss.amqp.AmqpConnectionManager.getPublisher(AmqpConnectionManager.java:162) ~[amqp-api-1.2-SNAPSHOT.jar:na] at org.zenoss.amqp.AmqpConnectionManager.publish(AmqpConnectionManager.java:193) ~[amqp-api-1.2-SNAPSHOT.jar:na] at org.zenoss.zep.impl.EventPublisherImpl.publishEvent(EventPublisherImpl.java:42) ~[zep-core-1.2-SNAPSHOT.jar:na] at org.zenoss.zep.impl.HeartbeatProcessorImpl.sendHeartbeatEvents(HeartbeatProcessorImpl.java:91) ~[zep-core-1.2-SNAPSHOT.jar:na] at org.zenoss.zep.impl.Application$5.run(Application.java:418) ~[zep-core-1.2-SNAPSHOT.jar:na] at org.zenoss.zep.impl.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:43) [zep-core-1.2-SNAPSHOT.jar:na] at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:51) [spring-context-3.0.5.RELEASE.jar:3.0.5.RELEASE] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [na:1.6.0_31] at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source) [na:1.6.0_31] at java.util.concurrent.FutureTask.runAndReset(Unknown Source) [na:1.6.0_31] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source) [na:1.6.0_31] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source) [na:1.6.0_31] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) [na:1.6.0_31] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) [na:1.6.0_31] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.6.0_31] at java.lang.Thread.run(Unknown Source) [na:1.6.0_31]
<<this repeats a LOT>>
zopectl Log File
Error: no log file exists
zenhub Log File
raise Exception("Could not publish message to RabbitMQ: %s" % lastexc) Exception: Could not publish message to RabbitMQ: [Errno 104] Connection reset by peer 2013-06-05 12:00:40,049 INFO zen.ZenHub: Worker (30003) reports 2013-06-05 12:00:40,049 CRITICAL zen.zenoss.protocols.amqp: Could not use exchange $RawZenEvents: [Errno 104] Connection reset by peer 2013-06-05 12:00:40,050 INFO zen.ZenHub: Worker (30003) reports 2013-06-05 12:00:40,050 CRITICAL zen.Events: Unable to publish event to <Products.ZenMessaging.queuemessaging.publisher.EventPublisher object at 0xc6c2e50>: [Errno 104] Connection reset by peer 2013-06-05 12:00:42,971 INFO zen.ZenHub: Worker (30199) reports 2013-06-05 12:00:42,970 CRITICAL zen.zenoss.protocols.amqp: Could not use exchange $RawZenEvents: [Errno 104] Connection reset by peer 2013-06-05 12:00:42,971 INFO zen.ZenHub: Worker (30199) reports 2013-06-05 12:00:42,970 CRITICAL zen.Events: Unable to publish event to <Products.ZenMessaging.queuemessaging.publisher.EventPublisher object at 0x1376ee50>: [Errno 104] Connection reset by peer 2013-06-05 12:00:43,055 INFO zen.ZenHub: Worker (30003) reports 2013-06-05 12:00:43,055 INFO zen.zenoss.protocols.amqp: RabbitMQ connection was closed: [Errno 104] Connection reset by peer 2013-06-05 12:00:45,975 INFO zen.ZenHub: Worker (30199) reports 2013-06-05 12:00:45,974 INFO zen.zenoss.protocols.amqp: RabbitMQ connection was closed: [Errno 104] Connection reset by peer 2013-06-05 12:00:46,059 INFO zen.ZenHub: Worker (30003) reports 2013-06-05 12:00:46,058 INFO zen.zenoss.protocols.amqp: RabbitMQ connection was closed: [Errno 104] Connection reset by peer 2013-06-05 12:00:46,060 INFO zen.ZenHub: Worker (30003) reports 2013-06-05 12:00:46,059 ERROR zen.Events: Unable to publish event to <Products.ZenMessaging.queuemessaging.publisher.EventPublisher object at 0xc6c2e50> Traceback (most recent call last): File "/opt/zenoss/Products/ZenEvents/MySqlSendEvent.py", line 43, in sendEvents self._publishEvent(event, publisher) File "/opt/zenoss/Products/ZenEvents/MySqlSendEvent.py", line 92, in _publishEvent publisher.publish(event) File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 288, in publish mandatory=mandatory, immediate=immediate) File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 319, in _publish mandatory=mandatory, immediate=immediate, createQueues=createQueues) File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 392, in publish headers=headers, declareExchange=declareExchange) File "/opt/zenoss/lib/python/zenoss/protocols/amqp.py", line 156, in publish raise Exception("Could not publish message to RabbitMQ: %s" % lastexc) Exception: Could not publish message to RabbitMQ: [Errno 104] Connection reset by peer 2013-06-05 12:00:48,978 INFO zen.ZenHub: Worker (30199) reports 2013-06-05 12:00:48,978 INFO zen.zenoss.protocols.amqp: RabbitMQ connection was closed: [Errno 104] Connection reset by peer 2013-06-05 12:00:48,979 INFO zen.ZenHub: Worker (30199) reports 2013-06-05 12:00:48,978 ERROR zen.Events: Unable to publish event to <Products.ZenMessaging.queuemessaging.publisher.EventPublisher object at 0x1376ee50> Traceback (most recent call last): File "/opt/zenoss/Products/ZenEvents/MySqlSendEvent.py", line 43, in sendEvents self._publishEvent(event, publisher) File "/opt/zenoss/Products/ZenEvents/MySqlSendEvent.py", line 92, in _publishEvent publisher.publish(event) File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 288, in publish mandatory=mandatory, immediate=immediate) File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 319, in _publish mandatory=mandatory, immediate=immediate, createQueues=createQueues) File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 392, in publish headers=headers, declareExchange=declareExchange) File "/opt/zenoss/lib/python/zenoss/protocols/amqp.py", line 156, in publish
I'm thinking the dependancy on the RabbitMQ service is the problem here...
Thoughts anyone?
--------------------------------------------------------------
Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/73472#73472]
Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]
"Zenoss No-longer Recording Event Data"
To view the discussion, visit: http://community.zenoss.org/message/73472#73472
--------------------------------------------------------------
Hello,
I have had a rather successful installation of Zenoss 4.2.3 Core; collecting device information for all of our Production estate (switches, routers, access points, printers, VMware hosts + VMs, SANs and LTO tape loaders).. all impressive stuff when we bundled a few ZenPacks..
I have 101 devices currently monitored.. and all seemed well until:
I cleared ALL events out; and added two additional IPs in to the discovery pool. One of those two IPs is not a real IP (the IP was changed on the device by someone else.. so it remained un-contactable). Zenoss never recovered from these instructions (aka Jobs)... It stayed for 6+ days showing '2 jobs remaining'.
I cleared all jobs and stopped and started every service/deamon.. much to no avail.
I've even looked around through the log files - what I can see the ZenJob and ZenCtrl logs are quite chatty.. the zenoss tool is definately still communicating with all my network nodes. It's just not storing the events in the database for me to poll through.
I'm thinking it's something to do with RabbitMQ..
Here is some (hopefully) useful information:
Software Component Versions
| Zenoss (http://www.zenoss.com/) | Zenoss 4.2.3 |
| OS (http://www.tldp.org/) | Linux (x86_64) 2.6.18 (Linux srvmanzen01.oakland.local 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009 x86_64) |
| Zope (http://www.zope.org/) | Zope 2.13.13 |
| Python (http://www.python.org/) | Python 2.7.2 |
| Database (http://www.mysql.com/) | MySQL 5.5.13 (5.5.13) |
| RRD (http://oss.oetiker.ch/rrdtool) | RRDtool 1.4.7 |
| Twisted (http://twistedmatrix.com/trac) | Twisted 11.0.0 |
| RabbitMQ (http://www.rabbitmq.com/) | Unavailable |
| Erlang (http://www.erlang.org/) | Erlang 5.6.5 |
| NetSnmp (http://net-snmp.sourceforge.net/) | NetSnmp 5.3.2 |
| PyNetSnmp (http://www.zenoss.com/) | PyNetSnmp 0.30.6 |
| WMI (http://www.zenoss.com/) | Wmi 1.3.15 |
Loaded ZenPacks
||
|| Pack || Package || Author || Version || Egg ||
| | ZenPacks.TwoNMS.PrinterMIB (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.TwoNMS.PrinterMIB) | TwoNMS | 2NMS - Maarten Wallraf / Modified for Zenoss 4 by Jane Curry | 1.1 | Yes |
| | ZenPacks.zenoss.ActiveDirectory (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.ActiveDirectory) | zenoss | Zenoss | 2.1.0 | Yes |
| | ZenPacks.zenoss.ApacheMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.ApacheMonitor) | zenoss | Zenoss | 2.1.3 | Yes |
| | ZenPacks.zenoss.DellMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.DellMonitor) | zenoss | Zenoss | 2.2.0 | Yes |
| | ZenPacks.zenoss.DeviceSearch (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.DeviceSearch) | zenoss | Zenoss | 1.2.0 | Yes |
| | ZenPacks.zenoss.DigMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.DigMonitor) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.DnsMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.DnsMonitor) | zenoss | Zenoss | 2.1.0 | Yes |
| | ZenPacks.zenoss.EsxTop (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.EsxTop) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.FtpMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.FtpMonitor) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.HPMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.HPMonitor) | zenoss | Zenoss | 2.1.0 | Yes |
| | ZenPacks.zenoss.IISMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.IISMonitor) | zenoss | Zenoss | 2.0.2 | Yes |
| | ZenPacks.zenoss.IRCDMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.IRCDMonitor) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.JabberMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.JabberMonitor) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.LDAPMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.LDAPMonitor) | zenoss | Zenoss | 1.4.0 | Yes |
| | ZenPacks.zenoss.LinuxMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.LinuxMonitor) | zenoss | Zenoss | 1.2.0 | Yes |
| | ZenPacks.zenoss.MSExchange (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.MSExchange) | zenoss | Zenoss | 2.0.4 | Yes |
| | ZenPacks.zenoss.MSMQMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.MSMQMonitor) | zenoss | Zenoss | 1.2.1 | Yes |
| | ZenPacks.zenoss.MSSQLServer (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.MSSQLServer) | zenoss | Zenoss | 2.0.3 | Yes |
| | ZenPacks.zenoss.MySqlMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.MySqlMonitor) | zenoss | Zenoss | 2.2.0 | Yes |
| | ZenPacks.zenoss.NNTPMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.NNTPMonitor) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.NtpMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.NtpMonitor) | zenoss | Zenoss | 2.2.0 | Yes |
| | ZenPacks.zenoss.PySamba (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.PySamba) | zenoss | Zenoss | 1.0.0 | Yes |
| | ZenPacks.zenoss.WindowsMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.WindowsMonitor) | zenoss | Zenoss | 1.0.5 | Yes |
| | ZenPacks.zenoss.XenMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.XenMonitor) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.ZenAWS (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.ZenAWS) | zenoss | Zenoss | 1.1.0 | Yes |
| | ZenPacks.zenoss.ZenJMX (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.ZenJMX) | zenoss | Zenoss | 3.9.3 | Yes |
| | ZenPacks.zenoss.ZenossVirtualHostMonitor (http://monitoring.cvsuk.com:8080/zport/dmd/ZenPackManager/packs/ZenPacks.zenoss.ZenossVirtualHostMonitor) | zenoss | Zenoss | 2.4.0 | Yes |
zeneventserver Log File
2013-06-05T07:52:01.168 [ZEP_HEARTBEAT_PROCESSOR] WARN org.zenoss.zep.impl.Application - Failed to process heartbeat events java.lang.NullPointerException: null at org.zenoss.amqp.AmqpConnectionManager.openChannel(AmqpConnectionManager.java:90) ~[amqp-api-1.2-SNAPSHOT.jar:na] at org.zenoss.amqp.AmqpConnectionManager.getPublisher(AmqpConnectionManager.java:162) ~[amqp-api-1.2-SNAPSHOT.jar:na] at org.zenoss.amqp.AmqpConnectionManager.publish(AmqpConnectionManager.java:193) ~[amqp-api-1.2-SNAPSHOT.jar:na] at org.zenoss.zep.impl.EventPublisherImpl.publishEvent(EventPublisherImpl.java:42) ~[zep-core-1.2-SNAPSHOT.jar:na] at org.zenoss.zep.impl.HeartbeatProcessorImpl.sendHeartbeatEvents(HeartbeatProcessorImpl.java:91) ~[zep-core-1.2-SNAPSHOT.jar:na] at org.zenoss.zep.impl.Application$5.run(Application.java:418) ~[zep-core-1.2-SNAPSHOT.jar:na] at org.zenoss.zep.impl.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:43) [zep-core-1.2-SNAPSHOT.jar:na] at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:51) [spring-context-3.0.5.RELEASE.jar:3.0.5.RELEASE] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [na:1.6.0_31] at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source) [na:1.6.0_31] at java.util.concurrent.FutureTask.runAndReset(Unknown Source) [na:1.6.0_31] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source) [na:1.6.0_31] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source) [na:1.6.0_31] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) [na:1.6.0_31] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) [na:1.6.0_31] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.6.0_31] at java.lang.Thread.run(Unknown Source) [na:1.6.0_31]
<<this repeats a LOT>>
zopectl Log File
Error: no log file exists
zenhub Log File
raise Exception("Could not publish message to RabbitMQ: %s" % lastexc) Exception: Could not publish message to RabbitMQ: [Errno 104] Connection reset by peer 2013-06-05 12:00:40,049 INFO zen.ZenHub: Worker (30003) reports 2013-06-05 12:00:40,049 CRITICAL zen.zenoss.protocols.amqp: Could not use exchange $RawZenEvents: [Errno 104] Connection reset by peer 2013-06-05 12:00:40,050 INFO zen.ZenHub: Worker (30003) reports 2013-06-05 12:00:40,050 CRITICAL zen.Events: Unable to publish event to <Products.ZenMessaging.queuemessaging.publisher.EventPublisher object at 0xc6c2e50>: [Errno 104] Connection reset by peer 2013-06-05 12:00:42,971 INFO zen.ZenHub: Worker (30199) reports 2013-06-05 12:00:42,970 CRITICAL zen.zenoss.protocols.amqp: Could not use exchange $RawZenEvents: [Errno 104] Connection reset by peer 2013-06-05 12:00:42,971 INFO zen.ZenHub: Worker (30199) reports 2013-06-05 12:00:42,970 CRITICAL zen.Events: Unable to publish event to <Products.ZenMessaging.queuemessaging.publisher.EventPublisher object at 0x1376ee50>: [Errno 104] Connection reset by peer 2013-06-05 12:00:43,055 INFO zen.ZenHub: Worker (30003) reports 2013-06-05 12:00:43,055 INFO zen.zenoss.protocols.amqp: RabbitMQ connection was closed: [Errno 104] Connection reset by peer 2013-06-05 12:00:45,975 INFO zen.ZenHub: Worker (30199) reports 2013-06-05 12:00:45,974 INFO zen.zenoss.protocols.amqp: RabbitMQ connection was closed: [Errno 104] Connection reset by peer 2013-06-05 12:00:46,059 INFO zen.ZenHub: Worker (30003) reports 2013-06-05 12:00:46,058 INFO zen.zenoss.protocols.amqp: RabbitMQ connection was closed: [Errno 104] Connection reset by peer 2013-06-05 12:00:46,060 INFO zen.ZenHub: Worker (30003) reports 2013-06-05 12:00:46,059 ERROR zen.Events: Unable to publish event to <Products.ZenMessaging.queuemessaging.publisher.EventPublisher object at 0xc6c2e50> Traceback (most recent call last): File "/opt/zenoss/Products/ZenEvents/MySqlSendEvent.py", line 43, in sendEvents self._publishEvent(event, publisher) File "/opt/zenoss/Products/ZenEvents/MySqlSendEvent.py", line 92, in _publishEvent publisher.publish(event) File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 288, in publish mandatory=mandatory, immediate=immediate) File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 319, in _publish mandatory=mandatory, immediate=immediate, createQueues=createQueues) File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 392, in publish headers=headers, declareExchange=declareExchange) File "/opt/zenoss/lib/python/zenoss/protocols/amqp.py", line 156, in publish raise Exception("Could not publish message to RabbitMQ: %s" % lastexc) Exception: Could not publish message to RabbitMQ: [Errno 104] Connection reset by peer 2013-06-05 12:00:48,978 INFO zen.ZenHub: Worker (30199) reports 2013-06-05 12:00:48,978 INFO zen.zenoss.protocols.amqp: RabbitMQ connection was closed: [Errno 104] Connection reset by peer 2013-06-05 12:00:48,979 INFO zen.ZenHub: Worker (30199) reports 2013-06-05 12:00:48,978 ERROR zen.Events: Unable to publish event to <Products.ZenMessaging.queuemessaging.publisher.EventPublisher object at 0x1376ee50> Traceback (most recent call last): File "/opt/zenoss/Products/ZenEvents/MySqlSendEvent.py", line 43, in sendEvents self._publishEvent(event, publisher) File "/opt/zenoss/Products/ZenEvents/MySqlSendEvent.py", line 92, in _publishEvent publisher.publish(event) File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 288, in publish mandatory=mandatory, immediate=immediate) File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 319, in _publish mandatory=mandatory, immediate=immediate, createQueues=createQueues) File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 392, in publish headers=headers, declareExchange=declareExchange) File "/opt/zenoss/lib/python/zenoss/protocols/amqp.py", line 156, in publish
I'm thinking the dependancy on the RabbitMQ service is the problem here...
Thoughts anyone?
--------------------------------------------------------------
Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/73472#73472]
Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]