Discussion:
zenoss 4.2.3 upgrade fail
Thomas Pollet
2013-04-08 15:49:09 UTC
Permalink
Thomas Pollet [http://community.zenoss.org/people/Thomax] created the discussion

"zenoss 4.2.3 upgrade fail"

To view the discussion, visit: http://community.zenoss.org/message/72720#72720

--------------------------------------------------------------
Hi,

I installed the zenoss 4.2.3 rpm but zenoss doesn't work.
If I go to 127.0.0.1:8080 , I get a redirect to http://10.32.25.66:8080/zport/acl_users/cookieAuthHelper/login http://10.32.25.66:8080/zport/acl_users/cookieAuthHelper/login and I see the source as txt with the error

<div id="more_info">
    <div id="request_toggle" onclick="toggleReqBox()">View Error Details</div>
    <div id="request_box">
  <p>
      <strong>Type: &lt;class 'socket.error'&gt;
      </strong><br />
      <strong>Value: [Errno 104] Connection reset by peer
      </strong><br />
      <pre>Traceback (most recent call last):
  File "/opt/zenoss/lib/python/ZPublisher/Publish.py", line 134, in publish
    transactions_manager.commit()
  File "/opt/zenoss/lib/python/Zope2/App/startup.py", line 301, in commit
    transaction.commit()
  File "/opt/zenoss/lib/python/transaction/_manager.py", line 89, in commit
    return self.get().commit()
  File "/opt/zenoss/lib/python/transaction/_transaction.py", line 327, in commit
    self._callBeforeCommitHooks()
  File "/opt/zenoss/lib/python/transaction/_transaction.py", line 397, in _callBeforeCommitHooks
    hook(*args, **kws)
  File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 236, in beforeCompletionHook
    dataManager = AmqpDataManager(self._queuePublisher.channel, tx._manager)
  File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 396, in channel
    return self._client.getChannel()
  File "/opt/zenoss/lib/python/zenoss/protocols/amqp.py", line 69, in getChannel
    ssl=self._connectionInfo.usessl)
  File "/opt/zenoss/lib/python/amqplib/client_0_8/connection.py", line 144, in __init__
    (10, 30), # tune
  File "/opt/zenoss/lib/python/amqplib/client_0_8/abstract_channel.py", line 95, in wait
    self.channel_id, allowed_methods)
  File "/opt/zenoss/lib/python/amqplib/client_0_8/connection.py", line 202, in _wait_method
    self.method_reader.read_method()
  File "/opt/zenoss/lib/python/amqplib/client_0_8/method_framing.py", line 221, in read_method
    raise m
error: [Errno 104] Connection reset by peer
</pre>
  </p>
</div>


also, the event.log contains the error

2013-04-08T17:40:40 ERROR Zope.SiteErrorLog 1365435640.120.475256859915 http://10.32.25.66:8080/zport/acl_users/cookieAuthHelper/login http://10.32.25.66:8080/zport/acl_users/cookieAuthHelper/login
Traceback (innermost last):
  Module ZPublisher.Publish, line 134, in publish
  Module Zope2.App.startup, line 301, in commit
  Module transaction._manager, line 89, in commit
  Module transaction._transaction, line 327, in commit
  Module transaction._transaction, line 397, in _callBeforeCommitHooks
  Module Products.ZenMessaging.queuemessaging.publisher, line 236, in beforeCompletionHook
  Module Products.ZenMessaging.queuemessaging.publisher, line 396, in channel
  Module zenoss.protocols.amqp, line 69, in getChannel
  Module amqplib.client_0_8.connection, line 144, in __init__
  Module amqplib.client_0_8.abstract_channel, line 95, in wait
  Module amqplib.client_0_8.connection, line 202, in _wait_method
  Module amqplib.client_0_8.method_framing, line 221, in read_method
error: [Errno 104] Connection reset by peer



zeneventd.log:
2013-04-08 17:46:45,917 ERROR zen.maintenance: Maintenance failed. Message from hub: Could not publish message to RabbitMQ: [Errno 104] Connection reset by peer
2013-04-08 17:46:45,923 WARNING zen.protocols.twisted: Error authenticating to localhost as zenoss


zenhub.log:
2013-04-08 17:47:14,219 INFO zen.ZenHub: Worker (7609) reports 2013-04-08 17:47:14,219 ERROR zen.Events: Unable to publish event to <Products.ZenMessaging.queuemessaging.publisher.EventPublisher object at 0x5c230d0>
Traceback (most recent call last):
  File "/opt/zenoss/Products/ZenEvents/MySqlSendEvent.py", line 43, in sendEvents
    self._publishEvent(event, publisher)
  File "/opt/zenoss/Products/ZenEvents/MySqlSendEvent.py", line 89, in _publishEvent
    return self._sendHeartbeat(event)
  File "/opt/zenoss/Products/ZenEvents/MySqlSendEvent.py", line 107, in _sendHeartbeat
    publisher.publish('$Heartbeats', 'zenoss.heartbeat.%s' % heartbeat.monitor, heartbeat)
  File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 392, in publish
    headers=headers, declareExchange=declareExchange)
  File "/opt/zenoss/lib/python/zenoss/protocols/amqp.py", line 156, in publish
    raise Exception("Could not publish message to RabbitMQ: %s" % lastexc)
Exception: Could not publish message to RabbitMQ: [Errno 104] Connection reset by peer



etc etc etc

please help? My os is CentOS release 6.4, I use zenoss-4.2.3-1695.el6.x86_64 .
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/72720#72720]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]
nilie
2013-04-09 18:56:52 UTC
Permalink
nilie [http://community.zenoss.org/people/nilie] created the discussion

"Re: zenoss 4.2.3 upgrade fail"

To view the discussion, visit: http://community.zenoss.org/message/72754#72754

--------------------------------------------------------------
Is rabbitmq-server running on your machine ?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/72754#72754]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]
Thomas Pollet
2013-04-10 12:17:40 UTC
Permalink
Thomas Pollet [http://community.zenoss.org/people/Thomax] created the discussion

"Re: zenoss 4.2.3 upgrade fail"

To view the discussion, visit: http://community.zenoss.org/message/72757#72757

--------------------------------------------------------------
rabbitmq is running i suppose :

rabbitmq  2051     1  0 11:32 ?        00:00:00 /usr/lib64/erlang/erts-5.8.5/bin/epmd -daemon
root      2064     1  0 11:32 ?        00:00:00 runuser rabbitmq --session-command /usr/sbin/rabbitmq-server
rabbitmq  2077  2064  0 11:32 ?        00:00:00 /bin/sh /usr/sbin/rabbitmq-server
rabbitmq  2092  2077  0 11:32 ?        00:00:05 /usr/lib64/erlang/erts-5.8.5/bin/beam.smp -W w -K true -A30 -P 1048576 -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -noshell -noinput -sname ***@XXX -boot /var/lib/rabbitmq/mnesia/***@XXX-plugins-expand/rabbit -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -sasl sasl_error_logger false -rabbit error_logger {file,"/var/log/rabbitmq/***@XXX.log"} -rabbit sasl_error_logger {file,"/var/log/rabbitmq/***@XXX-sasl.log"} -os_mon start_cpu_sup false -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/***@XXX"
rabbitmq  2181  2092  0 11:32 ?        00:00:00 inet_gethost 4
rabbitmq  2182  2181  0 11:32 ?        00:00:00 inet_gethost 4
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/72757#72757]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]
Thomas Pollet
2013-04-10 12:20:24 UTC
Permalink
Thomas Pollet [http://community.zenoss.org/people/Thomax] created the discussion

"Re: zenoss 4.2.3 upgrade fail"

To view the discussion, visit: http://community.zenoss.org/message/72758#72758

--------------------------------------------------------------
I don't know rabbit-mq, but there seem to be authentication errors in its logs:

=ERROR REPORT==== 10-Apr-2013::11:23:16 ===
closing AMQP connection <0.1506.2> (127.0.0.1:44799 -> 127.0.0.1:5672):
{channel0_error,starting,
                {amqp_error,access_refused,
                            "PLAIN login refused: user 'zenoss' - invalid credentials",
                            'connection.start_ok'}}


any idea where the authentication credentials ara stored?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/72758#72758]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]
Thomas Pollet
2013-04-10 13:05:58 UTC
Permalink
Thomas Pollet [http://community.zenoss.org/people/Thomax] created the discussion

"Re: zenoss 4.2.3 upgrade fail"

To view the discussion, visit: http://community.zenoss.org/message/72759#72759

--------------------------------------------------------------
rabbitmqctl change_password zenoss zenoss

I get a different error now lol.

This rpm is a joke, why even make it available for download when it doesn't work
(zenoss_core-4.2.3.el6.x86_64.rpm)
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/72759#72759]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]
nilie
2013-04-10 13:36:33 UTC
Permalink
nilie [http://community.zenoss.org/people/nilie] created the discussion

"Re: zenoss 4.2.3 upgrade fail"

To view the discussion, visit: http://community.zenoss.org/message/72765#72765

--------------------------------------------------------------
It it exactly the same rpm I used to upgrade from 4.2.0 to 4.2.3 in a distributed setup (master + collector) and I did not encounter any problem. I assume there must be a difference between our systems them.

If I understand correctly, you did an upgrade to 4.2.3 not a new installation. Did you also upgrade rabbitmq prior to upgrading Zenoss ? What is the version you have running right now ?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/72765#72765]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]
Thomas Pollet
2013-04-10 14:16:20 UTC
Permalink
Thomas Pollet [http://community.zenoss.org/people/Thomax] created the discussion

"Re: zenoss 4.2.3 upgrade fail"

To view the discussion, visit: http://community.zenoss.org/message/72760#72760

--------------------------------------------------------------
I had zenoss 3.2 installed on this vm.

I installed the zenoss_core-4.2.3.el6.x86_64.rpm, I removed /opt/zenoss and installed the rpm.

If you use 4.2.3 on el6 then I suppose it has to work somehow. My dependencies are ok since the rpm installed without errors. I'm gonna save this snapshot, look for the 4.2.0 rpm and try install + upgrade.

atm, the zeo doesn't start, zeo.log:
2013-04-10T15:59:51 INFO root pid 31474: exit status 2; exiting now
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/72760#72760]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]
Thomas Pollet
2013-04-10 14:50:02 UTC
Permalink
Thomas Pollet [http://community.zenoss.org/people/Thomax] created the discussion

"Re: zenoss 4.2.3 upgrade fail"

To view the discussion, visit: http://community.zenoss.org/message/72777#72777

--------------------------------------------------------------
ok, I didn't remove the mysql databases before upgrading, causing all sorts of issues I guess

Installation is OK now :x
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/72777#72777]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]
Loading...