Migrate from 3.2.1 to 4.2.0

[sconos]

sconos [http://community.zenoss.org/people/sconos] created the discussion

"Migrate from 3.2.1 to 4.2.0"

To view the discussion, visit: http://community.zenoss.org/message/69627#69627

--------------------------------------------------------------
Hi all,

Looking for some help moving from an estalished Zenoss 3.2.1 running instance to a different server running an instance of 4.2.0 SP1

Both 3.2.1 and 4.2.0 are running on supported OS installs (CentOS)

I've done a zenbackup from the old 3.2.1 server, scp'd the backup straight to the new 4.2.0 server, ran zenrestore then zenmigrate with what looks like no issues, but starting up zenoss on the 4.2.0 server is another story.

[***@zenoss2 bin]$ zenoss stop
Daemon: zenwin stopping...
Daemon: zeneventlog stopping...
Daemon: zenwinperf stopping...
Daemon: zenjmx stopping...
Daemon: zenrrdcached stopping...
Daemon: zenprocess stopping...
Daemon: zencommand stopping...
Daemon: zenperfsnmp stopping...
Daemon: zenmodeler stopping...
Daemon: zentrap stopping...
Daemon: zenactiond stopping...
Daemon: zenstatus stopping...
Daemon: zensyslog stopping...
Daemon: zenping stopping...
Daemon: zeneventd stopping...
Daemon: zenjobs stopping...
Daemon: zenhub stopping...
Daemon: zopectl .
daemon process stopped
Daemon: zeneventserver stopping...
[***@zenoss2 bin]$ ./zenrestore --file=/opt/zenoss/backups/zb10302012.tgz
[***@zenoss2 bin]$ ./zenmigrate
INFO:zen.migrate:Installing AddUserInterfaceSettings (4.2.0)
INFO:zen.migrate:Installing ChangeSnmpDefaults (4.2.0)
INFO:zen.migrate:Installing ClearUserSettings (4.2.0)
INFO:zen.migrate:Installing DeviceSearchIpAddressCatalogCleanup (4.2.0)
INFO:zen.migrate:Installing FileSystemIndexName (4.2.0)
INFO:zen.migrate:Installing JobsDelete (4.2.0)
INFO:zen.migrate:Removing old job records
INFO:zen.migrate:Removing old job relationship
INFO:zen.migrate:Installing RemoveDCSHFromCommand (4.2.0)
INFO:zen.migrate:Installing RemoveDeviceZenMenus (4.2.0)
INFO:zen.migrate:Installing RemoveModificationsMenuItems (4.2.0)
INFO:zen.migrate:Installing SetIgnoreParamsOnZenDaemons (4.2.0)
INFO:zen.migrate:Installing SnmpWalkWithPort (4.2.0)
INFO:zen.migrate:Installing addIgnoreParametersWhenModeling (4.2.0)
INFO:zen.migrate:Adding ignoreParametersWhenModeling to all process classes.
INFO:zen.migrate:Installing addzInterfaceMapIgnoreDescriptions (4.2.0)
INFO:zen.migrate:Installing discoveryMaxOidsPerRequest (4.2.0)
INFO:zen.migrate:Installing fixEmailNotificationClearSubjectFormat (4.2.0)
INFO:zen.migrate:Setting default values for E-mail notifications.
INFO:zen.migrate:Installing ifOperStatusEthernetCsmacd64 (4.2.0)
INFO:zen.migrate:Installing ifOperStatusRRDCreateCommand (4.2.0)
WARNING:zen.migrate:
Changing the definition for ifOperStatus. Please
remove all rrd files with this name: ifOperStatus_ifOperStatus.rrd.

You can remove all the rrd files with the following command as the zenoss user:
find $ZENHOME -name "ifOperStatus_ifOperStatus.rrd" -delete
INFO:zen.migrate:Installing moveZepConfig (4.2.0)
INFO:zen.migrate:Installing perDeviceCompCatalog (4.2.0)
INFO:zen.migrate:Installing removeClearHeartbeatsMenuItem (4.2.0)
INFO:zen.migrate:Installing setDefaultsOnEmailNotifications (4.2.0)
INFO:zen.migrate:Setting default values for E-mail notifications.
INFO:zen.migrate:Installing zRemoveAutoAllocate (4.2.0)
INFO:zen.migrate:Installing zenmodelerModeledDevices (4.2.0)
INFO:zen.migrate:Installing ifOperStatusEvents (4.2.0)
INFO:zen.migrate:Loading Reports
[***@zenoss2 bin]$ find $ZENHOME -name "ifOperStatus_ifOperStatus.rrd" -delete
[***@zenoss2 bin]$ zenoss start
Daemon: zeneventserver starting...
Waiting for zeneventserver to start....................................................................................................................................................................................
zeneventserver failed to start within 180 seconds.
Please check the following log file for more details:
  /opt/zenoss/log/zeneventserver.log
Daemon: zopectl .
daemon process started, pid=15290
Daemon: zenhub starting...
Daemon: zenjobs starting...
Daemon: zeneventd starting...
WARN: unable to read config file /opt/zenoss/etc/zeneventd.conf                 -- skipping. (IOError: [Errno 2] No such file or directory: '/opt/zenoss/etc/zeneventd.conf')
Daemon: zenping starting...
Daemon: zensyslog starting...
Daemon: zenstatus starting...
Daemon: zenactiond starting...
WARN: unable to read config file /opt/zenoss/etc/zenactiond.conf                 -- skipping. (IOError: [Errno 2] No such file or directory: '/opt/zenoss/etc/zenactiond.conf')
Daemon: zentrap starting...
Daemon: zenmodeler starting...
Daemon: zenperfsnmp starting...
Daemon: zencommand starting...
Daemon: zenprocess starting...
Daemon: zenrrdcached starting...
Daemon: zenjmx starting...
Daemon: zenwinperf starting...
WARN: unable to read config file /opt/zenoss/etc/zenwinperf.conf                 -- skipping. (IOError: [Errno 2] No such file or directory: '/opt/zenoss/etc/zenwinperf.conf')
Daemon: zeneventlog starting...
Daemon: zenwin starting...
[***@zenoss2 bin]$


my first backup from the 3.2.1 box was run with "./zenbackup --file=zb10302012.tgz --no-eventsdb --no-perfdata"

This is becasue I just want the devices, no old events and the device performance data to start fresh.  After the restore and migrate I could not even hit the login page on the new 4.2.0 instance.  This is a VM so I just restored snapshot back to baseline and am going to try again after zeopacking the old DB and getting as much event history cleared out, I'll try a straight ./zenbackup - I looked at the zeneventserver.log, but could not see anything and it was huge, so I did not post it.

I'm only posting this because everything I read says that the above should have worked with no issues.  Does anyone have any comments or suggestions?

TIA, Mike
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/69627#69627]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[pwarren]

pwarren [http://community.zenoss.org/people/pwarren] created the discussion

"Re: Migrate from 3.2.1 to 4.2.0"

To view the discussion, visit: http://community.zenoss.org/message/69625#69625

--------------------------------------------------------------
Are there any error messages in the zeneventserver-stdio-YYYY-MM-DD.log file (replace YYYY-MM-DD with the date when the install started)? In some cases, early failures in starting zeneventserver will be logged to this file (things like inability to bind to the HTTP port, etc.).

--
Philip Warren
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/69625#69625]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[sconos]

sconos [http://community.zenoss.org/people/sconos] created the discussion

"Re: Migrate from 3.2.1 to 4.2.0"

To view the discussion, visit: http://community.zenoss.org/message/69638#69638

--------------------------------------------------------------
Philip,

Thanks for the reply, it sort of pointed me in the right direction, the 4.2.0 server I have is backup and running.  After looking through the startup errors and stdio log its evident the zenbackup and zenrestore is not a migration tool from one version to another.

I traced the errors in startup back to the files in $ZENHOME/etc and zenrestore does not do a version compare during a restore, it wipes out the old /etc and lays down what was backed up in the tar file, the file structure is different between 3.2.1 and 4.2.0, zeneventserver has its own subfolder in /etc along with a /jetty folder for its jmx engine, both of these are wipped out when restoring from 3.2.1 to 4.2.0 - hence, zeneventserver had no startup config.  Snapshot back to my baseline running blank 4.2.0 install, made a copy of the $ZENHOME/etc folder to /tmp, ran the restore and tried a compare and copy back, - that got all the services to start without error, but the site page would not come up.  I just restored the $ZENHOME/etc folder completely and recursively and everything comes up including the site page, but I get no devices in the db, no users, no alert rules, nothing ...

Back to the drawing board .. thanks for the pointer it helped me understand the errors, but I'm still back to step one - no clean migration of data from a running 3.2.1 server to a new running 4.2.0 server.   I does not make sense to me, Zenoss does not have a data miration path that I can see ..

Regards,

Mike
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/69638#69638]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[jmp242]

jmp242 [http://community.zenoss.org/people/jmp242] created the discussion

"Re: Migrate from 3.2.1 to 4.2.0"

To view the discussion, visit: http://community.zenoss.org/message/69639#69639

--------------------------------------------------------------
Zenoss doesn't have a migration path. What you should do is install the older version on the new server, make sure the restore works, then do an upgrade in place. You can never restore a backup to a different version of Zenoss, never have been able to.

Probably a reasonable feature request, but I have no idea if they will add it.

--
James Pulver
ZCA Member
LEPP Computer Group
Cornell University
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/69639#69639]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[sconos]

sconos [http://community.zenoss.org/people/sconos] created the discussion

"Re: Migrate from 3.2.1 to 4.2.0"

To view the discussion, visit: http://community.zenoss.org/message/69657#69657

--------------------------------------------------------------
Hi James,

Thanks for the reply, you've given me very helpful information in the past.  I understand your comment comes from experience, the thing that confuses me is right in the 4.2 Core Admin Guide under Backup and Recovery its states:

"If you use these tools to go from an older version to a newer version, you should run zenmigrate after the
restore operation." Then .. "If restoring to a different Zenoss Core installation (one that differs from the backup version), make sure file paths in the $ZENHOME/etc/*.conf files are appropriate for the new environment after you restore."

The above makes me think I can migrate from one version to the next, the excersize I just went through plus your comments tell me I can't ... just confusing, thats all.

Before I kill the whole 4.2 install and go the route you suggested I wanted to see if I could  just dump all the devices and load them on the 4.2 instance to actually see them come over .. but zendevicedump does not like me either ...

[***@zenoss backups]$ ../bin/zendevicedump --outfile=/opt/zenoss/backups/mydevicelist.xml
Traceback (most recent call last):
  File "/opt/zenoss/Products/ZenRelations/ExportDevices.py", line 229, in <module>
    ex.export()
  File "/opt/zenoss/Products/ZenRelations/ExportDevices.py", line 215, in export
    doc = parseString(buffer.getvalue())
  File "/opt/zenoss/lib/python/_xmlplus/dom/minidom.py", line 1925, in parseString
    return expatbuilder.parseString(string)
  File "/opt/zenoss/lib/python/_xmlplus/dom/expatbuilder.py", line 942, in parseString
    return builder.parseString(string)
  File "/opt/zenoss/lib/python/_xmlplus/dom/expatbuilder.py", line 223, in parseString
    parser.Parse(string, True)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 966911, column 7

I took a look at http://community.zenoss.org/message/62321#62321 http://community.zenoss.org/message/62321 - which is close, but not quite the same.  If anyone has any ideas about zendevicedump that would help.

James, thanks for the input.

Regards,

Mike
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/69657#69657]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]