Scalability to Larger Systems

[Adrian Cox]

Adrian Cox [http://community.zenoss.org/people/AdrianCox] created the discussion

"Scalability to Larger Systems"

To view the discussion, visit: http://community.zenoss.org/message/63889#63889

--------------------------------------------------------------
I'm considering a Zenoss Community install that will monitor up to 3000 devices, with 40 data points on each device. I've previously run a smaller system that with Zenoss 2.5 for a couple of years, that had 200 devices, but many more data points per device. Are these performance points useful, and what have I missed?

1. Edit mysql tuning parameters, aggresively trim events database.
2. If RRD storage rate gets too high, move RRD files into a large ramdisk, and back-up to disk every 15 minutes.
3. If zenperfsnmp is then CPU bound, install multiple collectors.

With 3000 sites, will ZODB become a performance problem?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/63889#63889]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[jmp242]

jmp242 [http://community.zenoss.org/people/jmp242] created the discussion

"Re: Scalability to Larger Systems"

To view the discussion, visit: http://community.zenoss.org/message/63911#63911

--------------------------------------------------------------
Well, if you have enough hardware, a single Zenoss box can handle 3000 devices. But we are talking 32+GB RAM, 8+ core processor and rather fast disk access.

The Core 4 archetecture which is supposed to be betaing Q1-Q2 this year will scale much better. If you can't get a big enough box for 3.2.1, you might try distributed collectors.

--
James Pulver
ZCA Member
LEPP Computer Group
Cornell University
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/63911#63911]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: Scalability to Larger Systems"

To view the discussion, visit: http://community.zenoss.org/message/64105#64105

--------------------------------------------------------------
Adrian:

The newest version of Zenoss, v4.x, stores the ZODB in relstorage which helps with portal performance. Concerning scalability, collection of points is almost exclusively limited by disk IO. For example, my larger DC's with > 5,000 devices have 3 to 6 dedicated collectors, each equipped with Fusion IO disks for RRD storage.

Ultimately the resource requirements boil down in this manner:
* Zenoss master (zope, hub, message queue host)* More processors / cores increases speed substantially
* More RAM, the better. Aim to cache 80-90% of the ZODB
* Reasonable speed RAID 10 recomended, but don't go crazy if you have a seperate db host

* Zenoss db host (mysql server)* More processors / cores is better, but doesn't necessarily increase speed after a point
* More RAM, the better. Aim to cache everything possible. Increases speed substantially
* Reasonable speed RAID 10 recomended, but don't go crazy

* Zenoss collectors (runs performance collection daemons)* Reasonable number of cores / processors, but don't go crazy since they'll go unused
* Reasonable amount of RAM, but don't go crazy since it will go unused
* Fastest disk array affordable. Array of Fusion IO cards highly reccomended


If you have any scalability questions, feel free to ask.

Best,
--Shane (Hackman238)
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64105#64105]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Adrian Cox]

Adrian Cox [http://community.zenoss.org/people/AdrianCox] created the discussion

"Re: Scalability to Larger Systems"

To view the discussion, visit: http://community.zenoss.org/message/64108#64108

--------------------------------------------------------------
Thanks Shane,

that's made me suspect that ZODB might be the bottleneck on one of my existing installations. I'm now pretty sure that Zenoss can do what I need by using multiple collectors.

Adrian
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64108#64108]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]