How many Devices can Zenoss Core Handle ?

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64647#64647

--------------------------------------------------------------
Hi,

I am running Zenoss Core 3.2.1, and I was just wondering how manye devices Zenoss can run without giving problems ? ATM I have about 3000 on, with graphing on them. About 60 of them are servers and 20 Routers, the rest are just ping devices with uptime graphing.

The reason I am asking is zenoss stopped graphing my devices after the devices reached 1000+ . I checked all the daemons are still running I will try a restart and check how it goes.

The device count will top 10000, so that is why I am asking the forums just to see what other people have.

Spesifications of the server are high end, but I will post if you guys thinks it is needed.

Regards,
Stefan
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64647#64647]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[jmp242]

jmp242 [http://community.zenoss.org/people/jmp242] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64640#64640

--------------------------------------------------------------
This can be a tricky question, but with 10,000 devices you're going to need distributed collectors. Core 4 is going to be much more scalable, but in 3.2.1, you're looking at about 5000 per collector on really high end hardware with fusionIO cards etc.

However, your environment is quite different in that you've got really 80 devices or so running SNMP checks, and 900 running ping checks. Basic ping checks don't stress the system near as much - especially if they're not graphing anything (RRD IO is one biggie in performance). At this scale, have you considered either the enterprise product or a community consultant to help you get going?

--
James Pulver
ZCA Member
LEPP Computer Group
Cornell University
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64640#64640]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64649#64649

--------------------------------------------------------------
Currently we are using zabbix, and it works fine. Being it devices that we are just pinging and doing some uptime graphing did not think it will be that bad.

P.S I am using a script for all the graphing.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64649#64649]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[jmp242]

jmp242 [http://community.zenoss.org/people/jmp242] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64641#64641

--------------------------------------------------------------
Command datasources  for all your graphs? Those are very performance intensive and should be used judiciously. Are there a bunch of scripts hung / zencommand instances?

--
James Pulver
ZCA Member
LEPP Computer Group
Cornell University
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64641#64641]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64650#64650

--------------------------------------------------------------
I use them for my ping devices,because it is a requirement to graph if the device is up or down. For instance it goes down during the night.

What do you mean by scripts hung / zencommand instances ?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64650#64650]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64651#64651

--------------------------------------------------------------
Jsut checked my Zenoss server got this error, idea on how to fix this:

threshold of high event queue exceeded: current value 1052.00
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64651#64651]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[jmp242]

jmp242 [http://community.zenoss.org/people/jmp242] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64642#64642

--------------------------------------------------------------
Basically, you can't really do that many scripts. If its the same script for all devices, you might want to look into coding a daemon. The event queue exceeded isn't good either, you're not able to process events fast enough. What events are you getting?

Every time you run a command based datasource, you're actually spawing a shell to run the command, waiting for it to complete, and return. You might be close to "fork-bombing" your server if you're doing 10,000 shells + command every 60 seconds.

--
James Pulver
ZCA Member
LEPP Computer Group
Cornell University
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64642#64642]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64656#64656

--------------------------------------------------------------
Stefan:

To properly scale ZenOSS core to very large data centers you'll need to put a lot of hardware in to it. Collections should be limited to a set number of datapoints and be assigned their own dedicated zenhubs. A combination of disk IO and collector configuration handling are the most difficult hurdles. At larger scales enterprise ZenOSS pays off due to enhanced distributed collector support and improved zenhub<->daemon configuration handling. You'll want to use a configuration similar to ours here at Rackspace since we tend to have the largest instances. If you have specific questions, ask away.

--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64656#64656]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64657#64657

--------------------------------------------------------------
Stefan:

I just happened to notice you're scripting tests. Please don't do that. It's criminally inefficient. Try to use the specialized daemons for each test. If you have tests which require special activities otherwise unsupported, adapt an existing daemon to do the work. Only use zencommand for tests which aren't implemented on a massive scale. ZenOSS does come with some overhead, but the programmatic functionality pays off big. There is not another product with similar programatic functionality.

--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64657#64657]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64695#64695

--------------------------------------------------------------
The problem is I have no idea how to write a deamon nor to implement it. :(
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64695#64695]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64698#64698

--------------------------------------------------------------
What if I fool zenoss, basiclly create copies of the script make say 15 of them and each category of ping devices has its own template that executes that seperate script, not all of them going down on one script.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64698#64698]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[jmp242]

jmp242 [http://community.zenoss.org/people/jmp242] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64701#64701

--------------------------------------------------------------
The issue is not Zenoss exactly. It's that your OS can only handle so many scripts running, and it needs to be run for each check. It's not a disk IO issue, it's a running processes issue. Making copies won't help. Distributed collectors could I guess - each different computer is running only so many devices. However, it's far more efficient to use a daemon. If you need help creating a daemon, you can ask questions on the forum, or there are community developers who are available for consulting engagements.

You can also just ask for such a daemon to be created, but I don't think there's much support - you're the first or second person in years I've seen ask for such a thing.

--
James Pulver
ZCA Member
LEPP Computer Group
Cornell University
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64701#64701]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64702#64702

--------------------------------------------------------------
It is just weird, because it is obvious that most companies would want to see the uptime of their devices. If the device went down during the night etc. Even if it is only pinging the device it is still a very important feature.

The only thing I can not understand is that zenoss can create a report on device availability, so it needs to store when the device was up and down to create such a report. If only that data can be found somewhere ?

I really need the graphs otherwise we are going to have to fall back to zabbix :( , and I do not think distributed collectors is going to be the anwser, because I only have one server to do this on.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64702#64702]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64721#64721

--------------------------------------------------------------
Stefan:

Zenping automatically collects availability data. Look under reports. As for uptime, it's being collected by snmp.

--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64721#64721]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64715#64715

--------------------------------------------------------------
Shane:

I am only using Ping. Is there anyway I can use the availability data to graph ?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64715#64715]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64733#64733

--------------------------------------------------------------
Stefan:

Graph availability? Do you mean instead to collect and graph latency?

--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64733#64733]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64716#64716

--------------------------------------------------------------
Shane:

I want to graph when the device was available and when not, so that I can show the needed people when the device went down, at what time etc. So I want it to graph like this.

Device 1


        ____________________________________
1|     |                                                               |
  |     |                                                               |                      
  |     |                                                               |
0|___|____________________________________|___

So I powered the device on so it went to 1, and then it went down again it goes to 0.

And thats what I want to do with the graphs.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64716#64716]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64734#64734

--------------------------------------------------------------
Stefan:

Hum. I think a good way to do this would be to write a custom datasource for ping availability. The custom provider could check d.pingStatus() which passes back an integer for up down. The data would then be put in to an RRD which is easily graphable. Whats more is this method doesn't require an extra test and is substantially less overhead than an extra test.

--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64734#64734]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64717#64717

--------------------------------------------------------------
Shane:

Thank so much for your help so far, but can you maybe shed some light on how this would look ?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64717#64717]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64735#64735

--------------------------------------------------------------
Stefan:

I would disassemble the Core DNS monitor pack and model off of that. It contains a custom datasource for collecting dns repsonse.

http://community.zenoss.org/docs/DOC-3511 http://community.zenoss.org/docs/DOC-3511

--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64735#64735]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64718#64718

--------------------------------------------------------------
Isn't it possible to just create a command datasource that pulls a script for instance.

test.sh ${here/manageIp}

and then in the script have something like.

return [getPingStatus()]

The problem is I have no idea how to put this together so that it will actually work, if this is the solution it would be awesome!
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64718#64718]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64736#64736

--------------------------------------------------------------
Stefan:

It is possible, however, it's a script. Running a script to do this will easily septuple the load of the test and create a huge amount of scheduler competition. Zencommand based tests also don't scale well.

--shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64736#64736]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64737#64737

--------------------------------------------------------------
Stefan:

On small instances it doesn't hurt to make use of command based tests. If I understand right you're looking for a test that scales- you'll need to make use of ZenOSSs special functionality to do that. Zencommand doesn't scale for the same reason that nagios and zabbix don't scale- they run individual tests and create scheduler competion and chaos.

--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64737#64737]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64719#64719

--------------------------------------------------------------
Shane:

I understand what you are saying, only problem is I have no idea how to disassemble the DNS monitor and assemble it again,nor any idea how to code a daemon.

The DNS monitor package, does it monitor a IP to check if it is up, then I can just use it as it is for all my devices to do the graphing etc without changing any code ?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64719#64719]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64738#64738

--------------------------------------------------------------
Stefan:

The package is a zipped up egg file. Egg files are just zip files renamed to .egg. To disassemble it just unzip it twice. Unfortunately there is little way to add scalable functionality that hasn't already been coded without knowing python. I highly reccomend learning the basics of python if you're going to be using ZenOSS as a large scale. If development is not an option there are more than a few of us on here that do development contracting.

--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64738#64738]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64751#64751

--------------------------------------------------------------
I installed the Zenpack Fping, I hope this will fix all my troubles :)

Thanks for your help!
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64751#64751]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64739#64739

--------------------------------------------------------------
Stefan:

I'm pretty sure the fping pack uses command datasources.

Anytime.

--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64739#64739]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64752#64752

--------------------------------------------------------------
Sigh, so this will not fix it then. On that thread they suggested decreasing the RRD values (If you have many devices), which I did. From 0.5 to 0.15. And also increasing the parralel jobs. Maybe to like 30-50 ?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64752#64752]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64740#64740

--------------------------------------------------------------
Stefan:

Changing the number in parallel only helps reduce load across the polling cycle. You're still ultimately limited in scale.

--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64740#64740]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64753#64753

--------------------------------------------------------------
Okay I am going back to my uptime script, but with a few changes.

Going to create a seperate script for each group.

Each group is going to have different intervals at which they pull the data, up to 1500sec.

Except all the small groups which will remain at 300, to make sure they finish their job quickly.

This will hopefully help,because I am running out of ideas haha.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64753#64753]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64766#64766

--------------------------------------------------------------
Stefan:

Those are good changes. When I have some free time I'll create a test for this.

--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64766#64766]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[TitoOrtega]

TitoOrtega [http://community.zenoss.org/people/TitoOrtega] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64785#64785

--------------------------------------------------------------
hi stefan

i do monitoring for rackspace with zabbix. we have 300,000 nodes and it works wonderfully. please pm me for mor info.

thanks,
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64785#64785]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64772#64772

--------------------------------------------------------------
Stefan:

Heads up, Tito is a troll.

--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64772#64772]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64779#64779

--------------------------------------------------------------
haha!

Shane:

So far it has been working in setting the intervals at different times, graphs are wroking. I have about 2000 devices to go, so I am almost there. Still using the one file(script), must probably create copy of it, to run the last 2000 devices from it.

Stefan
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64779#64779]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64832#64832

--------------------------------------------------------------
Stefan:

Nice. Keep us posted on your progress. I'll be interested to see zencommand scale up and be stable! (I've come to not trust zencommand since zenoss v2.5)

--Shane (Hackman238)
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64832#64832]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64836#64836

--------------------------------------------------------------
Hey man,

So far I am successfully graphing 2077 devices with zencommand. I am having problems with two groups, one 200 devices and the other 600 devices both reporting back a nan to the graph. Changed the times to 5000 now :p

Hopefully that will work, because the others keeps graphing so I do not think it is a overloading issue.

I monitored the server with top -zenoss and I can se the scripts running with some breathing room between the bunch of scripts.

Only three left to go. 600+200+1500(ouch) . Then the devices will total 4377. So lets hold thumbs :)

Stefan
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64836#64836]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64856#64856

--------------------------------------------------------------
Stefan:

*Crosses fingers*
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64856#64856]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64934#64934

--------------------------------------------------------------
Okay so, I have doubts that this will work in the long run, because the devices are GSM based. And it is extra bandwidth to ping uptime the devices, and also for Zenoss to ping the device to check if it is down or up. So ATM it is doing the same thing twice, only for me to get a datapoint :( .
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64934#64934]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64952#64952

--------------------------------------------------------------
Stefan:

Add these devices to a custom device class and set it to zPingMonitorIfnore = True. This will stop zenping from hitting them. Next add your ping tests you've written. Create a custom copy of the template for this new device class and change the failure event class to /Status/Ping. On /Status/Ping add a transform that reads the events from your test and modifies them to look more like standard ping up/downs.

--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64952#64952]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64963#64963

--------------------------------------------------------------
My script ends like this at the moment:

TheResult="Datasource|Datapoint1="$checkloss     
echo $TheResult                                  
exit $ExitVal                                    

I geuss I need to add something that will return a event if checkloss is 0 ?

Also made the failure event class to /Status/Ping for my script.

I just have no idea to add the transform to /Status/Ping that it will read my event from my script.

--Stefan
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64963#64963]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64973#64973

--------------------------------------------------------------
Stefan:

You'll want your script to return a critical error code when the ping fails. This will cause a critical event in /Status/Ping.

Best,
--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64973#64973]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/65454#65454

--------------------------------------------------------------
Hi man,

After long struggles, forcing it to work. I decided against using this method, as it is not viable option. I can see it having a affect on Zenoss, and is not accurate when it needs to check 6000 devices. My only option left now is the following. Program something so that it can export all the events from the event console. Then use the firstseen and lastseen columns in the excel sheet to graph. All this will need to be done outside of zenoss now, and this is really starting to become a headace. If this fails, it is of to Zabbix. Which is really the last thing I want to do, because Zenoss is really a awesome monitoring mechanism.

Do you think the exporting will be any good ?

Regards,
Stefan
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/65454#65454]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/65518#65518

--------------------------------------------------------------
Stefan:

Yeah unfortunately command datasources don't scale well. What is it you're trying to do again? We can find a native way to do it.

To export reports or event console data your best bet is a simple python script using ZenScriptBase. ZenScriptBase allows one to open a dmd connection and work directly with everything.

Best,
--Shane (Hackman238)
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/65518#65518]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/65523#65523

--------------------------------------------------------------
Well I am trying to graph the uptime of ping devices. To basiclly create a graph to show the clients when their devices went down and for how long. Command datasource was doing that, but It was manipulating the server into what I want it to do without it doing it by itself and so causing strain. What other methods are there, when you say dmd connection ?

Regards,
Stefan
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/65523#65523]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/65524#65524

--------------------------------------------------------------
Stefan:

Can you post a screenshot of what you're doing?

--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/65524#65524]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/65525#65525

--------------------------------------------------------------
Okay so here is the script that I call, that returns a 0(Down) or 1(UP) value. With a datapoint and graph.

Loading Image... Loading Image...

Regards,
Stefan
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/65525#65525]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/65526#65526

--------------------------------------------------------------
Okay so here is the script I call through zenoss. It returns a 0 (Down) or 1 (Up) value, which then goes to a datapoint and then graphed.

Loading Image... Loading Image...

Regards,
Stefan
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/65526#65526]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/65599#65599

--------------------------------------------------------------
Stefan:

Got it. You can create a python datasource that calls here.getPingStatus().  You can do here.getSnmpStatus() too.

Best,
--Shane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/65599#65599]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/65590#65590

--------------------------------------------------------------
Okay, so no need to use a script anymore ?

And how do I create a python datasource?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/65590#65590]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[kb8u]

kb8u [http://community.zenoss.org/people/kb8u] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/65708#65708

--------------------------------------------------------------
You can create a command datasource like so:

/bin/sh -c "if [ ${dev/getPingStatus} = 0 ]; then echo 'OK|ping=1'; else echo 'OK|ping=0'; fi"

I don't think there is a purely python datasource unless it's in some zenpack I don't know about.  There doesn't seem to be any way around running sh for each device short of writing your own datasource type.  The command above takes about 6 milliseconds per device on my zenoss system so it wouldn't scale to 6000 hosts.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/65708#65708]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[kb8u]

kb8u [http://community.zenoss.org/people/kb8u] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/65710#65710

--------------------------------------------------------------
Correction, about 60 milliseconds per host, not 6.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/65710#65710]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/65713#65713

--------------------------------------------------------------
I am testing it out now! It should give exactly what I want :D .( Datapoint: Ping ), without the actual script file it has to run to create a datapoint.

I will let you know how it scales!

Thanks man!
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/65713#65713]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/65770#65770

--------------------------------------------------------------
Hi http://community.zenoss.org/people/kb8u kb8u,

This solition worked perfectly :)

Thanks a lot!
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/65770#65770]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Stefan Reinke]

Stefan Reinke [http://community.zenoss.org/people/Stefan] created the discussion

"Re: How many Devices can Zenoss Core Handle ?"

To view the discussion, visit: http://community.zenoss.org/message/64648#64648

--------------------------------------------------------------
Okay, I have restarted. Devices started graphing and then after 10minutes all the devices stopped graphing.

All deamons still running and zenoss still monitors, it is just the graphing that randomly stops.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/64648#64648]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]