Optimal filesystem partitioning for Zenoss installation

Discussion:

Mihail Lukin

2013-02-21 09:31:39 UTC

Mihail Lukin [http://community.zenoss.org/people/m_lukin] created the discussion

"Optimal filesystem partitioning for Zenoss installation"

To view the discussion, visit: http://community.zenoss.org/message/71628#71628

--------------------------------------------------------------
Hello, ZenGurus!

My question is about optimal linux filesystem partitioning for fresh Zenoss Core installation.

Right now we're using 2.5.2 and I'm going to install 4.2 from scratch. Currently our metrics are:
* 106 devices (growing slowly)
* 3500 components (dmd.Devices.getSubComponents)
* 18000 RRD files (700MB)
* 120MB Data.fs + 125MB Data.fs.old (auto-saved before weekly packing)
* 14GB MySQL data (90-day event history retention)
* 600MB Zenoss logs
Overall Zenoss installation is about 16GB.

From release notes I know that there is no Data.fs anymore and all ZODB data is now stored in MySQL. It should affect the size of MySQL data in some way.

I googled around but didn't find good recommendations about partitioning filesystem for Zenoss installation, only filesystem tuning for better performance. I would like to discuss here how server administrator can estimate Zenoss needs based on current monitoring metrics and efficiently partition filesystem.

As a result, it may become a good part of Installation Notes :)
Thanks in advance!
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/71628#71628]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

themactech

2013-02-21 17:58:55 UTC

Permalink

themactech [http://community.zenoss.org/people/themactech] created the discussion

"Re: Optimal filesystem partitioning for Zenoss installation"

To view the discussion, visit: http://community.zenoss.org/message/71631#71631

--------------------------------------------------------------
I think both subjects should be tackled simultaneously.Â I am working on a very low end server build for Zenoss for small deployments (a 1RU sub-600$ server with RAID and IPMI).Â My biggest worry as far as bottleneck goes is for file I/O.Â I will need to dive into file system optimization and would love for folks to discuss partitioning schemes.

Right now I only do 3 partitions:

/Â Â Â Â Â Â Â Â Â Â Â Â Â Â (root file system)
/imagesÂ Â Â Â (second partition where I can hold FSarchive backups of the root partition and copy off to other server)
/swap

I am pretty sure there are smarter ways to do this and would like to get opinions from folks smarter than I on the subject.

Manuel
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/71631#71631]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

Mihail Lukin

2013-02-22 04:36:37 UTC

Permalink

themactech

2013-09-19 13:59:30 UTC

Permalink

Mihail Lukin

2013-09-25 15:57:45 UTC

Permalink

Mihail Lukin [http://community.zenoss.org/people/m_lukin] created the discussion

"Re: Optimal filesystem partitioning for Zenoss installation"

To view the discussion, visit: http://community.zenoss.org/message/74752#74752

--------------------------------------------------------------
I see no direct benefits of putting zenoss' events database on separate
disk partition. Maybe someone who made the virtual appliance does. I assume
that the file system tuning is possible, though. Using mount options
"noatime" and "async" is mentioned at
https://dev.mysql.com/doc/refman/5.5/en/disk-issues.html https://dev.mysql.com/doc/refman/5.5/en/disk-issues.html . Also, common
considerations are still valid:
http://www.cyberciti.biz/tips/the-importance-of-linux-partitions.html http://www.cyberciti.biz/tips/the-importance-of-linux-partitions.html .

Being up for about 6 months I noticed that root file system is also being
extensively used. "Hot" things are rabbitmq persistent storage and
rrd_cached journal. I believe that overall performance can be improved by
placing this directories (or whole /var directory) on another specially
tuned partition. At this point I don't know what mount options or software
configuration changes can be applied for this without risk of significant
data loss or corruption.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/74752#74752]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

Doug Syer

2013-09-26 04:49:48 UTC

Permalink

Doug Syer [http://community.zenoss.org/people/dsyer%40nwnit.com] created the discussion

"Re: Optimal filesystem partitioning for Zenoss installation"

To view the discussion, visit: http://community.zenoss.org/message/74771#74771

--------------------------------------------------------------
I dont think with that sized system it really matters much either way if you have decently fast SAS drives orunless you are collecting stats at an insane rate.

On some version of rabbit it will shut down(with no error ugh)Â unless you have 10% of your memory available as free disk space, so wherever rabbit is, dont run low on space.

the only thing i see doing spikey disk usage sometimes is the lucene indexes for the catalog and events, those you can move later if you want to put it on a high performance.Â i think my default mysql will limit itself to something like 200 IOPS anyways unless you change that setting.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/74771#74771]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

Doug Syer

2013-09-26 04:50:49 UTC

Permalink

Photon

2013-09-26 13:13:57 UTC

Permalink

Photon [http://community.zenoss.org/people/Photon] created the discussion

"Re: Optimal filesystem partitioning for Zenoss installation"

To view the discussion, visit: http://community.zenoss.org/message/74775#74775

--------------------------------------------------------------
I recenly read an article in Zenoss Support about making Zenoss highly available. I saw in the steps that they created 4 partitions mounted at :
/opt/zenoss
/opt/zenoss/perf
/opt/zenoss/var/zeneventserver
/opt/zends (DataStore) : I used it to store RabbitMQ and MySQL in my Core version).
These were LVM partitions. All in the same LVM group. Easier management, I guess...
That said, i don't think creating a partition for /opt/zenoss is not really useful in your case. It was for replication purposes in the documentation.
Here's the link, it could be useful.

https://support.zenoss.com/ics/support/default.asp?deptID=16059 https://support.zenoss.com/ics/support/default.asp?deptID=16059
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/74775#74775]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

themactech

2013-09-26 13:15:52 UTC

Permalink