[smokeping-users] Smokeping/FPing scalability guidelines?

[Tobias Oetiker]

Hi Ged,

Today G.W. Haywood wrote:

> Hi there,
>
> On Thu, 19 Jun 2008, Allan Eising wrote:
>
> > ... 20 FPing processes ... 1997 nodes to be monitored ... the FPing
> > processes take too long ... any guidelines for a recommended number
> > of hosts per FPing process, so I might be able to write this into my
> > system, to maintain scalability.
>
> Unfortunately object-oriented Perl, while great for prototyping and
> coding on the hoof, is fairly costly in terms of computer resources.
> I think that's at least partly what you're up against.

> My feeling is that the entire setup isn't really scalable.  Looking at
> the processes running on my master smokeping host, about 1500MBytes of
> RAM (25 percent of the memory in the host) is used by approximately
> thirty processes (mostly FPing) in order to do little but sleep for
> the next few hundred seconds.  Between them, in a couple of days they
> have used nearly as much CPU as the X server has in a month - almost
> three minutes.  The network I'm monitoring has only a few dozen hosts.
[...]

> It is certainly feasible to ping a few thousand hosts using a single
> process without gobbling up gigabytes of memory and seconds of CPU.
> I'm tempted to code something but I'm convinced that it must already
> have been done.  If I did write something, it would be in C.  I'd be
> very disappointed if it used more than a couple of megabytes of RAM,
> and more than a few milliseconds of CPU per host per ping cycle which
> would include logging the results to a database.
>
> I hope that these back-of-an-envelope figures give you an idea of what
> should ultimately be achievable.  Are you doing a lot of logging?  You
> might be able to trim things by eliminating unnecessary disc accesses.
>
> None of this should be relevant to the Web front-end of course, most
> of the time you don't even need to be running it.

not sure about these numbers you have there, you say your smokeping
perl collector process is takeing up 1.5GB virtual memory, and that
it is increasing over time ? This would be some sort of leak then.

Or is it the fping binary ? They are started fresh for every run,
so they can not realy grow ...

if you want to code, how about improving on fping and makeing it
scale better ?

cheers
tobi

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten
http://it.oetiker.ch tobi at oetiker.ch ++41 62 213 9902