[rrd-users] rrdtool backed mrtg using massive amounts of memory

[Tobias Oetiker]

tobi




Today Steven Hartland wrote:

> Hmm I may well have jumped the gun with the diagnosis there.
> I've just seen it doing the same thing with mmap disabled.
>
> I setup full debug and restarting the the thing that stands
> out is the following:-
>
> 2009-05-13 15:54:58 -- --base: Check for Thresholds
> 2009-05-13 15:54:58 -- --base: Act on Router/Target as01.gswams_26
> 2009-05-13 15:54:59 -- --base: Get Current values: in:1524189419937, out:1124283784349, up:undef, name:undef, time:1242225809
> 2009-05-13 15:54:59 -- --base: Create Graphics
> 2009-05-13 15:54:59 -- --base: start RRDtool section
> 2009-05-13 15:54:59 -- --base: maxi:125000000, maxo:125000000
> 2009-05-13 15:54:59 -- --log: RRDs::tune(as01.gswams_26.rrd -a ds0:125000000 -a ds1:125000000 -d ds0:COUNTER -d ds1:COUNTER)
> 2009-05-13 15:54:59 -- --prof: RRDs::tune as01.gswams_26.rrd - 0.091s
> 2009-05-13 15:54:59 -- --log: RRDs::update(as01.gswams_26.rrd, '1242225809:1524189419937:1124283784349')
> 2009-05-13 15:59:50 -- --prof: RRDs::update as01.gswams_26.rrd - 290.345s
> 2009-05-13 15:59:50 -- --log:  got: ???/???
>
> So 290 seconds to do an single row update when this is the only
> thing the machine is doing?
>
> What could possibly be causing this?

have a look at iostat -x to see if there is high load on the disk
... the bit where your process is taking more and more memory is
rather odd though ... you are using the latest version of rrdtool
and there is not some old copy of the perl bindings kicking around
your system ?

cheers
tobi


>     Regards
>     Steve
>
> ----- Original Message -----
> From: "Steven Hartland" <killing at multiplay.co.uk>
> To: <rrd-users at lists.oetiker.ch>
> Sent: Wednesday, May 13, 2009 2:13 AM
> Subject: [rrd-users] rrdtool backed mrtg using massive amounts of memory
>
>
> > Having an issue here where mrtg which rrdtool backed in our
> > case is using a massive amount memory 5GB+ after a short
> > period of time.
> >
> > The config isn't that big just 371 targets. I seem to remember
> > a thread a while back suspecting mmap causing an issue so
> > I disabled it on the off chance this would help and sure
> > enough the mrtg perl process is now ~500MB.
> >
> > So why would mmap option cause such massive memory usage?
> >
> > rrdtool version: 1.3.7
> > mrtg version: 2.16.2,1
> > FreeBSD version: 7.0-RELEASE AMD64
> >
> > Its not just memory though actual processing speeds are
> > massively effected. The machine this runs on does two things
> > mrtg network port starts and server stats read from mysql DB
> > tables.
> >
> > With mmap enabled the a run of the server stats was taking
> > over 15mins without it ~1 1/2mins, so a massive difference.
> >
> > The machine has only got 4GB of ram so I expect the memory
> > usage is seriously dragging down the speed but obviously
> > the end result is a major slow down when enabling an option
> > which should be increasing performance.
> >
> > Comments, thoughts etc?
> >
> >    Regards
> >    Steve
> >
> > ================================================
> > This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the
> > event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any
> > information contained in it.
> >
> > In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
> > or return the E.mail to postmaster at multiplay.co.uk.
> >
> > _______________________________________________
> > rrd-users mailing list
> > rrd-users at lists.oetiker.ch
> > https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
> >
>
>
> ================================================
> This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it.
>
> In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
> or return the E.mail to postmaster at multiplay.co.uk.
>
> _______________________________________________
> rrd-users mailing list
> rrd-users at lists.oetiker.ch
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
>
>

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch tobi at oetiker.ch ++41 62 775 9902 / sb: -9900