[rrd-users] How to Create the rrd that storeor update million of requests per second

[Alex van den Bogaerdt]

----- Original Message ----- 
From: "Kashif Qazi" <kaqazi at etilizepak.com>
To: <rrd-users at lists.oetiker.ch>
Sent: Friday, February 13, 2009 7:25 AM
Subject: [rrd-users] How to Create the rrd that storeor update million of 
requests per second


>
> Hi All,
>
> I have to store each request data to rrd database. the request granularity
> can be more or less than one million hits per second.
> can rrd support granularity in milli/micro seconds.

It will do sub-second precision when updating, but not in DSes and RRAs. In 
other words: it will take the average (or max, min) of these updates.

> 1) How much time the request takes on average for each: minute, hour, work
> day (6am-8pm), day (24 hour), week, month

That's not a million per second, is it?

Are you perhaps saying that in a certain second you could get many (that 
million) requests but you do not really care about which request came first, 
and how many microseconds before the next?

> 2) Scatter graph of all requests so I can plot the time of every request
> over a period of time (day, week, month, year) and see where the majority 
> of
> requests perform and what the outliers are (really slow, really fast).

I don't think rrdtool can do this, or it has to be a new feature which I did 
not yet play with.

> 3) How many requests per second, minute, hour, word day, day, week, month
>
> 4) Number of requests in each “performance group”. For example, how many
> requests are 0-100ms, 101-200ms, 201-300ms, 301-400ms, 401-500ms, etc ...
>
> 5) When am I having requests > 1000ms? < 5000ms?  What is associated with
> these? Certain times of day? Certain traffic volumes?

Easy, easy, easy.

I don't think rrdtool is the right tool for you, even if you could work 
around certain issues.  It comes close though, so if you have not given a 
correct problem description, rrdtool may prove to be useful after all. Only 
that scatter graph will be a problem (unless, again, there is a new feature 
I have not yet explored).

HTH
Alex