Graphing/Plotting over time

[Chuck Fox]

What about using perl to massage the data into a file (or hash) and then
using gnuplot ( or Graph::Plot )

Chuck

[email]dwall@fastmail.fm[/email] wrote:

>
>
> --On Wednesday, August 27, 2003 1:29 PM -0500 "Akens, Anthony"
> <AAkens@egh.org> wrote:
>

>> Just wanted to look into a "for fun" project, after a
>> recent project that wasn't much fun at all... Our
>> organization got hit by the blaster worm, which hit
>> many, many windows boxes. The *nix boxes (which I
>> manage) were of course unaffected, except by the
>> total lack of bandwidth available to them. Except for
>> one. We have the syslog on our PIX firewall forward
>> on to one of my boxes, so I have an interested detailed
>> log of how the blaster worm spread on our network.
>>
>> So much for the history, now on to some ideas... I
>> thought it would be interesting to plot two things -
>> 1) How many hits per minute, and 2) Total compromised
>> systems over time.
>>
>> I thought of perl immediately as a good tool to break
>> this rather large file down, but being a newbie I'm
>> not sure how to begin.

>
>
> As much as I like Perl, my first thought for something like this would
> not be Perl, but some sort of statistical package that has routines
> already built-in to handle descriptive statistics. I'd use SAS, since
> I have it available at work and I'm familiar with it, but SPSS,
> Minitab, and other packages would work well, too. You might also try
> R <URL:http://www.r-project.org/>, which is an open-source (GNU Public
> License) language for statistical programming and graphics. It's
> modelled on the S language (a commercial product). (I still might use
> Perl just to extract the parts I wanted to analyze and put them in a
> format the stat package could read easily.)
>
> But if you're doing it just for practice, I won't argue. :-)
>
>
>

[K Old]

On Wed, 2003-08-27 at 14:29, Akens, Anthony wrote:

> Hello all,
>
> Just wanted to look into a "for fun" project, after a
> recent project that wasn't much fun at all... Our
> organization got hit by the blaster worm, which hit
> many, many windows boxes. The *nix boxes (which I
> manage) were of course unaffected, except by the
> total lack of bandwidth available to them. Except for
> one. We have the syslog on our PIX firewall forward
> on to one of my boxes, so I have an interested detailed
> log of how the blaster worm spread on our network.
>
> So much for the history, now on to some ideas... I
> thought it would be interesting to plot two things -
> 1) How many hits per minute, and 2) Total compromised
> systems over time.
>
> I thought of perl immediately as a good tool to break
> this rather large file down, but being a newbie I'm
> not sure how to begin. The format of each line is
> as follows (IPs changed to protect the lazy):
>
> Aug 20 16:57:28 pix %PIX-3-106011: Deny inbound (No xlate)
> icmp src inside:10.0.0.10 dst inside:10.1.1.23 (type 8, code 0)
>
>
> For the first bit I know I would need to just create a counter
> for each minute, probably using a regex to increment the counter?
>
> For the second I would need to count the source machine IPs, and
> use a hash(?) to keep track of them, and when each first appears
> in the logs, then plot that over time?

Tony,

Well, my recommendation is the GD::Graph module(look for it on CPAN).
It's a pretty good module once you get everything working behind the
scenes(check the pod docs for info on setting it up).

If you're graphing over time, you'll need to collect the data and store
it in another source (files, database, etc).

Use GD::Graph to put the calculated data into arrays/hashes and it will
build graphs and charts out of it. It handles all all different types
of charts and graphs(line, pie, bar, etc)

Hope this helps,
Kevin

>
> Can anyone give me some ideas where to start? This worm spread
> incredibly fast in our network, should be interesting to see it
> charted.
>
> Tony

--
K Old <kold@kold.homelinux.com>

[Nyimi Jose]

Found interesting info from here :
[url]http://www.manning.com/getpage.html?project=verbruggen&filename=source.ht ml[/url]

Others are welcome :-)

José.

-----Original Message-----
From: NYIMI Jose (BMB)
Sent: Friday, August 29, 2003 10:44 AM
To: K Old; Akens, Anthony
Cc: [email]beginners@perl.org[/email]
Subject: RE: Graphing/Plotting over time


Cool !

I was just dealing with following needs:

* parse a text files on daily basis
* put the result into database
* buit a web application upon above database
* provide the client a web form where he can choose which graph to see
* on submit, connect to database, plot the graph and render it to the client as an image

For the time beeing, i'm busy writing the parser using Parse::RecDescent module. The text file is a little bit complex, so we choose to parse it via buiding grammar. The second part will be, how to render the graph to the web client.

Is GD::Graph module sufficient to handle above requirements ?

Any ideas/advices is welcome.

Thanks in advance.

José.

-----Original Message-----
From: K Old [mailto:kold@kold.homelinux.com]
Sent: Friday, August 29, 2003 5:49 AM
To: Akens, Anthony
Cc: [email]beginners@perl.org[/email]
Subject: Re: Graphing/Plotting over time


On Wed, 2003-08-27 at 14:29, Akens, Anthony wrote:

> Hello all,
>
> Just wanted to look into a "for fun" project, after a
> recent project that wasn't much fun at all... Our organization got
> hit by the blaster worm, which hit many, many windows boxes. The *nix
> boxes (which I
> manage) were of course unaffected, except by the
> total lack of bandwidth available to them. Except for
> one. We have the syslog on our PIX fire
> For the second I would need to count the source machine IPs, and use a
> hash(?) to keep track of them, and when each first appears in the
> logs, then plot that over time?

Tony,

Well, my recommendation is the GD::Graph module(look for it on CPAN).
It's a pretty good module once you get everything working behind the scenes(check the pod docs for info on setting it up).

If you're graphing over time, you'll need to collect the data and store it in another source (files, database, etc).

Use GD::Graph to put the calculated data into arrays/hashes and it will build graphs and charts out of it. It handles all all different types of charts and graphs(line, pie, bar, etc)

Hope this helps,
Kevin

>
> Can anyone give me some ideas where to start? This worm spread
> incredibly fast in our network, should be interesting to see it
> charted.
>
> Tony

--
K Old <kold@kold.homelinux.com>


--
To unsubscribe, e-mail: [email]beginners-unsubscribe@perl.org[/email]
For additional commands, e-mail: [email]beginners-help@perl.org[/email]



**** DISCLAIMER ****

"This e-mail and any attachment thereto may contain information which is confidential and/or protected by intellectual property rights and are intended for the sole use of the recipient(s) named above.
Any use of the information contained herein (including, but not limited to, total or partial reproduction, communication or distribution in any form) by other persons than the designated recipient(s) is prohibited.
If you have received this e-mail in error, please notify the sender either by telephone or by e-mail and delete the material from any computer".

Thank you for your cooperation.

For further information about Proximus mobile phone services please see our website at [url]http://www.proximus.be[/url] or refer to any Proximus agent.


--
To unsubscribe, e-mail: [email]beginners-unsubscribe@perl.org[/email]
For additional commands, e-mail: [email]beginners-help@perl.org[/email]