data filtering

[Guruguhan]

Hi All,
I have a data set ( only a portion) as follows:
No. x1 x2 x3 y1 y2 y3 y4
1 0.200000 0.200000 0.200000 0.765685 75881.9 29289.3 -46592.6
2 0.200345 0.200000 0.200345 0.766661 75766.0 29268.4 -46497.6
3 0.200000 0.200345 0.200000 0.766030 75867.1 29259.8 -46607.4
4 0.359575 0.253987 0.359575 1.271019 43898.7 19675.6 -24223.1
5 0.359921 0.253987 0.359921 1.271995 43861.3 19666.1 -24195.2

This data set has to be filtered using the following conditions ( example only. in reality this can be anything):

0.2 < x1 < 0.3
y1 > 0.8
15000 < y3 < 30000

I have to filter the data set satisfying all the conditions.

Can somebody help me out to do the same. Any help in this regard is solicited.

Thanks
Regards
Guruguhan
EACoE, India.

[David]

Guruguhan,

Can I assume you have the data in memory?
Can I further assume the data is contained in an array of arrays?
Can I also further assume that the conditions you mentioned are AND'd?
(ie
the final result is the intersection of all conditions?)

Thus $data[0][0] = 0.200000 (aka, X1 from row 1)

Consider the following: (purposely written in readable perl)

my @filtered = ();
for( @data ){
push @filtered,$_ if
$$_[0] > 0.2 && $$_[0] < 0.3 && # 0.2 < x1 < 0.3
$$_[3] > 0.8 && # y1 > 0.8
$$_[5] > 15000 && $$_[5] < 30000; # 15000 < y3 < 30000
}

#@filtered data contains filtered result result.

Is that what you were thinking of?

Regards,

David le Blanc

--
Senior Technical Specialist
I d e n t i t y S o l u t i o n s

Level 1, 369 Camberwell Road, Melbourne, Vic 3124
Ph 03 9813 1388 Fax 03 9813 1688 Mobile 0417 595 550
Email com.au

-----Original Message-----
From: N, Guruguhan (GEAE, Foreign National, EACOE)
[mailto:ge.com]
Sent: Monday, 16 February 2004 10:47 PM
To: org
Subject: data filtering

Hi All,
I have a data set ( only a portion) as follows:
No. x1 x2 x3 y1 y2 y3
y4
1 0.200000 0.200000 0.200000 0.765685 75881.9 29289.3
-46592.6
2 0.200345 0.200000 0.200345 0.766661 75766.0 29268.4
-46497.6
3 0.200000 0.200345 0.200000 0.766030 75867.1 29259.8
-46607.4
4 0.359575 0.253987 0.359575 1.271019 43898.7 19675.6
-24223.1
5 0.359921 0.253987 0.359921 1.271995 43861.3 19666.1
-24195.2

This data set has to be filtered using the following conditions (
example only. in reality this can be anything):

0.2 < x1 < 0.3
y1 > 0.8
15000 < y3 < 30000

I have to filter the data set satisfying all the conditions.

Can somebody help me out to do the same. Any help in this regard is
solicited.

Thanks
Regards
Guruguhan
EACoE, India.


--
To unsubscribe, e-mail: org For additional
commands, e-mail: org <http://learn.perl.org/>
<http://learn.perl.org/first-response>

[Rob]

Guruguhan N wrote: 

Hi.

Unfortunately none of your data satisfy the criteria! But taking just the
limits on x1 the program below should help. It works by grabbing the column
headers and using them as the keys of a hash for the following data. That allows
you to write the conditions in meaningful terms.

I hope it helps,

Rob



use strict;
use warnings;

my @keys = split ' ', <DATA>;

my %data;

while (<DATA>) {

@data{@keys} = split ' ';

next unless
0.2 < $data{x1} && $data{x1} < 0.3; # &&
# $data{y1} > 0.8;
# 15000 < $data{y3} && $data{y3} < 30000;

print "@data{@keys}\n";

}

__DATA__
No. x1 x2 x3 y1 y2 y3 y4
1 0.200000 0.200000 0.200000 0.765685 75881.9 29289.3 -46592.6
2 0.200345 0.200000 0.200345 0.766661 75766.0 29268.4 -46497.6
3 0.200000 0.200345 0.200000 0.766030 75867.1 29259.8 -46607.4
4 0.359575 0.253987 0.359575 1.271019 43898.7 19675.6 -24223.1
5 0.359921 0.253987 0.359921 1.271995 43861.3 19666.1 -24195.2

**OUTPUT

2 0.200345 0.200000 0.200345 0.766661 75766.0 29268.4 -46497.6

[R.]

David le Blanc wrote:
 

I'm afraid the one assumption that you have not specified is the killer
here--that the conditions per field are a constant. The OP indicated below [why
below? please bottom-post] that the conditions could be anything. Therefore a
solution will have to be sufficiently flexible to take an arbitrary number of
rules for any column.

I think your third assumption is pretty fair. The others can be adjusted fairly
readily, but the and condition would be crucial to the logic. So the OP needs
at least two stages for this process. The first would store a hash or array
references keyed to column name. Each rule to be applied would be pushed into
the array for the relevant column during the first Of course, he might also
need an array for multifield functions if any conditions are based on
relationships between fields.


Then of course, the second phase would iterate through the records, and reject
any record as soon as it finds one condition that is not met.

Joseph

[David]

Aha. The problem here is
*how do I implement a parser to process a table of data based on a user
supplied query*

Ok, look at the following:-

#!/usr/bin/perl

use strict; - matter of style
use warnings; - and again

# This DATASET has the following columns, with these names in this
order.
my @col = qw( x1 x2 x3 y1 y2 y3 y4 );

# Fabricate a set of functions (subs) in the 'filter' name space which
parallel
# the column names, and enchant each with the ability to look up a data
element.
# Note the use of 'my $i=$_' This tweaks some magic in perl and causes
the automatically
# generated functions to be *closures* and not just normal functions.

map{ my $i=$_; no strict 'refs'; *{"filter::$col[$i]"} = sub{$_->[$i]} }
0..$#col;

# This is your data set. If you want 'line number as the first col
remember to add it to
# '@col' above.

my @data=(
[qw( 0.200000 0.200000 0.200000 0.765685 75881.9
29289.3 -46592.6)],
[qw( 0.200345 0.200000 0.200345 0.966661 75766.0
29268.4 -46497.6)],
[qw( 0.200000 0.200345 0.200000 0.766030 75867.1
29259.8 -46607.4)],
[qw( 0.359575 0.253987 0.359575 1.271019 43898.7
19675.6 -24223.1)],
[qw( 0.359921 0.253987 0.359921 1.271995 43861.3
19666.1 -24195.2)]
);

# Perform filtering. First parameter is a $coderef which is passed to
GREP for the
# actual filtering task.
sub filterdata($@)
{
my $cref=shift;
grep &$cref,@_
}

# Build the coderef for the specified data filter. The (only) parameter
is the
# text string 'x1 > 5' etc.
sub filterspec
{
my $e = shift;
map{ $e =~ s/\b$_\b/filter::$_()/g } @col;
eval "sub{ $e }"
}

# Here goes.......

# I want to filter according to the following spec:
my $expr = "x1 > 0.2 && x1 < 0.3 && y1 > 0.8 && y3 > 15000 && y3 <
30000";

# Compile the spec into a handler... The DIE will catch errors compiling
the filter.
# This usually occurs because you do silly things, call non-existent
functions, or
# use column names which don't exist.

my $filter_handler = filterspec( $expr ) or die $@;

# Perform the filter using the handler.
my @result = filterdata $filter_handler,@data;

# Display the table of results.
print join("$/",map{join(" ",@$_)}@result).$/;

This outputs one row:
0.200345 0.200000 0.200345 0.966661 75766.0 29268.4 -46497.6

Now remember. I fiddled the data and expression so they that WOULD match
one line. Your sample data matched nothing.

How does this help? I'm sorry if it is less readable than the specific
implementation of a particular filter.
------------------------------------------------------------------------
----------------------------------

Thanks Mr.David for the response.
I want the same logic in a generalized manner, because the conditions
keep changing as the data changes. Also in each condition string, the
delimiters can be anything (like <, >, <=, >=, or =). Hence I would
like to have a more generalized way.

Your solution has given me a way to think further ahead. Further help in
this regard is solicited.

Thanks
Regards
Guruguhan
EACoE, India.

-----Original Message-----
From: David le Blanc [mailto:com.au]
Sent: Monday, February 16, 2004 6:34 PM
To: N, Guruguhan (GEAE, Foreign National, EACOE); org
Subject: RE: data filtering



Guruguhan,

Can I assume you have the data in memory?
Can I further assume the data is contained in an array of arrays?
Can I also further assume that the conditions you mentioned are AND'd?
(ie
the final result is the intersection of all conditions?)

Thus $data[0][0] = 0.200000 (aka, X1 from row 1)

Consider the following: (purposely written in readable perl)

my @filtered = ();
for( @data ){
push @filtered,$_ if
$$_[0] > 0.2 && $$_[0] < 0.3 && # 0.2 < x1 < 0.3
$$_[3] > 0.8 && # y1 > 0.8
$$_[5] > 15000 && $$_[5] < 30000; # 15000 < y3 < 30000
}

#@filtered data contains filtered result result.

Is that what you were thinking of?

Regards,

David le Blanc

--
Senior Technical Specialist
I d e n t i t y S o l u t i o n s

Level 1, 369 Camberwell Road, Melbourne, Vic 3124
Ph 03 9813 1388 Fax 03 9813 1688 Mobile 0417 595 550
Email com.au

-----Original Message-----
From: N, Guruguhan (GEAE, Foreign National, EACOE)
[mailto:ge.com]
Sent: Monday, 16 February 2004 10:47 PM
To: org
Subject: data filtering

Hi All,
I have a data set ( only a portion) as follows:
No. x1 x2 x3 y1 y2 y3
y4
1 0.200000 0.200000 0.200000 0.765685 75881.9 29289.3
-46592.6
2 0.200345 0.200000 0.200345 0.766661 75766.0 29268.4
-46497.6
3 0.200000 0.200345 0.200000 0.766030 75867.1 29259.8
-46607.4
4 0.359575 0.253987 0.359575 1.271019 43898.7 19675.6
-24223.1
5 0.359921 0.253987 0.359921 1.271995 43861.3 19666.1
-24195.2

This data set has to be filtered using the following conditions (
example only. in reality this can be anything):

0.2 < x1 < 0.3
y1 > 0.8
15000 < y3 < 30000

I have to filter the data set satisfying all the conditions.

Can somebody help me out to do the same. Any help in this regard is
solicited.

Thanks
Regards
Guruguhan
EACoE, India.


--
To unsubscribe, e-mail: org For additional
commands, e-mail: org <http://learn.perl.org/>
<http://learn.perl.org/first-response>

[David]

What are you expecting the user to enter as a requirement to filter the
data?

I think what you want is to make the expression *less* generic. Ie,
make it specific
to how your users enter filter requirements.

If this is so, please elaborate.


-----Original Message-----
From: N, Guruguhan (GEAE, Foreign National, EACOE)
[mailto:ge.com]
Sent: Tuesday, 17 February 2004 11:29 PM
To: David le Blanc
Subject: RE: data filtering

Mr.David,
Once again I thank for your valuable suggestion. One more
help I need from you. I would like to know how can I make this part of
your code more generic?

# I want to filter according to the following spec:
my $expr = "x1 > 0.2 && x1 < 0.3 && y1 > 0.8 && y3 > 15000 && y3 <
30000";

As this is going to be supplied by user, I will not know information
about filters beforehand. The filters can be typically anything. I would
like to know, how can I group all the filters into a single variable
($expr) as you have done, if the filter expression keeps changing?

I hope I am clear to you. Please help me out to do this also.


Thanks
Regards
Guruguhan
EACoE, India.
* - *91-80-5031516
* - *901-1516 ( Dial Com)


-----Original Message-----
From: David le Blanc [mailto:com.au]
Sent: Tuesday, February 17, 2004 10:02 AM
To: N, Guruguhan (GEAE, Foreign National, EACOE); org
Subject: RE: data filtering



Aha. The problem here is
*how do I implement a parser to process a table of data based on a user
supplied query*

Ok, look at the following:-

#!/usr/bin/perl

use strict; - matter of style
use warnings; - and again

# This DATASET has the following columns, with these names in this
order.
my @col = qw( x1 x2 x3 y1 y2 y3 y4 );

# Fabricate a set of functions (subs) in the 'filter' name space which
parallel # the column names, and enchant each with the ability to look
up a data element.
# Note the use of 'my $i=$_' This tweaks some magic in perl and causes
the automatically # generated functions to be *closures* and not just
normal functions.

map{ my $i=$_; no strict 'refs'; *{"filter::$col[$i]"} = sub{$_->[$i]} }
0..$#col;

# This is your data set. If you want 'line number as the first col
remember to add it to # '@col' above.

my @data=(
[qw( 0.200000 0.200000 0.200000 0.765685 75881.9
29289.3 -46592.6)],
[qw( 0.200345 0.200000 0.200345 0.966661 75766.0
29268.4 -46497.6)],
[qw( 0.200000 0.200345 0.200000 0.766030 75867.1
29259.8 -46607.4)],
[qw( 0.359575 0.253987 0.359575 1.271019 43898.7
19675.6 -24223.1)],
[qw( 0.359921 0.253987 0.359921 1.271995 43861.3
19666.1 -24195.2)]
);

# Perform filtering. First parameter is a $coderef which is passed to
GREP for the # actual filtering task.
sub filterdata($@)
{
my $cref=shift;
grep &$cref,@_
}

# Build the coderef for the specified data filter. The (only) parameter
is the # text string 'x1 > 5' etc.
sub filterspec
{
my $e = shift;
map{ $e =~ s/\b$_\b/filter::$_()/g } @col;
eval "sub{ $e }"
}

# Here goes.......

# I want to filter according to the following spec:
my $expr = "x1 > 0.2 && x1 < 0.3 && y1 > 0.8 && y3 > 15000 && y3 <
30000";

# Compile the spec into a handler... The DIE will catch errors compiling
the filter.
# This usually occurs because you do silly things, call non-existent
functions, or # use column names which don't exist.

my $filter_handler = filterspec( $expr ) or die $@;

# Perform the filter using the handler.
my @result = filterdata $filter_handler,@data;

# Display the table of results.
print join("$/",map{join(" ",@$_)}@result).$/;

This outputs one row:
0.200345 0.200000 0.200345 0.966661 75766.0 29268.4 -46497.6

Now remember. I fiddled the data and expression so they that WOULD match
one line. Your sample data matched nothing.

How does this help? I'm sorry if it is less readable than the specific
implementation of a particular filter.
------------------------------------------------------------------------
----------------------------------

Thanks Mr.David for the response.
I want the same logic in a generalized manner, because the conditions
keep changing as the data changes. Also in each condition string, the
delimiters can be anything (like <, >, <=, >=, or =). Hence I would
like to have a more generalized way.

Your solution has given me a way to think further ahead. Further help in
this regard is solicited.

Thanks
Regards
Guruguhan
EACoE, India.

-----Original Message-----
From: David le Blanc [mailto:com.au]
Sent: Monday, February 16, 2004 6:34 PM
To: N, Guruguhan (GEAE, Foreign National, EACOE); org
Subject: RE: data filtering



Guruguhan,

Can I assume you have the data in memory?
Can I further assume the data is contained in an array of arrays?
Can I also further assume that the conditions you mentioned are AND'd?
(ie
the final result is the intersection of all conditions?)

Thus $data[0][0] = 0.200000 (aka, X1 from row 1)

Consider the following: (purposely written in readable perl)

my @filtered = ();
for( @data ){
push @filtered,$_ if
$$_[0] > 0.2 && $$_[0] < 0.3 && # 0.2 < x1 < 0.3
$$_[3] > 0.8 && # y1 > 0.8
$$_[5] > 15000 && $$_[5] < 30000; # 15000 < y3 < 30000
}

#@filtered data contains filtered result result.

Is that what you were thinking of?

Regards,

David le Blanc

--
Senior Technical Specialist
I d e n t i t y S o l u t i o n s

Level 1, 369 Camberwell Road, Melbourne, Vic 3124
Ph 03 9813 1388 Fax 03 9813 1688 Mobile 0417 595 550
Email com.au

-----Original Message-----
From: N, Guruguhan (GEAE, Foreign National, EACOE)
[mailto:ge.com]
Sent: Monday, 16 February 2004 10:47 PM
To: org
Subject: data filtering

Hi All,
I have a data set ( only a portion) as follows:
No. x1 x2 x3 y1 y2 y3
y4
1 0.200000 0.200000 0.200000 0.765685 75881.9 29289.3
-46592.6
2 0.200345 0.200000 0.200345 0.766661 75766.0 29268.4
-46497.6
3 0.200000 0.200345 0.200000 0.766030 75867.1 29259.8
-46607.4
4 0.359575 0.253987 0.359575 1.271019 43898.7 19675.6
-24223.1
5 0.359921 0.253987 0.359921 1.271995 43861.3 19666.1
-24195.2

This data set has to be filtered using the following conditions (
example only. in reality this can be anything):

0.2 < x1 < 0.3
y1 > 0.8
15000 < y3 < 30000

I have to filter the data set satisfying all the conditions.

Can somebody help me out to do the same. Any help in this regard is
solicited.

Thanks
Regards
Guruguhan
EACoE, India.


--
To unsubscribe, e-mail: org For additional
commands, e-mail: org <http://learn.perl.org/>
<http://learn.perl.org/first-response>

[David]

Based on what you have described, the code below will generate an
expression string
from a series of filter requirements.


#!/usr/bin/perl

sub expr_tostr{
(my $expr = shift) =~s/\s+//g;
@c = split /([<=>]+)/, $expr;
@flop{("<","<=",">",">=")}=(">",">=","<","<=");
@c = (
"(",$c[2],$c[3],$c[4],")&&(",$c[2],$flop{$c[1]},$c[0],")") if @c == 5;
join("",("(",map{${_}eq"="?"==":$_}@c,")"));
}

print "expression ? ";
while(<>){
chomp;
last unless $_;
push @filters, expr_tostr($_);
} continue {
print "expression ? ";
}

$expr = join("&&",@filters);

print "Filter string is $expr".$/;

Eg,

../build_expr
expression ? 0.2 < x1 < 0.3 <enter>
expression ? y1 > 0.8 <enter>
expression ? 15000 < y3 < 30000 <enter>
expression ? <enter on blank line>

Filter string is
((x1<0.3)&&(x1>0.2))&&(y1>0.8)&&((y3<30000)&&(y3>1 5000))

The resulting filter string can be used directly to build a filter spec
in the script below.
The only complication in the above script is making sure x=5 is
translated into x==5

Secondly, you ask:

Also can you please explain me in words what does this line of your code
means? I am fairly new to this concept.

map{ my $i=$_; no strict 'refs'; *{"filter::$col[$i]"} = sub{$_->[$i]} }
0..$#col;

Translates to:
for(0 .. @col-1 ) {
my $axis=$col[$_];
my $index = $_;

my $code = sub { $_->[$index] }

*{"filter::$axis"} = $code
}

Which in perlglish sound like:
define an anonymous function referring to a closure ($index)
which
contains the index which this function will use to lookup the
data.

Assign the anonymous function to a package symbol by the same
name
as the axis, in the package 'filter'.

Thus, if @col = ( x1 ), then there is a function called 'x1()' in the
'filter'
package which can retrieve the data for column 'x1' from a row of data.

You will note that in the later code, names in @col (such as x1) are
translated to
filter::x1() before compiling to remove any ambiguity.

Also note that function calls to 'cos()' or 'sin()' or 'system("halt")'
will
also execute correctly. Consider using the Safe.pm wrapper to make
sure the
expression runs in a restricted sandbox when it does get run.

Cheers.
David




-----Original Message-----
From: N, Guruguhan (GEAE, Foreign National, EACOE)
[mailto:ge.com]
Sent: Wednesday, 18 February 2004 12:07 AM
To: David le Blanc
Subject: RE: data filtering

Mr.David,
The user has the flexibility to 1. Give the number of
filters i.e. the user can specify one or more filters 2. The specified
filters can be either one sided or two sided i.e. "x1 >5" or "3< x1 <5"
3. The delimiters can be <, <=, >, >=, =, or a combination of these.

Also can you please explain me in words what does this line of your code
means? I am fairly new to this concept.

map{ my $i=$_; no strict 'refs'; *{"filter::$col[$i]"} = sub{$_->[$i]} }
0..$#col;

I hope I am clear to you. Once again I thank you for your interest and
help rendered to me.

Thanks
Regards
Guruguhan
EACoE, India.
* - *91-80-5031516
* - *901-1516 ( Dial Com)


-----Original Message-----
From: David le Blanc [mailto:com.au]
Sent: Tuesday, February 17, 2004 6:44 PM
To: N, Guruguhan (GEAE, Foreign National, EACOE); org
Subject: RE: data filtering




What are you expecting the user to enter as a requirement to filter the
data?

I think what you want is to make the expression *less* generic. Ie,
make it specific to how your users enter filter requirements.

If this is so, please elaborate.


-----Original Message-----
From: N, Guruguhan (GEAE, Foreign National, EACOE)
[mailto:ge.com]
Sent: Tuesday, 17 February 2004 11:29 PM
To: David le Blanc
Subject: RE: data filtering

Mr.David,
Once again I thank for your valuable suggestion. One more
help I need from you. I would like to know how can I make this part of
your code more generic?

# I want to filter according to the following spec:
my $expr = "x1 > 0.2 && x1 < 0.3 && y1 > 0.8 && y3 > 15000 && y3 <
30000";

As this is going to be supplied by user, I will not know information
about filters beforehand. The filters can be typically anything. I would
like to know, how can I group all the filters into a single variable
($expr) as you have done, if the filter expression keeps changing?

I hope I am clear to you. Please help me out to do this also.


Thanks
Regards
Guruguhan
EACoE, India.
* - *91-80-5031516
* - *901-1516 ( Dial Com)


-----Original Message-----
From: David le Blanc [mailto:com.au]
Sent: Tuesday, February 17, 2004 10:02 AM
To: N, Guruguhan (GEAE, Foreign National, EACOE); org
Subject: RE: data filtering



Aha. The problem here is
*how do I implement a parser to process a table of data based on a user
supplied query*

Ok, look at the following:-

#!/usr/bin/perl

use strict; - matter of style
use warnings; - and again

# This DATASET has the following columns, with these names in this
order.
my @col = qw( x1 x2 x3 y1 y2 y3 y4 );

# Fabricate a set of functions (subs) in the 'filter' name space which
parallel # the column names, and enchant each with the ability to look
up a data element.
# Note the use of 'my $i=$_' This tweaks some magic in perl and causes
the automatically # generated functions to be *closures* and not just
normal functions.

map{ my $i=$_; no strict 'refs'; *{"filter::$col[$i]"} = sub{$_->[$i]} }
0..$#col;

# This is your data set. If you want 'line number as the first col
remember to add it to # '@col' above.

my @data=(
[qw( 0.200000 0.200000 0.200000 0.765685 75881.9
29289.3 -46592.6)],
[qw( 0.200345 0.200000 0.200345 0.966661 75766.0
29268.4 -46497.6)],
[qw( 0.200000 0.200345 0.200000 0.766030 75867.1
29259.8 -46607.4)],
[qw( 0.359575 0.253987 0.359575 1.271019 43898.7
19675.6 -24223.1)],
[qw( 0.359921 0.253987 0.359921 1.271995 43861.3
19666.1 -24195.2)]
);

# Perform filtering. First parameter is a $coderef which is passed to
GREP for the # actual filtering task.
sub filterdata($@)
{
my $cref=shift;
grep &$cref,@_
}

# Build the coderef for the specified data filter. The (only) parameter
is the # text string 'x1 > 5' etc.
sub filterspec
{
my $e = shift;
map{ $e =~ s/\b$_\b/filter::$_()/g } @col;
eval "sub{ $e }"
}

# Here goes.......

# I want to filter according to the following spec:
my $expr = "x1 > 0.2 && x1 < 0.3 && y1 > 0.8 && y3 > 15000 && y3 <
30000";

# Compile the spec into a handler... The DIE will catch errors compiling
the filter.
# This usually occurs because you do silly things, call non-existent
functions, or # use column names which don't exist.

my $filter_handler = filterspec( $expr ) or die $@;

# Perform the filter using the handler.
my @result = filterdata $filter_handler,@data;

# Display the table of results.
print join("$/",map{join(" ",@$_)}@result).$/;

This outputs one row:
0.200345 0.200000 0.200345 0.966661 75766.0 29268.4 -46497.6

Now remember. I fiddled the data and expression so they that WOULD match
one line. Your sample data matched nothing.

How does this help? I'm sorry if it is less readable than the specific
implementation of a particular filter.
------------------------------------------------------------------------
----------------------------------

Thanks Mr.David for the response.
I want the same logic in a generalized manner, because the conditions
keep changing as the data changes. Also in each condition string, the
delimiters can be anything (like <, >, <=, >=, or =). Hence I would
like to have a more generalized way.

Your solution has given me a way to think further ahead. Further help in
this regard is solicited.

Thanks
Regards
Guruguhan
EACoE, India.

-----Original Message-----
From: David le Blanc [mailto:com.au]
Sent: Monday, February 16, 2004 6:34 PM
To: N, Guruguhan (GEAE, Foreign National, EACOE); org
Subject: RE: data filtering



Guruguhan,

Can I assume you have the data in memory?
Can I further assume the data is contained in an array of arrays?
Can I also further assume that the conditions you mentioned are AND'd?
(ie
the final result is the intersection of all conditions?)

Thus $data[0][0] = 0.200000 (aka, X1 from row 1)

Consider the following: (purposely written in readable perl)

my @filtered = ();
for( @data ){
push @filtered,$_ if
$$_[0] > 0.2 && $$_[0] < 0.3 && # 0.2 < x1 < 0.3
$$_[3] > 0.8 && # y1 > 0.8
$$_[5] > 15000 && $$_[5] < 30000; # 15000 < y3 < 30000
}

#@filtered data contains filtered result result.

Is that what you were thinking of?

Regards,

David le Blanc

--
Senior Technical Specialist
I d e n t i t y S o l u t i o n s

Level 1, 369 Camberwell Road, Melbourne, Vic 3124
Ph 03 9813 1388 Fax 03 9813 1688 Mobile 0417 595 550
Email com.au

-----Original Message-----
From: N, Guruguhan (GEAE, Foreign National, EACOE)
[mailto:ge.com]
Sent: Monday, 16 February 2004 10:47 PM
To: org
Subject: data filtering

Hi All,
I have a data set ( only a portion) as follows:
No. x1 x2 x3 y1 y2 y3
y4
1 0.200000 0.200000 0.200000 0.765685 75881.9 29289.3
-46592.6
2 0.200345 0.200000 0.200345 0.766661 75766.0 29268.4
-46497.6
3 0.200000 0.200345 0.200000 0.766030 75867.1 29259.8
-46607.4
4 0.359575 0.253987 0.359575 1.271019 43898.7 19675.6
-24223.1
5 0.359921 0.253987 0.359921 1.271995 43861.3 19666.1
-24195.2

This data set has to be filtered using the following conditions (
example only. in reality this can be anything):

0.2 < x1 < 0.3
y1 > 0.8
15000 < y3 < 30000

I have to filter the data set satisfying all the conditions.

Can somebody help me out to do the same. Any help in this regard is
solicited.

Thanks
Regards
Guruguhan
EACoE, India.


--
To unsubscribe, e-mail: org For additional
commands, e-mail: org <http://learn.perl.org/>
<http://learn.perl.org/first-response>