Big hash question

[S. Heiling]

I have a very large hash inside a script.
The $values of the $keys can be very large.

When the script runs, it takes a long time to load the hash.
And this eats up memory...really fast!
At least I'm assuming that's what it's doing.

My question is, is there a way to stop this?
How can I keep the whole hash from loading at startup?

while ( ($key, $value) = each %hash) {

# This is what I'm using for the hash

}

Thanks.

[S. Heiling]

"Tie your hash to a file, or use a database rather than a hash."

Well this is a rather odd experiment I'm tring.
And I'm trying to work it from a single script.

"Don't load the whole hash at startup."

How can I do that? Just simply being inside the script makes it load up
memory as far as I can tell. A long - long wait depending on size. I'd like
it to not load. I've tried to put it in a sub routine as well.

"If you want us to help you not load the whole hash at start-up,
you should show us the code that loads the hash, not the code that
later iterates over the hash."

I'm not loading it. It's just there.
And the script ends by running the hash.
This is what it looks like:

#####################################
%hash = (

key0 => "very long value.......",
key1 => "valy long value.......",

);
while ( ($key, $value) = each %hash) {

# The keys are file names or directory paths.
# The values are the file data unpacked into hex.."H"

}
#####################################

I've tried putting the hash after the while statement but still, a long
wait.
This is quite unconventional, I know, but is there a way?

Thanks.

[Tad McClellan]

S. Heiling <newsreadermail@charter.net> wrote:

> And I'm trying to work it from a single script.

Perhaps you could just put the data in a __DATA__ section.

> "Don't load the whole hash at startup."
>
> How can I do that?

The post you quoted had code that does that.

I do not see that you even need a hash at all...

> %hash = (
>
> key0 => "very long value.......",
> key1 => "valy long value.......",
>
> );

Remove that code that loads the hash, put the data in __DATA__ instead.

> while ( ($key, $value) = each %hash) {
>
> # The keys are file names or directory paths.
> # The values are the file data unpacked into hex.."H"
>
> }

> I've tried putting the hash after the while statement but still, a long
> wait.

Perl compiles your entire program, so where it is located in the
source code will not matter.

> is there a way?

--------------------------------------
# untested
while ( <DATA> ) {
chomp;
my($key, $value) = split / => /;

# do stuff with only 1 pair at a time in memory
}

__DATA__
key0 => very long value.......
key1 => valy long value.......
--------------------------------------


See the "Scalar value constructors" section in perldata.pod
for info on the DATA filehandle and the __DATA__ token.


--
Tad McClellan SGML consulting
[email]tadmc@augustmail.com[/email] Perl programming
Fort Worth, Texas

[S. Heiling]

"Perhaps you could just put the data in a __DATA__ section."

Thanks. I'm working on using __DATA__, and it stops anything after that from
"compiling", but I'm having some trouble getting anything out of my hash.


"Tad McClellan" <tadmc@augustmail.com> wrote in message
news:slrnbh0ga9.anm.tadmc@magna.augustmail.com...

> S. Heiling <newsreadermail@charter.net> wrote:
>

> > And I'm trying to work it from a single script.

>
>
> Perhaps you could just put the data in a __DATA__ section.
>
>

> > "Don't load the whole hash at startup."
> >
> > How can I do that?

>
>
> The post you quoted had code that does that.
>
> I do not see that you even need a hash at all...
>
>

> > %hash = (
> >
> > key0 => "very long value.......",
> > key1 => "valy long value.......",
> >
> > );

>
>
> Remove that code that loads the hash, put the data in __DATA__ instead.
>
>

> > while ( ($key, $value) = each %hash) {
> >
> > # The keys are file names or directory paths.
> > # The values are the file data unpacked into hex.."H"
> >
> > }

>
>

> > I've tried putting the hash after the while statement but still, a long
> > wait.

>
>
> Perl compiles your entire program, so where it is located in the
> source code will not matter.
>
>

> > is there a way?

>
> --------------------------------------
> # untested
> while ( <DATA> ) {
> chomp;
> my($key, $value) = split / => /;
>
> # do stuff with only 1 pair at a time in memory
> }
>
> __DATA__
> key0 => very long value.......
> key1 => valy long value.......
> --------------------------------------
>
>
> See the "Scalar value constructors" section in perldata.pod
> for info on the DATA filehandle and the __DATA__ token.
>
>
> --
> Tad McClellan SGML consulting
> [email]tadmc@augustmail.com[/email] Perl programming
> Fort Worth, Texas

[Tad McClellan]

S. Heiling <newsreadermail@charter.net> wrote:

> "Perhaps you could just put the data in a __DATA__ section."

Who said that?

Please provide an attribution when you quote someone.

Please mark quoted text following Usenet convention, so you don't
confuse people's newsreaders.

Have you seen the Posting Guidelines that are posted here frequently?

> Thanks.

If you mean that, then please learn to compose followups properly.

Soon.



[ snip 75 lines of TOFU ]


--
Tad McClellan SGML consulting
[email]tadmc@augustmail.com[/email] Perl programming
Fort Worth, Texas

[Ingo Fellner]

"S. Heiling" <newsreadermail@charter.net> wrote in message news:<vgu39at0610747@corp.supernews.com>...

> I have a very large hash inside a script.
> The $values of the $keys can be very large.
>
> When the script runs, it takes a long time to load the hash.
> And this eats up memory...really fast!
> At least I'm assuming that's what it's doing.
>
> My question is, is there a way to stop this?
> How can I keep the whole hash from loading at startup?
>
> while ( ($key, $value) = each %hash) {
>
> # This is what I'm using for the hash
>
> }
>

foreach $key (keys %hash) {

$hash{$key} = "blabla";
# or:
print "$hash{$key}\n";

}


Rgds

Ingo ;-))

> Thanks.