I killed my system with grep

[Ramiro Aceves]

Hello FreeBSD friends:

I am running a FreeBSD 5.3 system with 64MB RAM and 150 MB swap.

Yesterday I entered the command:

# grep -R something /

and after a while, my system did not respond. I do not remember the exact
messages as I am on a winbugs at the University. The error was about
swapping. I could switch among terminals but the system was dead. I needed
to reboot.

I rebooted and tried again watching "top" output and I could see as swap
usage was incresing very quickly until it ran out of swap space and the swap
pager failed.

Was my sytem dead? or, is it possible to recover from that state without
rebooting? How is it possible that a simple command like this could
auto-kill the machine?

What is the recomended fix for this?:

a- Asigning more swap.
b- Not executing that command anymore.


Thank you very much for your advices and help.

Ramiro

[Lowell Gilbert]

"Ramiro Aceves" <ea1abz@wanadoo.es> writes:

> Hello FreeBSD friends:
>
> I am running a FreeBSD 5.3 system with 64MB RAM and 150 MB swap.
>
> Yesterday I entered the command:
>
> # grep -R something /
>
> and after a while, my system did not respond. I do not remember the exact
> messages as I am on a winbugs at the University. The error was about
> swapping. I could switch among terminals but the system was dead. I needed
> to reboot.
>
> I rebooted and tried again watching "top" output and I could see as swap
> usage was incresing very quickly until it ran out of swap space and the swap
> pager failed.
>
> Was my sytem dead? or, is it possible to recover from that state without
> rebooting? How is it possible that a simple command like this could
> auto-kill the machine?
>
> What is the recomended fix for this?:
>
> a- Asigning more swap.
> b- Not executing that command anymore.

c- Setting user limits in login.conf(5).

[Loren M. Lang]

On Fri, Feb 25, 2005 at 12:14:04PM +0100, Ramiro Aceves wrote:

> Hello FreeBSD friends:
>
> I am running a FreeBSD 5.3 system with 64MB RAM and 150 MB swap.
>
> Yesterday I entered the command:
>
> # grep -R something /

Running a grep on an entire system as root is a bad idea. At least
limit to certain filesystems. You probably hit a file under /dev/ and
caused grep to hang. It's possible that as root, certain device files
might hang the system, but nothing comes to mind at the moment unless
/dev/io could do it. Also, think about what happens when grep hit's
/dev/zero. It will never finish.

>
> and after a while, my system did not respond. I do not remember the exact
> messages as I am on a winbugs at the University. The error was about
> swapping. I could switch among terminals but the system was dead. I needed
> to reboot.
>
> I rebooted and tried again watching "top" output and I could see as swap
> usage was incresing very quickly until it ran out of swap space and the swap
> pager failed.
>
> Was my sytem dead? or, is it possible to recover from that state without
> rebooting? How is it possible that a simple command like this could
> auto-kill the machine?
>
> What is the recomended fix for this?:
>
> a- Asigning more swap.
> b- Not executing that command anymore.
>
>
> Thank you very much for your advices and help.
>
> Ramiro
>
>
> _______________________________________________
> [email]freebsd-questions@freebsd.org[/email] mailing list
> [url]http://lists.freebsd.org/mailman/listinfo/freebsd-questions[/url]
> To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org"

--
I sense much NT in you.
NT leads to Bluescreen.
Bluescreen leads to downtime.
Downtime leads to suffering.
NT is the path to the darkside.
Powerful Unix is.

Public Key: [url]ftp://ftp.tallye.com/pub/lorenl_pubkey.asc[/url]
Fingerprint: B3B9 D669 69C9 09EC 1BCD 835A FAF3 7A46 E4A3 280C

[Parv]

in message <20050225135707.GC18789@alzatex.com>,
wrote Loren M. Lang thusly...

>
> On Fri, Feb 25, 2005 at 12:14:04PM +0100, Ramiro Aceves wrote:

> >
> > I am running a FreeBSD 5.3 system with 64MB RAM and 150 MB swap.
> >
> > Yesterday I entered the command:
> >
> > # grep -R something /

>
> You probably hit a file under /dev/ and caused grep to hang. It's
> possible that as root, certain device files might hang the system,
> but nothing comes to mind at the moment unless /dev/io could do
> it. Also, think about what happens when grep hit's /dev/zero. It
> will never finish.

Would using -I option (not search text-like files) help to avoid
above described hang ups in /dev?


- Parv

--

[John]

On Fri, 25 Feb 2005 12:14:04 +0100, Ramiro Aceves wrote

> Hello FreeBSD friends:
>
> I am running a FreeBSD 5.3 system with 64MB RAM and 150 MB swap.
>
> Yesterday I entered the command:
>
> # grep -R something /
>
> and after a while, my system did not respond. I do not remember the exact
> messages as I am on a winbugs at the University. The error was about
> swapping. I could switch among terminals but the system was dead. I needed
> to reboot.
>
> I rebooted and tried again watching "top" output and I could see as swap
> usage was incresing very quickly until it ran out of swap space and
> the swap pager failed.
>
> Was my sytem dead? or, is it possible to recover from that state without
> rebooting? How is it possible that a simple command like this could
> auto-kill the machine?
>
> What is the recomended fix for this?:
>
> a- Asigning more swap.
> b- Not executing that command anymore.

I had a very similar problem. I fixed it by reading up on auxillary swapfile.
I set one of those up as indicated, at about 2.5x RAM (750MB, roughly) and the
problem went away after I edited /etc/fstab and commented out the old swap
entry, then rebooted.

The problem was caused by the hard drive having failing blocks where the swap
partition resides. I got a few errors in /var/log/messages where the swap
daemon couldnt zero pages or inodes or whatever. If the swapper cant liberate
old pages then swap will fill up. My machine uses a lot of perl, and
openwebmail makes it swap under load.

It seems that the bad blocks are only on the old swap so hopefully the hard
drive still has some life in it yet.
--
[email]lists@reiteration.net[/email]

[Ramiro Aceves]

Ramiro Aceves wrote:

> Hello FreeBSD friends:
>
> I am running a FreeBSD 5.3 system with 64MB RAM and 150 MB swap.
>
> Yesterday I entered the command:
>
> # grep -R something /
>
> and after a while, my system did not respond. I do not remember the exact
> messages as I am on a winbugs at the University. The error was about
> swapping. I could switch among terminals but the system was dead. I needed
> to reboot.
>
> I rebooted and tried again watching "top" output and I could see as swap
> usage was incresing very quickly until it ran out of swap space and the swap
> pager failed.
>
> Was my sytem dead? or, is it possible to recover from that state without
> rebooting? How is it possible that a simple command like this could
> auto-kill the machine?
>
> What is the recomended fix for this?:
>
> a- Asigning more swap.
> b- Not executing that command anymore.
>
>
> Thank you very much for your advices and help.
>
> Ramiro

Thanks all for your responses.

I understand that I should avoid "greping" into /dev. I will do more
accurate searchs into the directories.

Ramiro.

[Loren M. Lang]

On Fri, Feb 25, 2005 at 11:42:48PM -0500, Parv wrote:

> in message <20050225135707.GC18789@alzatex.com>,
> wrote Loren M. Lang thusly...

> >
> > On Fri, Feb 25, 2005 at 12:14:04PM +0100, Ramiro Aceves wrote:

> > >
> > > I am running a FreeBSD 5.3 system with 64MB RAM and 150 MB swap.
> > >
> > > Yesterday I entered the command:
> > >
> > > # grep -R something /

> >
> > You probably hit a file under /dev/ and caused grep to hang. It's
> > possible that as root, certain device files might hang the system,
> > but nothing comes to mind at the moment unless /dev/io could do
> > it. Also, think about what happens when grep hit's /dev/zero. It
> > will never finish.

>
> Would using -I option (not search text-like files) help to avoid
> above described hang ups in /dev?

No, it still searches all files, it just doesn't print the usual line
that it matched, only whether there was success or not. You really just
need to make sure grep never goes into /dev. Since your running 5.x,
/dev is it's own filesystem of a unique type, so the following command
will run grep on only filesystems of type ufs, which won't include
network filesystems, or /dev:

find / -fstype ufs -exec grep -H something {} \;

>
>
> - Parv
>
> --

--
I sense much NT in you.
NT leads to Bluescreen.
Bluescreen leads to downtime.
Downtime leads to suffering.
NT is the path to the darkside.
Powerful Unix is.

Public Key: [url]ftp://ftp.tallye.com/pub/lorenl_pubkey.asc[/url]
Fingerprint: B3B9 D669 69C9 09EC 1BCD 835A FAF3 7A46 E4A3 280C