awk -v col=2 '{ total += $col } END { print total }' test awk: syntax error near line 1 awk: bailing out near line 1 Any ideas? [allowsmilie] => 1 [showsignature] => 0 [ipaddress] => [iconid] => 0 [visible] => 1 [attach] => 0 [infraction] => 0 [reportthreadid] => 0 [isusenetpost] => 1 [msgid] => <1138011394.623402.30750@g14g2000cwa.googlegroups.com> [ref] => <1138005467.920703.54950@g47g2000cwa.googlegroups.com><17iea3-f18.ln1@teksavvy.com> [htmlstate] => on_nl2br [postusername] => cconnell_1@lycos.com [ip] => cconnell_1@lyco [isdeleted] => 0 [usergroupid] => [membergroupids] => [displaygroupid] => [password] => [passworddate] => [email] => [styleid] => [parentemail] => [homepage] => [icq] => [aim] => [yahoo] => [msn] => [skype] => [showvbcode] => [showbirthday] => [usertitle] => [customtitle] => [joindate] => [daysprune] => [lastvisit] => [lastactivity] => [lastpost] => [lastpostid] => [posts] => [reputation] => [reputationlevelid] => [timezoneoffset] => [pmpopup] => [avatarid] => [avatarrevision] => [profilepicrevision] => [sigpicrevision] => [options] => [akvbghsfs_optionsfield] => [birthday] => [birthday_search] => [maxposts] => [startofweek] => [referrerid] => [languageid] => [emailstamp] => [threadedmode] => [autosubscribe] => [pmtotal] => [pmunread] => [salt] => [ipoints] => [infractions] => [warnings] => [infractiongroupids] => [infractiongroupid] => [adminoptions] => [profilevisits] => [friendcount] => [friendreqcount] => [vmunreadcount] => [vmmoderatedcount] => [socgroupinvitecount] => [socgroupreqcount] => [pcunreadcount] => [pcmoderatedcount] => [gmmoderatedcount] => [assetposthash] => [fbuserid] => [fbjoindate] => [fbname] => [logintype] => [fbaccesstoken] => [newrepcount] => [vbseo_likes_in] => [vbseo_likes_out] => [vbseo_likes_unread] => [temp] => [field1] => [field2] => [field3] => [field4] => [field5] => [subfolders] => [pmfolders] => [buddylist] => [ignorelist] => [signature] => [searchprefs] => [rank] => [icontitle] => [iconpath] => [avatarpath] => [hascustomavatar] => 0 [avatardateline] => [avwidth] => [avheight] => [edit_userid] => [edit_username] => [edit_dateline] => [edit_reason] => [hashistory] => [pagetext_html] => [hasimages] => [signatureparsed] => [sighasimages] => [sigpic] => [sigpicdateline] => [sigpicwidth] => [sigpicheight] => [postcount] => 3 [islastshown] => [isfirstshown] => [attachments] => [allattachments] => ) --> > awk -v col=2 '{ total += $col } END { print total }' test > awk: syntax error near line 1 > awk: bailing out near line 1 > > Any ideas?[/ref] Are you using Solaris? If so, use nawk or xpg4/awk (I forget the exact location). -- Chris F.A. Johnson, author | Shell Scripting Recipes: | My code in this post, if any, A Problem-Solution Approach | is released under the 2005, Apress | GNU General Public Licence [allowsmilie] => 1 [showsignature] => 0 [ipaddress] => [iconid] => 0 [visible] => 1 [attach] => 0 [infraction] => 0 [reportthreadid] => 0 [isusenetpost] => 1 [msgid] => [ref] => <1138005467.920703.54950@g47g2000cwa.googlegroups.com> <17iea3-f18.ln1@teksavvy.com> <1138011394.623402.30750@g14g2000cwa.googlegroups.com> [htmlstate] => on_nl2br [postusername] => Chris [ip] => cfajohnson@gmai [isdeleted] => 0 [usergroupid] => [membergroupids] => [displaygroupid] => [password] => [passworddate] => [email] => [styleid] => [parentemail] => [homepage] => [icq] => [aim] => [yahoo] => [msn] => [skype] => [showvbcode] => [showbirthday] => [usertitle] => [customtitle] => [joindate] => [daysprune] => [lastvisit] => [lastactivity] => [lastpost] => [lastpostid] => [posts] => [reputation] => [reputationlevelid] => [timezoneoffset] => [pmpopup] => [avatarid] => [avatarrevision] => [profilepicrevision] => [sigpicrevision] => [options] => [akvbghsfs_optionsfield] => [birthday] => [birthday_search] => [maxposts] => [startofweek] => [referrerid] => [languageid] => [emailstamp] => [threadedmode] => [autosubscribe] => [pmtotal] => [pmunread] => [salt] => [ipoints] => [infractions] => [warnings] => [infractiongroupids] => [infractiongroupid] => [adminoptions] => [profilevisits] => [friendcount] => [friendreqcount] => [vmunreadcount] => [vmmoderatedcount] => [socgroupinvitecount] => [socgroupreqcount] => [pcunreadcount] => [pcmoderatedcount] => [gmmoderatedcount] => [assetposthash] => [fbuserid] => [fbjoindate] => [fbname] => [logintype] => [fbaccesstoken] => [newrepcount] => [vbseo_likes_in] => [vbseo_likes_out] => [vbseo_likes_unread] => [temp] => [field1] => [field2] => [field3] => [field4] => [field5] => [subfolders] => [pmfolders] => [buddylist] => [ignorelist] => [signature] => [searchprefs] => [rank] => [icontitle] => [iconpath] => [avatarpath] => [hascustomavatar] => 0 [avatardateline] => [avwidth] => [avheight] => [edit_userid] => [edit_username] => [edit_dateline] => [edit_reason] => [hashistory] => [pagetext_html] => [hasimages] => [signatureparsed] => [sighasimages] => [sigpic] => [sigpicdateline] => [sigpicwidth] => [sigpicheight] => [postcount] => 4 [islastshown] => [isfirstshown] => [attachments] => [allattachments] => ) --> adding up field columns in a file - Linux / Unix Administration

adding up field columns in a file - Linux / Unix Administration

Hello, I have a file with some columns of numeric data e.g. 5 5 5 6 5 7 I need to total them, currently im using the following method for i in cat <filename> | awk print column2 (for example) do total=$i total=xpr $total + $i` done However this method is really slow on large files, could someone recommend a quicker way? Thanks...

  1. #1

    Default adding up field columns in a file

    Hello,
    I have a file with some columns of numeric data e.g.

    5 5
    5 6
    5 7

    I need to total them, currently im using the following method


    for i in cat <filename> | awk print column2 (for example)
    do
    total=$i
    total=xpr $total + $i`
    done

    However this method is really slow on large files, could someone
    recommend a quicker way?

    Thanks

    cconnell_1@lycos.com Guest

  2. #2

    Default Re: adding up field columns in a file

    On 2006-01-23, com wrote: 

    To add the second column:

    awk -v col=2 '{ total += $col } END { print total }' FILENAME

    To add the first column, change col=2 to col=1

    --
    Chris F.A. Johnson, author | <http://cfaj.freeshell.org>
    Shell Scripting Recipes: | My code in this post, if any,
    A Problem-Solution Approach | is released under the
    2005, Apress | GNU General Public Licence
    Chris Guest

  3. #3

    Default Re: adding up field columns in a file


    Chris F.A. Johnson wrote: 
    >
    > To add the second column:
    >
    > awk -v col=2 '{ total += $col } END { print total }' FILENAME
    >
    > To add the first column, change col=2 to col=1
    >
    > --
    > Chris F.A. Johnson, author | <http://cfaj.freeshell.org>
    > Shell Scripting Recipes: | My code in this post, if any,
    > A Problem-Solution Approach | is released under the
    > 2005, Apress | GNU General Public Licence[/ref]


    Thanks, I think there is a typo somewhere as I get byteosn01:/tmp-->
    awk -v col=2 '{ total += $col } END { print total }' test
    awk: syntax error near line 1
    awk: bailing out near line 1

    Any ideas?

    cconnell_1@lycos.com Guest

  4. #4

    Default Re: adding up field columns in a file

    On 2006-01-23, com wrote: 
    >>
    >> To add the second column:
    >>
    >> awk -v col=2 '{ total += $col } END { print total }' FILENAME
    >>
    >> To add the first column, change col=2 to col=1[/ref]
    >
    > Thanks, I think there is a typo somewhere as I get byteosn01:/tmp-->
    > awk -v col=2 '{ total += $col } END { print total }' test
    > awk: syntax error near line 1
    > awk: bailing out near line 1
    >
    > Any ideas?[/ref]

    Are you using Solaris? If so, use nawk or xpg4/awk (I forget the
    exact location).


    --
    Chris F.A. Johnson, author | <http://cfaj.freeshell.org>
    Shell Scripting Recipes: | My code in this post, if any,
    A Problem-Solution Approach | is released under the
    2005, Apress | GNU General Public Licence
    Chris Guest

  5. #5

    Default Re: adding up field columns in a file

    Yes its solaris.
    nawk works fantasic.Thx

    cconnell_1@lycos.com Guest

  6. #6

    Default Re: adding up field columns in a file

    >>>>> "cconnell" == cconnell 1 <com> writes:

    cconnell> for i in cat <filename> | awk print column2 (for example)

    Why the unnecessary cat?

    Why not just:

    cat filename | cat | cat | cat | tr | cat | awk ...

    --
    Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
    <com> <URL:http://www.stonehenge.com/merlyn/>
    Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
    See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
    Randal Guest

  7. #7

    Default Re: adding up field columns in a file

    On 2006-01-23, Randal L. Schwartz wrote: [/ref]
    >
    >cconnell> for i in cat <filename> | awk print column2 (for example)
    >
    > Why the unnecessary cat?
    >
    > Why not just:
    >
    > cat filename | cat | cat | cat | tr | cat | awk ...[/ref]

    usage: tr [-csu] string1 string2
    tr [-cu] -d string1
    tr [-cu] -s string1
    tr [-cu] -ds string1 string2

    --
    Chris F.A. Johnson, author | <http://cfaj.freeshell.org>
    Shell Scripting Recipes: | My code in this post, if any,
    A Problem-Solution Approach | is released under the
    2005, Apress | GNU General Public Licence
    Chris Guest

  8. #8

    Default Re: adding up field columns in a file

    While on the subject, if one wants to print selective columns in a file
    using awk, is it possible to do it without cat to make it faster?
    i.e I always pipe the output from a file into awk from cat, can you use
    awk on its own?
    Thanks

    cconnell_1@lycos.com Guest

  9. #9

    Default Re: adding up field columns in a file

    com wrote: 

    Yes. In general, you can almost always eliminate redundant "cat"s
    by changing this form:

    cat foo | bar

    into this form:

    bar < foo

    With awk, there is an even shorter way, because awk can take an input
    filename as a command line argument directly (just like cat does), so
    you can change this

    cat /etc/passwd | awk -F: '{print $1}'

    into this:

    awk -F: '{print $1}' /etc/passwd

    You could of course use redirection so that the shell opens the file
    and makes it awk's standard input:

    awk -F: '{print $1}' < /etc/passwd

    but that more typing.

    - Logan
    Logan Guest

  10. #10

    Default Re: adding up field columns in a file

    On 2006-01-24, com wrote: 

    Did you look at the solutions you were given? Don't they answer
    your question?

    --
    Chris F.A. Johnson, author | <http://cfaj.freeshell.org>
    Shell Scripting Recipes: | My code in this post, if any,
    A Problem-Solution Approach | is released under the
    2005, Apress | GNU General Public Licence
    Chris Guest

  11. #11

    Default Re: adding up field columns in a file

    com wrote: 

    A large percentage of the time using cat to prime a pipeline
    will end up being labelled "gratitous use of cat". You should
    nearly always be able to avoid it with I/O redirection other
    than a pipe, putting filenames on the command line and
    so on.

    And still, after decades of writing scripts I often prime a
    pipeline with a cat. I especially like cat ... | while ... Why?
    Stylistic reasons. I don't like while ... one < file

    Doug Guest

  12. #12

    Default Re: adding up field columns in a file

    Thats great to know. Thanks for the help, I will experiment with it.
    And Chris yes your answer did answer my question as there was the
    filename at the end of the awk statement- doh!

    cconnell_1@lycos.com Guest

  13. #13

    Default Re: adding up field columns in a file

    com wrote: 

    you can sum up the 2nd column by:
    awk '{t+=$2}END{print t}' FILENAME

    sum up two columns:
    awk '{t1+=$1;t2+=$2}END{print "total:",t1,t2}' FILENAME

    sum up all columns:
    perl -alne '$t[$_]+=$F[$_]for(0..$#F);END{print "total: t\n"}'
    FILENAME

    Xicheng

    Xicheng Guest

Similar Threads

  1. Adding Columns
    By jenn in forum Coldfusion - Getting Started
    Replies: 2
    Last Post: May 17th, 07:55 PM
  2. Adding Custom Columns to DataGrid
    By flexbay in forum Macromedia Flex General Discussion
    Replies: 2
    Last Post: March 7th, 04:49 PM
  3. Adding columns dynamically
    By Peter Kamitz in forum ASP.NET Data Grid Control
    Replies: 2
    Last Post: March 2nd, 06:11 PM
  4. modifying and adding columns to a big table
    By Cris in forum Oracle Server
    Replies: 6
    Last Post: November 12th, 09:49 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •