Professional Web Applications Themes

small db app - what to use ? - UNIX Programming

Hey I need to make a rather small application, I must use some data for this which need to be located somewhere on the disk, the app should be able to delete data, add data and upgrade data. I don't want to use a rational db like mysql or postgresql ... using plain old files would be ok , but maybe that would be to slow .. are there other solutions , I've heard about berkely DB would this fit in the application ? The data will look something like this; foobar;some string can be up to 256characters barfoo;some string ...

  1. #1

    Default small db app - what to use ?

    Hey
    I need to make a rather small application,
    I must use some data for this which need to be located somewhere on the
    disk, the app should be able to delete data, add data and upgrade data.
    I don't want to use a rational db like mysql or postgresql ... using
    plain old files would be ok , but maybe that would be to slow ..
    are there other solutions , I've heard about berkely DB would this fit
    in the application ?

    The data will look something like this;

    foobar;some string can be up to 256characters
    barfoo;some string can be up to 256characters
    bfooar;some string can be up to 256characters

    this can be at max 4000lines ..
    this could be at min 200lines ...

    Geiregat Guest

  2. #2

    Default Re: small db app - what to use ?

    On Sun, 4 Jan 2004, Geiregat Jonas wrote:
     

    Yes, the ndbm (dbm_open etc al) man page should tell all.

    --
    Rich Teer, SCNA, SCSA . * * . * .* .
    . * . .*
    President, * . . /\ ( . . *
    Rite Online Inc. . . / .\ . * .
    .*. / * \ . .
    . /* o \ .
    Voice: +1 (250) 979-1638 * '''||''' .
    URL: http://www.rite-online.net ******************
    Rich Guest

  3. #3

    Default Re: small db app - what to use ?

    Geiregat Jonas <org> wrote: 

    1. http://www.daemon.de/dbtool
    man dbtool
    DESCRIPTION
    This manual page doents the program dbtool. dbtool can be
    used to store and retrieve data in a key/value format in a
    hash database. Perl compatible regular expressions are
    supported both for storing and retrieving of data. It's main
    advantages are the ability to maintain huge amounts of data
    and speed.

    2. Text file with sed, awk, grep, ed, bash, ... ouch! :-)

    --
    William Park, Open Geometry Consulting, <ca>
    Linux solution for data management and processing.
    William Guest

  4. #4

    Default Re: small db app - what to use ?

    On Sun, 04 Jan 2004 16:07:37 -0500, Geiregat Jonas wrote:
     

    So a table of data? How about character separated values:

    http://www.ioplex.com/~miallen/libmba/dl/src/csv.c

    Mike
    Michael Guest

  5. #5

    Default Re: small db app - what to use ?

    On Mon, 05 Jan 2004 at 06:34 GMT, Michael B Allen wrote: 
    >
    > So a table of data? How about character separated values:
    >
    > http://www.ioplex.com/~miallen/libmba/dl/src/csv.c[/ref]

    Which can also be handled easily in a bash script (or a ksh script
    with a little modification):

    set_values() {
    local IFS="${separator:-" "}"
    case $db_type in
    *CSV*)
    csv_split "$1"
    ;;
    *)
    set -f
    set -- $1
    values=( "$" )
    set +f
    ;;
    esac
    num_values=${#values[]}
    }

    csv_split() {
    local record=${1%"${CR}"}
    local right
    local vnum=0
    unset values
    while [ -n "$record" ]
    do
    case $record in
    \"*) right=${record#*\",}
    value=${record%%\",*}
    values[$vnum]=${value#\"}
    ;;
    *) values[$vnum]=${record%%,*}
    right=${record#*,}
    ;;
    esac
    case $record in
    *,*) record=${right} ;;
    *) record=${record#\"}
    values[$vnum]=${record%\"}
    break;;
    esac
    vnum=$(( $vnum + 1 ))
    done
    }

    --
    Chris F.A. Johnson http://cfaj.freeshell.org
    ================================================== =================
    My code (if any) in this post is copyright 2004, Chris F.A. Johnson
    and may be copied under the terms of the GNU General Public License
    Chris Guest

  6. #6

    Default Re: small db app - what to use ?

    On Mon, 05 Jan 2004 06:13:46 -0500, Chris F.A. Johnson wrote: 
    >
    > Which can also be handled easily in a bash script (or a ksh script
    > with a little modification):[/ref]
    <snip bash subroutines>

    Beware however that CSV can have quotes and escaped quotes like:

    one, " two", ""three"", " ""four"""
    q q e e q e e q

    where q is a quote but e is just an escape which makes the following
    quote part of the value. Granted CSV is not a standard outside of what is
    observed in Excel and Gnumeric. The code I cited properly handles quotes.

    Mike
    Michael Guest

  7. #7

    Default Re: small db app - what to use ?

    On Mon, 05 Jan 2004 15:39:21 -0500, Michael B Allen wrote:
     
    >>
    >> Which can also be handled easily in a bash script (or a ksh script
    >> with a little modification):[/ref]
    > <snip bash subroutines>
    >
    > Beware however that CSV can have quotes and escaped quotes like:
    >
    > one, " two", ""three"", " ""four"""
    > q q e e q e e q[/ref]

    Does the middle version actually happen? I assumed that it would be...

    """three"""
    qe e q

    ....or was there supposed to be another q somewhere (I see a dangling one,
    so maybe so, Hmm)...
     

    ....and that is a good reason I wouldn't recommend CSV data unless you want
    to move it into apps. that understand CSV already. For instance doing a
    "simple CSV" you might do...

    1. ',' seperates values.
    2. '\\' escapes values to be literal.

    ....so you'd get...

    foo bar\,baz, blah said "whatever",and another line\
    was made

    ....instead of...

    "foo bar",baz," blah said ""whatever""","and another line
    was made"

    ....which is simpler, or even just use a netstr[1] for each line, and
    another for each column...

    70:7:foo bar,3:baz,21: blah said "whatever",25:and another line
    was made,,

    ....which is much easier to p. Of course, as with many things, network
    effects can be a higher priority than simplicity.

    [1] http://cr.yp.to/proto/netstrings.txt

    --
    James Antill -- org
    Need an efficient and powerful string library for C?
    http://www.and.org/vstr/

    James Guest

  8. #8

    Default Re: small db app - what to use ?

    On Mon, 05 Jan 2004 18:11:04 -0500, James Antill wrote: 
    >> <snip bash subroutines>
    >>
    >> Beware however that CSV can have quotes and escaped quotes like:
    >>
    >> one, " two", ""three"", " ""four"""
    >> q q e e q e e q[/ref]
    >
    > Does the middle version actually happen? I assumed that it would be...
    >
    > """three"""
    > qe e q
    >
    > ...or was there supposed to be another q somewhere (I see a dangling
    > one, so maybe so, Hmm)...[/ref]

    If I use braces to show the four values explicitly from the CSV line
    above they are:

    {one}
    { two}
    {"three"}
    { "four"}

    So yes, the middle version does actually happen if you wish to preserve
    quotes. Also, I don't believe anything is "dangling". The way to think
    about quotes in CSV is this; if there are two quotes next to each other,
    the first is discarded but the second is treated litereally. If a third
    quote appears after the two quotes or if a single quote appears alone
    it should be discarded but used to escape sections of arbitrary text
    that should be treated literally.
     
    >
    > ...and that is a good reason I wouldn't recommend CSV data unless you
    > want to move it into apps. that understand CSV already. For instance
    > doing a "simple CSV" you might do...
    >
    > 1. ',' seperates values.
    > 2. '\\' escapes values to be literal.[/ref]

    Using a single '\' escape character is not ideal for escaping sections
    text that should be treated literally (e.g. UTF-8).

    Mike
    Michael Guest

  9. #9

    Default Re: small db app - what to use ?

    On Tue, 06 Jan 2004 03:10:17 -0500, Michael B Allen wrote:
     
     
    >> Does the middle version actually happen? I assumed that it would be...
    >>
    >> """three"""
    >> qe e q[/ref]
    >
    > yes, the middle version does actually happen if you wish to preserve
    > quotes.[/ref]

    I just checked your code, and it doesn't p the above line (the
    <""three> part is treat like <" "three> without the space ... Ie. a quoted
    string with stuff at the end). Which make sense as I don't see how you
    could p the line...

    """one, two""", ""three""
     
    > Also, I don't believe anything is "dangling". The way to think[/ref]

    Yeh, sorry. I saw ... < ""four"""> for the last element, for some reason.
     
    >
    > Using a single '\' escape character is not ideal for escaping sections
    > text that should be treated literally (e.g. UTF-8).[/ref]

    Anything that isn't a '\\', ',' or '\n' would be unchanged. UTF-8 makes
    this simple as no multi-byte character will have any of those three values
    in it (AIUI some other multi-byte encodings make this more painful, but
    utf-8 is fine).
    So just a loop searching for a '\\', or ',' and passing the next
    character or noting a new column has started respectively _is all_ that's
    required for utf-8.

    --
    James Antill -- org
    Need an efficient and powerful string library for C?
    http://www.and.org/vstr/

    James Guest

  10. #10

    Default Re: small db app - what to use ?

    On Tue, 06 Jan 2004 12:31:44 -0500, James Antill wrote: 
    >>
    >> yes, the middle version does actually happen if you wish to preserve
    >> quotes.[/ref]
    >
    > I just checked your code, and it doesn't p the above line (the
    > <""three> part is treat like <" "three> without the space ... Ie. a
    > quoted string with stuff at the end). Which make sense as I don't see[/ref]

    Opps. You're right.

    src/csv.c:169:csv_p_str: bad end quote in element 3
    tmba.c:172:run_suite: failure

    If you have escaped quotes you need the outer quotes. Like you said this
    example would need to be written:

    one, " two", """three""", " ""four"""

    which regardless of the trim parameter gives:

    {one}{ two}{"three"}{ "four"}

    Mmm, I really need a routine to *serialize* CSV too...

    Thanks,
    Mike
    Michael Guest

Similar Threads

  1. Small cry for help
    By Grant_J_Dixon@adobeforums.com in forum Adobe Photoshop 7, CS, CS2 & CS3
    Replies: 13
    Last Post: September 30th, 11:18 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139