Professional Web Applications Themes

UNICODE and special characters in MySQL & Python - MySQL

I have an MySQL database called zingers. The structure is: zid - integer, key, autoincrement keyword - varchar citation - text quotation - text I am having trouble storing text, as typed in latter two fields. Special characters and punctuation all seem not to be stored and retrieved correctly. Special apostrophes and single quotes from Microsoft Word are causing a special problem, even though I have ''ed all 's perhaps the encoding of the database itself should be different? it is currenlty latin_swedish_ci Input and output is through a browser. I think my problem may be that I need to ...

  1. #1

    Default UNICODE and special characters in MySQL & Python

    I have an MySQL database called zingers. The structure is:

    zid - integer, key, autoincrement
    keyword - varchar
    citation - text
    quotation - text

    I am having trouble storing text, as typed in latter two fields.
    Special characters and punctuation all seem not to be stored and
    retrieved correctly.

    Special apostrophes and single quotes from Microsoft Word are causing a
    special problem, even though I have ''ed all 's

    perhaps the encoding of the database itself should be different? it is
    currenlty latin_swedish_ci

    Input and output is through a browser.

    I think my problem may be that I need to encode the string before
    saving it in the databse. Can anyone point me in the right direction
    here?



    here's the error message:

    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position
    95: ordinal not in range(128)
    args = ('ascii', "update zingers set keywords =
    'a;Action;b;Religi... \n \n \n ' where zid = 422", 95, 96, 'ordinal not
    in range(128)')
    encoding = 'ascii'
    end = 96
    object = "update zingers set keywords = 'a;Action;b;Religi... \n
    \n \n ' where zid = 422"
    reason = 'ordinal not in range(128)'
    start = 95



    the characters I am trying to add are startquote and endquote copied
    and pasted from Microsoft Word.



    Can anyone help me on this?

    bests,

    -rsr-

    Ron Guest

  2. #2

    Default UNICODE and special characters in MySQL & Python

    I have an MySQL database called zingers. The structure is:

    zid - integer, key, autoincrement
    keyword - varchar
    citation - text
    quotation - text

    I am having trouble storing text, as typed in latter two fields.
    Special characters and punctuation all seem not to be stored and
    retrieved correctly.

    Special apostrophes and single quotes from Microsoft Word are causing a
    special problem, even though I have ''ed all 's

    perhaps the encoding of the database itself should be different? it is
    currenlty latin_swedish_ci

    Input and output is through a browser.

    I think my problem may be that I need to encode the string before
    saving it in the databse. Can anyone point me in the right direction
    here?



    here's the error message:

    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position
    95: ordinal not in range(128)
    args = ('ascii', "update zingers set keywords =
    'a;Action;b;Religi... \n \n \n ' where zid = 422", 95, 96, 'ordinal not
    in range(128)')
    encoding = 'ascii'
    end = 96
    object = "update zingers set keywords = 'a;Action;b;Religi... \n
    \n \n ' where zid = 422"
    reason = 'ordinal not in range(128)'
    start = 95



    the characters I am trying to add are startquote and endquote copied
    and pasted from Microsoft Word.



    Can anyone help me on this?

    bests,

    -rsr-

    Ron Guest

  3. #3

    Default Re: UNICODE and special characters in MySQL & Python

    On 1 Jan 2007 21:56:51 -0800, Ron wrote: 

    Right problem, but wrong solution. The "special apostrophes" from Word
    aren't the same as "'" either.
     

    That's a collation, not an encoding, nor a character set. It's also
    missing the '1' after 'latin'. (It's also not your actual problem, so
    I'm not going to go further into those.)
     

    You need to *either* strip those special apostrophies out and make them
    normal puctuation, or ALTER the column's character set to one that does
    have those "special apostrophies" in it. You'll find them at U+2018,
    U+2019 and U+201C, U+201D. Which means you'll probably want to use utf8
    as your character set.
     

    This should be yout first clue. ascii's pretty limited: it's only got
    128 characters, and anything with an accent or diacritical of any kind
    won't fit into the definition.

    --
    Every fleeting thought you've ever had in your life, no matter how bizarre,
    is someone's lifelong obsession. And he has a website.
    -- Skif's Internet Theorem
    Peter Guest

Similar Threads

  1. Special Characters & Access w/ Unicode
    By smithjo in forum Macromedia ColdFusion
    Replies: 19
    Last Post: September 13th, 07:08 PM
  2. Special character / Unicode problems
    By mxc in forum Adobe Flash, Flex & Director
    Replies: 5
    Last Post: April 29th, 12:46 PM
  3. Unicode characters and ado.
    By Bilal Abbasi in forum ASP Database
    Replies: 1
    Last Post: January 10th, 12:41 AM
  4. PHP/Mysql/special characters problem
    By Mosher in forum PHP Development
    Replies: 12
    Last Post: January 7th, 04:41 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139