Professional Web Applications Themes

Convert / Decimal / Float - Microsoft SQL / MS SQL Server

I need to convert this example in 16 decimals number. CREATE TABLE #TMP_APLICACAO ( [VL_APLICACAO] [MONEY], [NR_QUANTIDADE] [NUMERIC] (28,16), [VL_PU_OPERACAO] [NUMERIC] (28,16) ) declare vl_pu_operacao numeric(28,16) INSERT INTO #TMP_APLICACAO VALUES ( 3103.40, 3.00, 0) select vl_pu_operacao = VL_APLICACAO / NR_QUANTIDADE FROM #TMP_APLICACAO SELECT VL_PU_OPERACAO PS. How can I have to do to see the result in 16 decimal number : 1034.4666666666666667 ? I use select vl_pu_operacao = CONVERT(decimal (28,16),VL_APLICACAO / NR_QUANTIDADE) FROM #TMP_APLICACAO , but the results is 1034.4666666666 and itīs wrong. DROP TABLE #TMP_APLICACAO...

  1. #1

    Default Convert / Decimal / Float

    I need to convert this example in 16 decimals number.

    CREATE TABLE #TMP_APLICACAO
    (
    [VL_APLICACAO] [MONEY],
    [NR_QUANTIDADE] [NUMERIC] (28,16),
    [VL_PU_OPERACAO] [NUMERIC] (28,16)
    )

    declare vl_pu_operacao numeric(28,16)

    INSERT INTO #TMP_APLICACAO VALUES ( 3103.40, 3.00, 0)

    select vl_pu_operacao = VL_APLICACAO / NR_QUANTIDADE FROM #TMP_APLICACAO

    SELECT VL_PU_OPERACAO

    PS. How can I have to do to see the result in 16 decimal number :
    1034.4666666666666667 ?

    I use select vl_pu_operacao = CONVERT(decimal
    (28,16),VL_APLICACAO / NR_QUANTIDADE) FROM #TMP_APLICACAO
    , but the results is 1034.4666666666 and itīs wrong.

    DROP TABLE #TMP_APLICACAO







    Marco Guest

  2. #2

    Default Re: Convert / Decimal / Float

    This is by design per bol.

    -- e1 / e2 Result Precision: p1 - s1 + s2 + max(6, s1 + p2 + 1); Result
    Scale: max(6, s1 + p2 + 1)
    -- * The result precision and scale have an absolute maximum of 38. When a
    result precision is greater than 38, the corresponding scale is reduced to
    prevent the integral part of a result from being truncated.

    select result_prec=19-4+16+4+28+1, result_scale=4+28+1

    Workaround:

    select VL_PU_OPERACAO =
    convert(numeric(28,16),convert(numeric(28,16),VL_A PLICACAO) /
    convert(money,NR_QUANTIDADE))
    FROM #TMP_APLICACAO


    --
    -oj
    RAC v2.2 & QALite!
    http://www.rac4sql.net


    "Marco" <com.br> wrote in message
    news:#phx.gbl... 


    oj Guest

  3. #3

    Default Re: Convert / Decimal / Float

    Marco,

    In addition to oj's solution, you can lower the precision in your
    table, if this is possible. You will get better results with

    [NR_QUANTIDADE] [NUMERIC] (22,6),

    Because you are dividing by NUMERIC(22,16), the calculation is done
    to make room for many places on the left of the decimal point, in case
    you divide by 0.0000000000000001, and places on the right of the
    decimal are sacrificed.

    Or you can force the arithmetic to use floating point, but then you
    may get different answers that are not exact:

    select vl_pu_operacao = 1e0 * VL_...

    -- Steve Kass
    -- Drew University
    -- Ref: A62F699B-36FE-49D6-86F7-AC3377CA7B39

    Marco wrote:
     

    Steve Guest

Similar Threads

  1. Convert float value to integer
    By afroman22 in forum Coldfusion - Getting Started
    Replies: 2
    Last Post: October 27th, 07:12 PM
  2. Convert Decimal to hh:mm
    By JulianForniss in forum Macromedia ColdFusion
    Replies: 3
    Last Post: March 11th, 06:29 PM
  3. how to convert decimal to hexadecimal in perl
    By indie in forum Perl / CGI
    Replies: 2
    Last Post: July 12th, 11:47 AM
  4. How to convert string to float?
    By Jason Chan in forum ASP
    Replies: 4
    Last Post: September 4th, 12:51 PM
  5. Which data type decimal, money, float...?
    By JCardinal in forum Microsoft SQL / MS SQL Server
    Replies: 2
    Last Post: August 6th, 04:18 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139