Professional Web Applications Themes

What would you consider as a large table? - MySQL

Is 2.5 million rows (about 400 MB CSV file size, has 25 about columns which probably average 10 characters per column) large? How long would it take to import this file? I know it is 2.5 million rows because I counted the rows (using unix's wc -l filename). I tried importing it but gave up after it imported only 800K plus records in about 24 hours using an Athlon 3200, lots of disk space and, 2 GB's of memory....

  1. #1

    Default What would you consider as a large table?

    Is 2.5 million rows (about 400 MB CSV file size, has 25 about columns
    which probably average 10 characters per column) large? How long
    would it take to import this file? I know it is 2.5 million rows
    because I counted the rows (using unix's wc -l filename).

    I tried importing it but gave up after it imported only 800K plus
    records in about 24 hours using an Athlon 3200, lots of disk space
    and, 2 GB's of memory.


    Pete Guest

  2. #2

    Default Re: What would you consider as a large table?

    Hi,

    did you try to show number of approx table rows in SQL-administrator?

    Maybe it innodb as storage engine is better for you.

    1 day for 2.5 million rows is not too much,
    especially if all is on the same hard disk.

    regards,

    Toni

    In article <com>,
    Pete <com> wrote:
     

    --
    Posen fuer Anfaenger: http://www.w-klch.med.uni-muenchen.de/dischner
    A. Guest

  3. #3

    Default Re: What would you consider as a large table?

    Pete <com> wrote: 

    This is not large. Ten times that size is largish. Hundred times that
    size is large ;-)
     

    Probably a misconfigured MySQL and/or wrong usage. The default MySQL
    configuration uses only very little memory, so most of your shiny 2GB
    of memory will be unused. One very common error is to bulk load data
    into InnoDB table with AUTO_COMMIT turned on.

    I suggest you read here first:
    http://dev.mysql.com/doc/refman/5.0/en/insert-speed.html

    Then tune your my.cnf and try again. Since your data is CSV, I suggest
    using LOAD DATA INFILE.


    XL
    --
    Axel Schwenke, Support Engineer, MySQL AB

    Online User Manual: http://dev.mysql.com/doc/refman/5.0/en/
    MySQL User Forums: http://forums.mysql.com/
    Axel Guest

Similar Threads

  1. Going for a LARGE Table: Any Tips?
    By Good Man in forum MySQL
    Replies: 1
    Last Post: November 11th, 05:36 PM
  2. How to search a large table?
    By Jonah in forum Microsoft SQL / MS SQL Server
    Replies: 2
    Last Post: July 29th, 09:30 AM
  3. Large SQL table causes web page to hang
    By Michael Netherton in forum ASP Database
    Replies: 2
    Last Post: July 16th, 11:49 AM
  4. Large table creation
    By Yan-Hong Huang[MSFT] in forum ASP.NET General
    Replies: 0
    Last Post: June 25th, 07:02 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139