Home » Performance » MySQL » Out of memory when query big table
Out of memory when query big table [message #607] Thu, 18 January 2007 21:47 Go to next message
bc19148
Messages: 2
Registered: January 2007
Junior Member
Hi Peter,

I am new to MySQL. I installed MySQL 5.0 and used the InnoDB engine. It currently have just one table with 39 columns, combination of varchar, datetime, double, and text data-types. At the DOS prompt, I ran this simple query (“select * from main where name like ‘%abc%’ or description like ‘%abc%’ order by cost;) with 20k rows, it ran pretty fast (2 seconds); but when the table grows to 800k rows, the query took over a minute. I tried the EXPLAIN method, but not sure what those values meant.

Another thing, while the SQL is executing, I monitored the Windows Task Manger and saw the mysqld-nt.exe went from 27K Memory Usage to 306K, and ran out of memory on 512MB RAM. I also added a primary key field and index it, but it doesn’t help. I read through your “Why MySQL could be slow with large tables?” blog, but still not sure what to do. There must be a way to execute the sql faster because the table will grows to 2M rows or more. Please help and let me know what I need to do.

Thanks,
Bryan
Re: Out of memory when query big table [message #608 is a reply to message #607 ] Fri, 19 January 2007 06:29 Go to previous messageGo to next message
Peter  is currently offline Peter
Messages: 405
Registered: August 2006
Senior Member
Super Guru
You should not use LIKE search for large sizes. Use MySQL Full Text Search or some external search engine.

And buy some good MySQL book or read MySQL Manual Online.

You're asking a question which is well documented.


Peter Zaitsev, MySQL Performance Expert
MySQL Performance Blog - http://www.mysqlperformanceblog.com
MySQL Consulting http://www.mysqlperformanceblog.com/mysql-consulting/
Re: Out of memory when query big table [message #609 is a reply to message #608 ] Fri, 19 January 2007 08:56 Go to previous messageGo to next message
bc19148
Messages: 2
Registered: January 2007
Junior Member
Hi,

Thanks for information. I read the Full-Text Search functions in MySQL 5.0 reference manual before, and I'm sure it said the Full-Text Search functions ONLY apply to MyISAM tables and not InnoDB tables. The table is currently in InnoDB engine. You also recommended to use external search engine, please provide a list of search engine so I can look for it.
I apologize if I asked the question to again, but after spent three days reading stuff through reference manual and web, and it still doesn't work, it can be painful.

Thanks again,
Bryan
Re: Out of memory when query big table [message #610 is a reply to message #609 ] Fri, 19 January 2007 09:44 Go to previous message
Peter  is currently offline Peter
Messages: 405
Registered: August 2006
Senior Member
Super Guru
Try out sphinx

http://www.sphinxsearch.com

Or create "shadow" MyISAM table for search


Peter Zaitsev, MySQL Performance Expert
MySQL Performance Blog - http://www.mysqlperformanceblog.com
MySQL Consulting http://www.mysqlperformanceblog.com/mysql-consulting/
Previous Topic:faster not using Index????
Next Topic:Need some help with big tables
Goto Forum:
  


Current Time: Fri Jul 10 03:43:30 EDT 2009

Total time taken to generate the page: 0.01098 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 2.7.7.
Copyright ©2001-2007 FUD Forum Bulletin Board Software

MySQL is a trademark of Sun Microsystems.
InnoDB is a trademark of Oracle Corp.

Percona Performance Forums are a service of Percona, Inc.
Not affiliated with Sun Microsystems or Oracle Corp.