Home » Performance » MySQL » Storing InnoDB tables on RAM disk
Storing InnoDB tables on RAM disk [message #2077] Wed, 17 October 2007 07:15 Go to next message
Nomad  is currently offline Nomad
Messages: 4
Registered: October 2007
Location: Ukraine
Junior Member
Greetings

Currently I'm using MySQL 5.0 and InnoDB to store small database (1.1-1.8GB). I need extremely fast read access and moderate insert/update speed. On my dedicated server there is 4 GB RAM and I use 2GB as RAM drive where InnoDB bases are stored. Also InnoDB memory usage is set to minimal settings and 128MB log file (also placed on RAM disk).
Such combination gives me required speed but limits database size and requires extended backup features, etc.
My question is simple - is it possible to implement similar performance without RAM drive, just giving MySQL RAM that is now wasted on virtual drive.
Please, make comparative tests and post the results instead of just telling to "increase log size, may be it would help".
Russian version of my question can be found here: http://www.mysqlperformanceblog.com/about/#comment-178034
Re: Storing InnoDB tables on RAM disk [message #2079 is a reply to message #2077 ] Wed, 17 October 2007 07:49 Go to previous messageGo to next message
AlexN  is currently offline AlexN
Messages: 20
Registered: October 2007
Location: Moscow,Russia
Junior Member
It depends.
As I understand it, MySQL caches query results, not data itself. So if you have repeated similar queries, maybe increasing memory size for query cache will help. If queries are all different, RAM disk is better. BTW, you can also use in-memory tables.

If you need fast SELECTs, why are you using InnoDB, not MyISAM ?
Re: Storing InnoDB tables on RAM disk [message #2080 is a reply to message #2079 ] Wed, 17 October 2007 07:56 Go to previous messageGo to next message
Nomad  is currently offline Nomad
Messages: 4
Registered: October 2007
Location: Ukraine
Junior Member
All queries are different and retrieved data is cached on app side, so we don't need to increase query cache. I've tried increasing it but nothing happened.
About InnoDB vs MyISAM. I still think that InnoDB is somewhat faster in complex queries and it give better result on my 4-CPU server with multi-threaded client applications.
Re: Storing InnoDB tables on RAM disk [message #2086 is a reply to message #2077 ] Wed, 17 October 2007 15:52 Go to previous messageGo to next message
sterin  is currently offline sterin
Messages: 324
Registered: March 2007
Location: Sweden
Senior Member
How did you really end up with the RAM disk solution?

Yes, MySQL has a query cache that caches the results of queries and uses a hashing mechanism to return the result without the need to actually execute the query again.

But InnoDB also has a caching to avoid disk reads and that is what the innodb_buffer_size variable is controlling.
Recommended is that you set this to about 80% of the available RAM on a dedicated server.
And if data in the tables are smaller than this MySQL won't even touch the disk and perform all queries internally in RAM.

Writes is another matter.
Here you have the flushing of log to disk after each query(commit) that slows things down. Which is needed to be ACID compliant.
But that is just a default setting.

If you need more speed and are prepared to sacrifice a bit for it then you set:
innodb_flush_log_at_trx_commit = 0

Which configures InnoDB to not perform the flushing of the log after each write and that speeds things up significantly.
Re: Storing InnoDB tables on RAM disk [message #2087 is a reply to message #2077 ] Wed, 17 October 2007 16:00 Go to previous messageGo to next message
Nomad  is currently offline Nomad
Messages: 4
Registered: October 2007
Location: Ukraine
Junior Member
innodb_flush_log_at_trx_commit = 0
already used.
You see, I'm not going to make experiments on working server just because "setting cache size to 80% RAM would help", etc. May be someone could test different options and configurations to compare performance difference to RAM disk solution.
RAM drive solution came from using Firebird - with DB on RAM disk it really rocks on select operations.
Re: Storing InnoDB tables on RAM disk [message #2088 is a reply to message #2087 ] Wed, 17 October 2007 17:20 Go to previous messageGo to next message
sterin  is currently offline sterin
Messages: 324
Registered: March 2007
Location: Sweden
Senior Member
Nomad wrote on Wed, 17 October 2007 22:00

innodb_flush_log_at_trx_commit = 0
already used.


Good

Nomad wrote on Wed, 17 October 2007 22:00


You see, I'm not going to make experiments on working server just because "setting cache size to 80% RAM would help", etc.


Well the whole reason why the innodb_buffer_pool exist in the first place is to cache data to avoid the disk reads which is what you want to avoid.
And it is the single most important configuration variable to get speed out of a normal InnoDB database setup.

Nomad wrote on Wed, 17 October 2007 22:00

May be someone could test different options and configurations to compare performance difference to RAM disk solution.


You are free to perform the homework.

Nomad wrote on Wed, 17 October 2007 22:00


RAM drive solution came from using Firebird - with DB on RAM disk it really rocks on select operations.

That is actually a bit surprising because the OS cache usually does a very good job avoiding disk reads and on cache aggressive OS like Linux the speed difference is about 5%. And 5% penalty compared to the flexibility it offers is usually a very easy choice.

The only time when there really is a difference is if an application is using fsync like in the innodb_flush_log_at_trx_commit case.
But at some times the HW/OS is faking this anyway and at those cases there isn't a speed difference anyway.


Out of curiousity, how many queries per second do you have and how many of them are writes?
Re: Storing InnoDB tables on RAM disk [message #2093 is a reply to message #2087 ] Thu, 18 October 2007 03:09 Go to previous messageGo to next message
AlexN  is currently offline AlexN
Messages: 20
Registered: October 2007
Location: Moscow,Russia
Junior Member
Nomad wrote on Wed, 17 October 2007 16:00

May be someone could test different options and configurations to compare performance difference to RAM disk solution.


I'm sorry - do you mean someone would test it using YOUR queries ?

Re: Storing InnoDB tables on RAM disk [message #2094 is a reply to message #2093 ] Thu, 18 October 2007 03:58 Go to previous messageGo to next message
Nomad  is currently offline Nomad
Messages: 4
Registered: October 2007
Location: Ukraine
Junior Member
AlexN wrote on Thu, 18 October 2007 10:09

Nomad wrote on Wed, 17 October 2007 16:00

May be someone could test different options and configurations to compare performance difference to RAM disk solution.


I'm sorry - do you mean someone would test it using YOUR queries ?




I'm asking you not to optimize my queries and configs, but to perform some tests (and for site owners - to write an article) comparing RAM disk solution vs optimized memory usage on heavy load.

2 sterin
I'll try to measure this today
Re: Storing InnoDB tables on RAM disk [message #2096 is a reply to message #2094 ] Thu, 18 October 2007 05:35 Go to previous messageGo to next message
AlexN  is currently offline AlexN
Messages: 20
Registered: October 2007
Location: Moscow,Russia
Junior Member
Nomad wrote on Thu, 18 October 2007 03:58



I'm asking you not to optimize my queries and configs, but to perform some tests (and for site owners - to write an article) comparing RAM disk solution vs optimized memory usage on heavy load.




It is easy to predict that the answer will depend on queries you are using. In fact, I can easily propose a setup where there will be
no difference between RAM disc and query cache performance.
After some thoughts and experiments I think I'll be able to show
a setup where this difference will be huge.


Re: Storing InnoDB tables on RAM disk [message #2101 is a reply to message #2094 ] Fri, 19 October 2007 00:51 Go to previous messageGo to next message
scoundrel  is currently offline scoundrel
Messages: 58
Registered: August 2006
Location: Toronto, ON, Canada
Member

Nomad wrote on Thu, 18 October 2007 03:58

AlexN wrote on Thu, 18 October 2007 10:09

Nomad wrote on Wed, 17 October 2007 16:00

May be someone could test different options and configurations to compare performance difference to RAM disk solution.


I'm sorry - do you mean someone would test it using YOUR queries ?




I'm asking you not to optimize my queries and configs, but to perform some tests (and for site owners - to write an article) comparing RAM disk solution vs optimized memory usage on heavy load.



If you'd like us to perform these tests for you, I can send you our hourly rates info.


Alexey Kovyrin, MySQL Performance Expert
MySQL Performance Blog
MySQL Consulting
Re: Storing InnoDB tables on RAM disk [message #2102 is a reply to message #2077 ] Fri, 19 October 2007 00:58 Go to previous messageGo to next message
scoundrel  is currently offline scoundrel
Messages: 58
Registered: August 2006
Location: Toronto, ON, Canada
Member

Nomad wrote on Wed, 17 October 2007 07:15

Greetings

Currently I'm using MySQL 5.0 and InnoDB to store small database (1.1-1.8GB).


First of all, if you use in-memory db, you don't really need innodb because the most useful feature there is reliability and in your case it has no advantages at all. So, I'd suggest you to try myisam tables (they'd use less "disk" space because of compression) or MEMORY(HEAP) storage engine, which is in-memory by default.

As for tests, your approach is REALLY strange and I personally don't see any reasons to test such configuration because even if we'd be able to show that it is really fast, no one would use such approach because mysql has its own in-memory db implementation for such needs.


Alexey Kovyrin, MySQL Performance Expert
MySQL Performance Blog
MySQL Consulting
Re: Storing InnoDB tables on RAM disk [message #2104 is a reply to message #2102 ] Fri, 19 October 2007 02:45 Go to previous message
AlexN  is currently offline AlexN
Messages: 20
Registered: October 2007
Location: Moscow,Russia
Junior Member
scoundrel wrote on Fri, 19 October 2007 00:58


First of all, if you use in-memory db, you don't really need innodb because the most useful feature there is reliability and in your case it has no advantages at all.




Earlier he explained his reasons for using InnoDB.

scoundrel wrote on Fri, 19 October 2007 00:58


As for tests, your approach is REALLY strange and I personally don't see any reasons to test such configuration because even if we'd be able to show that it is really fast, no one would use such approach because mysql has its own in-memory db implementation for such needs.


The only reason for using RAM disc instead of in-memory tables
I see is faster warm-up. Initialization. Data load. In case of
memory table there should be a great number of insert statements
(even if packed into one dump file). In case of RAM disc just copy few files.


Previous Topic:Query tuning - Performance Issue
Next Topic:more RAM, less performance
Goto Forum:
  


Current Time: Thu Jul 9 20:45:51 EDT 2009

Total time taken to generate the page: 0.01637 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 2.7.7.
Copyright ©2001-2007 FUD Forum Bulletin Board Software

MySQL is a trademark of Sun Microsystems.
InnoDB is a trademark of Oracle Corp.

Percona Performance Forums are a service of Percona, Inc.
Not affiliated with Sun Microsystems or Oracle Corp.