Вопрос: Оптимизация MySQL по результатам mysqltuner.pl
Добрый день всем, честно говоря я не профи в настройке mysql но после переноса части сайтов мускул стал грузить процессор под 300-500 процентов что и заставило меня полезть в более глубокую настройку.
Временные файлы вынес уже в ОЗУ, таблицы чекал раза четыре и разными способами но количество фрагментированных таблиц остается в пределах 300-500 таблиц, как ни крути. Не знаю почему, но вопрос в другом. Следую плавно советам mysqltuner и подкручиваю значения, паралелльно читая что за что отвечает и где какие должны быть оптимальные значения но уже начинаю волноватся за некоторые параметры которые кажутся "большими" скромно говоря. Можете проверить и подсказать что где не так или на что стоит обратить внимание, буду очень признателен.
вот содержимое файла my.cnf
#
# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html
# This will be passed to all mysql clients
# It has been reported that passwords should be enclosed with ticks/quotes
# escpecially if they contain "#" chars...
# Remember to edit /etc/mysql/debian.cnf when changing the socket location.
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
default-character-set = utf8
# Here is entries for some specific programs
# The following values assume you have at least 32M ram
# This was formally known as [safe_mysqld]. Both versions are currently parsed.
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
#
# * Basic Settings
#
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /dev/shm
language = /usr/share/mysql/english
skip-external-locking
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#
# * Fine Tuning
#
key_buffer = 3048M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 32
join_buffer_size = 256K
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover = BACKUP
max_connections = 300
table_cache = 8096
open_files_limit = 16192
#interactive_timeout = 28800
#wait_timeout = 28800
#thread_concurrency = 10
#
# * Query Cache Configuration
#
query_cache_limit = 18M
query_cache_size = 896M
table_open_cache = 8000
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file = /var/log/mysql/mysql.log
#general_log = 1
#
# Error logging goes to syslog due to /etc/mysql/conf.d/mysqld_safe_syslog.cnf.
#
# Here you can see queries with especially long duration
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 2
log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
# other settings you may need to change.
#server-id = 1
#log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 10
max_binlog_size = 100M
default-character-set = utf8
#binlog_do_db = include_database_name
#binlog_ignore_db = include_database_name
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem
tmp_table_size = 512M
max_heap_table_size = 256M
table_cache = 4048
innodb_buffer_pool_size = 2048M
[mysqldump]
quick
quote-names
max_allowed_packet = 16M
default-character-set = utf8
[mysql]
#no-auto-rehash # faster start of mysql but no tab completition
default-character-set = utf8
[isamchk]
key_buffer = 2048M
#
# * IMPORTANT: Additional settings that can override those from this file!
# The files must end with '.cnf', otherwise they'll be ignored.
#
!includedir /etc/mysql/conf.d/
вот что выдает myslqtuner
-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.1.63-0+squeeze1-log
[OK] Operating on 64-bit architecture
-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in MyISAM tables: 11G (Tables: 3504)
[--] Data in InnoDB tables: 1G (Tables: 292)
[--] Data in MEMORY tables: 0B (Tables: 4)
[!!] Total fragmented tables: 461
-------- Security Recommendations -------------------------------------------
[OK] All database users have passwords assigned
-------- Performance Metrics -------------------------------------------------
[--] Up for: 29m 5s (523K q [300.139 qps], 18K conn, TX: 2B, RX: 63M)
[--] Reads / Writes: 94% / 6%
[--] Total buffers: 6.1G global + 2.8M per thread (300 max threads)
[OK] Maximum possible memory usage: 6.9G (22% of installed RAM)
[OK] Slow queries: 1% (10K/523K)
[OK] Highest usage of available connections: 21% (65/300)
[OK] Key buffer size / total MyISAM indexes: 3.0G/1.5G
[OK] Key buffer hit rate: 100.0% (966M cached / 219K reads)
[OK] Query cache efficiency: 64.7% (284K cached / 439K selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 0% (260 temp sorts / 54K sorts)
[!!] Joins performed without indexes: 1227
[!!] Temporary tables created on disk: 42% (12K on disk / 28K total)
[OK] Thread cache hit rate: 99% (65 created / 18K connections)
[OK] Table cache hit rate: 26% (4K open / 15K opened)
[OK] Open file limit used: 44% (7K/16K)
[OK] Table locks acquired immediately: 99% (191K immediate / 191K locks)
[OK] InnoDB data size / buffer pool: 1.5G/2.0G
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
MySQL started within last 24 hours - recommendations may be inaccurate
Adjust your join queries to always utilize indexes
Temporary table size is already large - reduce result set size
Reduce your SELECT DISTINCT queries without LIMIT clauses
Variables to adjust:
join_buffer_size (> 256.0K, or always use indexes with joins)
вот что творится в топе
top - 00:48:28 up 237 days, 7:29, 1 user, load average: 5.15, 6.64, 7.16
Tasks: 307 total, 5 running, 302 sleeping, 0 stopped, 0 zombie
Cpu(s): 75.1%us, 14.2%sy, 0.0%ni, 9.8%id, 0.7%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 32703444k total, 31550156k used, 1153288k free, 2300076k buffers
Swap: 33553332k total, 8296k used, 33545036k free, 21243024k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
15750 mysql 20 0 6620m 2.6g 7896 S 345 8.4 70:06.29 mysqld
23246 www-data 20 0 344m 127m 7276 R 73 0.4 0:35.28 apache2
24004 www-data 20 0 290m 77m 5928 S 44 0.2 0:06.39 apache2
23957 www-data 20 0 292m 77m 5976 R 41 0.2 0:19.32 apache2
24025 www-data 20 0 272m 60m 4292 S 30 0.2 0:00.90 apache2
23913 www-data 20 0 295m 83m 6716 R 29 0.3 0:37.49 apache2
23912 www-data 20 0 277m 65m 6768 S 25 0.2 0:47.11 apache2
24003 www-data 20 0 273m 60m 6232 S 24 0.2 0:16.04 apache2
23292 www-data 20 0 290m 78m 7556 S 21 0.2 1:06.56 apache2
23935 www-data 20 0 284m 69m 6780 S 19 0.2 0:32.12 apache2
23922 www-data 20 0 291m 78m 6120 R 16 0.2 1:02.37 apache2
23920 www-data 20 0 278m 65m 6872 S 15 0.2 0:21.00 apache2
23958 www-data 20 0 279m 67m 6920 S 11 0.2 0:18.90 apache2
24023 www-data 20 0 268m 57m 4784 S 10 0.2 0:01.65 apache2
23286 www-data 20 0 281m 67m 6816 S 9 0.2 0:07.07 apache2
23291 www-data 20 0 288m 74m 7092 S 1 0.2 0:33.66 apache2
9591 root 20 0 0 0 0 S 0 0.0 1:51.62 kworker/4:1
18429 root 20 0 0 0 0 S 0 0.0 0:37.95 kworker/1:1
23311 www-data 20 0 277m 66m 6568 S 0 0.2 0:03.41 apache2
24024 root 20 0 19256 1456 948 R 0 0.0 0:00.02 top
26329 gitlab 20 0 943m 133m 2168 S 0 0.4 167:56.34 ruby1.9.1
28214 www-data 20 0 56732 28m 1788 S 0 0.1 1:08.11 nginx
1 root 20 0 8404 716 592 S 0 0.0 2:09.28 init
2 root 20 0 0 0 0 S 0 0.0 0:00.22 kthreadd
3 root 20 0 0 0 0 S 0 0.0 9:10.40 ksoftirqd/0
6 root RT 0 0 0 0 S 0 0.0 0:26.19 migration/0
7 root RT 0 0 0 0 S 0 0.0 0:47.69 watchdog/0
8 root RT 0 0 0 0 S 0 0.0 0:32.35 migration/1
10 root 20 0 0 0 0 S 0 0.0 5:38.45 ksoftirqd/1
12 root RT 0 0 0 0 S 0 0.0 0:31.18 watchdog/1
13 root RT 0 0 0 0 S 0 0.0 0:23.97 migration/2
15 root 20 0 0 0 0 S 0 0.0 5:21.56 ksoftirqd/2
16 root RT 0 0 0 0 S 0 0.0 0:23.93 watchdog/2
17 root RT 0 0 0 0 S 0 0.0 0:18.57 migration/3
19 root 20 0 0 0 0 S 0 0.0 5:15.37 ksoftirqd/3
в целом в данный момент показатели вроде в норме но иногда без видимой причины резко растет la и вырастает до 20, иногда до 50 и mysqltuner выдает совет увеличить
query_cache_limit = 18M
query_cache_size = 896M
которые на мой взгляд уже нереально большие обосновывая это большим количеством
Query cache prunes per day
Баз на сервере очень много, 32гб памяти, поэтому все так сложно. Подскажите, буду очень признателен за любые советы и мнения.
Ответ:
ну вот, при кеше в 128мб уже через 15 минут полезли prunes
-------- Performance Metrics -------------------------------------------------
[--] Up for: 14m 53s (328K q [368.365 qps], 12K conn, TX: 1B, RX: 43M)
[--] Reads / Writes: 93% / 7%
[--] Total buffers: 4.4G global + 6.6M per thread (300 max threads)
[OK] Maximum possible memory usage: 6.3G (20% of installed RAM)
[OK] Slow queries: 2% (8K/328K)
[OK] Highest usage of available connections: 16% (49/300)
[OK] Key buffer size / total MyISAM indexes: 2.0G/1.5G
[OK] Key buffer hit rate: 100.0% (545M cached / 178K reads)
[OK] Query cache efficiency: 63.3% (176K cached / 278K selects)
[!!] Query cache prunes per day: 1996875
[OK] Sorts requiring temporary tables: 0% (141 temp sorts / 35K sorts)
[!!] Joins performed without indexes: 1069
[!!] Temporary tables created on disk: 41% (7K on disk / 18K total)
[OK] Thread cache hit rate: 99% (49 created / 12K connections)
[OK] Table cache hit rate: 25% (4K open / 15K opened)
[OK] Open file limit used: 44% (7K/16K)
[OK] Table locks acquired immediately: 99% (128K immediate / 128K locks)
[OK] InnoDB data size / buffer pool: 1.5G/2.0G
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
MySQL started within last 24 hours - recommendations may be inaccurate
Adjust your join queries to always utilize indexes
Temporary table size is already large - reduce result set size
Reduce your SELECT DISTINCT queries without LIMIT clauses
Variables to adjust:
query_cache_size (> 128M)
join_buffer_size (> 4.0M, or always use indexes with joins)
Сейчас поставлю 256 но уверен что и этого будет мало и будет просить дальше, до примерно отметки в 1гб (1гб долго не гонял, возможно и там будет лезть)
Удобный поиск информации с популярных форумов в одном месте