insert - Mysql commit sometimes takes minutes to finish -
ive been having problem on 1 week now. ive allready gone though similar problems other people had here on stackoverflow. im running mysql 5.6.10.
- "query end" step long @ random times
- update statements in "query end state"
- lots of "query end" states in mysql, connections used in matter of minutes
it happends randomly. every 5 minutes or so, commit takes very long time. , size of transation insert around 1-2000 rows.
ive been changing settings much, ive kind of lost decided create questions here.
my inserts/updates sqlfiles have been generated program. have windows service runs every 1,5 minutes processing files. build string of many sql files in transaction mode. works 4 out of 5 time smooth, 1 out of 5 jams.
it happends on same table, , of times regarding inserts happends on updates. doubt has structure of tables never know until solved :)
this table structure
create table `int_links_root_1` ( `intlinksrootid` bigint(20) not null auto_increment, `urlid` int(11) not null, `link` varchar(2000) not null, `anchortext` varchar(500) default null, `title` varchar(2000) default null, `pagetext` varchar(8000) default null, `added` datetime default null, `lastmodifieddatetime` datetime default null, primary key (`intlinksrootid`), key `int_links_urlid_idx_1` (`urlid`), fulltext key `fulltext` (`anchortext`,`title`,`pagetext`), constraint `int_links_urlid_idx_1` foreign key (`urlid`) references `urls` (`urlid`) on delete cascade on update no action ) engine=innodb auto_increment=1708708 default charset=utf8$$
these 'show variables'
auto_increment_increment 1 auto_increment_offset 1 autocommit on automatic_sp_privileges on back_log 90 basedir c:/program files/mysql/mysql server 5.6/ big_tables off bind_address * binlog_cache_size 32768 binlog_checksum crc32 binlog_direct_non_transactional_updates off binlog_format statement binlog_max_flush_queue_time 0 binlog_order_commits on binlog_row_image full binlog_rows_query_log_events off binlog_stmt_cache_size 32768 bulk_insert_buffer_size 8388608 character_set_client utf8 character_set_connection utf8 character_set_database latin1 character_set_filesystem binary character_set_results utf8 character_set_server latin1 character_set_system utf8 character_sets_dir c:\program files\mysql\mysql server 5.6\share\charsets\ collation_connection utf8_general_ci collation_database latin1_swedish_ci collation_server latin1_swedish_ci completion_type no_chain concurrent_insert auto connect_timeout 10 core_file off datadir d:\mysql6\ date_format %y-%m-%d datetime_format %y-%m-%d %h:%i:%s default_storage_engine innodb default_tmp_storage_engine innodb default_week_format 0 delay_key_write on delayed_insert_limit 100 delayed_insert_timeout 300 delayed_queue_size 1000 disconnect_on_expired_password on div_precision_increment 4 end_markers_in_json off enforce_gtid_consistency off eq_range_index_dive_limit 10 error_count 0 event_scheduler off expire_logs_days 0 explicit_defaults_for_timestamp off external_user flush off flush_time 0 foreign_key_checks on ft_boolean_syntax + -><()~*:""&| ft_max_word_len 84 ft_min_word_len 3 ft_query_expansion_limit 20 ft_stopword_file (built-in) general_log on general_log_file d:\mysql6\win-0pccfi7ppin.log group_concat_max_len 1024 gtid_executed gtid_mode off gtid_next automatic gtid_owned gtid_purged have_compress yes have_crypt no have_dynamic_loading yes have_geometry yes have_openssl disabled have_profiling yes have_query_cache yes have_rtree_keys yes have_ssl disabled have_symlink yes host_cache_size 328 hostname win-0pccfi7ppin identity 0 ignore_builtin_innodb off ignore_db_dirs init_connect init_file init_slave innodb_adaptive_flushing on innodb_adaptive_flushing_lwm 10 innodb_adaptive_hash_index on innodb_adaptive_max_sleep_delay 150000 innodb_additional_mem_pool_size 8388608 innodb_api_bk_commit_interval 5 innodb_api_disable_rowlock off innodb_api_enable_binlog off innodb_api_enable_mdl off innodb_api_trx_level 0 innodb_autoextend_increment 64 innodb_autoinc_lock_mode 1 innodb_buffer_pool_dump_at_shutdown off innodb_buffer_pool_dump_now off innodb_buffer_pool_filename ib_buffer_pool innodb_buffer_pool_instances 8 innodb_buffer_pool_load_abort off innodb_buffer_pool_load_at_startup off innodb_buffer_pool_load_now off innodb_buffer_pool_size 524288000 innodb_change_buffer_max_size 25 innodb_change_buffering innodb_checksum_algorithm innodb innodb_checksums on innodb_cmp_per_index_enabled off innodb_commit_concurrency 0 innodb_compression_failure_threshold_pct 5 innodb_compression_level 6 innodb_compression_pad_pct_max 50 innodb_concurrency_tickets 5000 innodb_data_file_path ibdata1:10m:autoextend innodb_data_home_dir d:\mysql6 innodb_disable_sort_file_cache off innodb_doublewrite on innodb_fast_shutdown 1 innodb_file_format antelope innodb_file_format_check on innodb_file_format_max antelope innodb_file_per_table on innodb_flush_log_at_timeout 1 innodb_flush_log_at_trx_commit 0 innodb_flush_method innodb_flush_neighbors 1 innodb_flushing_avg_loops 30 innodb_force_load_corrupted off innodb_force_recovery 0 innodb_ft_aux_table innodb_ft_cache_size 8000000 innodb_ft_enable_diag_print off innodb_ft_enable_stopword on innodb_ft_max_token_size 84 innodb_ft_min_token_size 3 innodb_ft_num_word_optimize 2000 innodb_ft_server_stopword_table innodb_ft_sort_pll_degree 2 innodb_ft_user_stopword_table innodb_io_capacity 200 innodb_io_capacity_max 2000 innodb_large_prefix off innodb_lock_wait_timeout 30 innodb_locks_unsafe_for_binlog off innodb_log_buffer_size 2097152 innodb_log_file_size 262144000 innodb_log_files_in_group 2 innodb_log_group_home_dir .\ innodb_lru_scan_depth 1024 innodb_max_dirty_pages_pct 0 innodb_max_dirty_pages_pct_lwm 0 innodb_max_purge_lag 0 innodb_max_purge_lag_delay 0 innodb_mirrored_log_groups 1 innodb_monitor_disable innodb_monitor_enable innodb_monitor_reset innodb_monitor_reset_all innodb_old_blocks_pct 37 innodb_old_blocks_time 1000 innodb_online_alter_log_max_size 134217728 innodb_open_files 2000 innodb_optimize_fulltext_only off innodb_page_size 16384 innodb_print_all_deadlocks off innodb_purge_batch_size 300 innodb_purge_threads 1 innodb_random_read_ahead off innodb_read_ahead_threshold 56 innodb_read_io_threads 4 innodb_read_only off innodb_replication_delay 0 innodb_rollback_on_timeout off innodb_rollback_segments 128 innodb_sort_buffer_size 1048576 innodb_spin_wait_delay 6 innodb_stats_auto_recalc on innodb_stats_method nulls_equal innodb_stats_on_metadata off innodb_stats_persistent on innodb_stats_persistent_sample_pages 20 innodb_stats_sample_pages 8 innodb_stats_transient_sample_pages 8 innodb_strict_mode off innodb_support_xa off innodb_sync_array_size 1 innodb_sync_spin_loops 30 innodb_table_locks on innodb_thread_concurrency 10 innodb_thread_sleep_delay 0 innodb_undo_directory . innodb_undo_logs 128 innodb_undo_tablespaces 0 innodb_use_native_aio on innodb_use_sys_malloc on innodb_version 1.2.10 innodb_write_io_threads 20 insert_id 0 interactive_timeout 28800 join_buffer_size 262144 keep_files_on_create off key_buffer_size 67108864 key_cache_age_threshold 300 key_cache_block_size 1024 key_cache_division_limit 100 large_files_support on large_page_size 0 large_pages off last_insert_id 0 lc_messages en_us lc_messages_dir c:\program files\mysql\mysql server 5.6\share\ lc_time_names en_us license gpl local_infile on lock_wait_timeout 31536000 log_bin off log_bin_basename log_bin_index log_bin_trust_function_creators off log_bin_use_v1_row_events off log_error d:\mysql6\error_log.txt log_output table log_queries_not_using_indexes off log_slave_updates off log_throttle_queries_not_using_indexes 0 log_warnings 1 long_query_time 3.000000 low_priority_updates off lower_case_file_system on lower_case_table_names 1 master_info_repository file master_verify_checksum off max_allowed_packet 4194304 max_binlog_cache_size 18446744073709547520 max_binlog_size 1073741824 max_binlog_stmt_cache_size 18446744073709547520 max_connect_errors 100 max_connections 200 max_delayed_threads 20 max_error_count 64 max_heap_table_size 16777216 max_insert_delayed_threads 20 max_join_size 18446744073709551615 max_length_for_sort_data 1024 max_prepared_stmt_count 16382 max_relay_log_size 0 max_seeks_for_key 4294967295 max_sort_length 1024 max_sp_recursion_depth 0 max_tmp_tables 32 max_user_connections 0 max_write_lock_count 4294967295 metadata_locks_cache_size 1024 metadata_locks_hash_instances 8 min_examined_row_limit 0 multi_range_count 256 myisam_data_pointer_size 6 myisam_max_sort_file_size 107374182400 myisam_mmap_size 18446744073709551615 myisam_recover_options off myisam_repair_threads 1 myisam_sort_buffer_size 209715200 myisam_stats_method nulls_unequal myisam_use_mmap off named_pipe on net_buffer_length 16384 net_read_timeout 30 net_retry_count 10 net_write_timeout 60 new off old off old_alter_table off old_passwords 0 open_files_limit 7048 optimizer_prune_level 1 optimizer_search_depth 62 optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on,index_condition_pushdown=on,mrr=on,mrr_cost_based=on,block_nested_loop=on,batched_key_access=off,materialization=on,semijoin=on,loosesc... optimizer_trace enabled=off,one_line=off optimizer_trace_features greedy_search=on,range_optimizer=on,dynamic_range=on,repeated_subselect=on optimizer_trace_limit 1 optimizer_trace_max_mem_size 16384 optimizer_trace_offset -1 performance_schema on performance_schema_accounts_size 100 performance_schema_digests_size 5000 performance_schema_events_stages_history_long_size 1000 performance_schema_events_stages_history_size 10 performance_schema_events_statements_history_long_size 1000 performance_schema_events_statements_history_size 10 performance_schema_events_waits_history_long_size 1000 performance_schema_events_waits_history_size 10 performance_schema_hosts_size 100 performance_schema_max_cond_classes 80 performance_schema_max_cond_instances 1215 performance_schema_max_file_classes 50 performance_schema_max_file_handles 32768 performance_schema_max_file_instances 1750 performance_schema_max_mutex_classes 200 performance_schema_max_mutex_instances 4429 performance_schema_max_rwlock_classes 30 performance_schema_max_rwlock_instances 2286 performance_schema_max_socket_classes 10 performance_schema_max_socket_instances 300 performance_schema_max_stage_classes 150 performance_schema_max_statement_classes 167 performance_schema_max_table_handles 2858 performance_schema_max_table_instances 556 performance_schema_max_thread_classes 50 performance_schema_max_thread_instances 358 performance_schema_session_connect_attrs_size 512 performance_schema_setup_actors_size 100 performance_schema_setup_objects_size 100 performance_schema_users_size 100 port 3330 preload_buffer_size 32768 profiling off profiling_history_size 15 protocol_version 10 proxy_user pseudo_slave_mode off pseudo_thread_id 1520 query_alloc_block_size 8192 query_cache_limit 1048576 query_cache_min_res_unit 4096 query_cache_size 0 query_cache_type off query_cache_wlock_invalidate off query_prealloc_size 8192 rand_seed1 0 rand_seed2 0 range_alloc_block_size 4096 read_buffer_size 65536 read_only off read_rnd_buffer_size 262144 relay_log relay_log_basename relay_log_index relay_log_info_file relay-log.info relay_log_info_repository file relay_log_purge on relay_log_recovery off relay_log_space_limit 0 report_host report_password report_user secure_auth on secure_file_priv server_id 0 server_id_bits 32 server_uuid 48f32a6c-9836-11e2-9e13-984be17fa006 shared_memory off shared_memory_base_name mysql skip_external_locking on skip_name_resolve off skip_networking off skip_show_database off slave_allow_batching off slave_checkpoint_group 512 slave_checkpoint_period 300 slave_compressed_protocol off slave_exec_mode strict slave_load_tmpdir d:\mysql6\temp slave_max_allowed_packet 1073741824 slave_net_timeout 3600 slave_parallel_workers 0 slave_pending_jobs_size_max 16777216 slave_rows_search_algorithms table_scan,index_scan slave_skip_errors off slave_sql_verify_checksum on slave_transaction_retries 10 slave_type_conversions slow_launch_time 2 slow_query_log on slow_query_log_file d:\mysql6\win-0pccfi7ppin-slow.log socket mysqlpipe6 sort_buffer_size 524288 sql_auto_is_null off sql_big_selects on sql_buffer_result off sql_log_bin on sql_log_off off sql_mode strict_trans_tables,no_auto_create_user,no_engine_substitution sql_notes on sql_quote_show_create on sql_safe_updates on sql_select_limit 18446744073709551615 sql_slave_skip_counter 0 sql_warnings off ssl_ca ssl_capath ssl_cert ssl_cipher ssl_crl ssl_crlpath ssl_key storage_engine innodb stored_program_cache 256 sync_binlog 0 sync_frm on sync_master_info 10000 sync_relay_log 10000 sync_relay_log_info 10000 system_time_zone w. europe daylight time table_definition_cache 1400 table_open_cache 2000 table_open_cache_instances 1 thread_cache_size 8 thread_concurrency 10 thread_handling one-thread-per-connection thread_stack 262144 time_format %h:%i:%s time_zone system timed_mutexes off timestamp 1368530016.924189 tmp_table_size 211812352 tmpdir d:/mysql6/temp transaction_alloc_block_size 8192 transaction_allow_batching off transaction_prealloc_size 4096 tx_isolation repeatable-read tx_read_only off unique_checks on updatable_views_with_limit yes version 5.6.10-log version_comment mysql community server (gpl) version_compile_machine x86_64 version_compile_os win64 wait_timeout 28800 warning_count 0
this show status
aborted_clients 26 aborted_connects 1 binlog_cache_disk_use 0 binlog_cache_use 0 binlog_stmt_cache_disk_use 0 binlog_stmt_cache_use 0 bytes_received 1203 bytes_sent 1936957 com_admin_commands 17 com_assign_to_keycache 0 com_alter_db 0 com_alter_db_upgrade 0 com_alter_event 0 com_alter_function 0 com_alter_procedure 0 com_alter_server 0 com_alter_table 0 com_alter_tablespace 0 com_alter_user 0 com_analyze 0 com_begin 0 com_binlog 0 com_call_procedure 0 com_change_db 1 com_change_master 0 com_check 0 com_checksum 0 com_commit 0 com_create_db 0 com_create_event 0 com_create_function 0 com_create_index 0 com_create_procedure 0 com_create_server 0 com_create_table 0 com_create_trigger 0 com_create_udf 0 com_create_user 0 com_create_view 0 com_dealloc_sql 0 com_delete 0 com_delete_multi 0 com_do 0 com_drop_db 0 com_drop_event 0 com_drop_function 0 com_drop_index 0 com_drop_procedure 0 com_drop_server 0 com_drop_table 0 com_drop_trigger 0 com_drop_user 0 com_drop_view 0 com_empty_query 0 com_execute_sql 0 com_flush 0 com_get_diagnostics 0 com_grant 0 com_ha_close 0 com_ha_open 0 com_ha_read 0 com_help 0 com_insert 0 com_insert_select 0 com_install_plugin 0 com_kill 0 com_load 0 com_lock_tables 0 com_optimize 0 com_preload_keys 0 com_prepare_sql 0 com_purge 0 com_purge_before_date 0 com_release_savepoint 0 com_rename_table 0 com_rename_user 0 com_repair 0 com_replace 0 com_replace_select 0 com_reset 0 com_resignal 0 com_revoke 0 com_revoke_all 0 com_rollback 0 com_rollback_to_savepoint 0 com_savepoint 0 com_select 5 com_set_option 6 com_signal 0 com_show_binlog_events 0 com_show_binlogs 0 com_show_charsets 0 com_show_collations 0 com_show_create_db 0 com_show_create_event 0 com_show_create_func 0 com_show_create_proc 0 com_show_create_table 0 com_show_create_trigger 0 com_show_databases 0 com_show_engine_logs 0 com_show_engine_mutex 0 com_show_engine_status 0 com_show_events 0 com_show_errors 0 com_show_fields 0 com_show_function_code 0 com_show_function_status 0 com_show_grants 0 com_show_keys 0 com_show_master_status 0 com_show_open_tables 0 com_show_plugins 0 com_show_privileges 0 com_show_procedure_code 0 com_show_procedure_status 0 com_show_processlist 0 com_show_profile 0 com_show_profiles 0 com_show_relaylog_events 0 com_show_slave_hosts 0 com_show_slave_status 0 com_show_status 3 com_show_storage_engines 0 com_show_table_status 0 com_show_tables 0 com_show_triggers 0 com_show_variables 9 com_show_warnings 0 com_slave_start 0 com_slave_stop 0 com_stmt_close 0 com_stmt_execute 0 com_stmt_fetch 0 com_stmt_prepare 0 com_stmt_reprepare 0 com_stmt_reset 0 com_stmt_send_long_data 0 com_truncate 0 com_uninstall_plugin 0 com_unlock_tables 0 com_update 0 com_update_multi 0 com_xa_commit 0 com_xa_end 0 com_xa_prepare 0 com_xa_recover 0 com_xa_rollback 0 com_xa_start 0 compression off connection_errors_accept 0 connection_errors_internal 0 connection_errors_max_connections 0 connection_errors_peer_address 0 connection_errors_select 0 connection_errors_tcpwrap 0 connections 152 created_tmp_disk_tables 0 created_tmp_files 5 created_tmp_tables 9 delayed_errors 0 delayed_insert_threads 0 delayed_writes 0 flush_commands 1 handler_commit 0 handler_delete 0 handler_discover 0 handler_external_lock 8 handler_mrr_init 0 handler_prepare 0 handler_read_first 0 handler_read_key 0 handler_read_last 0 handler_read_next 0 handler_read_prev 0 handler_read_rnd 1122 handler_read_rnd_next 2463 handler_rollback 0 handler_savepoint 0 handler_savepoint_rollback 0 handler_update 0 handler_write 1333 innodb_buffer_pool_dump_status not started innodb_buffer_pool_load_status not started innodb_buffer_pool_pages_data 259 innodb_buffer_pool_bytes_data 4243456 innodb_buffer_pool_pages_dirty 170 innodb_buffer_pool_bytes_dirty 2785280 innodb_buffer_pool_pages_flushed 122362 innodb_buffer_pool_pages_free 380 innodb_buffer_pool_pages_misc 1 innodb_buffer_pool_pages_total 640 innodb_buffer_pool_read_ahead_rnd 0 innodb_buffer_pool_read_ahead 14036 innodb_buffer_pool_read_ahead_evicted 370 innodb_buffer_pool_read_requests 15129702 innodb_buffer_pool_reads 196618 innodb_buffer_pool_wait_free 152 innodb_buffer_pool_write_requests 727352 innodb_data_fsyncs 18008 innodb_data_pending_fsyncs 1 innodb_data_pending_reads 0 innodb_data_pending_writes 0 innodb_data_read 3462483968 innodb_data_reads 211501 innodb_data_writes 155812 innodb_data_written 4099368960 innodb_dblwr_pages_written 122602 innodb_dblwr_writes 2140 innodb_have_atomic_builtins on innodb_log_waits 0 innodb_log_write_requests 157303 innodb_log_writes 29814 innodb_os_log_fsyncs 4332 innodb_os_log_pending_fsyncs 0 innodb_os_log_pending_writes 1 innodb_os_log_written 83195392 innodb_page_size 16384 innodb_pages_created 3118 innodb_pages_read 211072 innodb_pages_written 122514 innodb_row_lock_current_waits 0 innodb_row_lock_time 261518 innodb_row_lock_time_avg 411 innodb_row_lock_time_max 30420 innodb_row_lock_waits 635 innodb_rows_deleted 0 innodb_rows_inserted 103754 innodb_rows_read 3291528 innodb_rows_updated 86170 innodb_num_open_files 52 innodb_truncated_status_writes 0 innodb_available_undo_logs 128 key_blocks_not_flushed 0 key_blocks_unused 53581 key_blocks_used 4 key_read_requests 248 key_reads 4 key_write_requests 0 key_writes 0 last_query_cost 10.499000 last_query_partial_plans 1 max_used_connections 8 not_flushed_delayed_rows 0 open_files 22 open_streams 0 open_table_definitions 86 open_tables 96 opened_files 207 opened_table_definitions 0 opened_tables 0 performance_schema_accounts_lost 0 performance_schema_cond_classes_lost 0 performance_schema_cond_instances_lost 0 performance_schema_digest_lost 0 performance_schema_file_classes_lost 0 performance_schema_file_handles_lost 0 performance_schema_file_instances_lost 0 performance_schema_hosts_lost 0 performance_schema_locker_lost 0 performance_schema_mutex_classes_lost 0 performance_schema_mutex_instances_lost 0 performance_schema_rwlock_classes_lost 0 performance_schema_rwlock_instances_lost 0 performance_schema_session_connect_attrs_lost 0 performance_schema_socket_classes_lost 0 performance_schema_socket_instances_lost 0 performance_schema_stage_classes_lost 0 performance_schema_statement_classes_lost 0 performance_schema_table_handles_lost 0 performance_schema_table_instances_lost 0 performance_schema_thread_classes_lost 0 performance_schema_thread_instances_lost 0 performance_schema_users_lost 0 prepared_stmt_count 0 qcache_free_blocks 1 qcache_free_memory 10468392 qcache_hits 0 qcache_inserts 0 qcache_lowmem_prunes 0 qcache_not_cached 107 qcache_queries_in_cache 0 qcache_total_blocks 1 queries 56190 questions 25 select_full_join 0 select_full_range_join 0 select_range 0 select_range_check 0 select_scan 10 slave_heartbeat_period slave_last_heartbeat slave_open_temp_tables 0 slave_received_heartbeats slave_retried_transactions slave_running off slow_launch_threads 0 slow_queries 2 sort_merge_passes 0 sort_range 0 sort_rows 1122 sort_scan 2 ssl_accept_renegotiates 0 ssl_accepts 0 ssl_callback_cache_hits 0 ssl_cipher ssl_cipher_list ssl_client_connects 0 ssl_connect_renegotiates 0 ssl_ctx_verify_depth 0 ssl_ctx_verify_mode 0 ssl_default_timeout 0 ssl_finished_accepts 0 ssl_finished_connects 0 ssl_server_not_after ssl_server_not_before ssl_session_cache_hits 0 ssl_session_cache_misses 0 ssl_session_cache_mode none ssl_session_cache_overflows 0 ssl_session_cache_size 0 ssl_session_cache_timeouts 0 ssl_sessions_reused 0 ssl_used_session_cache_entries 0 ssl_verify_depth 0 ssl_verify_mode 0 ssl_version table_locks_immediate 50440 table_locks_waited 0 table_open_cache_hits 4 table_open_cache_misses 0 table_open_cache_overflows 0 tc_log_max_pages_used 0 tc_log_page_size 0 tc_log_page_waits 0 threads_cached 3 threads_connected 5 threads_created 8 threads_running 3 uptime 3329 uptime_since_flush_status 3329
i have tried different innodb_flush_log_at_trx_commit settings. im queries optimal run fast occasionaly, im has flushing. ive tried disabling autocommit still happends. query cache disabled. ideas more welcome. maybe disks slow?
the system not in production can restart , test suggestions.
please let me know if can provide more details.
are sure previous commits finished ? not faniliar inner structure of mysql couldnt rebuild indexing, slow large tables. or can run multiple queries @ time fifth 1 out of rhreads count , in queue waiting rest finish.
Comments
Post a Comment