Write a log event to the general log before executing a statemment in PS mode
This change fixes failure of the test main.log_tables when it is run in PS mode
Updated the test main.events_logs_tests to query mysql.general_log
for the command_type 'Query' and 'Execute' in order to have consistent
test result both in case the test is run with --ps-protocol and without it
Withing this task the following changes were made:
- Added sending of metadata info in prepare phase for the admin related
command (check table, checksum table, repair, optimize, analyze).
- Refactored implmentation of HELP command to support its execution in
PS mode
- Added support for execution of LOAD INTO and XA- related statements
in PS mode
- Modified mysqltest.cc to run statements in PS mode unconditionally
in case the option --ps-protocol is set. Formerly, only those statements
were executed using PS protocol that matched the hard-coded regular expression
- Fixed the following issues:
The statement
explain select (select 2)
executed in regular and PS mode produces different results:
MariaDB [test]> prepare stmt from "explain select (select 2)";
Query OK, 0 rows affected (0,000 sec)
Statement prepared
MariaDB [test]> execute stmt;
+------+-------------+-------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------+------+---------------+------+---------+------+------+----------------+
| 1 | PRIMARY | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
| 2 | SUBQUERY | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
+------+-------------+-------+------+---------------+------+---------+------+------+----------------+
2 rows in set (0,000 sec)
MariaDB [test]> explain select (select 2);
+------+-------------+-------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------+------+---------------+------+---------+------+------+----------------+
| 1 | SIMPLE | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
+------+-------------+-------+------+---------------+------+---------+------+------+----------------+
1 row in set, 1 warning (0,000 sec)
In case the statement
CREATE TABLE t1 SELECT * FROM (SELECT 1 AS a, (SELECT a+0)) a
is run in PS mode it fails with the error
ERROR 1054 (42S22): Unknown column 'a' in 'field list'.
- Uniform handling of read-only variables both in case the SET var=val
statement is executed as regular or prepared statememt.
- Fixed assertion firing on handling LOAD DATA statement for temporary tables
- Relaxed assert condition in the function lex_end_stage1() by adding
the commands SQLCOM_ALTER_EVENT, SQLCOM_CREATE_PACKAGE,
SQLCOM_CREATE_PACKAGE_BODY to a list of supported command
- Removed raising of the error ER_UNSUPPORTED_PS in the function
check_prepared_statement() for the ALTER VIEW command
- Added initialization of the data memember st_select_lex_unit::last_procedure
(assign NULL value) in the constructor
Without this change the test case main.ctype_utf8 fails with the following
report in case it is run with the optoin --ps-protocol.
mysqltest: At line 2278: query 'VALUES (_latin1 0xDF) UNION VALUES(_utf8'a' COLLATE utf8_bin)' failed: 2013: Lost connection
- The following bug reports were fixed:
MDEV-24460: Multiple rows result set returned from stored
routine over prepared statement binary protocol is
handled incorrectly
CONC-519: mariadb client library doesn't handle server_status and
warnign_count fields received in the packet
COM_STMT_EXECUTE_RESPONSE.
Reasons for these bug reports have the same nature and caused by
missing loop iteration on results sent by server in response to
COM_STMT_EXECUTE packet.
Enclosing of statements for processing of COM_STMT_EXECUTE response
in the construct like
do
{
...
} while (!mysql_stmt_next_result());
fixes the above mentioned bug reports.
In commit 1c5ae9919463ee0fc1d65fdfc7ec39534495ecaf (MDEV-25666)
we had changed Mariabackup so that it would no longer skip files
whose names start with #sql. This turned out to be wrong.
Because operations on such named files are not protected by any
locks in the server, it is not safe to copy them.
Not copying the files may make the InnoDB data dictionary
inconsistent with the file system. So, we must do something
in InnoDB to adjust for that.
If InnoDB is being started up without the redo log (ib_logfile0)
or with a zero-length log file, we will assume that the server
was restored from a backup, and adjust things as follows:
dict_check_sys_tables(), fil_ibd_open(): Do not complain about
missing #sql files if they would be dropped a little later.
dict_stats_update_if_needed(): Never add #sql tables to
the recomputing queue. This avoids a potential race condition when
dropping the garbage tables.
drop_garbage_tables_after_restore(): Try to drop any garbage tables.
innodb_ddl_recovery_done(): Invoke drop_garbage_tables_after_restore()
if srv_start_after_restore (a new flag) was set and we are not in
read-only mode (innodb_read_only=ON or innodb_force_recovery>3).
The tests and dbug_mariabackup_event() instrumentation
were developed by Vladislav Vaintroub, who also reviewed this.
Due to wsrep uses its own xid format for its recovery,
the xid hashing has to be refined.
When a xid object is not in the server "mysql" format,
the hash record made to contain the xid also in the full format.
In commit 1bd681c8b3c5213ce1f7976940a7dc38b48a0d39 we introduced
a work-around for the missing MDL protection when the internal
tables of FULLTEXT INDEX are dropped during DDL operations.
That work-around suffered from a race condition. A purge thread
could have narrowly passed purge_sys.check_stop_FTS() and then
(while holding dict_sys.mutex) acquire a table reference right
after fts_lock_table() determined that no references were being
held.
fts_lock_table(): Protect the reference check with dict_sys.mutex.
Thanks to Thirunarayanan Balathandayuthapani for repeating the
failure and testing the fix.
Before any committed DDL operation returns control to the caller,
we must ensure that it will have been durably committed before the
ddl_log state may be changed, no matter if
innodb_flush_log_at_trx_commit=0 is being used.
Operations that would involve deleting files were already safe,
because the durable write of the FILE_DELETE record that would precede
the file deletion would also have made the commit durable.
Furthermore, when we clean up ADD INDEX stubs that were left behind
by a previous ha_innobase::commit_inplace_alter_table(commit=false)
when MDL could not be acquired, we will use the same interface as
for dropping indexes. This safety measure should be dead code,
because ADD FULLTEXT INDEX is not supported online, and dropping indexes
only involves file deletion for FULLTEXT INDEX.
ha_innobase::extra(HA_EXTRA_END_ALTER_COPY): Flush the redo log
because the main transaction will not have written any log and
thus will not initiate any log write either. But the DDL recovery
layer expects that the operation will have been committed.
1. work around MDEV-25912 to not apply assert
at wsrep running time;
2. handle wsrep mode of the server recovery
3. convert hton calls to static binlog_commit ones.
4. satisfy MSAN complain on uninitialized std::pair
- Replace strncpy() with strmake() to ensure things are \0 terminated
- Initialized some variables that caused compiler warnings
- Increased buffer that caused warnings from strncpy
One warning from InnoDB, others from Connect.
This is used by InnoDB to detect if CREATE...SELECT is used
Other things:
- Changed InnoDB to use thd_ddl_options()
- Removed lock checking code for create...select (Approved by Marko)
Test times when using --valgrind are now 4 hours and server start/shutdown
time 180 seconds.
The whole test suite time is caped at 1 day instead of 7 days
Reviewer: Monty
As a part of this MDEV following changes were made:
1) Mariadb named executables used instead of mysql named executables in scripts
2) renamed mysql-test-run and mysql-stress-test to mariadb-test-run and
mariadb-stress-test and created a symlink.
In commit 49e2c8f0a6fefdeac50925f758090d6bd099768d (MDEV-25743)
we made dict_sys_t::find() incompatible with the rest of the
table name hash table operations in case the table name contains
non-ASCII octets (using a compatibility mode that facilitates the
upgrade into the MySQL 5.0 filename-safe encoding) and the target
platform implements signed char.
ut_fold_string(): Remove; replace with my_crc32c(). This also makes
table name hash value calculations independent on whether char
is unsigned or signed.
The server still may abort if there is no enough free space in the
ring buffer to resubmit the IO job, but the behavior is equal to
the failure of os_aio() -> submit_io().
In commit de407e7cb4d12574806bf4f7830e73908d5ca9e2 a debug assertion
was added that would not always hold: We could have TRX_STATE_PREPARED
here. But, in that case, the transaction should not have been chosen
as a deadlock victim.
Problem:
=======
When the semisync master is crashed and restarted as slave it could
recover transactions that former slaves may never have seen.
A known method existed to clear out all prepared transactions
with --tc-heuristic-recover=rollback does not care to adjust
binlog accordingly.
Fix:
===
The binlog-based recovery is made to concern of the slave semisync role of
post-crash restarted server.
No changes in behavior is done to the "normal" binloggging server
and the semisync master.
When the restarted server is configured with
--rpl-semi-sync-slave-enabled=1
the refined recovery attempts to roll back prepared transactions
and truncate binlog accordingly.
In case of a partially committed (that is committed at least
in one of the engine participants) such transaction gets committed.
It's guaranteed no (partially as well) committed transactions
exist beyond the truncate position.
In case there exists a non-transactional replication event
(being in a way a committed transaction) past the
computed truncate position the recovery ends with an error.
As after master crash and failover to slave, the demoted-to-slave
ex-master must be ready to face and accept its own (generated by)
events, without generally necessary --replicate-same-server-id.
So the acceptance conditions are relaxed for the semisync slave
to accept own events without that option.
While gtid_strict_mode ON ensures no duplicate transaction can be
(re-)executed the master_use_gtid=none slave has to be
configured with --replicate-same-server-id.
*NOTE* for reviewers.
This patch does not handle the user XA which is done
in next git commit.
the test tests whether short options work on the server command line
* remove 'show variables' for variables not affected by short options
* remove options, that are not short
* remove options, that cannot be tested from SQL
* in particular, -T12 doesn't affect the test output,
but cases ~30sec delay on shutdown
* use -W1 as -W2 is the default, so doesn't affect the test output
also avoid an oxymoron of using `MYSQL_PLUGIN_IMPORT` under
`#ifdef MYSQL_SERVER`, and empty_clex_str is so trivial that a plugin
can define it if needed.
We no longer maintain the unstable-tests collection.
In commit 3635280cf7ef74ec18cfb21b21e779725aab186d we added
a smoke test collection, which independent MariaDB package maintainers
can expect to pass.
should be updated only when adaptive hashing is turned-on
Currently, btr_cur_n_non_sea is used to track the search that missed
adaptive hash index. adaptive hash index is turned off by default
but the said variable is updated always though the value of it makes sense
only when an adaptive index is enabled. It is meant to check how many
searches didn't go through an adaptive hash index.
Given a global variable that is updated on each search path it causes
a contention with a multi-threaded workload.
Patch moves the said variables inside a loop that is now updated
only when the adaptive hash index is enabled and that in theory should
also, reduce the update frequency of the said variable as the majority of
the request should be serviced through the adaptive hash index.
Variables (btr_cur_n_non_sea and btr_cur_n_sea) are also converted to
use distributed counter to avoid contention.
User visible changes:
This also means that user will now see
Innodb_adaptive_hash_non_hash_searches (viewed as part of show status)
only if code is compiled with DWITH_INNODB_AHI=ON (default) and it will
be updated only if innodb_adaptive_hash_index=1 else it reported as 0.
lock_wait_rpl_report(): Tolerate the loss of a lock wait.
commit c68007d958aaab0953a01c96eb02326f19d3c9d7 (MDEV-24738)
had introduced this rare failure.
