Also, include fixes by Vladislav Vaintroub to the
aws_key_management plugin. The AWS C++ SDK specifically depends on
OPENSSL_LIBRARIES, not generic SSL_LIBRARIES (such as YaSSL).
During merge of innodb code QC invalidation was removed from innodb part but not added to server part.
We decided to keep it in innodb to keep server/engine interface the same.
This only merges MDEV-12253, adapting it to MDEV-12602 which is already
present in 10.2 but not yet in the 10.1 revision that is being merged.
TODO: Error handling in crash recovery needs to be improved.
If a page cannot be decrypted (or read), we should cleanly abort
the startup. If innodb_force_recovery is specified, we should
ignore the problematic page and apply redo log to other pages.
Currently, the test encryption.innodb-redo-badkey randomly fails
like this (the last messages are from cmake -DWITH_ASAN):
2017-05-05 10:19:40 140037071685504 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1635994
2017-05-05 10:19:40 140037071685504 [ERROR] InnoDB: Missing MLOG_FILE_NAME or MLOG_FILE_DELETE before MLOG_CHECKPOINT for tablespace 1
2017-05-05 10:19:40 140037071685504 [ERROR] InnoDB: Plugin initialization aborted at srv0start.cc[2201] with error Data structure corruption
2017-05-05 10:19:41 140037071685504 [Note] InnoDB: Starting shutdown...
i=================================================================
==5226==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x612000018588 in thread T0
#0 0x736750 in operator delete(void*) (/mariadb/server/build/sql/mysqld+0x736750)
#1 0x1e4833f in LatchCounter::~LatchCounter() /mariadb/server/storage/innobase/include/sync0types.h:599:4
#2 0x1e480b8 in LatchMeta<LatchCounter>::~LatchMeta() /mariadb/server/storage/innobase/include/sync0types.h:786:17
#3 0x1e35509 in sync_latch_meta_destroy() /mariadb/server/storage/innobase/sync/sync0debug.cc:1622:3
#4 0x1e35314 in sync_check_close() /mariadb/server/storage/innobase/sync/sync0debug.cc:1839:2
#5 0x1dfdc18 in innodb_shutdown() /mariadb/server/storage/innobase/srv/srv0start.cc:2888:2
#6 0x197e5e6 in innobase_init(void*) /mariadb/server/storage/innobase/handler/ha_innodb.cc:4475:3
Port of mysql changeset by Georgi Kodinov <Georgi.Kodinov@Oracle.com>:
Bug #34325 : --add-drop-trigger option for mysqldump
Implemented the --add-drop-trigger option to prepend each
CREATE TRIGGER in the dump file with DROP TRIGGER.
The option is off by default. Added a test case.
The usage of windows functions when all tables were optimized away
by min/max optimization were not supported. As result a result,
the queries that used window functions with min/max aggregation
over the whole table returned wrong result sets.
The patch fixed this problem.
(from: http://buildbot.askmonty.org/buildbot/builders/p8-rhel6-bintar/builds/820/steps/test/logs/stdio)
Errors like the following indicate a potential endian storage issue:
rocksdb.rocksdb_range w1 [ fail ]
Test ended at 2017-04-27 18:56:11
CURRENT_TEST: rocksdb.rocksdb_range
--- /home/buildbot/maria-slave/p8-rhel6-bintar/build/storage/rocksdb/mysql-test/rocksdb/r/rocksdb_range.result 2017-04-27 17:41:27.740050347 -0400
+++ /home/buildbot/maria-slave/p8-rhel6-bintar/build/storage/rocksdb/mysql-test/rocksdb/r/rocksdb_range.reject 2017-04-27 18:56:11.230050346 -0400
@@ -25,15 +25,15 @@
select * from t2 force index (a) where a=0;
pk a b
0 0 0
-1 0 1
-2 0 2
-3 0 3
-4 0 4
-5 0 5
-6 0 6
-7 0 7
-8 0 8
-9 0 9
+16777216 0 1
+33554432 0 2
+50331648 0 3
+67108864 0 4
+83886080 0 5
+100663296 0 6
+117440512 0 7
+134217728 0 8
+150994944 0 9
# The rest are for code coverage:
explain
select * from t2 force index (a) where a=2;
@@ -41,23 +41,23 @@
1 SIMPLE t2 ref a a 4 const #
select * from t2 force index (a) where a=2;
pk a b
-20 2 20
-21 2 21
-22 2 22
-23 2 23
-24 2 24
-25 2 25
-26 2 26
-27 2 27
-28 2 28
-29 2 29
+335544320 2 20
+352321536 2 21
+369098752 2 22
+385875968 2 23
+402653184 2 24
+419430400 2 25
+436207616 2 26
+452984832 2 27
+469762048 2 28
+486539264 2 29
explain
select * from t2 force index (a) where a=3 and pk=33;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t2 const a a 8 const,const #
select * from t2 force index (a) where a=3 and pk=33;
pk a b
-33 3 33
+553648128 3 33
select * from t2 force index (a) where a=99 and pk=99;
pk a b
select * from t2 force index (a) where a=0 and pk=0;
...
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
innochecksum uses global variables. great, let's use them all the
way down, instead of passing them as arguments to innodb internals,
conditionally modifying function prototypes with #ifdefs
The bug happened when the specification of a recursive CTE had
no recursive references at the top level of the specification.
In this case the regular processing of derived table references
of the select containing a non-recursive reference to this
recursive CTE misses handling the specification unit.
At the preparation stage any non-recursive reference to a
recursive CTE must be handled after the preparation of the
specification unit for this CTE. So we have to force this
preparation when regular handling of derived tables does not
do it.
This fixes a regression caused by MDEV-12428.
When we introduced a variant of fil_space_acquire() that could
increment space->n_pending_ops after space->stop_new_ops was set,
the logic of fil_check_pending_operations() was broken.
fil_space_t::n_pending_ios: A new field to track read or write
access from the buffer pool routines immediately before a block
write or after a block read in the file system.
fil_space_acquire_for_io(), fil_space_release_for_io(): Similar
to fil_space_acquire_silent() and fil_space_release(), but
modify fil_space_t::n_pending_ios instead of fil_space_t::n_pending_ops.
Adjust a number of places accordingly, and remove some redundant
tablespace lookups.
The following parts of this fix differ from the 10.2 version of this fix:
buf_page_get_corrupt(): Add a tablespace parameter.
In 10.2, we already had a two-phase process of freeing fil_space objects
(first, fil_space_detach(), then release fil_system->mutex, and finally
free the fil_space and fil_node objects).
fil_space_free_and_mutex_exit(): Renamed from fil_space_free().
Detach the tablespace from the fil_system cache, release the
fil_system->mutex, and then wait for space->n_pending_ios to reach 0,
to avoid accessing freed data in a concurrent thread.
During the wait, future calls to fil_space_acquire_for_io() will
not find this tablespace, and the count can only be decremented to 0,
at which point it is safe to free the objects.
fil_node_free_part1(), fil_node_free_part2(): Refactored from
fil_node_free().
This fixes a regression caused by MDEV-12428.
When we introduced a variant of fil_space_acquire() that could
increment space->n_pending_ops after space->stop_new_ops was set,
the logic of fil_check_pending_operations() was broken.
fil_space_t::n_pending_ios: A new field to track read or write
access from the buffer pool routines immediately before a block
write or after a block read in the file system.
fil_space_acquire_for_io(), fil_space_release_for_io(): Similar
to fil_space_acquire_silent() and fil_space_release(), but
modify fil_space_t::n_pending_ios instead of fil_space_t::n_pending_ops.
fil_space_free_low(): Wait for space->n_pending_ios to reach 0,
to avoid accessing freed data in a concurrent thread. Future
calls to fil_space_acquire_for_io() will not find this tablespace,
because it will already have been detached from fil_system.
Adjust a number of places accordingly, and remove some redundant
tablespace lookups.
FIXME: buf_page_check_corrupt() should take a tablespace from
fil_space_acquire_for_io() as a parameter. This will be done
in the 10.1 version of this patch and merged from there.
That depends on MDEV-12253, which has not been merged from 10.1 yet.
and make sure that private ca key is not deleted at the end of
the procedure, so that we could generate additional certificates
any time without regenerating everything
These changes are comparable to Percona's modifications in innodb in the
Percona Xtrabackup repository.
- If functions are used in backup as well as in innodb, make them non-static.
- Define IS_XTRABACKUP() macro for special handling of innodb running
inside backup.
- Extend some functions for backup.
fil_space_for_table_exists_in_mem() gets additional parameter
'remove_from_data_dict_if_does_not_exist', for partial backups
fil_load_single_table_tablespaces() gets an optional parameter predicate
which tells whether to load tablespace based on database or table name,
also for partial backups.
srv_undo_tablespaces_init() gets an optional parameter 'backup_mode'
- Allow single redo log file (for backup "prepare")
- Do not read doublewrite buffer pages in backup, they are outdated
- Add function fil_remove_invalid_table_from_data_dict(), to remove non-existing
tables from data dictionary in case of partial backups.
- On Windows, fix file share modes when opening tablespaces,
to allow mariabackup to read tablespaces while server is online.
- Avoid access to THDVARs in backup, because innodb plugin is not loaded,
and THDVAR would crash in this case.
- Backup will load encryption plugins outside of mysqld. Thus, do not
force loading MyISAM plugin in plugin_load.
- init_signals() will be used in backup, make it global, not static.
- Do not throw output of exec command, if disable_result_log is set
save and dump it if exec fails. Need tha to meaningfully analyze
errors from mariabackup.
- rmdir now removes the entire tree. need that because xtrabackup tests
clean the whole directory.
- all filesystem modifying commands now require the argument to
be under MYSQLTEST_VARDIR or MYSQL_TMP_DIR.
Do not exporting mysqld entry points directly.
This is needed for mariabackup, to load encryption plugins on Windows.
All plugins are "pure" by default. To mark plugin "impure"
it should use RECOMPILE_FOR_EMBEDDED or STORAGE_ENGINE keyword.
