START REPLIA was issued during initializing which mean that
even before /docker-entrypoint/initdb.d there was initializtion
going on.
Entrypoints that needed data initialzation didn't complete with
this nicely. Also if there wasn't any initialization there
would be little time for the replication to acheive anything
before being shutdown ready for the final start.
Moved --skip-slave-start to the default docker_temp_server_start
implementation.
Technically this is a compaibility break that is likely to be
of significants if:
* /docker-entrypoint/initdb.d contains a script waiting
for replication to catch up.
The recitifcation to the previous behaviour is:
/docker-entrypoint/initdb.d contains a SQL to START REPLICA.
Recommend also having another script that is:
until healthcheck.sh --replication_io \
--replication_sql \
--replication_seconds_behind_master=0 \
--replication
--no-connect;
do
sleep 1
done
Closes#614
require-secure-transport on the server mandates that tls or
unix socket be used. The healthcheck user doesn't have explict
tls credentials, so would have failed. 11.4+ would have
tls negiotated, except in #594 it was disabled for people that
didn't configure ssl-ca correctly.
To resolve this _process_sql adds an explict --protocol socket
to get around the default configuration of 'protocol=tcp' in
.my-healthcheck.sh. The protocol=tcp was there to catch people
who put `healthcheck.sh --innodb_initialized` to discover it
checked that in the starting phase of the container, without
a tcp connection being available, it still returned true.
We work around this my making a connection test always
occur in the healthcheck.
Remove the protocol=tcp from the generation of .my-healthcheck.cnf
files.
--connect, as a method that requires to test the connection,
we add a mechanims that examines @@skip_networking and considers
that if false, the connection is viable. We made a unix socket
connection to do the test, which is active the same time as tcp
sockets are.
This alternate --connect method would have only worked the
credentials of the healthcheck user where valid. If it isn't
fall back to looking for "Can't connect".
Closes: #596
There may be cases where specific files are readonly,
like .my-healthcheck.cnf due to filesystem mounts.
So lets make the ownership/permission changes optional.
Closes: #573
Correct healthcheck.sh comments.
Few minor errors in later versions corrected.
11.6 upgrade file corrected.
More resiliant to version changes.
Remove mysql/mariadb safe.cnf file that isn't distributed
from Dockerfile.
Factor our create_healthcheck_users.
Make sure that when recreating users, if they already exist, we just password
reset these and ensure the .my-healtcheck.cnf file is there for usage. We don't
want to clobber any existing grants if we happen not to have MARIADB_HEALTHCHECK_GRANTS
set.
Because creating users needs to move past --skip-grant-tables with FLUSH PRIVEGES,
and mariadb-upgrade also issue FLUSH PRIVILEGES, then unfortunately is yet another
restart.
Adjust test case to ensure there is no .cnf file, and create it on
restore.
As the backup may be from a different configuration requiring
specific parameters, we make the mysql user have a homedir of
/var/lib/mysql and place the backup my.cnf there.
Thanks Faustin for review and Martin for testing.
healthcheck@{127.0.0.1,::1,localhost} users are granted USAGE by default, which
is enough for the non-replication healthchecks in healtcheck.sh.
The env variable MARIADB_HEALTHCHECK_GRANTS can replace USAGE with any
comma separated set of grants.
On initialization a generated password is created and saved in
$DATADIR/.my-healthcheck.cnf along with the server port and socket. If the
command args or default configuration file changes this may become out
of date. Because the password is generated in configuration file the
'#', comment, and '=' characters cannot be part of this password.
The healthcheck.cnf configuration file also sets protocol=tcp to
enforce indirectly that --connect being a standard part of the test. This is
required as starts of the service under --skip-networking should
never be considered healthy.
The healthcheck script also has the --defaults-extra-file set to this
.my-healthcheck.cnf file, if it exists (backwards compatible on
previously created datadirs), so that all new healthcheck invokations
use the authentication here by default.
The compatibility with old instances, without the .my-healthcheck.cnf is
preserved by non setting --defaults-extra-file.
The healthcheck --connect will increment the server status variable Aborted_connects
for each check, however now connection_error* counts are changed.
This also prevents any invalid password errors showing up in the
container log.
Closes#430
These are controled by:
* MARIADB_MASTER_HOST - if specified, the master, and the container is a replica
* MARIADB_REPLICATION_PORT - port number
* MARIADB_REPLICATION_USER - create or use this user
* MARIADB_REPLICATION_PASSWORD - create or use this password
* MARIADB_REPLICATION_PASSWORD_HASH (on master only)
If MARIADB_MASTER_HOST isn't specified, the container is a master. In
this case, the MARIADB_REPLICATION_USER is created, with password{,hash},
and given the REPLICATION REPLICA grant, or REPLICATION SLAVE (for
10.3, 10.4).
If the MARIADB_MASTER_HOST is specified, CHANGE MASTER TO is used to
connect to the MARIADB_MASTER_HOST:MARIADB_MASTER_PORT(default 3306)
using the MARIADB_REPLICATION_USER. As the password is needed
MARIADB_REPLICATION_PASSWORD_HASH cannot be used on a replica.
CHANGE MASTER TO is executed with MASTER_CONNECT_RETRY=10 and the
replica is started by default.
The creation of the replication user is replicated along with the
master's /docker-entrypoint-initdb.d/ contents and MARIADB_DATABASE
/MARIADB_USER. The MARIADB_MYSQL_LOCALHOST_USER isn't replicated and
neither is the timezone data.
Signed-off-by: MdSahil-oss <Mohdssahil1@gmail.com>
Reviewed-by: Daniel Black <daniel@mariadb.org>
This had significant follow on repercussions for other parts
of the script that depending on a $MARIADB_ROOT_PASSWORD for
initialization.
* docker_temp_server_stop always uses the PID now.
* creating users and databases are now part of the same session
that now secures the system users.
Note /docker-entrypoint-initdb.d/ is currently broken with
MARIADB_ROOT_PASSWORD_HASH as no password exists to process
these files.
The timezone initialization wasn't intended to be in the binary log.
Despite the --skip-write-binlog argument to mariadb-tzinfo-to-sql,
this only has an effect when galera is enabled.
A user wants to initialize the master with a bunch of SQL to be
replicated doesn't get binary logged due to the --skip-log-bin
on the temporary server start.
The commit that this reverts added the skip-log-bin to speed up
timezones. A later commit 013d851b19cee4a109c849bb45ae08ce4c974ac4
speed up the timezone initialization by disabling binary logs
when timezones where loaded.
The mariadb-upgrade does set `SET SQL_LOG_BIN=0` by default.
As such:
Revert "MDEV-27074: disable log-bin during temp server start"
This reverts commit 146e09c9c7aa5d301ac96c1939fdd112f802f4ae.
Thanks Markus Mäkelä for the bug report.
Initial testing showed that kill -0 sometimes
didn't kill mariadbd enough and that the
startup sill had the aria_control locked.
To work around this additional testing on
aquiring aria_control exclusive locks was
attempted. This also was insufficient and
an additional 2 seconds where added.
Falling back to the capabilites of bash to
use a wait builtin to wait for an examined
process to terminate was a better option
found upon further examination.
For note:
The docker_temp_server_stop uses the
mysqladmin shutdown mechanism. This waits
until the pid file created by the server
is removed. The server code shows this occurs
after the storage engines are shut down.
Daniel Rudolf from
https://github.com/MariaDB/mariadb-docker/pull/409#pullrequestreview-873937215:
IMHO a user implies a grant, not necessarily the other way round. Just think of
it as if we made the username (mysql) configurable: You can't grant privileges
without an user, you need the user first. The only reason we can do this is
because the username is hard coded, not because a grant implies its user.
Additionally I'm a bit concerned about unexpected behaviour: Just think about an
user passing --env MARIADB_MYSQL_LOCALHOST_USER= --env MARIADB_MYSQL_LOCALHOST_GRANTS=USAGE.
The user explicitly tells us not to create an user, but the dangling grant
overrules this statement. Sure, no human would ever pass this purposely, but users
don't necessarily execute commands manually, sometimes the options to pass are
computed and/or read from arbitrary sources - which then must all be aware of this
tight coupling.
We create a mysql@localhost user under MARIADB_MYSQL_LOCALHOST_USER=1
with its default USAGE privileges.
This gives access to SHOW GLOBAL STATUS/VARIABLES but no other
real access including database visibility or process visibility.
Being a @localhost this restricts access to via the unix socket.
The level of access can be increased controlled by the environment variable
MARIADB_MYSQL_LOCALHOST_GRANTS.
If you are using monitoring replication or processes addition
privileges are required, and the setting of the environment variable
is the comma separated list of grants possible. For the moment,
these are a set of global grants only.
If you share the unix socket location, /var/run/mysqld by default,
as a volume with another container, you have effectively given that
container the mysql@localhost provided they have the same uid map
or can create arbitrary users.
This can be good for things like backup. This would require the
datadir volume as well and the set of privileges
(https://mariadb.com/kb/en/mariabackup-overview/#authentication-and-privileges).
Any grants of UPDATE on mysql database will mean that the
mysql@localhost user can manipulate any other user, potentially
transparently adding unix_socket auth (in 10.4+), and the being able
to gain their privileges.
Grants of CREATE USER, or INSERT on the mysql database allow the
creation of user and privilege escalation.
For these reasons, ALL for MARIADB_MYSQL_LOCALHOST_GRANTS gains a
warning.
Many thanks to Daniel Rudolf for all the reviews and the support
to develop this feature.
mysql_install_db - both disabled
docker_temp_server_start, used for dump useful for if /docker-entrypoint-initdb.d
is loaded.
mariadb_upgrade - both disabled
innodb-buffer-pool-load disabled until final startup.
"loose" is used in case a user starts with --skip-innodb.
Future enhancement, change dump only if something in
/docker-entrypoint-initdb.d exists.
This saves a small bit of IO, more so with mariadb-upgrade, and
none of the IO/memory acheives much gain in theses temporary start
situtations.
Also by skipping on the temporary starts, the innodb_buffer_pool_loaded
option in the healthcheck script only gains the "loaded" status
on the final startup in the entrypoint.
