qtbase/cmake/QtBuildHelpers.cmake

496 lines
17 KiB
CMake
Raw Permalink Normal View History

CMake: Split QtBuild.cmake into smaller files v2 My motivation to do this: - it got big and tangled again - sometimes functions need to be added to QtBuild.cmake rather than to a separate file because they need to be called before some of the global variables are set, to determine the value of those global variables (in my case install paths needed to be modified when building with xcframework support) - some of the global variable assignments have dependencies on other variables already being set and it's hard to keep track where that happens Split the contents of the file into smaller functions and macros and place them into pre-existing files when appropriate, or into new files. The new files are: - QtBuildHelpers.cmake - QtBuildPathsHelpers.cmake - QtMkspecHelpers.cmake The idea is to have Helpers file only define functions and never call them, so it's easy to include the file where needed without being scared of side effects. QtBuild.cmake will just include the helpers and call one entry point function to set up everything that was done by the file before. QtBuild.cmake is not merged into QtSetup, to make it easier to git blame (it's hard to blame a removed file). No new features were added as part of the refactoring. Some function names were renamed (but not all of them) to include the qt_internal prefix. Some lines were reformatted so they don't pass 100 chars limit after the code was placed into a function / macro. The Helpers includes were re-sorted. Some function calls were re-ordered where the order call didn't matter. Some of the code in QtAndroidHelpers.cmake was wrapped into a macro so that including the file does not cause side-effects by default. I'd like to follow up with similar changes for QtSetup.cmake and QtBuildInternalsConfig.cmake where possible, because having a few "entry points" into building a Qt submodule is also confusing, especially for those that aren't familiar with the build system and why certain things go into certain places. The intent is to cherry-pick this also to 6.5 and 6.6. Amends 44cce1a2ea9dadd8b2de93f40de34269dda703c0 Task-number: QTBUG-86035 Change-Id: I02ceff8ceb9b6e9c78bc85d6a42deb02fca3e46b Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Orkun Tokdemir <orkun.tokdemir@qt.io>
2023-11-15 18:08:15 +01:00
# Copyright (C) 2023 The Qt Company Ltd.
# SPDX-License-Identifier: BSD-3-Clause
function(qt_internal_validate_cmake_generator)
get_property(warning_shown GLOBAL PROPERTY _qt_validate_cmake_generator_warning_shown)
if(NOT warning_shown
AND NOT CMAKE_GENERATOR MATCHES "Ninja"
AND NOT QT_SILENCE_CMAKE_GENERATOR_WARNING
AND NOT DEFINED ENV{QT_SILENCE_CMAKE_GENERATOR_WARNING})
set_property(GLOBAL PROPERTY _qt_validate_cmake_generator_warning_shown TRUE)
message(WARNING
"The officially supported CMake generator for building Qt is "
"Ninja / Ninja Multi-Config. "
"You are using: '${CMAKE_GENERATOR}' instead. "
"Thus, you might encounter issues. Use at your own risk.")
endif()
endfunction()
macro(qt_internal_set_qt_building_qt)
# Set the QT_BUILDING_QT variable so we can verify whether we are building
# Qt from source.
# Make sure not to set it when building a standalone test, otherwise
# upon reconfiguration we get an error about qt_internal_add_test
# not being found due the if(NOT QT_BUILDING_QT) check we have
# in each standalone test.
if(NOT QT_INTERNAL_IS_STANDALONE_TEST)
set(QT_BUILDING_QT TRUE CACHE BOOL
"When this is present and set to true, it signals that we are building Qt from source.")
endif()
endmacro()
macro(qt_internal_unset_extra_build_internals_vars)
# Reset content of extra build internal vars for each inclusion of QtSetup.
unset(QT_EXTRA_BUILD_INTERNALS_VARS)
endmacro()
macro(qt_internal_get_generator_is_multi_config)
# Save the global property in a variable to make it available to feature conditions.
get_property(QT_GENERATOR_IS_MULTI_CONFIG GLOBAL PROPERTY GENERATOR_IS_MULTI_CONFIG)
endmacro()
macro(qt_internal_setup_position_independent_code)
## Position independent code:
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
# Does the linker support position independent code?
include(CheckPIESupported)
check_pie_supported()
endmacro()
macro(qt_internal_set_link_depends_no_shared)
# Do not relink dependent libraries when no header has changed:
set(CMAKE_LINK_DEPENDS_NO_SHARED ON)
endmacro()
macro(qt_internal_set_qt_source_tree_var)
# Specify the QT_SOURCE_TREE only when building qtbase. Needed by some tests when the tests are
# built as part of the project, and not standalone. For standalone tests, the value is set in
# QtBuildInternalsExtra.cmake.
if(PROJECT_NAME STREQUAL "QtBase")
set(QT_SOURCE_TREE "${QtBase_SOURCE_DIR}" CACHE PATH
"A path to the source tree of the previously configured QtBase project." FORCE)
endif()
endmacro()
macro(qt_internal_include_qt_platform_android)
## Android platform settings
if(ANDROID)
include(QtPlatformAndroid)
endif()
endmacro()
macro(qt_internal_include_qt_properties)
include(QtProperties)
endmacro()
macro(qt_internal_set_compiler_optimization_flags)
include(QtCompilerOptimization)
endmacro()
macro(qt_internal_set_compiler_warning_flags)
include(QtCompilerFlags)
endmacro()
macro(qt_internal_set_skip_setup_deployment)
if(NOT QT_BUILD_EXAMPLES)
# Disable deployment setup to avoid warnings about missing patchelf with CMake < 3.21.
set(QT_SKIP_SETUP_DEPLOYMENT ON)
endif()
endmacro()
CMake: Split QtBuild.cmake into smaller files v2 My motivation to do this: - it got big and tangled again - sometimes functions need to be added to QtBuild.cmake rather than to a separate file because they need to be called before some of the global variables are set, to determine the value of those global variables (in my case install paths needed to be modified when building with xcframework support) - some of the global variable assignments have dependencies on other variables already being set and it's hard to keep track where that happens Split the contents of the file into smaller functions and macros and place them into pre-existing files when appropriate, or into new files. The new files are: - QtBuildHelpers.cmake - QtBuildPathsHelpers.cmake - QtMkspecHelpers.cmake The idea is to have Helpers file only define functions and never call them, so it's easy to include the file where needed without being scared of side effects. QtBuild.cmake will just include the helpers and call one entry point function to set up everything that was done by the file before. QtBuild.cmake is not merged into QtSetup, to make it easier to git blame (it's hard to blame a removed file). No new features were added as part of the refactoring. Some function names were renamed (but not all of them) to include the qt_internal prefix. Some lines were reformatted so they don't pass 100 chars limit after the code was placed into a function / macro. The Helpers includes were re-sorted. Some function calls were re-ordered where the order call didn't matter. Some of the code in QtAndroidHelpers.cmake was wrapped into a macro so that including the file does not cause side-effects by default. I'd like to follow up with similar changes for QtSetup.cmake and QtBuildInternalsConfig.cmake where possible, because having a few "entry points" into building a Qt submodule is also confusing, especially for those that aren't familiar with the build system and why certain things go into certain places. The intent is to cherry-pick this also to 6.5 and 6.6. Amends 44cce1a2ea9dadd8b2de93f40de34269dda703c0 Task-number: QTBUG-86035 Change-Id: I02ceff8ceb9b6e9c78bc85d6a42deb02fca3e46b Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Orkun Tokdemir <orkun.tokdemir@qt.io>
2023-11-15 18:08:15 +01:00
macro(qt_internal_reset_global_state)
qt_internal_clear_qt_repo_known_modules()
qt_internal_clear_qt_repo_known_plugin_types()
qt_internal_set_qt_known_plugins("")
set(QT_KNOWN_MODULES_WITH_TOOLS "" CACHE INTERNAL "Known Qt modules with tools" FORCE)
endmacro()
macro(qt_internal_set_qt_path_separator)
# For adjusting variables when running tests, we need to know what
# the correct variable is for separating entries in PATH-alike
# variables.
if(CMAKE_HOST_SYSTEM_NAME STREQUAL "Windows")
set(QT_PATH_SEPARATOR "\\;")
else()
set(QT_PATH_SEPARATOR ":")
endif()
endmacro()
macro(qt_internal_set_internals_extra_cmake_code)
# This is used to hold extra cmake code that should be put into QtBuildInternalsExtra.cmake file
# at the QtPostProcess stage.
set(QT_BUILD_INTERNALS_EXTRA_CMAKE_CODE "")
endmacro()
macro(qt_internal_set_top_level_source_dir)
# Save the value of the current first project source dir.
# This will be /path/to/qtbase for qtbase both in a super-build and a non super-build.
# This will be /path/to/qtbase/tests when building standalone tests.
set(QT_TOP_LEVEL_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}")
endmacro()
macro(qt_internal_set_apple_archiver_flags)
# Prevent warnings about object files without any symbols. This is a common
# thing in Qt as we tend to build files unconditionally, and then use ifdefs
# to compile out parts that are not relevant.
if(CMAKE_CXX_COMPILER_ID MATCHES "AppleClang")
foreach(lang ASM C CXX)
# We have to tell 'ar' to not run ranlib by itself, by passing the 'S' option
set(CMAKE_${lang}_ARCHIVE_CREATE "<CMAKE_AR> qcS <TARGET> <LINK_FLAGS> <OBJECTS>")
set(CMAKE_${lang}_ARCHIVE_APPEND "<CMAKE_AR> qS <TARGET> <LINK_FLAGS> <OBJECTS>")
set(CMAKE_${lang}_ARCHIVE_FINISH "<CMAKE_RANLIB> -no_warning_for_no_symbols <TARGET>")
endforeach()
endif()
endmacro()
macro(qt_internal_set_apple_privacy_manifest target manifest_file)
set_target_properties(${target} PROPERTIES _qt_privacy_manifest "${manifest_file}")
endmacro()
CMake: Split QtBuild.cmake into smaller files v2 My motivation to do this: - it got big and tangled again - sometimes functions need to be added to QtBuild.cmake rather than to a separate file because they need to be called before some of the global variables are set, to determine the value of those global variables (in my case install paths needed to be modified when building with xcframework support) - some of the global variable assignments have dependencies on other variables already being set and it's hard to keep track where that happens Split the contents of the file into smaller functions and macros and place them into pre-existing files when appropriate, or into new files. The new files are: - QtBuildHelpers.cmake - QtBuildPathsHelpers.cmake - QtMkspecHelpers.cmake The idea is to have Helpers file only define functions and never call them, so it's easy to include the file where needed without being scared of side effects. QtBuild.cmake will just include the helpers and call one entry point function to set up everything that was done by the file before. QtBuild.cmake is not merged into QtSetup, to make it easier to git blame (it's hard to blame a removed file). No new features were added as part of the refactoring. Some function names were renamed (but not all of them) to include the qt_internal prefix. Some lines were reformatted so they don't pass 100 chars limit after the code was placed into a function / macro. The Helpers includes were re-sorted. Some function calls were re-ordered where the order call didn't matter. Some of the code in QtAndroidHelpers.cmake was wrapped into a macro so that including the file does not cause side-effects by default. I'd like to follow up with similar changes for QtSetup.cmake and QtBuildInternalsConfig.cmake where possible, because having a few "entry points" into building a Qt submodule is also confusing, especially for those that aren't familiar with the build system and why certain things go into certain places. The intent is to cherry-pick this also to 6.5 and 6.6. Amends 44cce1a2ea9dadd8b2de93f40de34269dda703c0 Task-number: QTBUG-86035 Change-Id: I02ceff8ceb9b6e9c78bc85d6a42deb02fca3e46b Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Orkun Tokdemir <orkun.tokdemir@qt.io>
2023-11-15 18:08:15 +01:00
macro(qt_internal_set_debug_extend_target)
option(QT_CMAKE_DEBUG_EXTEND_TARGET "Debug extend_target calls in Qt's build system" OFF)
endmacro()
# These upstream CMake modules will be automatically include()'d when doing
# find_package(Qt6 COMPONENTS BuildInternals).
function(qt_internal_get_qt_build_upstream_cmake_modules out_var)
set(${out_var}
CMakeFindBinUtils
CMakePackageConfigHelpers
CheckCXXSourceCompiles
FeatureSummary
PARENT_SCOPE
)
endfunction()
# These helpers will be installed when building qtbase, and they will be automatically include()'d
# when doing find_package(Qt6 COMPONENTS BuildInternals).
# The helpers are expected to exist under the qtbase/cmake sub-directory and their file name
# extension should be '.cmake'.
function(qt_internal_get_qt_build_private_helpers out_var)
set(${out_var}
Qt3rdPartyLibraryHelpers
QtAndroidHelpers
QtAppHelpers
QtAutoDetectHelpers
QtAutogenHelpers
QtBuildInformation
QtBuildOptionsHelpers
QtBuildPathsHelpers
QtBuildRepoExamplesHelpers
QtBuildRepoHelpers
QtCMakeHelpers
QtCMakeVersionHelpers
QtDbusHelpers
QtDeferredDependenciesHelpers
QtDocsHelpers
QtExecutableHelpers
QtFindPackageHelpers
QtFlagHandlingHelpers
QtFrameworkHelpers
QtGlobalStateHelpers
QtHeadersClean
QtInstallHelpers
QtJavaHelpers
QtLalrHelpers
QtMkspecHelpers
QtModuleHelpers
QtNoLinkTargetHelpers
QtPkgConfigHelpers
QtPlatformTargetHelpers
QtPluginHelpers
QtPostProcessHelpers
QtPrecompiledHeadersHelpers
QtPriHelpers
QtPrlHelpers
QtProperties
QtQmakeHelpers
QtResourceHelpers
QtRpathHelpers
QtSanitizerHelpers
CMake: Generate an SPDX v2.3 SBOM file for each built repository This change adds a new -sbom configure option to allow generating and installing an SPDX v2.3 SBOM file when building a qt repo. The -sbom-dir option can be used to configure the location where each repo sbom file will be installed. By default it is installed into $prefix/$archdatadir/sbom/$sbom_lower_project_name.sdpx which is basically ~/Qt/sbom/qtbase-6.8.0.spdx The file is installed as part of the default installation rules, but it can also be installed manually using the "sbom" installation component, or "sbom_$lower_project_name" in a top-level build. For example: cmake install . --component sbom_qtbase CMake 3.19+ is needed to read the qt_attribution.json files for copyrights, license info, etc. When using an older cmake version, configuration will error out. It is possible to opt into using an older cmake version, but the generated sbom will lack all the attribution file information. Using an older cmake version is untested and not officially supported. Implementation notes. The bulk of the implementation is split into 4 new files: - QtPublicSbomHelpers.cmake - for Qt-specific collecting, processing and dispatching the generation of various pieces of the SBOM document e.g. a SDPX package associated with a target like Core, a SDPX file entry for each target binary file (per-config shared library, archive, executable, etc) - QtPublicSbomGenerationHelpers.cmake - for non-Qt specific implementation of SPDX generation. This also has some code that was taken from the cmake-sbom 3rd party project, so it is dual licensed under the usual Qt build system BSD license, as well as the MIT license of the 3rd party project - QtPublicGitHelpers.cmake - for git related features, mainly to embed queried hashes or tags into version strings, is dual-licensed for the same reasons as QtPublicSbomGenerationHelpers.cmake - QtSbomHelpers.cmake - Qt-specific functions that just forward arguments to the public functions. These are meant to be used in our Qt CMakeLists.txt instead of the public _qt_internal_add_sbom ones for naming consistency. These function would mostly be used to annotate 3rd party libraries with sbom info and to add sbom info for unusual target setups (like the Bootstrap library), because most of the handling is already done automatically via qt_internal_add_module/plugin/etc. The files are put into Public cmake files, with the future hope of making this available to user projects in some capacity. The distinction of Qt-specific and non-Qt specific code might blur a bit, and thus the separation across files might not always be consistent, but it was best effort. The main purpose of the code is to collect various information about targets and their relationships and generate equivalent SPDX info. Collection is currently done for the following targets: Qt modules, plugins, apps, tools, system libraries, bundled 3rd party libraries and partial 3rd party sources compiled directly as part of Qt targets. Each target has an equivalent SPDX package generated with information like version, license, copyright, CPE (common vulnerability identifier), files that belong to the package, and relationships on other SPDX packages (associated cmake targets), mostly gathered from direct linking dependencies. Each package might also contain files, e.g. libQt6Core.so for the Core target. Each file also has info like license id, copyrights, but also the list of source files that were used to generate the file and a sha1 checksum. SPDX documents can also refer to packages in other SPDX documents, and those are referred to via external document references. This is the case when building qtdeclarative and we refer to Core. For qt provided targets, we have complete information regarding licenses, and copyrights. For bundled 3rd party libraries, we should also have most information, which is usually parsed from the src/3rdparty/libfoo/qt_attribution.json files. If there are multiple attribution files, or if the files have multiple entries, we create a separate SBOM package for each of those entries, because each might have a separate copyright or version, and an sbom package can have only one version (although many copyrights). For system libraries we usually lack the information because we don't have attribution files for Find scripts. So the info needs to be manually annotated via arguments to the sbom function calls, or the FindFoo.cmake scripts expose that information in some form and we can query it. There are also corner cases like 3rdparty sources being directly included in a Qt library, like the m4dc files for Gui, or PCRE2 for Bootstrap. Or QtWebEngine libraries (either Qt bundled or Chromium bundled or system libraries) which get linked in by GN instead of CMake, so there are no direct targets for them. The information for these need to be annotated manually as well. There is also a distinction to be made for static Qt builds (or any static Qt library in a shared build), where the system libraries found during the Qt build might not be the same that are linked into the final user application or library. The actual generation of the SBOM is done by file(GENERATE)-ing one .cmake file for each target, file, external ref, etc, which will be included in a top-level cmake script. The top-level cmake script will run through each included file, to append to a "staging" spdx file, which will then be used in a configure_file() call to replace some final variables, like embedding a file checksum. There are install rules to generate a complete SBOM during installation, and an optional 'sbom' custom target that allows building an incomplete SBOM during the build step. The build target is just for convenience and faster development iteration time. It is incomplete because it is missing the installed file SHA1 checksums and the document verification code (the sha1 of all sha1s). We can't compute those during the build before the files are actually installed. A complete SBOM can only be achieved at installation time. The install script will include all the generated helper files, but also set some additional variables to ensure checksumming happens, and also handle multi-config installation, among other small things. For multi-config builds, CMake doesn't offer a way to run code after all configs are installed, because they might not always be installed, someone might choose to install just Release. To handle that, we rely on ninja installing each config sequentially (because ninja places the install rules into the 'console' pool which runs one task at a time). For each installed config we create a config-specific marker file. Once all marker files are present, whichever config ends up being installed as the last one, we run the sbom generation once, and then delete all marker files. There are a few internal variables that can be set during configuration to enable various checks (and other features) on the generated spdx files: - QT_INTERNAL_SBOM_VERIFY - QT_INTERNAL_SBOM_AUDIT - QT_INTERNAL_SBOM_AUDIT_NO_ERROR - QT_INTERNAL_SBOM_GENERATE_JSON - QT_INTERNAL_SBOM_SHOW_TABLE - QT_INTERNAL_SBOM_DEFAULT_CHECKS These use 3rd party python tools, so they are not enabled by default. If enabled, they run at installation time after the sbom is installed. We will hopefully enable them in CI. Overall, the code is still a bit messy in a few places, due to time constraints, but can be improved later. Some possible TODOs for the future: - Do we need to handle 3rd party libs linked into a Qt static library in a Qt shared build, where the Qt static lib is not installed, but linked into a Qt shared library, somehow specially? We can record a package for it, but we can't create a spdx file record for it (and associated source relationships) because we don't install the file, and spdx requires the file to be installed and checksummed. Perhaps we can consider adding some free-form text snippet to the package itself? - Do we want to add parsing of .cpp source files for Copyrights, to embed them into the packages? This will likely slow down configuration quite a bit. - Currently sbom info attached to WrapFoo packages in one repo is not exported / available in other repos. E.g. If we annotate WrapZLIB in qtbase with CPE_VENDOR zlib, this info will not be available when looking up WrapZLIB in qtimageformats. This is because they are IMPORTED libraries, and are not exported. We might want to record this info in the future. [ChangeLog][Build System] A new -sbom configure option can be used to generate and install a SPDX SBOM (Software Bill of Materials) file for each built Qt repository. Pick-to: 6.8 Task-number: QTBUG-122899 Change-Id: I9c730a6bbc47e02ce1836fccf00a14ec8eb1a5f4 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2024-03-07 18:02:56 +01:00
QtSbomHelpers
QtScopeFinalizerHelpers
QtSeparateDebugInfo
QtSimdHelpers
QtSingleRepoTargetSetBuildHelpers
QtSyncQtHelpers
QtTargetHelpers
QtTestHelpers
QtToolHelpers
QtToolchainHelpers
QtUnityBuildHelpers
QtWasmHelpers
QtWindowsHelpers
QtWrapperScriptHelpers
PARENT_SCOPE
)
endfunction()
# These files will be installed when building qtbase, but will NOT be automatically include()d
# when doing find_package(Qt6 COMPONENTS BuildInternals).
# The files are expected to exist under the qtbase/cmake sub-directory.
function(qt_internal_get_qt_build_private_files_to_install out_var)
set(${out_var}
ModuleDescription.json.in
PkgConfigLibrary.pc.in
Qt3rdPartyLibraryConfig.cmake.in
QtTransitiveExtras.cmake.in
QtBaseTopLevelHelpers.cmake
QtBuild.cmake
QtBuildHelpers.cmake
QtBuildStaticDocToolsScript.cmake
QtCMakePackageVersionFile.cmake.in
QtCompilerFlags.cmake
QtCompilerOptimization.cmake
QtConfigDependencies.cmake.in
QtConfigureTimeExecutableCMakeLists.txt.in
QtFileConfigure.txt.in
QtFindWrapConfigExtra.cmake.in
QtFindWrapHelper.cmake
QtFinishPkgConfigFile.cmake
QtFinishPrlFile.cmake
QtGenerateExtPri.cmake
QtGenerateLibHelpers.cmake
QtGenerateLibPri.cmake
QtGenerateVersionScript.cmake
QtModuleConfig.cmake.in
QtModuleConfigPrivate.cmake.in
QtModuleDependencies.cmake.in
QtModuleHeadersCheck.cmake
QtModuleToolsConfig.cmake.in
QtModuleToolsDependencies.cmake.in
QtModuleToolsVersionlessTargets.cmake.in
QtPlatformAndroid.cmake
QtPlatformSupport.cmake
QtPluginConfig.cmake.in
QtPluginDependencies.cmake.in
QtPlugins.cmake.in
QtPostProcess.cmake
QtProcessConfigureArgs.cmake
QtSeparateDebugInfo.Info.plist.in
QtSetup.cmake
QtStandaloneTestsConfig.cmake.in
QtVersionlessAliasTargets.cmake.in
QtVersionlessTargets.cmake.in
QtWriteArgsFile.cmake
modulecppexports.h.in
qbatchedtestrunner.in.cpp
qt-internal-config.redo.in
qt-internal-config.redo.bat.in
PARENT_SCOPE
)
endfunction()
# These helpers will be installed when building qtbase, and they will be automatically include()'d
# when doing find_package(Qt6 COMPONENTS BuildInternals).
# The helpers are expected to exist under the qtbase/cmake sub-directory and their file name
# extension should be '.cmake'.
# In addition, they are meant to be included when doing find_package(Qt6) as well.
function(qt_internal_get_qt_build_public_helpers out_var)
set(${out_var}
QtFeature
QtFeatureCommon
QtPublicAndroidHelpers
QtPublicAppleHelpers
QtPublicCMakeHelpers
QtPublicCMakeVersionHelpers
QtPublicDependencyHelpers
QtPublicExternalProjectHelpers
QtPublicFinalizerHelpers
QtPublicFindPackageHelpers
CMake: Generate an SPDX v2.3 SBOM file for each built repository This change adds a new -sbom configure option to allow generating and installing an SPDX v2.3 SBOM file when building a qt repo. The -sbom-dir option can be used to configure the location where each repo sbom file will be installed. By default it is installed into $prefix/$archdatadir/sbom/$sbom_lower_project_name.sdpx which is basically ~/Qt/sbom/qtbase-6.8.0.spdx The file is installed as part of the default installation rules, but it can also be installed manually using the "sbom" installation component, or "sbom_$lower_project_name" in a top-level build. For example: cmake install . --component sbom_qtbase CMake 3.19+ is needed to read the qt_attribution.json files for copyrights, license info, etc. When using an older cmake version, configuration will error out. It is possible to opt into using an older cmake version, but the generated sbom will lack all the attribution file information. Using an older cmake version is untested and not officially supported. Implementation notes. The bulk of the implementation is split into 4 new files: - QtPublicSbomHelpers.cmake - for Qt-specific collecting, processing and dispatching the generation of various pieces of the SBOM document e.g. a SDPX package associated with a target like Core, a SDPX file entry for each target binary file (per-config shared library, archive, executable, etc) - QtPublicSbomGenerationHelpers.cmake - for non-Qt specific implementation of SPDX generation. This also has some code that was taken from the cmake-sbom 3rd party project, so it is dual licensed under the usual Qt build system BSD license, as well as the MIT license of the 3rd party project - QtPublicGitHelpers.cmake - for git related features, mainly to embed queried hashes or tags into version strings, is dual-licensed for the same reasons as QtPublicSbomGenerationHelpers.cmake - QtSbomHelpers.cmake - Qt-specific functions that just forward arguments to the public functions. These are meant to be used in our Qt CMakeLists.txt instead of the public _qt_internal_add_sbom ones for naming consistency. These function would mostly be used to annotate 3rd party libraries with sbom info and to add sbom info for unusual target setups (like the Bootstrap library), because most of the handling is already done automatically via qt_internal_add_module/plugin/etc. The files are put into Public cmake files, with the future hope of making this available to user projects in some capacity. The distinction of Qt-specific and non-Qt specific code might blur a bit, and thus the separation across files might not always be consistent, but it was best effort. The main purpose of the code is to collect various information about targets and their relationships and generate equivalent SPDX info. Collection is currently done for the following targets: Qt modules, plugins, apps, tools, system libraries, bundled 3rd party libraries and partial 3rd party sources compiled directly as part of Qt targets. Each target has an equivalent SPDX package generated with information like version, license, copyright, CPE (common vulnerability identifier), files that belong to the package, and relationships on other SPDX packages (associated cmake targets), mostly gathered from direct linking dependencies. Each package might also contain files, e.g. libQt6Core.so for the Core target. Each file also has info like license id, copyrights, but also the list of source files that were used to generate the file and a sha1 checksum. SPDX documents can also refer to packages in other SPDX documents, and those are referred to via external document references. This is the case when building qtdeclarative and we refer to Core. For qt provided targets, we have complete information regarding licenses, and copyrights. For bundled 3rd party libraries, we should also have most information, which is usually parsed from the src/3rdparty/libfoo/qt_attribution.json files. If there are multiple attribution files, or if the files have multiple entries, we create a separate SBOM package for each of those entries, because each might have a separate copyright or version, and an sbom package can have only one version (although many copyrights). For system libraries we usually lack the information because we don't have attribution files for Find scripts. So the info needs to be manually annotated via arguments to the sbom function calls, or the FindFoo.cmake scripts expose that information in some form and we can query it. There are also corner cases like 3rdparty sources being directly included in a Qt library, like the m4dc files for Gui, or PCRE2 for Bootstrap. Or QtWebEngine libraries (either Qt bundled or Chromium bundled or system libraries) which get linked in by GN instead of CMake, so there are no direct targets for them. The information for these need to be annotated manually as well. There is also a distinction to be made for static Qt builds (or any static Qt library in a shared build), where the system libraries found during the Qt build might not be the same that are linked into the final user application or library. The actual generation of the SBOM is done by file(GENERATE)-ing one .cmake file for each target, file, external ref, etc, which will be included in a top-level cmake script. The top-level cmake script will run through each included file, to append to a "staging" spdx file, which will then be used in a configure_file() call to replace some final variables, like embedding a file checksum. There are install rules to generate a complete SBOM during installation, and an optional 'sbom' custom target that allows building an incomplete SBOM during the build step. The build target is just for convenience and faster development iteration time. It is incomplete because it is missing the installed file SHA1 checksums and the document verification code (the sha1 of all sha1s). We can't compute those during the build before the files are actually installed. A complete SBOM can only be achieved at installation time. The install script will include all the generated helper files, but also set some additional variables to ensure checksumming happens, and also handle multi-config installation, among other small things. For multi-config builds, CMake doesn't offer a way to run code after all configs are installed, because they might not always be installed, someone might choose to install just Release. To handle that, we rely on ninja installing each config sequentially (because ninja places the install rules into the 'console' pool which runs one task at a time). For each installed config we create a config-specific marker file. Once all marker files are present, whichever config ends up being installed as the last one, we run the sbom generation once, and then delete all marker files. There are a few internal variables that can be set during configuration to enable various checks (and other features) on the generated spdx files: - QT_INTERNAL_SBOM_VERIFY - QT_INTERNAL_SBOM_AUDIT - QT_INTERNAL_SBOM_AUDIT_NO_ERROR - QT_INTERNAL_SBOM_GENERATE_JSON - QT_INTERNAL_SBOM_SHOW_TABLE - QT_INTERNAL_SBOM_DEFAULT_CHECKS These use 3rd party python tools, so they are not enabled by default. If enabled, they run at installation time after the sbom is installed. We will hopefully enable them in CI. Overall, the code is still a bit messy in a few places, due to time constraints, but can be improved later. Some possible TODOs for the future: - Do we need to handle 3rd party libs linked into a Qt static library in a Qt shared build, where the Qt static lib is not installed, but linked into a Qt shared library, somehow specially? We can record a package for it, but we can't create a spdx file record for it (and associated source relationships) because we don't install the file, and spdx requires the file to be installed and checksummed. Perhaps we can consider adding some free-form text snippet to the package itself? - Do we want to add parsing of .cpp source files for Copyrights, to embed them into the packages? This will likely slow down configuration quite a bit. - Currently sbom info attached to WrapFoo packages in one repo is not exported / available in other repos. E.g. If we annotate WrapZLIB in qtbase with CPE_VENDOR zlib, this info will not be available when looking up WrapZLIB in qtimageformats. This is because they are IMPORTED libraries, and are not exported. We might want to record this info in the future. [ChangeLog][Build System] A new -sbom configure option can be used to generate and install a SPDX SBOM (Software Bill of Materials) file for each built Qt repository. Pick-to: 6.8 Task-number: QTBUG-122899 Change-Id: I9c730a6bbc47e02ce1836fccf00a14ec8eb1a5f4 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2024-03-07 18:02:56 +01:00
QtPublicGitHelpers
QtPublicPluginHelpers
QtPublicPluginHelpers_v2
QtPublicSbomAttributionHelpers
QtPublicSbomCpeHelpers
QtPublicSbomDepHelpers
QtPublicSbomFileHelpers
CMake: Generate an SPDX v2.3 SBOM file for each built repository This change adds a new -sbom configure option to allow generating and installing an SPDX v2.3 SBOM file when building a qt repo. The -sbom-dir option can be used to configure the location where each repo sbom file will be installed. By default it is installed into $prefix/$archdatadir/sbom/$sbom_lower_project_name.sdpx which is basically ~/Qt/sbom/qtbase-6.8.0.spdx The file is installed as part of the default installation rules, but it can also be installed manually using the "sbom" installation component, or "sbom_$lower_project_name" in a top-level build. For example: cmake install . --component sbom_qtbase CMake 3.19+ is needed to read the qt_attribution.json files for copyrights, license info, etc. When using an older cmake version, configuration will error out. It is possible to opt into using an older cmake version, but the generated sbom will lack all the attribution file information. Using an older cmake version is untested and not officially supported. Implementation notes. The bulk of the implementation is split into 4 new files: - QtPublicSbomHelpers.cmake - for Qt-specific collecting, processing and dispatching the generation of various pieces of the SBOM document e.g. a SDPX package associated with a target like Core, a SDPX file entry for each target binary file (per-config shared library, archive, executable, etc) - QtPublicSbomGenerationHelpers.cmake - for non-Qt specific implementation of SPDX generation. This also has some code that was taken from the cmake-sbom 3rd party project, so it is dual licensed under the usual Qt build system BSD license, as well as the MIT license of the 3rd party project - QtPublicGitHelpers.cmake - for git related features, mainly to embed queried hashes or tags into version strings, is dual-licensed for the same reasons as QtPublicSbomGenerationHelpers.cmake - QtSbomHelpers.cmake - Qt-specific functions that just forward arguments to the public functions. These are meant to be used in our Qt CMakeLists.txt instead of the public _qt_internal_add_sbom ones for naming consistency. These function would mostly be used to annotate 3rd party libraries with sbom info and to add sbom info for unusual target setups (like the Bootstrap library), because most of the handling is already done automatically via qt_internal_add_module/plugin/etc. The files are put into Public cmake files, with the future hope of making this available to user projects in some capacity. The distinction of Qt-specific and non-Qt specific code might blur a bit, and thus the separation across files might not always be consistent, but it was best effort. The main purpose of the code is to collect various information about targets and their relationships and generate equivalent SPDX info. Collection is currently done for the following targets: Qt modules, plugins, apps, tools, system libraries, bundled 3rd party libraries and partial 3rd party sources compiled directly as part of Qt targets. Each target has an equivalent SPDX package generated with information like version, license, copyright, CPE (common vulnerability identifier), files that belong to the package, and relationships on other SPDX packages (associated cmake targets), mostly gathered from direct linking dependencies. Each package might also contain files, e.g. libQt6Core.so for the Core target. Each file also has info like license id, copyrights, but also the list of source files that were used to generate the file and a sha1 checksum. SPDX documents can also refer to packages in other SPDX documents, and those are referred to via external document references. This is the case when building qtdeclarative and we refer to Core. For qt provided targets, we have complete information regarding licenses, and copyrights. For bundled 3rd party libraries, we should also have most information, which is usually parsed from the src/3rdparty/libfoo/qt_attribution.json files. If there are multiple attribution files, or if the files have multiple entries, we create a separate SBOM package for each of those entries, because each might have a separate copyright or version, and an sbom package can have only one version (although many copyrights). For system libraries we usually lack the information because we don't have attribution files for Find scripts. So the info needs to be manually annotated via arguments to the sbom function calls, or the FindFoo.cmake scripts expose that information in some form and we can query it. There are also corner cases like 3rdparty sources being directly included in a Qt library, like the m4dc files for Gui, or PCRE2 for Bootstrap. Or QtWebEngine libraries (either Qt bundled or Chromium bundled or system libraries) which get linked in by GN instead of CMake, so there are no direct targets for them. The information for these need to be annotated manually as well. There is also a distinction to be made for static Qt builds (or any static Qt library in a shared build), where the system libraries found during the Qt build might not be the same that are linked into the final user application or library. The actual generation of the SBOM is done by file(GENERATE)-ing one .cmake file for each target, file, external ref, etc, which will be included in a top-level cmake script. The top-level cmake script will run through each included file, to append to a "staging" spdx file, which will then be used in a configure_file() call to replace some final variables, like embedding a file checksum. There are install rules to generate a complete SBOM during installation, and an optional 'sbom' custom target that allows building an incomplete SBOM during the build step. The build target is just for convenience and faster development iteration time. It is incomplete because it is missing the installed file SHA1 checksums and the document verification code (the sha1 of all sha1s). We can't compute those during the build before the files are actually installed. A complete SBOM can only be achieved at installation time. The install script will include all the generated helper files, but also set some additional variables to ensure checksumming happens, and also handle multi-config installation, among other small things. For multi-config builds, CMake doesn't offer a way to run code after all configs are installed, because they might not always be installed, someone might choose to install just Release. To handle that, we rely on ninja installing each config sequentially (because ninja places the install rules into the 'console' pool which runs one task at a time). For each installed config we create a config-specific marker file. Once all marker files are present, whichever config ends up being installed as the last one, we run the sbom generation once, and then delete all marker files. There are a few internal variables that can be set during configuration to enable various checks (and other features) on the generated spdx files: - QT_INTERNAL_SBOM_VERIFY - QT_INTERNAL_SBOM_AUDIT - QT_INTERNAL_SBOM_AUDIT_NO_ERROR - QT_INTERNAL_SBOM_GENERATE_JSON - QT_INTERNAL_SBOM_SHOW_TABLE - QT_INTERNAL_SBOM_DEFAULT_CHECKS These use 3rd party python tools, so they are not enabled by default. If enabled, they run at installation time after the sbom is installed. We will hopefully enable them in CI. Overall, the code is still a bit messy in a few places, due to time constraints, but can be improved later. Some possible TODOs for the future: - Do we need to handle 3rd party libs linked into a Qt static library in a Qt shared build, where the Qt static lib is not installed, but linked into a Qt shared library, somehow specially? We can record a package for it, but we can't create a spdx file record for it (and associated source relationships) because we don't install the file, and spdx requires the file to be installed and checksummed. Perhaps we can consider adding some free-form text snippet to the package itself? - Do we want to add parsing of .cpp source files for Copyrights, to embed them into the packages? This will likely slow down configuration quite a bit. - Currently sbom info attached to WrapFoo packages in one repo is not exported / available in other repos. E.g. If we annotate WrapZLIB in qtbase with CPE_VENDOR zlib, this info will not be available when looking up WrapZLIB in qtimageformats. This is because they are IMPORTED libraries, and are not exported. We might want to record this info in the future. [ChangeLog][Build System] A new -sbom configure option can be used to generate and install a SPDX SBOM (Software Bill of Materials) file for each built Qt repository. Pick-to: 6.8 Task-number: QTBUG-122899 Change-Id: I9c730a6bbc47e02ce1836fccf00a14ec8eb1a5f4 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2024-03-07 18:02:56 +01:00
QtPublicSbomGenerationHelpers
QtPublicSbomHelpers
QtPublicSbomLicenseHelpers
QtPublicSbomOpsHelpers
QtPublicSbomPurlHelpers
QtPublicSbomPythonHelpers
QtPublicSbomQtEntityHelpers
QtPublicSbomSystemDepHelpers
QtPublicTargetHelpers
QtPublicTestHelpers
QtPublicToolHelpers
QtPublicWalkLibsHelpers
QtPublicWindowsHelpers
PARENT_SCOPE
)
endfunction()
# These files will be installed when building qtbase, but will NOT be automatically include()d
# when doing find_package(Qt6) nor find_package(Qt6 COMPONENTS BuildInternals).
# The files are expected to exist under the qtbase/cmake sub-directory.
function(qt_internal_get_qt_build_public_files_to_install out_var)
set(${out_var}
QtCopyFileIfDifferent.cmake
QtInitProject.cmake
CMake: Adjust CMP0156 policy handling for Apple platforms Backstory. 1) Starting with Qt 6.8, the prebuilt Qt for iOS SDK is built as static framework bundles instead of static libraries. That is done so that we can embed a privacy manifest into each framework bundle. 2) Up until CMake 3.28, CMake would not attempt to de-duplicate static libraries (or frameworks) on the command line. Starting with CMake 3.29, the CMP0156 policy was introduced to allow such de-duplication. 3) Since a while ago, Qt had its own policy handling for CMP0156, which it force sets to OLD, to disable any de-duplication. That was done to avoid possible regressions on platforms where link order matters. 4) A developer might add the -ObjC linker flag to a project, to ensure the linker retains all Objective-C categories it encounters in all the static libraries that were provided on the link line. Retaining in this case means that the /whole/ object file of a static library will be linked to the final executable. 5) The Apple ld linker (both the legacy and the new ld_prime one) can't cope with duplicate static frameworks on the link line when the -ObjC flag is used. It ends up throwing duplicate symbol errors, from trying to link the same object file from the same static framework more than once. The linker works just fine if the link line contains duplicate static libraries, rather than static frameworks. 6) When a project links against Qt6::Gui and Qt6::Core, the link line will contain Qt6::Core twice. This gets even more involved, when linking plugins, which cause static framework cycles, and thus a framework might appear multiple times. Thus, we have the situation that Qt forces the CMP0156 policy to OLD, Qt6::Core appears multiple times on the link line, no de-duplication is performed, the project adds the -ObjC flag, and the linker throws duplicate symbol errors. We can fix this by force setting the CMP0156 policy to NEW when targeting Apple platforms and using CMake 3.29+. A potential workaround for a project developer is to set the policy to NEW manually in their project. Unfortunately that doesn't work for older Qt versions. That's because CMake applies the policy value when add_executable is called, and the policy value in qt_add_executable is the one that is recorded when the function is defined. And the recorded policy is always OLD, because Qt6Config.cmake calls cmake_minimum_required with VERSION up to 3.21, which resets the policy value to OLD. So we have to force set the policy in qt_add_executable / qt_add_library via the existing __qt_internal_set_cmp0156 function. The __qt_internal_set_cmp0156 had some diagnostics to show a warning when the user modifies the policy themselves, but this never worked because of reason stated above: the policy value was always overridden in Qt6Config.cmake. To actually make the diagnostic work, there is now new code to save the policy value in the current directory scope, before Qt resets it. This only works if a project uses the find_package(Qt6 COMPONENTS Foo) signature. It won't work with a find_package(Qt6Core)-like signature. The policy value is not modified for platforms other than Apple ones for now. Amends 9702c3c78b2c16db6a9d0515d7d7698d9b064cd8 Pick-to: 6.5 6.8 6.9 Fixes: QTBUG-135978 Change-Id: I4d6e6c2a01e7092b417fc669d2aea40cf2dca578 Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2025-05-05 17:18:49 +02:00
QtPublicCMakeEarlyPolicyHelpers.cmake
# Public CMake files that are installed next Qt6Config.cmake, but are NOT included by it.
# Instead they are included by the generated CMake toolchain file.
QtPublicWasmToolchainHelpers.cmake
PARENT_SCOPE
)
endfunction()
# Includes all Qt CMake helper files that define functions and macros.
CMake: Split QtBuild.cmake into smaller files v2 My motivation to do this: - it got big and tangled again - sometimes functions need to be added to QtBuild.cmake rather than to a separate file because they need to be called before some of the global variables are set, to determine the value of those global variables (in my case install paths needed to be modified when building with xcframework support) - some of the global variable assignments have dependencies on other variables already being set and it's hard to keep track where that happens Split the contents of the file into smaller functions and macros and place them into pre-existing files when appropriate, or into new files. The new files are: - QtBuildHelpers.cmake - QtBuildPathsHelpers.cmake - QtMkspecHelpers.cmake The idea is to have Helpers file only define functions and never call them, so it's easy to include the file where needed without being scared of side effects. QtBuild.cmake will just include the helpers and call one entry point function to set up everything that was done by the file before. QtBuild.cmake is not merged into QtSetup, to make it easier to git blame (it's hard to blame a removed file). No new features were added as part of the refactoring. Some function names were renamed (but not all of them) to include the qt_internal prefix. Some lines were reformatted so they don't pass 100 chars limit after the code was placed into a function / macro. The Helpers includes were re-sorted. Some function calls were re-ordered where the order call didn't matter. Some of the code in QtAndroidHelpers.cmake was wrapped into a macro so that including the file does not cause side-effects by default. I'd like to follow up with similar changes for QtSetup.cmake and QtBuildInternalsConfig.cmake where possible, because having a few "entry points" into building a Qt submodule is also confusing, especially for those that aren't familiar with the build system and why certain things go into certain places. The intent is to cherry-pick this also to 6.5 and 6.6. Amends 44cce1a2ea9dadd8b2de93f40de34269dda703c0 Task-number: QTBUG-86035 Change-Id: I02ceff8ceb9b6e9c78bc85d6a42deb02fca3e46b Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Orkun Tokdemir <orkun.tokdemir@qt.io>
2023-11-15 18:08:15 +01:00
macro(qt_internal_include_all_helpers)
# Upstream cmake modules.
qt_internal_get_qt_build_upstream_cmake_modules(__qt_upstream_helpers)
foreach(__qt_file_name IN LISTS __qt_upstream_helpers)
include("${__qt_file_name}")
endforeach()
CMake: Split QtBuild.cmake into smaller files v2 My motivation to do this: - it got big and tangled again - sometimes functions need to be added to QtBuild.cmake rather than to a separate file because they need to be called before some of the global variables are set, to determine the value of those global variables (in my case install paths needed to be modified when building with xcframework support) - some of the global variable assignments have dependencies on other variables already being set and it's hard to keep track where that happens Split the contents of the file into smaller functions and macros and place them into pre-existing files when appropriate, or into new files. The new files are: - QtBuildHelpers.cmake - QtBuildPathsHelpers.cmake - QtMkspecHelpers.cmake The idea is to have Helpers file only define functions and never call them, so it's easy to include the file where needed without being scared of side effects. QtBuild.cmake will just include the helpers and call one entry point function to set up everything that was done by the file before. QtBuild.cmake is not merged into QtSetup, to make it easier to git blame (it's hard to blame a removed file). No new features were added as part of the refactoring. Some function names were renamed (but not all of them) to include the qt_internal prefix. Some lines were reformatted so they don't pass 100 chars limit after the code was placed into a function / macro. The Helpers includes were re-sorted. Some function calls were re-ordered where the order call didn't matter. Some of the code in QtAndroidHelpers.cmake was wrapped into a macro so that including the file does not cause side-effects by default. I'd like to follow up with similar changes for QtSetup.cmake and QtBuildInternalsConfig.cmake where possible, because having a few "entry points" into building a Qt submodule is also confusing, especially for those that aren't familiar with the build system and why certain things go into certain places. The intent is to cherry-pick this also to 6.5 and 6.6. Amends 44cce1a2ea9dadd8b2de93f40de34269dda703c0 Task-number: QTBUG-86035 Change-Id: I02ceff8ceb9b6e9c78bc85d6a42deb02fca3e46b Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Orkun Tokdemir <orkun.tokdemir@qt.io>
2023-11-15 18:08:15 +01:00
# Internal helpers available only while building Qt itself.
qt_internal_get_qt_build_private_helpers(__qt_private_helpers)
foreach(__qt_file_name IN LISTS __qt_private_helpers)
include("${__qt_file_name}")
endforeach()
CMake: Split QtBuild.cmake into smaller files v2 My motivation to do this: - it got big and tangled again - sometimes functions need to be added to QtBuild.cmake rather than to a separate file because they need to be called before some of the global variables are set, to determine the value of those global variables (in my case install paths needed to be modified when building with xcframework support) - some of the global variable assignments have dependencies on other variables already being set and it's hard to keep track where that happens Split the contents of the file into smaller functions and macros and place them into pre-existing files when appropriate, or into new files. The new files are: - QtBuildHelpers.cmake - QtBuildPathsHelpers.cmake - QtMkspecHelpers.cmake The idea is to have Helpers file only define functions and never call them, so it's easy to include the file where needed without being scared of side effects. QtBuild.cmake will just include the helpers and call one entry point function to set up everything that was done by the file before. QtBuild.cmake is not merged into QtSetup, to make it easier to git blame (it's hard to blame a removed file). No new features were added as part of the refactoring. Some function names were renamed (but not all of them) to include the qt_internal prefix. Some lines were reformatted so they don't pass 100 chars limit after the code was placed into a function / macro. The Helpers includes were re-sorted. Some function calls were re-ordered where the order call didn't matter. Some of the code in QtAndroidHelpers.cmake was wrapped into a macro so that including the file does not cause side-effects by default. I'd like to follow up with similar changes for QtSetup.cmake and QtBuildInternalsConfig.cmake where possible, because having a few "entry points" into building a Qt submodule is also confusing, especially for those that aren't familiar with the build system and why certain things go into certain places. The intent is to cherry-pick this also to 6.5 and 6.6. Amends 44cce1a2ea9dadd8b2de93f40de34269dda703c0 Task-number: QTBUG-86035 Change-Id: I02ceff8ceb9b6e9c78bc85d6a42deb02fca3e46b Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Orkun Tokdemir <orkun.tokdemir@qt.io>
2023-11-15 18:08:15 +01:00
# Helpers that are available in public projects and while building Qt itself.
qt_internal_get_qt_build_public_helpers(__qt_public_helpers)
foreach(__qt_file_name IN LISTS __qt_public_helpers)
include("${__qt_file_name}")
endforeach()
CMake: Split QtBuild.cmake into smaller files v2 My motivation to do this: - it got big and tangled again - sometimes functions need to be added to QtBuild.cmake rather than to a separate file because they need to be called before some of the global variables are set, to determine the value of those global variables (in my case install paths needed to be modified when building with xcframework support) - some of the global variable assignments have dependencies on other variables already being set and it's hard to keep track where that happens Split the contents of the file into smaller functions and macros and place them into pre-existing files when appropriate, or into new files. The new files are: - QtBuildHelpers.cmake - QtBuildPathsHelpers.cmake - QtMkspecHelpers.cmake The idea is to have Helpers file only define functions and never call them, so it's easy to include the file where needed without being scared of side effects. QtBuild.cmake will just include the helpers and call one entry point function to set up everything that was done by the file before. QtBuild.cmake is not merged into QtSetup, to make it easier to git blame (it's hard to blame a removed file). No new features were added as part of the refactoring. Some function names were renamed (but not all of them) to include the qt_internal prefix. Some lines were reformatted so they don't pass 100 chars limit after the code was placed into a function / macro. The Helpers includes were re-sorted. Some function calls were re-ordered where the order call didn't matter. Some of the code in QtAndroidHelpers.cmake was wrapped into a macro so that including the file does not cause side-effects by default. I'd like to follow up with similar changes for QtSetup.cmake and QtBuildInternalsConfig.cmake where possible, because having a few "entry points" into building a Qt submodule is also confusing, especially for those that aren't familiar with the build system and why certain things go into certain places. The intent is to cherry-pick this also to 6.5 and 6.6. Amends 44cce1a2ea9dadd8b2de93f40de34269dda703c0 Task-number: QTBUG-86035 Change-Id: I02ceff8ceb9b6e9c78bc85d6a42deb02fca3e46b Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Orkun Tokdemir <orkun.tokdemir@qt.io>
2023-11-15 18:08:15 +01:00
endmacro()
function(qt_internal_check_host_path_set_for_cross_compiling)
if(CMAKE_CROSSCOMPILING)
if(NOT IS_DIRECTORY "${QT_HOST_PATH}")
message(FATAL_ERROR "You need to set QT_HOST_PATH to cross compile Qt.")
endif()
endif()
endfunction()
macro(qt_internal_setup_find_host_info_package)
_qt_internal_determine_if_host_info_package_needed(__qt_build_requires_host_info_package)
_qt_internal_find_host_info_package("${__qt_build_requires_host_info_package}"
${INSTALL_CMAKE_NAMESPACE})
CMake: Split QtBuild.cmake into smaller files v2 My motivation to do this: - it got big and tangled again - sometimes functions need to be added to QtBuild.cmake rather than to a separate file because they need to be called before some of the global variables are set, to determine the value of those global variables (in my case install paths needed to be modified when building with xcframework support) - some of the global variable assignments have dependencies on other variables already being set and it's hard to keep track where that happens Split the contents of the file into smaller functions and macros and place them into pre-existing files when appropriate, or into new files. The new files are: - QtBuildHelpers.cmake - QtBuildPathsHelpers.cmake - QtMkspecHelpers.cmake The idea is to have Helpers file only define functions and never call them, so it's easy to include the file where needed without being scared of side effects. QtBuild.cmake will just include the helpers and call one entry point function to set up everything that was done by the file before. QtBuild.cmake is not merged into QtSetup, to make it easier to git blame (it's hard to blame a removed file). No new features were added as part of the refactoring. Some function names were renamed (but not all of them) to include the qt_internal prefix. Some lines were reformatted so they don't pass 100 chars limit after the code was placed into a function / macro. The Helpers includes were re-sorted. Some function calls were re-ordered where the order call didn't matter. Some of the code in QtAndroidHelpers.cmake was wrapped into a macro so that including the file does not cause side-effects by default. I'd like to follow up with similar changes for QtSetup.cmake and QtBuildInternalsConfig.cmake where possible, because having a few "entry points" into building a Qt submodule is also confusing, especially for those that aren't familiar with the build system and why certain things go into certain places. The intent is to cherry-pick this also to 6.5 and 6.6. Amends 44cce1a2ea9dadd8b2de93f40de34269dda703c0 Task-number: QTBUG-86035 Change-Id: I02ceff8ceb9b6e9c78bc85d6a42deb02fca3e46b Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Orkun Tokdemir <orkun.tokdemir@qt.io>
2023-11-15 18:08:15 +01:00
endmacro()
macro(qt_internal_setup_poor_mans_scope_finalizer)
# This sets up the poor man's scope finalizer mechanism.
# For newer CMake versions, we use cmake_language(DEFER CALL) instead.
if(CMAKE_VERSION VERSION_LESS "3.19.0")
variable_watch(CMAKE_CURRENT_LIST_DIR qt_watch_current_list_dir)
endif()
endmacro()
macro(qt_internal_set_qt_namespace)
set(QT_NAMESPACE "" CACHE STRING "Qt Namespace")
endmacro()
macro(qt_internal_set_qt_coord_type)
if(PROJECT_NAME STREQUAL "QtBase")
set(QT_COORD_TYPE double CACHE STRING "Type of qreal")
endif()
endmacro()
function(qt_internal_check_macos_host_version)
# macOS versions 10.14 and less don't have the implementation of std::filesystem API.
if(CMAKE_HOST_APPLE AND CMAKE_HOST_SYSTEM_VERSION VERSION_LESS "19.0.0")
message(FATAL_ERROR "macOS versions less than 10.15 are not supported for building Qt.")
endif()
endfunction()
function(qt_internal_setup_tool_path_command)
if(NOT CMAKE_HOST_WIN32)
return()
endif()
set(bindir "${QT_BUILD_INTERNALS_RELOCATABLE_INSTALL_PREFIX}/${INSTALL_BINDIR}")
file(TO_NATIVE_PATH "${bindir}" bindir)
list(APPEND command COMMAND)
list(APPEND command set PATH=${bindir}$<SEMICOLON>%PATH%)
set(QT_TOOL_PATH_SETUP_COMMAND "${command}" CACHE INTERNAL
"internal command prefix for tool invocations" FORCE)
# QT_TOOL_PATH_SETUP_COMMAND is deprecated. Please use _qt_internal_get_wrap_tool_script_path
# instead.
endfunction()
macro(qt_internal_setup_android_platform_specifics)
if(ANDROID)
qt_internal_setup_android_target_properties()
endif()
endmacro()
macro(qt_internal_setup_build_and_global_variables)
qt_internal_validate_cmake_generator()
qt_internal_set_qt_building_qt()
CMake: Simplify default CMAKE_BUILD_TYPE logic Previously we had four-ish locations where the CMAKE_BUILD_TYPE was force set. Twice in QtBuildInternalsExtra.cmake via qt_internal_force_set_cmake_build_type_conditionally(), depending on some conditions. This was executed right at find_package(Qt6 COMPONENTS BuildInternals) time. And twice in qt_internal_set_default_build_type() via qt_build_repo_begin() / qt_prepare_standalone_project() that goes through QtSetup.cmake. This was executed only if the relevant functions were called, rather than directly at find_package() time. The exact logic of which build type ended up being set was very confusing. Refactor the code to decide the build type in one single location when qt_build_repo_begin() / qt_prepare_standalone_project() are called, rather than directly at find_package() time. The actual logic when we override the build type depends on many factors: - when an explicit CMAKE_BUILD_TYPE is given, honor it, unless it's a multi-config build - when it's a multi-config build, don't set any CMAKE_BUILD_TYPE, use the value of CMAKE_CONFIGURATION_TYPES - when it's a qtbase build, compute a default unless an explicit value was given - the default is Debug if FEATURE_developer_build is ON - otherwise the default is Release - when it's a top-level build, only choose a build type for qtbase - when it's another repo build, use the original build type unless another was given explicitly (including in a top-level build) - when it's a standalone tests build - if qt is multi-config, the tests will be single config, due to various CI failure reasons, this hasn't changed - if qt is single config, use the original unless an explicit value was given - when it's a single standalone test build, use the original unless an explicit value was given To determine when an explicit CMAKE_BUILD_TYPE was given in contrast to when it was default initialized, we now have one single function that uses a few heuristics. The heuristics are needed because we can't reliably determine an explicitly given 'Debug' build on Windows, because CMake default initializes to that. The heuristics include: - checking whether CMAKE_BUILD_TYPE_INIT is different from CMAKE_BUILD_TYPE - checking what the CMAKE_BUILD_TYPE was before the first project() call when CMake default initializes - we save the previous value in the qt.toolchain.cmake file - also in QtAutoDetect during qtbase configuration - also when building the sqldrivers project - honoring the value of QT_NO_FORCE_SET_CMAKE_BUILD_TYPE As a result of the above changes, the build type will be set exactly zero or one times, for a particular build directory. Note that the configure script also has some logic on which CMAKE_BUILD_TYPE / CMAKE_CONFIGURATION_TYPES to pass to CMake depending on whether -debug / -release / -debug-and-release / -force-debug-info were passed. But once the values are passed, CMake will honor them. Amends 48841c34d2e86a741ec9992b9704c0fa5973503c Amends 8c912cddebe544010e7da3f87af5b21f3328d7ec Pick-to: 6.7 Task-number: QTBUG-114958 Task-number: QTBUG-120436 Change-Id: I30db14d1e8e9ff9bd2d7ea1d2256cdeb9493ca0d Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2024-01-22 18:11:31 +01:00
qt_internal_set_cmake_build_type()
qt_internal_set_message_log_level(CMAKE_MESSAGE_LOG_LEVEL)
qt_internal_unset_extra_build_internals_vars()
qt_internal_get_generator_is_multi_config()
CMake: Simplify default CMAKE_BUILD_TYPE logic Previously we had four-ish locations where the CMAKE_BUILD_TYPE was force set. Twice in QtBuildInternalsExtra.cmake via qt_internal_force_set_cmake_build_type_conditionally(), depending on some conditions. This was executed right at find_package(Qt6 COMPONENTS BuildInternals) time. And twice in qt_internal_set_default_build_type() via qt_build_repo_begin() / qt_prepare_standalone_project() that goes through QtSetup.cmake. This was executed only if the relevant functions were called, rather than directly at find_package() time. The exact logic of which build type ended up being set was very confusing. Refactor the code to decide the build type in one single location when qt_build_repo_begin() / qt_prepare_standalone_project() are called, rather than directly at find_package() time. The actual logic when we override the build type depends on many factors: - when an explicit CMAKE_BUILD_TYPE is given, honor it, unless it's a multi-config build - when it's a multi-config build, don't set any CMAKE_BUILD_TYPE, use the value of CMAKE_CONFIGURATION_TYPES - when it's a qtbase build, compute a default unless an explicit value was given - the default is Debug if FEATURE_developer_build is ON - otherwise the default is Release - when it's a top-level build, only choose a build type for qtbase - when it's another repo build, use the original build type unless another was given explicitly (including in a top-level build) - when it's a standalone tests build - if qt is multi-config, the tests will be single config, due to various CI failure reasons, this hasn't changed - if qt is single config, use the original unless an explicit value was given - when it's a single standalone test build, use the original unless an explicit value was given To determine when an explicit CMAKE_BUILD_TYPE was given in contrast to when it was default initialized, we now have one single function that uses a few heuristics. The heuristics are needed because we can't reliably determine an explicitly given 'Debug' build on Windows, because CMake default initializes to that. The heuristics include: - checking whether CMAKE_BUILD_TYPE_INIT is different from CMAKE_BUILD_TYPE - checking what the CMAKE_BUILD_TYPE was before the first project() call when CMake default initializes - we save the previous value in the qt.toolchain.cmake file - also in QtAutoDetect during qtbase configuration - also when building the sqldrivers project - honoring the value of QT_NO_FORCE_SET_CMAKE_BUILD_TYPE As a result of the above changes, the build type will be set exactly zero or one times, for a particular build directory. Note that the configure script also has some logic on which CMAKE_BUILD_TYPE / CMAKE_CONFIGURATION_TYPES to pass to CMake depending on whether -debug / -release / -debug-and-release / -force-debug-info were passed. But once the values are passed, CMake will honor them. Amends 48841c34d2e86a741ec9992b9704c0fa5973503c Amends 8c912cddebe544010e7da3f87af5b21f3328d7ec Pick-to: 6.7 Task-number: QTBUG-114958 Task-number: QTBUG-120436 Change-Id: I30db14d1e8e9ff9bd2d7ea1d2256cdeb9493ca0d Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2024-01-22 18:11:31 +01:00
# Depends on qt_internal_set_cmake_build_type
qt_internal_setup_cmake_config_postfix()
qt_internal_setup_position_independent_code()
qt_internal_set_link_depends_no_shared()
qt_internal_setup_default_install_prefix()
qt_internal_set_qt_source_tree_var()
qt_internal_set_export_compile_commands()
qt_internal_set_configure_from_ide()
# Depends on qt_internal_set_configure_from_ide
qt_internal_set_sync_headers_at_configure_time()
qt_internal_setup_build_benchmarks()
# Depends on qt_internal_setup_build_benchmarks
qt_internal_setup_build_tests()
qt_internal_setup_build_tools()
qt_internal_setup_sbom()
# Depends on qt_internal_setup_default_install_prefix
qt_internal_setup_build_examples()
qt_internal_set_qt_host_path()
qt_internal_setup_find_host_info_package()
qt_internal_setup_build_docs()
qt_internal_setup_build_java_docs_on_host()
qt_internal_include_qt_platform_android()
qt_internal_include_qt_properties()
# Depends on qt_internal_setup_default_install_prefix
CMake: Split QtBuild.cmake into smaller files v2 My motivation to do this: - it got big and tangled again - sometimes functions need to be added to QtBuild.cmake rather than to a separate file because they need to be called before some of the global variables are set, to determine the value of those global variables (in my case install paths needed to be modified when building with xcframework support) - some of the global variable assignments have dependencies on other variables already being set and it's hard to keep track where that happens Split the contents of the file into smaller functions and macros and place them into pre-existing files when appropriate, or into new files. The new files are: - QtBuildHelpers.cmake - QtBuildPathsHelpers.cmake - QtMkspecHelpers.cmake The idea is to have Helpers file only define functions and never call them, so it's easy to include the file where needed without being scared of side effects. QtBuild.cmake will just include the helpers and call one entry point function to set up everything that was done by the file before. QtBuild.cmake is not merged into QtSetup, to make it easier to git blame (it's hard to blame a removed file). No new features were added as part of the refactoring. Some function names were renamed (but not all of them) to include the qt_internal prefix. Some lines were reformatted so they don't pass 100 chars limit after the code was placed into a function / macro. The Helpers includes were re-sorted. Some function calls were re-ordered where the order call didn't matter. Some of the code in QtAndroidHelpers.cmake was wrapped into a macro so that including the file does not cause side-effects by default. I'd like to follow up with similar changes for QtSetup.cmake and QtBuildInternalsConfig.cmake where possible, because having a few "entry points" into building a Qt submodule is also confusing, especially for those that aren't familiar with the build system and why certain things go into certain places. The intent is to cherry-pick this also to 6.5 and 6.6. Amends 44cce1a2ea9dadd8b2de93f40de34269dda703c0 Task-number: QTBUG-86035 Change-Id: I02ceff8ceb9b6e9c78bc85d6a42deb02fca3e46b Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Orkun Tokdemir <orkun.tokdemir@qt.io>
2023-11-15 18:08:15 +01:00
qt_internal_setup_paths_and_prefixes()
qt_internal_reset_global_state()
# Depends on qt_internal_setup_paths_and_prefixes
CMake: Split QtBuild.cmake into smaller files v2 My motivation to do this: - it got big and tangled again - sometimes functions need to be added to QtBuild.cmake rather than to a separate file because they need to be called before some of the global variables are set, to determine the value of those global variables (in my case install paths needed to be modified when building with xcframework support) - some of the global variable assignments have dependencies on other variables already being set and it's hard to keep track where that happens Split the contents of the file into smaller functions and macros and place them into pre-existing files when appropriate, or into new files. The new files are: - QtBuildHelpers.cmake - QtBuildPathsHelpers.cmake - QtMkspecHelpers.cmake The idea is to have Helpers file only define functions and never call them, so it's easy to include the file where needed without being scared of side effects. QtBuild.cmake will just include the helpers and call one entry point function to set up everything that was done by the file before. QtBuild.cmake is not merged into QtSetup, to make it easier to git blame (it's hard to blame a removed file). No new features were added as part of the refactoring. Some function names were renamed (but not all of them) to include the qt_internal prefix. Some lines were reformatted so they don't pass 100 chars limit after the code was placed into a function / macro. The Helpers includes were re-sorted. Some function calls were re-ordered where the order call didn't matter. Some of the code in QtAndroidHelpers.cmake was wrapped into a macro so that including the file does not cause side-effects by default. I'd like to follow up with similar changes for QtSetup.cmake and QtBuildInternalsConfig.cmake where possible, because having a few "entry points" into building a Qt submodule is also confusing, especially for those that aren't familiar with the build system and why certain things go into certain places. The intent is to cherry-pick this also to 6.5 and 6.6. Amends 44cce1a2ea9dadd8b2de93f40de34269dda703c0 Task-number: QTBUG-86035 Change-Id: I02ceff8ceb9b6e9c78bc85d6a42deb02fca3e46b Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Orkun Tokdemir <orkun.tokdemir@qt.io>
2023-11-15 18:08:15 +01:00
qt_internal_set_mkspecs_dir()
qt_internal_setup_platform_definitions_and_mkspec()
qt_internal_check_macos_host_version()
_qt_internal_check_apple_sdk_and_xcode_versions()
qt_internal_check_msvc_versions()
CMake: Split QtBuild.cmake into smaller files v2 My motivation to do this: - it got big and tangled again - sometimes functions need to be added to QtBuild.cmake rather than to a separate file because they need to be called before some of the global variables are set, to determine the value of those global variables (in my case install paths needed to be modified when building with xcframework support) - some of the global variable assignments have dependencies on other variables already being set and it's hard to keep track where that happens Split the contents of the file into smaller functions and macros and place them into pre-existing files when appropriate, or into new files. The new files are: - QtBuildHelpers.cmake - QtBuildPathsHelpers.cmake - QtMkspecHelpers.cmake The idea is to have Helpers file only define functions and never call them, so it's easy to include the file where needed without being scared of side effects. QtBuild.cmake will just include the helpers and call one entry point function to set up everything that was done by the file before. QtBuild.cmake is not merged into QtSetup, to make it easier to git blame (it's hard to blame a removed file). No new features were added as part of the refactoring. Some function names were renamed (but not all of them) to include the qt_internal prefix. Some lines were reformatted so they don't pass 100 chars limit after the code was placed into a function / macro. The Helpers includes were re-sorted. Some function calls were re-ordered where the order call didn't matter. Some of the code in QtAndroidHelpers.cmake was wrapped into a macro so that including the file does not cause side-effects by default. I'd like to follow up with similar changes for QtSetup.cmake and QtBuildInternalsConfig.cmake where possible, because having a few "entry points" into building a Qt submodule is also confusing, especially for those that aren't familiar with the build system and why certain things go into certain places. The intent is to cherry-pick this also to 6.5 and 6.6. Amends 44cce1a2ea9dadd8b2de93f40de34269dda703c0 Task-number: QTBUG-86035 Change-Id: I02ceff8ceb9b6e9c78bc85d6a42deb02fca3e46b Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Orkun Tokdemir <orkun.tokdemir@qt.io>
2023-11-15 18:08:15 +01:00
qt_internal_check_host_path_set_for_cross_compiling()
qt_internal_setup_android_platform_specifics()
qt_internal_setup_tool_path_command()
qt_internal_setup_default_target_function_options()
qt_internal_set_default_rpath_settings()
qt_internal_set_qt_namespace()
qt_internal_set_qt_coord_type()
qt_internal_set_qt_path_separator()
qt_internal_set_internals_extra_cmake_code()
qt_internal_set_top_level_source_dir()
qt_internal_set_apple_archiver_flags()
qt_internal_set_debug_extend_target()
qt_internal_setup_poor_mans_scope_finalizer()
qt_internal_set_compiler_optimization_flags()
qt_internal_set_compiler_warning_flags()
qt_set_language_standards()
qt_internal_set_use_ccache()
qt_internal_set_unity_build()
qt_internal_set_allow_symlink_in_paths()
qt_internal_set_skip_setup_deployment()
qt_internal_set_qt_allow_download()
qt_internal_detect_dirty_features()
CMake: Split QtBuild.cmake into smaller files v2 My motivation to do this: - it got big and tangled again - sometimes functions need to be added to QtBuild.cmake rather than to a separate file because they need to be called before some of the global variables are set, to determine the value of those global variables (in my case install paths needed to be modified when building with xcframework support) - some of the global variable assignments have dependencies on other variables already being set and it's hard to keep track where that happens Split the contents of the file into smaller functions and macros and place them into pre-existing files when appropriate, or into new files. The new files are: - QtBuildHelpers.cmake - QtBuildPathsHelpers.cmake - QtMkspecHelpers.cmake The idea is to have Helpers file only define functions and never call them, so it's easy to include the file where needed without being scared of side effects. QtBuild.cmake will just include the helpers and call one entry point function to set up everything that was done by the file before. QtBuild.cmake is not merged into QtSetup, to make it easier to git blame (it's hard to blame a removed file). No new features were added as part of the refactoring. Some function names were renamed (but not all of them) to include the qt_internal prefix. Some lines were reformatted so they don't pass 100 chars limit after the code was placed into a function / macro. The Helpers includes were re-sorted. Some function calls were re-ordered where the order call didn't matter. Some of the code in QtAndroidHelpers.cmake was wrapped into a macro so that including the file does not cause side-effects by default. I'd like to follow up with similar changes for QtSetup.cmake and QtBuildInternalsConfig.cmake where possible, because having a few "entry points" into building a Qt submodule is also confusing, especially for those that aren't familiar with the build system and why certain things go into certain places. The intent is to cherry-pick this also to 6.5 and 6.6. Amends 44cce1a2ea9dadd8b2de93f40de34269dda703c0 Task-number: QTBUG-86035 Change-Id: I02ceff8ceb9b6e9c78bc85d6a42deb02fca3e46b Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Orkun Tokdemir <orkun.tokdemir@qt.io>
2023-11-15 18:08:15 +01:00
endmacro()