292 lines
9.3 KiB
Go
Raw Permalink Normal View History

build: pass BuildOptions around explicitly & fix multi-platform issues The big change here is to pass around an explicit `*BuildOptions` object as part of Compose operations like `up` & `run` that may or may not do builds. If the options object is `nil`, no builds whatsoever will be attempted. Motivation is to allow for partial rebuilds in the context of an `up` for watch. This was broken and tricky to accomplish because various parts of the Compose APIs mutate the `*Project` for convenience in ways that make it unusable afterwards. (For example, it might set `service.Build = nil` because it's not going to build that service right _then_. But we might still want to build it later!) NOTE: This commit does not actually touch the watch logic. This is all in preparation to make it possible. As part of this, a bunch of code moved around and I eliminated a bunch of partially redundant logic, mostly around multi-platform. Several edge cases have been addressed as part of this: * `DOCKER_DEFAULT_PLATFORM` was _overriding_ explicitly set platforms in some cases, this is no longer true, and it behaves like the Docker CLI now * It was possible for Compose to build an image for one platform and then try to run it for a different platform (and fail) * Errors are no longer returned if a local image exists but for the wrong platform - the correct platform will be fetched/built (if possible). Because there's a LOT of subtlety and tricky logic here, I've also tried to add an excessive amount of explanatory comments. Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-08-30 08:47:09 -04:00
/*
Copyright 2023 Docker Compose CLI authors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package compose
import (
"context"
build: pass BuildOptions around explicitly & fix multi-platform issues The big change here is to pass around an explicit `*BuildOptions` object as part of Compose operations like `up` & `run` that may or may not do builds. If the options object is `nil`, no builds whatsoever will be attempted. Motivation is to allow for partial rebuilds in the context of an `up` for watch. This was broken and tricky to accomplish because various parts of the Compose APIs mutate the `*Project` for convenience in ways that make it unusable afterwards. (For example, it might set `service.Build = nil` because it's not going to build that service right _then_. But we might still want to build it later!) NOTE: This commit does not actually touch the watch logic. This is all in preparation to make it possible. As part of this, a bunch of code moved around and I eliminated a bunch of partially redundant logic, mostly around multi-platform. Several edge cases have been addressed as part of this: * `DOCKER_DEFAULT_PLATFORM` was _overriding_ explicitly set platforms in some cases, this is no longer true, and it behaves like the Docker CLI now * It was possible for Compose to build an image for one platform and then try to run it for a different platform (and fail) * Errors are no longer returned if a local image exists but for the wrong platform - the correct platform will be fetched/built (if possible). Because there's a LOT of subtlety and tricky logic here, I've also tried to add an excessive amount of explanatory comments. Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-08-30 08:47:09 -04:00
"fmt"
"io"
"os"
"slices"
"sort"
"strings"
"text/tabwriter"
build: pass BuildOptions around explicitly & fix multi-platform issues The big change here is to pass around an explicit `*BuildOptions` object as part of Compose operations like `up` & `run` that may or may not do builds. If the options object is `nil`, no builds whatsoever will be attempted. Motivation is to allow for partial rebuilds in the context of an `up` for watch. This was broken and tricky to accomplish because various parts of the Compose APIs mutate the `*Project` for convenience in ways that make it unusable afterwards. (For example, it might set `service.Build = nil` because it's not going to build that service right _then_. But we might still want to build it later!) NOTE: This commit does not actually touch the watch logic. This is all in preparation to make it possible. As part of this, a bunch of code moved around and I eliminated a bunch of partially redundant logic, mostly around multi-platform. Several edge cases have been addressed as part of this: * `DOCKER_DEFAULT_PLATFORM` was _overriding_ explicitly set platforms in some cases, this is no longer true, and it behaves like the Docker CLI now * It was possible for Compose to build an image for one platform and then try to run it for a different platform (and fail) * Errors are no longer returned if a local image exists but for the wrong platform - the correct platform will be fetched/built (if possible). Because there's a LOT of subtlety and tricky logic here, I've also tried to add an excessive amount of explanatory comments. Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-08-30 08:47:09 -04:00
"github.com/compose-spec/compose-go/v2/cli"
"github.com/compose-spec/compose-go/v2/template"
"github.com/compose-spec/compose-go/v2/types"
"github.com/docker/cli/cli/command"
"github.com/docker/compose/v2/internal/tracing"
ui "github.com/docker/compose/v2/pkg/progress"
"github.com/docker/compose/v2/pkg/prompt"
build: pass BuildOptions around explicitly & fix multi-platform issues The big change here is to pass around an explicit `*BuildOptions` object as part of Compose operations like `up` & `run` that may or may not do builds. If the options object is `nil`, no builds whatsoever will be attempted. Motivation is to allow for partial rebuilds in the context of an `up` for watch. This was broken and tricky to accomplish because various parts of the Compose APIs mutate the `*Project` for convenience in ways that make it unusable afterwards. (For example, it might set `service.Build = nil` because it's not going to build that service right _then_. But we might still want to build it later!) NOTE: This commit does not actually touch the watch logic. This is all in preparation to make it possible. As part of this, a bunch of code moved around and I eliminated a bunch of partially redundant logic, mostly around multi-platform. Several edge cases have been addressed as part of this: * `DOCKER_DEFAULT_PLATFORM` was _overriding_ explicitly set platforms in some cases, this is no longer true, and it behaves like the Docker CLI now * It was possible for Compose to build an image for one platform and then try to run it for a different platform (and fail) * Errors are no longer returned if a local image exists but for the wrong platform - the correct platform will be fetched/built (if possible). Because there's a LOT of subtlety and tricky logic here, I've also tried to add an excessive amount of explanatory comments. Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-08-30 08:47:09 -04:00
)
func applyPlatforms(project *types.Project, buildForSinglePlatform bool) error {
defaultPlatform := project.Environment["DOCKER_DEFAULT_PLATFORM"]
for name, service := range project.Services {
build: pass BuildOptions around explicitly & fix multi-platform issues The big change here is to pass around an explicit `*BuildOptions` object as part of Compose operations like `up` & `run` that may or may not do builds. If the options object is `nil`, no builds whatsoever will be attempted. Motivation is to allow for partial rebuilds in the context of an `up` for watch. This was broken and tricky to accomplish because various parts of the Compose APIs mutate the `*Project` for convenience in ways that make it unusable afterwards. (For example, it might set `service.Build = nil` because it's not going to build that service right _then_. But we might still want to build it later!) NOTE: This commit does not actually touch the watch logic. This is all in preparation to make it possible. As part of this, a bunch of code moved around and I eliminated a bunch of partially redundant logic, mostly around multi-platform. Several edge cases have been addressed as part of this: * `DOCKER_DEFAULT_PLATFORM` was _overriding_ explicitly set platforms in some cases, this is no longer true, and it behaves like the Docker CLI now * It was possible for Compose to build an image for one platform and then try to run it for a different platform (and fail) * Errors are no longer returned if a local image exists but for the wrong platform - the correct platform will be fetched/built (if possible). Because there's a LOT of subtlety and tricky logic here, I've also tried to add an excessive amount of explanatory comments. Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-08-30 08:47:09 -04:00
if service.Build == nil {
continue
}
// default platform only applies if the service doesn't specify
if defaultPlatform != "" && service.Platform == "" {
if len(service.Build.Platforms) > 0 && !slices.Contains(service.Build.Platforms, defaultPlatform) {
return fmt.Errorf("service %q build.platforms does not support value set by DOCKER_DEFAULT_PLATFORM: %s", name, defaultPlatform)
build: pass BuildOptions around explicitly & fix multi-platform issues The big change here is to pass around an explicit `*BuildOptions` object as part of Compose operations like `up` & `run` that may or may not do builds. If the options object is `nil`, no builds whatsoever will be attempted. Motivation is to allow for partial rebuilds in the context of an `up` for watch. This was broken and tricky to accomplish because various parts of the Compose APIs mutate the `*Project` for convenience in ways that make it unusable afterwards. (For example, it might set `service.Build = nil` because it's not going to build that service right _then_. But we might still want to build it later!) NOTE: This commit does not actually touch the watch logic. This is all in preparation to make it possible. As part of this, a bunch of code moved around and I eliminated a bunch of partially redundant logic, mostly around multi-platform. Several edge cases have been addressed as part of this: * `DOCKER_DEFAULT_PLATFORM` was _overriding_ explicitly set platforms in some cases, this is no longer true, and it behaves like the Docker CLI now * It was possible for Compose to build an image for one platform and then try to run it for a different platform (and fail) * Errors are no longer returned if a local image exists but for the wrong platform - the correct platform will be fetched/built (if possible). Because there's a LOT of subtlety and tricky logic here, I've also tried to add an excessive amount of explanatory comments. Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-08-30 08:47:09 -04:00
}
service.Platform = defaultPlatform
}
if service.Platform != "" {
if len(service.Build.Platforms) > 0 {
if !slices.Contains(service.Build.Platforms, service.Platform) {
return fmt.Errorf("service %q build configuration does not support platform: %s", name, service.Platform)
build: pass BuildOptions around explicitly & fix multi-platform issues The big change here is to pass around an explicit `*BuildOptions` object as part of Compose operations like `up` & `run` that may or may not do builds. If the options object is `nil`, no builds whatsoever will be attempted. Motivation is to allow for partial rebuilds in the context of an `up` for watch. This was broken and tricky to accomplish because various parts of the Compose APIs mutate the `*Project` for convenience in ways that make it unusable afterwards. (For example, it might set `service.Build = nil` because it's not going to build that service right _then_. But we might still want to build it later!) NOTE: This commit does not actually touch the watch logic. This is all in preparation to make it possible. As part of this, a bunch of code moved around and I eliminated a bunch of partially redundant logic, mostly around multi-platform. Several edge cases have been addressed as part of this: * `DOCKER_DEFAULT_PLATFORM` was _overriding_ explicitly set platforms in some cases, this is no longer true, and it behaves like the Docker CLI now * It was possible for Compose to build an image for one platform and then try to run it for a different platform (and fail) * Errors are no longer returned if a local image exists but for the wrong platform - the correct platform will be fetched/built (if possible). Because there's a LOT of subtlety and tricky logic here, I've also tried to add an excessive amount of explanatory comments. Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-08-30 08:47:09 -04:00
}
}
if buildForSinglePlatform || len(service.Build.Platforms) == 0 {
// if we're building for a single platform, we want to build for the platform we'll use to run the image
// similarly, if no build platforms were explicitly specified, it makes sense to build for the platform
// the image is designed for rather than allowing the builder to infer the platform
service.Build.Platforms = []string{service.Platform}
}
}
// services can specify that they should be built for multiple platforms, which can be used
// with `docker compose build` to produce a multi-arch image
// other cases, such as `up` and `run`, need a single architecture to actually run
// if there is only a single platform present (which might have been inferred
// from service.Platform above), it will be used, even if it requires emulation.
// if there's more than one platform, then the list is cleared so that the builder
// can decide.
// TODO(milas): there's no validation that the platform the builder will pick is actually one
// of the supported platforms from the build definition
// e.g. `build.platforms: [linux/arm64, linux/amd64]` on a `linux/ppc64` machine would build
// for `linux/ppc64` instead of returning an error that it's not a valid platform for the service.
if buildForSinglePlatform && len(service.Build.Platforms) > 1 {
// empty indicates that the builder gets to decide
service.Build.Platforms = nil
}
project.Services[name] = service
build: pass BuildOptions around explicitly & fix multi-platform issues The big change here is to pass around an explicit `*BuildOptions` object as part of Compose operations like `up` & `run` that may or may not do builds. If the options object is `nil`, no builds whatsoever will be attempted. Motivation is to allow for partial rebuilds in the context of an `up` for watch. This was broken and tricky to accomplish because various parts of the Compose APIs mutate the `*Project` for convenience in ways that make it unusable afterwards. (For example, it might set `service.Build = nil` because it's not going to build that service right _then_. But we might still want to build it later!) NOTE: This commit does not actually touch the watch logic. This is all in preparation to make it possible. As part of this, a bunch of code moved around and I eliminated a bunch of partially redundant logic, mostly around multi-platform. Several edge cases have been addressed as part of this: * `DOCKER_DEFAULT_PLATFORM` was _overriding_ explicitly set platforms in some cases, this is no longer true, and it behaves like the Docker CLI now * It was possible for Compose to build an image for one platform and then try to run it for a different platform (and fail) * Errors are no longer returned if a local image exists but for the wrong platform - the correct platform will be fetched/built (if possible). Because there's a LOT of subtlety and tricky logic here, I've also tried to add an excessive amount of explanatory comments. Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-08-30 08:47:09 -04:00
}
return nil
}
// isRemoteConfig checks if the main compose file is from a remote source (OCI or Git)
func isRemoteConfig(dockerCli command.Cli, options buildOptions) bool {
if len(options.ConfigPaths) == 0 {
return false
}
remoteLoaders := options.remoteLoaders(dockerCli)
for _, loader := range remoteLoaders {
if loader.Accept(options.ConfigPaths[0]) {
return true
}
}
return false
}
// checksForRemoteStack handles environment variable prompts for remote configurations
func checksForRemoteStack(ctx context.Context, dockerCli command.Cli, project *types.Project, options buildOptions, assumeYes bool, cmdEnvs []string) error {
if !isRemoteConfig(dockerCli, options) {
return nil
}
if metrics, ok := ctx.Value(tracing.MetricsKey{}).(tracing.Metrics); ok && metrics.CountIncludesRemote > 0 {
if err := confirmRemoteIncludes(dockerCli, options, assumeYes); err != nil {
return err
}
}
displayLocationRemoteStack(dockerCli, project, options)
return promptForInterpolatedVariables(ctx, dockerCli, options.ProjectOptions, assumeYes, cmdEnvs)
}
// Prepare the values map and collect all variables info
type varInfo struct {
name string
value string
source string
required bool
defaultValue string
}
// promptForInterpolatedVariables displays all variables and their values at once,
// then prompts for confirmation
func promptForInterpolatedVariables(ctx context.Context, dockerCli command.Cli, projectOptions *ProjectOptions, assumeYes bool, cmdEnvs []string) error {
if assumeYes {
return nil
}
varsInfo, noVariables, err := extractInterpolationVariablesFromModel(ctx, dockerCli, projectOptions, cmdEnvs)
if err != nil {
return err
}
if noVariables {
return nil
}
displayInterpolationVariables(dockerCli.Out(), varsInfo)
// Prompt for confirmation
userInput := prompt.NewPrompt(dockerCli.In(), dockerCli.Out())
msg := "\nDo you want to proceed with these variables? [Y/n]: "
confirmed, err := userInput.Confirm(msg, true)
if err != nil {
return err
}
if !confirmed {
return fmt.Errorf("operation cancelled by user")
}
return nil
}
func extractInterpolationVariablesFromModel(ctx context.Context, dockerCli command.Cli, projectOptions *ProjectOptions, cmdEnvs []string) ([]varInfo, bool, error) {
cmdEnvMap := extractEnvCLIDefined(cmdEnvs)
// Create a model without interpolation to extract variables
opts := configOptions{
noInterpolate: true,
ProjectOptions: projectOptions,
}
model, err := opts.ToModel(ctx, dockerCli, nil, cli.WithoutEnvironmentResolution)
if err != nil {
return nil, false, err
}
// Extract variables that need interpolation
variables := template.ExtractVariables(model, template.DefaultPattern)
if len(variables) == 0 {
return nil, true, nil
}
var varsInfo []varInfo
proposedValues := make(map[string]string)
for name, variable := range variables {
info := varInfo{
name: name,
required: variable.Required,
defaultValue: variable.DefaultValue,
}
// Determine value and source based on priority
if value, exists := cmdEnvMap[name]; exists {
info.value = value
info.source = "command-line"
proposedValues[name] = value
} else if value, exists := os.LookupEnv(name); exists {
info.value = value
info.source = "environment"
proposedValues[name] = value
} else if variable.DefaultValue != "" {
info.value = variable.DefaultValue
info.source = "compose file"
proposedValues[name] = variable.DefaultValue
} else {
info.value = "<unset>"
info.source = "none"
}
varsInfo = append(varsInfo, info)
}
return varsInfo, false, nil
}
func extractEnvCLIDefined(cmdEnvs []string) map[string]string {
// Parse command-line environment variables
cmdEnvMap := make(map[string]string)
for _, env := range cmdEnvs {
parts := strings.SplitN(env, "=", 2)
if len(parts) == 2 {
cmdEnvMap[parts[0]] = parts[1]
}
}
return cmdEnvMap
}
func displayInterpolationVariables(writer io.Writer, varsInfo []varInfo) {
// Display all variables in a table format
_, _ = fmt.Fprintln(writer, "\nFound the following variables in configuration:")
w := tabwriter.NewWriter(writer, 0, 0, 3, ' ', 0)
_, _ = fmt.Fprintln(w, "VARIABLE\tVALUE\tSOURCE\tREQUIRED\tDEFAULT")
sort.Slice(varsInfo, func(a, b int) bool {
return varsInfo[a].name < varsInfo[b].name
})
for _, info := range varsInfo {
required := "no"
if info.required {
required = "yes"
}
_, _ = fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n",
info.name,
info.value,
info.source,
required,
info.defaultValue,
)
}
_ = w.Flush()
}
func displayLocationRemoteStack(dockerCli command.Cli, project *types.Project, options buildOptions) {
mainComposeFile := options.ProjectOptions.ConfigPaths[0] //nolint:staticcheck
if ui.Mode != ui.ModeQuiet && ui.Mode != ui.ModeJSON {
_, _ = fmt.Fprintf(dockerCli.Out(), "Your compose stack %q is stored in %q\n", mainComposeFile, project.WorkingDir)
}
}
func confirmRemoteIncludes(dockerCli command.Cli, options buildOptions, assumeYes bool) error {
if assumeYes {
return nil
}
var remoteIncludes []string
remoteLoaders := options.ProjectOptions.remoteLoaders(dockerCli) //nolint:staticcheck
for _, cf := range options.ProjectOptions.ConfigPaths { //nolint:staticcheck
for _, loader := range remoteLoaders {
if loader.Accept(cf) {
remoteIncludes = append(remoteIncludes, cf)
break
}
}
}
if len(remoteIncludes) == 0 {
return nil
}
_, _ = fmt.Fprintln(dockerCli.Out(), "\nWarning: This Compose project includes files from remote sources:")
for _, include := range remoteIncludes {
_, _ = fmt.Fprintf(dockerCli.Out(), " - %s\n", include)
}
_, _ = fmt.Fprintln(dockerCli.Out(), "\nRemote includes could potentially be malicious. Make sure you trust the source.")
msg := "Do you want to continue? [y/N]: "
confirmed, err := prompt.NewPrompt(dockerCli.In(), dockerCli.Out()).Confirm(msg, false)
if err != nil {
return err
}
if !confirmed {
return fmt.Errorf("operation cancelled by user")
}
return nil
}