I spent my Christmas break starting work on a tool that I want to use at work. The tool is a docker-compose like application, but intended to pass information to Podman.

docker-compose (or as it’s now named docker compose takes configuration information from a compose.yml or docker-compose.yml file (other named files can be used, but these are the defaults).

It’s a simple Extract Transform Load process, Extract the information from the YAML files, validate and Transform it into commands that podman will understand, then Load those commands (via a unix socket) into podman which will then go and do its magic.

Will I finish the project? Who knows :)

The first job, then, is extraction of the configuration information from the compose.yml or docker-compose.yml (or file[s] specified by the -f or --file flag on the commandline). There is an agreed structure to the YAML, that is, the sections, headings and types are specified in advance, and they can be found in the Compose spec.

That’s a very big document, with links to more documents for sub-sections.

I have created a collection of go structs that were meant to hold the data found in the YAML files. The process of converting the YAML to go struct(s) is called unmarshal (Note, for now I am linking to gopkg.in/yaml.v2 but I have not as yet decided whichYAML libraries will be used in the project.

The structs look something like this composeyaml.go :

type Config struct {
	File     string `yaml:"file"`
	External bool   `yaml:"external"`
	Name     string `yaml:"name"`
}

type Secret struct {
	File        string `yaml:"file,omitempty"`
	Environment string `yaml:"environment,omitempty"`
	External    bool   `yaml:"external,omitempty"`
	Name        string `yaml:"name,omitempty"`
}

type Compose struct {
	Version  string             `yaml:"version,omitempty"`
	Services map[string]Service `yaml:"services"`
	Networks []Network          `yaml:"networks,omitempty"`
	Volumes  []Volume           `yaml:"volumes,omitempty"`
	Configs  []Config           `yaml:"configs,omitempty"`
	Secrets  map[string]Secret  `yaml:"secrets,omitempty"`
}

Anonymous inner structs

In my first iteration I used anonymous inner structs in several places.

eg.:

type Service struct {
	Image  string            `yaml:"image,omitempty"`
	Ports  MapOrListOfString `yaml:"ports,omitempty"`
	Deploy struct {
		EndpointMode string            `yaml:"endpoint_mode,omitempty"`
		Labels       map[string]string `yaml:"labels,omitempty"`
		Mode         string            `yaml:"mode,omitempty"`
		Placement    struct {
			Constraints map[string]string `yaml:"constraints,omitempty"`
			Preferences map[string]string `yaml:"preferences,omitempty"`
		} `yaml:"placement,omitempty"`
		Replicas  int32 `yaml:"replicas,omitempty"`
		Resources struct {
			Limits       Resources `yaml:"limits,omitempty"`
			Reservations Resources `yaml:"reservations,omitempty"`
		} `yaml:"resources,omitempty"`
		RestartPolicy struct {
			Condition   string        `yaml:"condition,omitempty"`
			Delay       time.Duration `yaml:"delay,omitempty"`
			MaxAttempts int           `yaml:"max_attempts,omitempty"`
			Window      time.Duration `yaml:"window,omitempty"`
		} `yaml:"restart_policy,omitempty"`
		RollbackConfig DeployConfig `yaml:"rollback_config,omitempty"`
		UpdateConfig   DeployConfig `yaml:"update_config,omitempty"`
	} `yaml:"deploy,omitempty"`

When I started (validation) testing that the examples found in the spec loaded into my structs (as can be seen in composeyaml_test.go I had an issue where the runtime would error, telling me that I could not use the structs in the tests as the structs that were being tested, and the reason for that was that I was not copying the annotations correctly. That is, the names and types of the fields in the structs (and anonymous inner structs) in my tests were correct, but the annotations were missing, so the runtime could not use them.

Secondly, every test I had to copy and paste the relevant parts of the struct into the tests, making for an error prone mess.

The solution: Always use exportable structs for the inner structs.

Multitype fields

The spec allows multiple types for some of the fields, for example, the Ports field can be either a map[string]string or a []string; Go, as you know isn’t a fan of dynamic typing. I had hoped generics would help here but

type MapOrListOfString interface {
    map[string]string | []string
}

produced errors about constraints not being allowed.

For now I am using interface{} with the intention of either setting the type in bespoke unmarshal function,or a separate validation function.

Note: Some time after I started the project I discovered the JSON description of the spec, which can be used with JSON-to-Go to automagically create the structs. I haven’t used it except to check my work. I wouldn’t have had nearly as much learnings had I opted to use that tool.