EDIT: I’ve discovered this morning that, in fact, the issue still exists. I suspect a bug in the VCS code for Go, and have raised an issue (Although the issue bot gnome has misread what was being said and claimed it was a discussion not an issue)

Almost every write up I have seen for working with private repositories and Go have taken the point of view that the repository is hosted on one of the major public Git repository holders (Github, or Gitlab). My problem is that I am hosting repositories on my own machines. I can use git without those websites ;).

I was having a lot of trouble with Go interacting with those self-hosted repositories however, which I thought was odd because Go uses Git under the hood for things like go get and go mod download.

After a lot of hunting around I discovered that, in fact, Go has been created to deal with several possible Version Control Systems (VCS). You can see how it interacts with them in the internal VCS package.

The general purpose nature of the VCS usage in Go means that some assumptions have been made that make go get and go mod download not quite work the same as (say) git clone, and that’s why I was having a bit of trouble :).

This StackOverflow question about Private Repository with SSH was invaluable for diagnosing the problems and suggestions for solutions, including what I have finally got working.

There are three things that are needed in order for you to use the code hosted non-publicly.

Gitconfig

The very first problem for me was that my repositories are accessible via ssh only, not https.

Both git and go get can handle this just fine, by adding a couple of lines to the local ~/.gitconfig

[url "git@github.com:"]
        insteadOf = https://github.com/
[url "git@shanehoweath.com:"]
        insteadOf = https://shanehowearth.com/

GOPRIVATE

The Go download tools will try to find the repository/package using the module proxy, but my modules are private, and I don’t want them published/acknowledged to even exist on the public proxy.

Go uses an environment variable GOPRIVATE to determine if the public proxy should be consulted.

I use GOPRIVATE when I run go mod download (including in Dockerfiles)

GOPRIVATE=shanehowearth.com go mod download

Repository naming.

This was the most difficult issue to diagnose. None of the tools were telling me what the problem was, and it’s a function of the fact that Go’s vcs system is trying to cater to multiple different types of version control systems (git, mercurial, etc).

Git will happily clone, pull, push, to any directory on any host that it’s directed to. Go will too, except when it’s a private repository over ssh, then the name of the repository/directory becomes important.

Firstly, the name of the host must have a ‘.’ (period) in the name. so localhost/path/to/repo will not work. The stackoverflow link above recommends to use the ip if the host doesn’t have a period in the name (eg. use 192.168.1.72 instead of zeus).

Secondly, and this is quite painful, the name of the repository must have a .git suffix.

So shanehowearth.com/path/to/repo must be renamed to shanehowearth.com/path/to/repo.git in order for it to work.

Note that this also means any references to the repo in the code must also have the .git suffix too, so I have package imports like shanehowearth.com/path/to/repo.git/package/name in my code, which I find ugly, but it is what it is.

This includes the go.mod file, which has to have .git in the package name:

module shanehowearth/path/to/module.git

I am exasperated by this issue, it took me a couple of days to discover that the three things were combining to make it really difficult to host my own repo and use it via ssh with go, but I can now.

Hopefully this blog post saves someone else the pain I went through, and, of course, a massive thanks to StackOverflow user nshuytsa for publishing their solution as well.