Go Generate: Quick, Easy, and oh so Powerful

If you’ve had the misfortune of talking to me about development over the past two years, then you may well have noticed me evangelising about Go. It’s true: I’ve become guilty of something I previously mocked colleagues for.

Perhaps it’s because the book “The Go Programming Language” reminds me of the first ever programming book I picked up - “The C Programming Language”? Maybe it’s the simplicity and distinct lack of “magic” that the language has? Or maybe it’s even because it encourages verbosity… and if you’ve ever read anything I’ve written before, you’ll likely understand why that resonates with me!

Recently I learnt about a yet another thing to preach about though: go generate. Now this command isn’t a recent addition, on the contrary - it was added in v1.4 of the language . However it’s also not something I’ve often seen used, and that’s a real shame.

Go’s generate command allows you to execute commands at an early stage of the build process, and it gets it’s name from the fact it’s intended to generate new source files. In actual fact, you could use the command to do more than simple code generation… really though, you shouldn’t: that would be a bad idea, and that’s what make is for. Talking of bad ideas though..

Erhh… is code generation a good idea?#

I have mixed feelings about code generation, but when used appropriately it can be incredibly useful. I’m sure we’ve all worked with tools before that generate a certain amount of scaffolding - it’s common in scenarios where you’re using technologies like protobuf or Swagger/OpenAPI, and it’s quite common with database migration packages too.

Alternatively, if you’re using a microservices oriented architecture - then there may be other useful applications; i.e defining messages and models in a centralised repository using a language-agnostic method, and utilising code generators to pull/parse/validate the latest versions during build.

So with the obligatory “Erhh.. use it when needed and don’t do stupid things” disclaimer out of the way, lets look at a worked example.

Example: YAML -> Go#

The simplest possible example is (a) accepting a yaml file, and (b) generating the appropriate data-structure to make it accessible at compile-time. So we want to take this:

status:
  - code: -1
    message: "Operation Failed"
  - code: 1
    message: "Operation Accepted"
  - code: 2
    message: "Operation Queued"

… and generate this:

// Code generated by go generate; DO NOT EDIT.
// File generated at 2019-02-17 20:17:26.668250852 +0000 GMT m=+0.003175536

package main

var statusMap = map[StatusCode]StatusMessage{
	-1: "Operation Failed",
	1: "Operation Accepted",
	2: "Operation Queued",
}

In “real life”, we could retrieve our data from anywhere that is programmatically accessible, and in any concievable format. Similarly, we could structure our output without many constraints either. It may also be worth highlighting that the generators don’t even have to be written in Go: they just need to be an executable available on the build system. We’re using go though, and we’re going to build the generator automatically too.

In our repository we have 3 .go files, and 1 .yaml file. All of these source files are incredibly simple too: with status.go just containing a couple of type aliases, and main.go the main() function.

.generators/
------------ status_gen.go
main.go
status.go
statuses.yaml

You may be surprised to see that our generation is handled by two lines - or two comments to be precise. If we peak in to status.go then we’ll see //go:generate go run .generators/status_gen.go . This comment asks the go generate tool to run a command, and in our case - that’s go run .generators/status_gen.go.

If we look at .generators/status_gen.go then we’ll find another rather simple looking file, but also with a curious comment: // +build ignore. Like the comment found in status.go , this provides an instruction to the Go tooling; in this example, it tells the Go tooling to not use this file when performing go build.

In reality, it’s only these two comments which are responsible for the code generation. There really is nothing else to it. If you thought this was going to be a code heavy blog post then sadly I must disappoint; but definitely clone the repository and run through it for yourself. With two steps, it really couldn’t be simpler:

  1. Write a utility to generate your code, including // +build ignore to prevent inclusion at build time;
  2. Annotate a .go file with //go:generate go run ... to trigger execution during go generate.

That’s really all there is to it.

Conclusion#

Writing code generation, and - more importantly - binding it to the build process, is quick and easy with go generate. By using the native Go tooling, you can also save yourself headaches during the build process too - by avoiding any dependencies external to the Go ecosystem. (perfect for those nice multi-stage builds using the official Go Dockerimages!)

If you find yourself relying on copy/paste programming at times, or would like to generate source files in multiple projects from one centralised source - then this may well be a worthy suggestion to explore.