Writing Go command line applications with Cobra: Part I

For just about every little solution I build I use the command line. My theory is that if I can automate it through the command line, then I can use it just about anywhere. I also spend far less time tweaking UIs this way, and can keep my focus on solving the problem.

If I want a UI for my solution, I will expose my functionality through an API and write my UI in something like dotNET. Alternatively, I may just opt to write my solution in dotNET instead. Go is not my be all and end all. It may be different for you though.

When I started with Go, I created stock command line applications. Here’s what a typical Hello World looks like in Go for a command line application:

package main

import "fmt"

func main() {
	fmt.Println("Hello, World!")
}

Straight forward and that is all the code that is needed. The above code is in a file called main.go, so here’s how to run the above application:

$ go run main.go
Hello, World!

Predictable and not that interesting, since you can achieve the same result with many languages. There’s little compelling reason to think Go is the way here. I struggled with this too as I can do the same thing in a dotnet core application, or C application. Guess what the default sample code is on the golang.org website? Pretty much what I just wrote. A Hello World application is pretty meaningless when stating the reason you’d want to use a language over another, but it at least gives you a comparable starting point. And no, I do not miss the irony in that I used the same example in my Part I and Part II articles.

Rounding back to my original point, I write all of my utility applications as command line applications (or CLI apps). I started with the above template and started writing code when I first came to Go. It will take you far so there is no need to go much further, but I started writing more complicated solutions that needed more command-line options, and some options that needed other options, and flags, and flags that are only relevant in one command and not another. Go allows you to set this all up via its flag library. So, importing no 3rd party libraries, you can process flags like so:

package main

import (
	"flag"
	"fmt"
)

func main() {
	flag.Parse()
	args := flag.Args()
	fmt.Println(args)
}

Run it and you see the following:

$ go run main.go init --include-blob
[init --include-blob]

It’s difficult to produce consistency in your application this way. I was keen on trying to get my applications to behave more like the git CLI, so you see something closer to this:

$ git
usage: git [--version] [--help] [-C <path>] [-c <name>=<value>]
           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
           [-p | --paginate | --no-pager] [--no-replace-objects] [--bare]
           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
           <command> [<args>]

These are common Git commands used in various situations:

start a working area (see also: git help tutorial)
   clone      Clone a repository into a new directory
   init       Create an empty Git repository or reinitialize an existing one

work on the current change (see also: git help everyday)
   add        Add file contents to the index
   mv         Move or rename a file, a directory, or a symlink
   reset      Reset current HEAD to the specified state
   rm         Remove files from the working tree and from the index

examine the history and state (see also: git help revisions)
   bisect     Use binary search to find the commit that introduced a bug
   grep       Print lines matching a pattern
   log        Show commit logs
   show       Show various types of objects
   status     Show the working tree status

grow, mark and tweak your common history
   branch     List, create, or delete branches
   checkout   Switch branches or restore working tree files
   commit     Record changes to the repository
   diff       Show changes between commits, commit and working tree, etc
   merge      Join two or more development histories together
   rebase     Reapply commits on top of another base tip
   tag        Create, list, delete or verify a tag object signed with GPG

collaborate (see also: git help workflows)
   fetch      Download objects and refs from another repository
   pull       Fetch from and integrate with another repository or a local branch
   push       Update remote refs along with associated objects

'git help -a' and 'git help -g' list available subcommands and some
concept guides. See 'git help <command>' or 'git help <concept>'
to read about a specific subcommand or concept.

Seems like a far fetch, and loads of work using the standard flag package, doesn’t it? Well, turns out it’s not that hard when someone has done the hard work for you. I was trying to solve the issue where I wanted consistency in my CLI applications with flag dependencies and make it predictable in how it worked. I searched for some Go libraries for command line packages and stumbled across a few. One stood out for me, called Cobra, and so I tried to figure out how to get that working. I read through the README on the Cobra site, and here’s the thing you need to do to install it:

go get -u github.com/spf13/cobra/cobra

Okay, run that in your terminal and all is good? Let’s see:

$ go get -u github.com/spf13/cobra/cobra
$

Well, that seemed innocuous, let’s see how we create the application. It’s in the generator documentation. I wanted to let Cobra do the hard work instead of me importing the library and going the hard way:

$ cobra init github.com/spf13/newApp
Using config file: /Users/brian/.cobra.yaml
Your Cobra application is ready at
/Users/brian/go/src/github.com/spf13/newApp

Give it a try by going there and running `go run main.go`.
Add commands to it by running `cobra add [cmdname]`.

Okay, what was easy, I followed those instructions, and it all worked. Note where it put the source code? Since I copied the example, I probably should think about a better location, so let’s try that again:

$ cobra init bitbucket.com/brianj-za/cli1
Using config file: /Users/brian/.cobra.yaml
Your Cobra application is ready at
/Users/brian/go/src/bitbucket.com/brianj-za/cli1

Give it a try by going there and running `go run main.go`.
Add commands to it by running `cobra add [cmdname]`.

That’s better. Complete with a fancy name! Let’s see what the code structure looks like:

 $ tree
.
├── LICENSE
├── cmd
│   └── root.go
└── main.go

1 directory, 3 files

Slightly more complex than my simple starting application, but in reality hardly tough. This excited me as I didn’t want loads of generated code lying around. Having a look at the code it seems straightforward. The main.go file becomes a static file now, as you will use other files for your code:

package main

import "bitbucket.com/brianj-za/cli1/cmd"

func main() {
	cmd.Execute()
}

So where’s the rest? Well, the import "bitbucket.com/brianj-za/cli1/cmd" statement gives a clue. Let’s have a look in my cmd/ folder, in the only file there called root.go:

package cmd

import (
	"fmt"
	"os"

	homedir "github.com/mitchellh/go-homedir"
	"github.com/spf13/cobra"
	"github.com/spf13/viper"
)

var cfgFile string

// rootCmd represents the base command when called without any subcommands
var rootCmd = &cobra.Command{
	Use:   "cli1",
	Short: "A brief description of your application",
	Long: `A longer description that spans multiple lines and likely contains
examples and usage of using your application. For example:

Cobra is a CLI library for Go that empowers applications.
This application is a tool to generate the needed files
to quickly create a Cobra application.`,
	// Uncomment the following line if your bare application
	// has an action associated with it:
	//	Run: func(cmd *cobra.Command, args []string) { },
}

// Execute adds all child commands to the root command and sets flags appropriately.
// This is called by main.main(). It only needs to happen once to the rootCmd.
func Execute() {
	if err := rootCmd.Execute(); err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
}

func init() {
	cobra.OnInitialize(initConfig)

	// Here you will define your flags and configuration settings.
	// Cobra supports persistent flags, which, if defined here,
	// will be global for your application.
	rootCmd.PersistentFlags().StringVar(&cfgFile, "config", "", "config file (default is $HOME/.cli1.yaml)")

	// Cobra also supports local flags, which will only run
	// when this action is called directly.
	rootCmd.Flags().BoolP("toggle", "t", false, "Help message for toggle")
}

// initConfig reads in config file and ENV variables if set.
func initConfig() {
	if cfgFile != "" {
		// Use config file from the flag.
		viper.SetConfigFile(cfgFile)
	} else {
		// Find home directory.
		home, err := homedir.Dir()
		if err != nil {
			fmt.Println(err)
			os.Exit(1)
		}

		// Search config in home directory with name ".cli1" (without extension).
		viper.AddConfigPath(home)
		viper.SetConfigName(".cli1")
	}

	viper.AutomaticEnv() // read in environment variables that match

	// If a config file is found, read it in.
	if err := viper.ReadInConfig(); err == nil {
		fmt.Println("Using config file:", viper.ConfigFileUsed())
	}
}

Whoa! That’s a lot of generated code! Do I really want this in my application? I started off with a simple command line app using one file, and now I have this huge root.go file under the cmd/folder. What have I done!? I had this reservation early on. However, I also know how hard it is to get command line flag parsing working well, and consistently. Although the Go standard library provides us with flags, I couldn’t find commands, which I wanted. I also wanted the ability to nest flags in to specific commands. For example, git init will have different flags to git commit.

The good news is that there’s a lot of good work that goes on in Cobra. It extends the flag library a bit to allow for a consistent flag approach, along with being able to add commands to your application. In the git sample, a command might be commit. A flag to that command may be -m or --message. This is where the extra code comes from. You could do this yourself but it’s tricky to get command-line options right. So I opted in to the Cobra way of thinking and went with it. I still do occasionally just create a main.go and write simple utilities that way, but you almost always start hitting the case where you really need to have a better command line parser. I’ve started making Cobra apps my default now, even if the app is simple.

Next up is how we create the commands you want, possibly with flags. Since I want to replace git with my new awesome tool called cli1, a new awesome source control application, let’s emulate that. One bit of output from the Cobra generator gave a clue as to the next step:

Add commands to it by running `cobra add [cmdname]`.

Let’s try that. Our cli1 source control probably needs an init as well, so let’s go:

$ cobra add init
Using config file: /Users/brian/.cobra.yaml
init created at /Users/brian/go/src/bitbucket.com/brianj-za/cli1/cmd/init.go

Nice, what happens when I run this application:

$ go run main.go
A longer description that spans multiple lines and likely contains
examples and usage of using your application. For example:

Cobra is a CLI library for Go that empowers applications.
This application is a tool to generate the needed files
to quickly create a Cobra application.

Usage:
  cli1 [command]

Available Commands:
  help        Help about any command
  init        A brief description of your command

Flags:
      --config string   config file (default is $HOME/.cli1.yaml)
  -h, --help            help for cli1
  -t, --toggle          Help message for toggle

Use "cli1 [command] --help" for more information about a command.

That really is looking good. I can see my init command now. Although the descriptions all look generated. Where is that set? The root.go listing from above:

var rootCmd = &cobra.Command{
	Use:   "cli1",
	Short: "A brief description of your application",
	Long: `A longer description that spans multiple lines and likely contains
examples and usage of using your application. For example:

Cobra is a CLI library for Go that empowers applications.
This application is a tool to generate the needed files
to quickly create a Cobra application.`,
	// Uncomment the following line if your bare application
	// has an action associated with it:
	//	Run: func(cmd *cobra.Command, args []string) { },
}

Let’s clean that up.

var rootCmd = &cobra.Command{
	Use:   "cli1",
	Short: "This application is a better replacement for the git utility",
	Long: `Git, although cool, is not awesome. This utility takes the ideas
from git, and makes them awesome! It's a better source control than git.
In fact, it's git but awesome.`,
	// Uncomment the following line if your bare application
	// has an action associated with it:
	//	Run: func(cmd *cobra.Command, args []string) { },
}

Right, much better. I noticed that the description of my init command was also bland. Let’s fix that up:

var initCmd = &cobra.Command{
	Use:   "init",
	Short: "Create an empty cli1 repository or reinitialize an existing one",
	Long: `This command creates an empty cli1 repository - basically a .cli1 directory with subdirectories for
objects, refs/heads, refs/tags, and template files. An initial HEAD file that references the HEAD of the
master branch is also created.

Running cli1 init in an existing repository is safe. It will not overwrite things that are already there.
The primary reason for rerunning cli1 init is to pick up newly added templates (or to move the repository
to another place if --separate-cli1-dir is given).`,
	Run: func(cmd *cobra.Command, args []string) {
		fmt.Println("init called")
	},
}

Much better now that I’ve literally copied the git init text and crossed out the word “git” and put in the word “cli1”:

$ go run main.go
Git, although cool, is not awesome. This utility takes the ideas
from git, and makes them awesome! It's a better source control than git.
In fact, it's git but awesome.

Usage:
  cli1 [command]

Available Commands:
  help        Help about any command
  init        Create an empty cli1 repository or reinitialize an existing one

Flags:
      --config string   config file (default is $HOME/.cli1.yaml)
  -h, --help            help for cli1
  -t, --toggle          Help message for toggle

Use "cli1 [command] --help" for more information about a command.

So much better. Although, what is that --toggle option? I didn’t put that there. I don’t need it, so I’ll go find it and remove it. Aha! It’s in the root.go file:

func init() {
	cobra.OnInitialize(initConfig)

	// Here you will define your flags and configuration settings.
	// Cobra supports persistent flags, which, if defined here,
	// will be global for your application.
	rootCmd.PersistentFlags().StringVar(&cfgFile, "config", "", "config file (default is $HOME/.cli1.yaml)")

	// Cobra also supports local flags, which will only run
	// when this action is called directly.
	rootCmd.Flags().BoolP("toggle", "t", false, "Help message for toggle")
}

I will comment that out. Now we have a nice, clean, CLI ready to go.

You may have noticed that the toggle is in Flags, and a little above that is a config in PersistentFlags? The difference here is that the persistent flags are global to the command and sub commands. All commands have them. They are useful for cases where you need something for all your commands. I typically find something like --database-server could fit here. Although, that could arguably sit in the config file (which I will describe in a later post).

Here’s what the application outputs when I run the init command:

 go run main.go init
init called

How do I see more information about this command? Cobra adds a --help global flag that automatically prints out usage. Let’s run that:

$ go run main.go init --help
This command creates an empty cli1 repository - basically a .cli1 directory with subdirectories for
objects, refs/heads, refs/tags, and template files. An initial HEAD file that references the HEAD of the
master branch is also created.

Running cli1 init in an existing repository is safe. It will not overwrite things that are already there.
The primary reason for rerunning cli1 init is to pick up newly added templates (or to move the repository
to another place if --separate-cli1-dir is given).

Usage:
  cli1 init [flags]

Flags:
  -h, --help   help for init

Global Flags:
      --config string   config file (default is $HOME/.cli1.yaml)

Great. My application is now ready to do its magic. One final step is to have init do something different. Here’s what I did for now, which is just a slightly worse version than was already there, since I am lying (I didn’t really create that folder at all):

	Run: func(cmd *cobra.Command, args []string) {
		fmt.Println("Initialized empty Cli1 repository in /Users/brian/go/src/bitbucket.com/brianj-za/cli1/.cli1/")
	},

Running that looks like:

$ go run main.go init
Initialized empty Cli1 repository in /Users/brian/go/src/bitbucket.com/brianj-za/cli1/.cli1/

In a future post I will take you through more complex scenarios, elaborate on how you add your code to the commands properly, and how you read your much needed flags. Cobra is a very simple way to create your command line applications and allows you to expand on them later without having to reorganise your code. Pick one for yourself (any package really, I just went with Cobra as it works for me) but I will use Cobra in all later articles.