DevOps Your Desktop

So, you got a new developer machine? Whatever the reason; you now have hours of updating, installing and configuring ahead of you... yay... haven't we all been there?

There is an easier™️ way and in this post I'll be walking you through how I personally go about doing it. I'm also going to assume you are familiar with the basics of git, speak some bash and have heard of DevOps before.

The first time I tried setting up my machine in a repeatable way was to use dotfiles.

Dotfiles only got me so far and before I knew it I was writing bash scripts to handle more complex installations and they were all but indempotent. Ansible to the rescue. I still have bash files, but they are only used to run the initial setup and updates and because the underlying tech is Ansible I can run them as many times as I want without breaking anything.

As I'm doing this on my MacBook Pro and running macOS X, I'm going to use Homebrew as my package manager. This should also work on Linux.

Head on over to brew.sh and follow the installation instructions and I'll meet you back here.

Creating a dedicated git repo

We also need a git repo. I've gone and setup a GitHub repository but feel free to chose another provider. You also don't need to make it public but for the purpose of this post I did.

Now I have a git repo with a README.md to which I will add basic information on how to clone and get setup as we progress.

Generating an SSH Key and adding it to your GitHub account.

To clone the new repo I'll be using ssh key authentication instead of username/password, so we'll have to generate a public/private key pair.

$ ssh-keygen -o -a 100 -t ed25519 -f ~/.ssh/id_ed25519 -C "joe@blog.net"

It'll prompt you to enter a passphrase, you can opt out by leaving it empty.
Once they keys have been successfully generated you can copy the public key into your clipboard.

$ cat ~/.ssh/id_ed25519.pub | pbcopy

With the content in the clipboard now add this key to your GitHub account under settings > SSH and GPG Keys. Click "New SSH Key", give it a descriptive title and paste the clipboard content into the "Key" field. Now hit "Add SSH Key" to save. Okay, now we're ready to clone the new repository.

If you're on a Mac and haven't got the Xcode command line developer tools installed it'll prompt you to do so when git is run for the first time.

Cloning the bare git repo and adding configuration

$ git clone git@github.com:ck3mp3r/desktop-devops.git --bare ~/.cfg

This will now have cloned a bare repo into ~/.cfg but before we can start working on adding files to the repo we need to add a little more configuration to your current terminal session. We won't be calling the git command directly anymore but instead use a preconfigured alias.

$ alias dotfiles='/usr/bin/git --git-dir=$HOME/.cfg/ --work-tree=$HOME/'
$ dotfiles config --local status.showUntrackedFiles no
$ echo "alias dotfiles='/usr/bin/git --git-dir=\$HOME/.cfg/ --work-tree=\$HOME'" >> $HOME/.zshrc

This will create the alias we need to interact with the dotfiles repo. It should behave in much the same way the git command would. Next it will ensure that untracked files are ignored, we don't want to be adding everything in your home directory. Finally we ensure the alias is persisted for subsequent sessions.

Now lets test it.

$ dotfiles status
On branch main
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
	deleted:    README.md

Untracked files not listed (use -u option to show untracked files)

Because we cloned a bare repository the README.md is missing and marked as deleted. We can fix this by doing:

$ dotfiles restore --staged README.md
$ dotfiles restore README.md
$ dotfiles status
On branch main
nothing to commit (use -u to show untracked files)

Now we'll add a way to initialize the system in such a way that we have the tooling we need to provision the workstation further.

Adding a directory structure

To continue with the setup we'll create a hidden folder structure:

$ mkdir ~/.dotfiles/{bin,ansible}

This will contain everything else we need that doesn't have a specific home.

Next up is creating a shell script for the setup.

$ touch ~/.dotfiles/bin/df_setup.sh
$ chmod +x ~/.dotfiles/bin/df_setup.sh
$ echo "export PATH=~/.dotfiles/bin:$PATH" >> .zshrc
$ exec $SHELL

Here we just created the scaffolding for the df_setup command. We created an empty shell script, made it executable, added it to the system path for the current user and reloaded the shell. Now we can start editing the script.

#!/usr/bin/env zsh

df_init () {
  if [[ ! `command -v brew` ]] then;
    /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
  fi
  sudo -H python3 -m pip install --upgrade pip ansible
}

while [[ "$#" -gt 0 ]]; do case $1 in
  -i|--init) df_init;;
  *) echo "Unknown parameter passed: $1"; exit 1;;
esac; shift; done

Now is a good time to commit our changes.

Committing changes for the first time

$ git config --global user.name "Joe Blogs"
$ git config --global user.email "joe@blogs.net"
$ dotfiles add .zshrc .dotfiles/bin/df_setup
$ dotfiles commit -m "basic initialisation of setup"
$ dotfiles push --set-upstream origin main

If you execute df_setup -i it will run the newly created script. Once it has completed you should have ansible and homebrew installed.

Homebrew is the missing package manager for macOS (also available for linux now) and ansible is the jack of all trades when it comes to provisioning software and even infrastructure.

Adding ansible into the mix

- hosts: all
  tasks:
    - debug: msg="You are running {{ ansible_distribution }}"

We also need to update the content of ~/.dotfiles/bin/df_setup in order to use ansible in our setup with a convenient command.

#!/usr/bin/env zsh

df_init () {
  if [[ ! `command -v brew` ]] then;
    /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
  fi
  sudo -H python3 -m pip install --upgrade pip ansible
}

df_ansible () {
  cwd=`pwd`
  cd ~/.dotfiles/ansible
  ansible-playbook playbook.yaml -i localhost, -c local -e ansible_python_interpreter=auto_silent
  cd $cwd
}

while [[ "$#" -gt 0 ]]; do case $1 in
  -i|--init) df_init;;
  -a|--ansible) df_ansible;;
  *) echo "Unknown parameter passed: $1"; exit 1;;
esac; shift; done

Now running df_setup -a will start an ansible run and configure your workstation from scratch or later if something needs to change.

Our simple playbook will just output a debug message... for now:

$ df_setup -a

PLAY [all] ************************************************************************************************

TASK [Gathering Facts] ************************************************************************************
ok: [localhost]

TASK [debug] **********************************************************************************************
ok: [localhost] => {
    "msg": "You are running MacOSX"
}

PLAY RECAP ************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

Getting organized with roles

$ mkdir -p ~/.dotfiles/ansible/roles/common/{tasks,files}

---
- name: Update Homebrew inventory.
  homebrew:
    update_homebrew: yes
    

In order for our new role to be picked up we need to reference it in ~/.dotfiles/ansible/playbook.yaml as follows:

- hosts: all
  roles:
    - common
    

As you can see we removed the debug task as it isn't really necessary. If you run df_setup -a now you should see the role execute and cause homebrew to do an update.

Now is as good a time as any to commit our changes.

$ dotfiles status
On branch main
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   .dotfiles/bin/df_setup

no changes added to commit (use "git add" and/or "git commit -a")

Using our alias we can see it is only aware of the changes in one file. We need to add the other files to the repo explicitly.

$ dotfiles add .dotfiles/bin/df_setup .dotfiles/ansible/playbook.yaml .dotfiles/ansible/roles/common/tasks/main.yaml
$ dotfiles status
On branch main
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
	new file:   .dotfiles/ansible/playbook.yaml
	new file:   .dotfiles/ansible/roles/common/tasks/main.yaml
	modified:   .dotfiles/bin/df_setup

Untracked files not listed (use -u option to show untracked files)

That looks more like it.

$ dotfiles commit -m "adding ansible playbook and common role"
$ dotfiles push 

Oh My ZSH

ZSH can be nicely enhanced using Oh My ZSH, a community driven framework for managing Zsh configuration. We'll also add zplug into the mix.

---
- name: Create ~/.zshrc.d directory
  file:
    path: ~/.zshrc.d
    state: directory

- name: Ensure all files in ~/.zshrc.d get sourced
  lineinfile:
    path: ~/.zshrc
    line: for n in `ls ~/.zshrc.d/*.zsh`; do source ~/.zshrc.d/$n; done;

- name: Check for Oh My ZSH Installation
  stat:
    path: ~/.oh-my-zsh
  register: omz_installed

- name: Install Oh My ZSH...
  git:
    repo: 'git://github.com/robbyrussell/oh-my-zsh.git'
    dest: ~/.oh-my-zsh
  when: omz_installed.stat.exists == False
    
- name: Install zsh autosuggestions and zplug
  homebrew:
    package:
      - zsh-autosuggestions
      - zplug
    state: latest
    
- name: Ensure ZSH config for Oh My ZSH and zplug is loaded
  copy:
    src: files/zshrc.d/oh-my.zsh
    dest: ~/.zshrc.d/oh-my.zsh

export ZPLUG_HOME=/usr/local/opt/zplug
source $ZPLUG_HOME/init.zsh

export ZSH=~/.oh-my-zsh
ZSH_DISABLE_COMPFIX="true"
COMPLETION_WAITING_DOTS="true"
plugins=(git)
source $ZSH/oh-my-zsh.sh

# Load zsh autosuggestion functionality
source /usr/local/share/zsh-autosuggestions/zsh-autosuggestions.zsh

# Load theme file
zplug 'dracula/zsh', as:theme

# Install plugins if there are plugins that have not been installed
if ! zplug check --verbose; then
    printf "Install? [y/N]: "
    if read -q; then
        echo; zplug install
    fi
fi

# Then, source plugins and add commands to $PATH
zplug load --verbose

---
- name: Update Homebrew inventory.
  homebrew:
    update_homebrew: yes

- include_tasks: zsh.yaml

Running df_setup -a will add Oh My ZSH. To load the changes exec $SHELL or just restart your terminal. You'll notice your shell look and behave a little differently. If you had a look at the Oh My ZSH documentation you'll also know that there are plenty of new cli shortcuts to learn.

$ dotfiles add ~/.zshrc ~/.dotfiles
$ dotfiles commit -m "adding oh-my-zsh and zplug"
$ dotfiles push

Terminal Multiplexer

When I use a terminal I usually end up with different open terminals. I find it a bit clunky having to switch between tabs and/or windows by having to leave the "home row" and use a trackpad or mouse. This is where tmux comes into play. There are plenty other use cases for tmux however.

---
- name: Install tmuxinator
  gem:
    name: tmuxinator
    state: latest

- name: Install tmux
  homebrew:
    package:
      - tmux
      - tmuxinator-completion
      - reattach-to-user-namespace
    state: latest

- name: Check if tmuxinator plugin manager is installed.
  stat:
    path: ~/.tmux/plugins/tpm
  register: tpm_installed

- name: Create tmux plugin directory
  file:
    state: directory
    path: ~/.tmux/plugins

- name: Install tmuxinator plugin manager...
  git:
    repo: 'https://github.com/tmux-plugins/tpm'
    dest: ~/.tmux/plugins/tpm
  when: tpm_installed.stat.exists == False

- name: Copy tmux.conf in place
  copy:
    src: tmux.conf
    dest: ~/.tmux.conf

- name: Add alias for tmuxinator in ~/.zshrc
  lineinfile:
    path: ~/.zshrc
    line: alias mux=tmuxinator 

# set Zsh as your default Tmux shell
set-option -g default-shell /bin/zsh
set-option -g default-command "reattach-to-user-namespace -l zsh"
set-option -sa terminal-overrides ',xterm-256color:RGB'

# Tmux should be pretty, we need 256 color for that
set -g default-terminal "screen-256color"

# Tmux uses a 'control key', let's set it to 'Ctrl-a'
# Reason: 'Ctrl-a' is easier to reach than 'Ctrl-b'
set -g prefix C-a
unbind C-b

# make sure new windows are in the same directory as last pane
bind-key c new-window -c "#{pane_current_path}"

# command delay? We don't want that, make it short
set -sg escape-time 1

# Set the numbering of windows to go from 1 instead
# of 0 - silly programmers :|
set-option -g base-index 1
setw -g pane-base-index 1

# Allow us to reload our Tmux configuration while
# using Tmux
bind r source-file ~/.tmux.conf \; display "Reloaded!"

# Getting interesting now, we use the vertical and horizontal
# symbols to split the screen
bind | split-window -h
bind - split-window -v

# Remap window navigation to vim
unbind-key j
bind-key j select-pane -D
unbind-key k
bind-key k select-pane -U
unbind-key h
bind-key h select-pane -L
unbind-key l
bind-key l select-pane -R

set -g @plugin 'jimeh/tmux-themepack'
# set -g @themepack 'powerline/double/magenta'
set -g @plugin 'sei40kr/tmux-airline-dracula'

# Initialize TMUX plugin manager (keep this line at the very bottom of tmux.conf)
run '~/.tmux/plugins/tpm/tpm'

Chances are your PATH needs to be extended to also include ruby gems.

if which ruby >/dev/null && which gem >/dev/null; then
    PATH="$(ruby -r rubygems -e 'puts Gem.user_dir')/bin:$PATH"
fi

Now tmux and tmuxinator should be available after terminal restart or exec $SHELL.

$ dotfiles add ~/.zshrc ~/.dotfiles
$ dotfiles commit -m "adding tmux and tmuxinator with completions, adding ruby gem paths"
$ dotfiles push

Space Up Your Vim

This next part is for those of you wanting to up their vi experience a level. I started out configuring my installation manually from scratch... it resulted in the never ending tweaking saga and at some point I stumbled upon SpaceVim, a community-driven vim distribution.

We'll add this as a role in our playbook.

- hosts: all
  tasks:
    - debug: msg="You are running {{ ansible_distribution }}"
  roles:
    - common
    - spacevim

Also create the required directories:
mkdir -p ~/.dotfiles/ansible/roles/spacevim/{tasks,files}

---
- name: Install neovim
  homebrew:
    package:
      - neovim
    state: latest

- name: Tap cask-fonts
  homebrew_tap:
    name: homebrew/cask-fonts
    state: present

- name: Install nerd fonts
  homebrew_cask:
    name: font-hack-nerd-font
    state: upgraded

- name: Check if SpaceVim is installed.
  stat:
    path: ~/.SpaceVim
  register: spacevim_installed

- name: Load SpaceVim installer data
  uri: 
    url: https://spacevim.org/install.sh
    return_content: yes
  register: spaceviminstall
  when: spacevim_installed.stat.exists == False

- name: Install SpaceVim...
  command:
    argv:
      - "bash"
      - "-c"
      - "{{ spaceviminstall.content }}"
  when: spacevim_installed.stat.exists == False

- include_tasks: config.yaml

---
- name: Create ~/.SpaceVim.d/autoload
  file:
    path: ~/.SpaceVim.d/autoload
    state: directory
    mode: '0755'

- name: Copy init.toml in place
  copy:
    src: SpaceVim.d/init.toml
    dest: ~/.SpaceVim.d/init.toml

- name: Copy myspacevim.vim in place
  copy:
    src: SpaceVim.d/autoload/myspacevim.vim
    dest: ~/.SpaceVim.d/autoload/myspacevim.vim

Run df_setup -a to execute the new role. Before you start nvim change the terminal font to Hack Nerd Font. When run for the first time it'll install a few plugins.

Now is a good time to add and commit these changes...

What languages do you speak?

My daily driver needs support for more than just plain text. SpaceVim has the concept of layers, allowing you to add preset functionality by just adding specific layers to your init.toml. Layers go beyond just adding language support.

[[layers]]
  name = 'checkers#syntax-checking'

[[layers]]
  name = 'lang#toml'

[[layers]]
  name = "git"
  
[[layers]]
  name = "github"

[[layers]]
  name = 'lang#html'

[[layers]]
  name = 'lang#javascript'
  auto_fix = true
  enable_flow_syntax = true

[[layers]]
  name = 'lang#json'

[[layers]]
  name = 'lang#markdown'

---
- name: install nodejs and yarn
  homebrew:
    package:
      - nodejs
      - yarn
      - yarn-completion
    state: upgraded

- name: install nodejs neovim support
  yarn:
    name: neovim
    state: latest
    global: yes

- include_tasks: lang.yaml

After running df_setup -a and starting nvim again you should now have support for html, javascript, nodejs, json and markdown.

If you haven't been committing and pushing your changes now is the time...

Keep healthy

SpaceVim has a command to perform a health check. Typing :checkhealth will open another buffer and output a health summary. Most likely up till now you should only have 3 areas of concern: python 2 provider support, python 3 provider support and ruby support. None of these are mandatory, but if you are like me and you don't like failing health checks you'll add the support.

sudo -H python -m ensurepip

- name: install python neovim support
  pip:
    name: neovim
    state: latest
    extra_args: --user

- name: install python 2 neovim support
  pip:
    executable: pip2
    name: neovim
    state: latest
    extra_args: --user
    
- name: install ruby neovim support
  gem:
    name: neovim
    state: latest

Now execute df_setup -i -a for a updated init and ansible run.
A subsequent restart of neovim and a :checkhealth should show the result.

Time to commit our changes...

NB This is a work in progress. I'll be adding more posts as and when I discover new and better ways to achieve the ultimate desktop devops utopia...