Hey folks,

We’re in for a bit of an epic today! I’m going to take you through the whole process of building a reproducible, lightweight development environment for Laravel Lumen, using Vagrant and Puppet. If you’re not here to learn, and would prefer just to get started, you can find all the code in its repository on Github.

If you’re not familiar with Vagrant and Puppet, they’re both tools used extensively in the DevOps field. Vagrant allows you, as their website states to “Create and configure lightweight, reproducible, and portable development environments”.

Most developers have experienced the frustration of their development machine getting slower and slower as they add more services to it. Vagrant allows us to contain entire environments in a single, easily manageable space, at the expense of slightly higher overheads while running each one.

Puppet, according to the official website, “helps you make rapid, repeatable changes and automatically enforce the consistency of systems and devices across physical and virtual machines, on premise or in the cloud”. Essentially, it allows you to manage a diverse array of operating systems, features and applications via a (mostly) unified API. It will be the tool which installs all the software our development environment requires.

Requirements

You'll need to:

• Install Virtualbox
• Install Vagrant

Setting up your Vagrantfile

Vagrant environments are managed using something called a Vagrantfile. At its core, it’s just a Ruby script that leverages Vagrant’s powerful VM management libraries.

To initialise a directory as a Vagrant environment, navigate to it in your command line interface, and type vagrant init. This will place a stock Vagrantfile in the current working directory. You can pass options to Vagrant to preconfigure it, but for now we’re just going to create the default one. If you get a command not found error on Windows, make sure the vagrant.exe executable is in your %PATH% environment variable.

There’s loads of really useful documentation in this file, so if you’re new to Vagrant, take some time to read it through. The only actual code in the file currently though, is:

Vagrant.configure(2) do |config|
  config.vm.box = "base"
end

All this does is tell Vagrant to start a virtual machine, using the box “base”. Vagrant boxes are pre-installed operating system images, which allow you to spin up a box as fast as you can when purchasing cloud servers. The “base” box it states there isn’t valid, so if you try and start it, you’ll get an error. We need to choose a useful replacement box. The one we’re going to use is “puppetlabs/debian-7.8-64-puppet”. It uses Debian Wheezy, a tried and tested distribution, and has Puppet pre-installed, making our lives a bit easier.

To get this show on the road, replace “base” with "puppetlabs/debian-7.8-64-puppet" in your Vagrantfile, and type vagrant up. It’ll take a little while to download the box, but this only needs to happen the first time you boot any image. When it’s all done and you get back control of your terminal, either type vagrant ssh and play around in the VM, or move on to the next section. We don’t actually need it powered up, so feel free to shut it down with vagrant halt.

Setting up your Puppet environment

The next step is to get our Puppet provisioning system set up locally. We don’t even need Puppet installed on our development box, because it all runs inside the VM.

Start out by creating some structure for our environment. You want to create a folder called “puppet”, with subfolders called “hieradata”, “manifests” and “modules”. This is where we’ll develop our node definition, which will install all the software we need on the target VM.

Also create a folder called www, at the same level as puppet rather than inside it. We’ll get to why later on.

Now create some files, and insert the content as shown:

puppet/hiera.yaml

---
:hierarchy:
    - common
:backends:
    - yaml
:yaml:
    :datadir: /vagrant/puppet/hieradata

This file tells Puppet to look for configuration data in /vagrant/puppet/hieradata. When you spin up a Vagrant box, it tries to automatically mount your Vagrant environment’s working directory under /vagrant. This feature allows us to easily share files between the machines. The file also describes to Puppet that it should look for a file called common.yaml within there.

puppet/hieradata/common.yaml

Leave this file empty for now

First, power up your Vagrant box by typing vagrant up. Once it’s booted, log in with vagrant ssh. You will find yourself logged into your Vagrant box as the user “vagrant”. We need to install some modules which will help us manage the resources our development system requires. We can use the following command:

for MODULE in puppetlabs-apache puppetlabs-mysql mayflower-php; do
    puppet module install --modulepath=/vagrant/puppet/modules $MODULE
done

If you have a look at your Vagrant environment’s puppet folder on your host machine, you will see some more folders than the number of modules we installed. This is because all these modules have dependencies of their own. DRY is a core concept that drives the DevOps movement.

Now that we’ve got our modules installed, we can configure Vagrant to initialise Puppet as a local provisioner when it boots this machine. Go into your Vagrantfile and add the following lines under config.vm.box, but before the end of the Vagrant.configure block:

config.vm.provision "puppet" do |puppet|
  puppet.hiera_config_path = 'puppet/hiera.yaml'
  puppet.module_path = 'puppet/modules'
  puppet.manifests_path = 'puppet/manifests'
  puppet.manifest_file = 'site.pp'
  # puppet.options = '--verbose --debug'
end

There’s one file referenced there that we haven’t mentioned yet: site.pp. This is the generally accepted standard name for your node definition file. Go ahead and create this file in puppet/manifests, and insert the following code:

node default {

}

This tells Puppet that all machines that use this configuration should load the rules contained within the curly braces. You can also refer to machines by their FQDN - which is very useful when configuring multi-machine Vagrant environments - but that’s out of the scope of this tutorial.

To load these new rules into our Vagrant environment, we reload our box with vagrant reload. It will reboot, displaying a few messages describing the processes it has used to configure the Puppet provisioner. If you want to see more information about what it’s doing, you can uncomment the puppet.options line above and marvel at pages and pages of logging. Not much use right now, but if you need to debug the machine you’ll be thankful it exists!

Building the Provisioner

After all that faffing about (once you’re more familiar with this, you’ll be firing these things up in seconds), you’re finally ready to start telling the operating system about the state you want it to be in.

First of all, we want to make sure that every time vagrant runs the provisioner, we have an up to date package cache. We do that by inserting the following into your Vagrantfile:

config.vm.provision "shell", inline: "/usr/bin/apt-get update"

Make sure this is always the first provisioner you list in the file. They are executed in order.

Now enter the following into your site.pp’s default node definition:

  include ::php

  $default_path = '/usr/local/bin:/usr/bin:/usr/sbin:/bin:/sbin'

  Exec {
    path => $default_path
  }

  # For libapache2-mod-fastcgi
  apt::source { 'debian_wheezy_non-free':
    comment  => 'Core non-free mirror',
    location => 'http://http.debian.net/debian/',
    release  => 'wheezy',
    repos    => 'non-free',
    include  => {
      'src' => false,
      'deb' => true,
    },
    notify  => Exec['apt-update'],
    before  => Apache::Fastcgi::Server['php']
  }

  exec { 'apt-update':
    command     => '/usr/bin/apt-get update',
    refreshonly => true
  }

  class { '::apache':
    default_mods        => false,
    default_confd_files => false,
    default_vhost       => false,
    user                => $::web_user,
    group               => $::web_group,
    logroot             => $::web_root
  }

  include apache::mod::rewrite
  include apache::mod::actions
  include apache::mod::mime
  include apache::mod::alias
  include apache::mod::dir

  $php_mime_type = 'application/x-httpd-php'

  apache::fastcgi::server { 'php':
    host       => '127.0.0.1:9000',
    timeout    => 15,
    flush      => false,
    faux_path  => "${::web_root}/php.fcgi",
    fcgi_alias => '/php.fcgi',
    file_type  => $php_mime_type
  }

  file { 'log_store':
    path   => "${::web_root}/logs",
    ensure => directory,
    owner  => $::web_user,
    group  => $::web_group
  }

  $docroot = "${::web_root}/${::lumen_app_name}/public"

  apache::vhost { "${::lumen_app_name}":
    port            => 80,
    directoryindex  => 'index.php /index.php',
    override        => 'All',
    docroot         => $docroot,
    docroot_owner   => $::web_user,
    docroot_group   => $::web_group,
    custom_fragment => "AddType ${php_mime_type} .php",
    error_log_file  => "logs/${::lumen_app_name}_error.log",
    access_log_file => "logs/${::lumen_app_name}_access.log",
    require         => [
      Exec['install_lumen_app'],
      File['log_store']
    ]
  }

  file { '/var/lib/apache2/fastcgi':
    ensure => directory,
    owner  => $::web_user,
    group  => $::web_group,
    before => Apache::Fastcgi::Server['php']
  }

  $composer_env    = ["HOME=/home/${::web_user}"]
  $composer_bindir = '.composer/vendor/bin'

  # Allow users to run composer-installed CLI apps
  file { '/etc/profile.d/composer-path.sh':
    mode    => '0644',
    content => 'PATH=$PATH:$HOME/.composer/vendor/bin'
  }

  exec { 'install_lumen_installer':
    command     => "composer global require \"laravel/lumen-installer=${::lumen_version}\"",
    user        => $::web_user,
    environment => $composer_env,
    require     => File['/usr/local/bin/composer'],
    onlyif      => "test ! -x /home/${::web_user}/${composer_bindir}/lumen"
  } ->
  exec { 'install_lumen_app':
    command     => "lumen new ${::lumen_app_name}",
    cwd         => $::web_root,
    user        => $::web_user,
    environment => $composer_env,
    path        => "/home/${::web_user}/${composer_bindir}:${default_path}",
    onlyif      => "test ! -f ${docroot}/index.php"
  }

  if $::lumen_app_db_enabled {
    class { '::mysql::server':
      root_password           => 'dev',
      remove_default_accounts => true
    }

    mysql::db { "${::lumen_app_name}":
      user     => "${::lumen_app_name}_user",
      password => 'password'
    }
  }

This describes to Puppet exactly what resources to implement to ensure that the target machine assumes a given state at the end. The state we wish it to be in is as follows:

1. We need at least php 5.5.9 as that is Lumen’s minimum requirement
2. We need a web server which:
   1. Is configured to host a PHP application
   2. Supports our Vagrant environment
   3. Outputs its logs to a sensible place to allow us to debug more easily
3. Lumen is preinstalled, and we have access to the installer binary
4. We want a database, but don’t bother installing it if the user doesn’t need one

Now I could have written this file with all the values hard coded, and at this point we could actually boot the system and start developing, but to make this as reusable as possible, I decided to add some extra functionality, which will allow us to reconfigure the machine in a more easily manageable fashion than editing the logic every time we want to change some settings, or start a new project.

Configuring PHP with Hiera

Hiera is how Puppet looks up data from our YAML files. The mayflower-php module states in its documentation that you should configure the application via Hiera, so we insert the following into our common.yaml file:

php::ensure: latest
php::manage_repos: true
php::fpm: true
php::fpm::config::log_level: notice
php::fpm::config::error_log: "%{::web_root}/logs/php-fpm.log"
php::settings::fpm:
    PHP/memory_limit: 256M
    PHP/error_reporting: E_ALL
    PHP/display_errors: On
    PHP/display_startup_errors: On
php::settings::cli:
    PHP/memory_limit: 256M
php::composer: true
php::composer::auto_update: true

php::extensions:
    json: {}
    mysql: {}
    mcrypt: {}
    openssl: {}

php::fpm::pools:
    www:
        listen: 127.0.0.1:9000
        catch_workers_output: true
        user: "%{::web_user}"
        group: "%{::web_group}"

This provides the PHP module with all the information it needs to set up PHP so it will happily serve our Lumen application.

Configuring Facter from Vagrant

Facter provides something referred to as “facts”. Facts are basically environment-specific data that can be looked up by the client machine on the fly. Facts are often more than simply environment variables, and can represent any kind of local data parsing. To see all facts that are available on a system, type facter. This will show you tons of data that is available for you to leverage in your Puppet environments.

Insert the following into your Vagrantfile, right at the top of inside the config.vm.provision block:

puppet.facter = {
  'lumen_version'        => '~1.0',
  'lumen_app_name'       => app_config[:application][:name],
  'lumen_app_db_enabled' => app_config[:application][:db_enabled],
  'web_user'             => core_config[:web_user],
  'web_group'            => core_config[:web_group],
  'web_root'             => core_config[:web_root]
}

This tells the application to insert facts (referenced as variables prefixed with :: in our site.pp) into Puppet that resolve as stated. You may be wondering where the app_config and core_config variables came from - and you’re right to do so - we haven’t implemented them yet!

To achieve this, we’re going to tell Vagrant to parse a couple of YAML files into native Ruby objects. To do so, insert the following 4 lines to the very top of your Vagrantfile:

require 'yaml'
cur_dir     = File.dirname(File.expand_path(__FILE__))
app_config  = YAML.load_file("#{cur_dir}/app.yaml")
core_config = YAML.load_file("#{cur_dir}/config.yaml")

Now, they both need some content. Looking at the values the Vagrantfile injects, we can craft the following two files:

app.yaml

---
:local_port: 8080
:application:
    :name: tutorial
    :db_enabled: true

config.yaml

---
:web_root: /var/www
:web_user: vagrant
:web_group: vagrant

There are just two small things left to add to our Vagrantfile, and we’re ready to run! You may already have noticed we’ve got some unused data in the files I just told you to populate. Right below config.vm.box, insert the following line:

config.vm.synced_folder "www", core_config[:web_root]

This tells Vagrant that we want to mount a local, relative folder called “www” on core_config[:web_root], in our case “/var/www”. This lets us contain a particular set of data in one place. This will be where our application and our logs reside.

Directly below that, insert the following:

config.vm.network "forwarded_port", guest: 80, host: app_config[:local_port]

This causes Vagrant to use NAT to provide whatever service is running on the guest box on port 80 (apache) via app_config[:local_port], which in my case is 8080, on localhost.

We’re done! It’s been a trek, but now, if you issue the following command, you should be able to visit http://localhost:8080 and see Lumen waiting, ready for you to develop something awesome:

vagrant reload --provision

You can now access your application in www/tutorial, and all your Apache and PHP logs in www/logs.

Happy hacking!