DevOps

When Automations and Continuous Integrations Fail

Everyone has tasks they have to do but would rather not. There are a handful of things we can automate, and should like paying bills, reminders for appointments or testing code before deploying. Every one of these can fail if you do not have a plan to review them regularly.

Automatic bill pay

Automatic bill pay is a great way to make sure your bills get paid on time. But it is not fully automatic, or it should not be. If you are maxed out on your credit card or do not account for money in your checking account, the payment will fail and may add charges and penalties.

You can prevent penalties by having a budget and knowing when accounts are due. You can also create a buffer, so any unexpected charges do not take you off guard. Having an emergency fund and a backup plan will help you in the long run.

Calendar reminders

Writing appointments in a calendar is a great way to know what is coming up soon. Do you have a meeting Tuesday you need to complete preparation? A birthday or anniversary coming up you need a card or gift?

Regular reviews of your upcoming week will help you plan better; help you be agile. Knowing your schedule is key to not becoming overwhelmed when the unexpected happens.

Continuous Integration

Continuous Integration (CI) is vital for any developer. Correctly set up, the CI process will alert you when the unexcited happens. Using three levels of automatic testing, you can catch 90% or more of the issues that may occur with your site.

  • phpcs – Tests the cleanliness of your code; are you following standards?
  • PHPUnit – Tests the function of your code; does it work as you expect?
  • behat – Tests the front end

But automatic testing is not enough. Computers will only do what someone told them to do. That means if we wrote the test wrong you would see false positives, some of which may not be caught for days. You need to have a human look at the site to very everything is correct.

I believe you should have a three by three testing module, a combination of man and machine.

Dev Test Prod
Code CI/H CI/H H
Function CI/H CI/H H
Front end CI/H CI/H H

Code

  • Dev – A developer writes code and checks it locally with phpcs.
  • Test – The code is checked on every commit, and the results logged, before going live a code review is done by another developer.
  • Live – Periodically you pull the code form live and perform a diff to see if anything changed that you are not aware of.

Function

  • Dev – A developer writes tests and checks it locally with PHPUnit.
  • Test – PHPUnit tests the code before going live, and the results logged.
  • Live – As you are working on the site you note any issues you find so they can be fixed. If the site is somewhat static, create a schedule to test the site every month or two.

Front end

  • Dev – A developer writes behat tests and tests locally.
  • Test – Behat tests run before going live, the Project Manager or tester reviews the side and theme for issues.
  • Live – Periodically you walk through the site to see if the content has changed the layout.

In summary; automate what you can and schedule a time to review your systems.

Using Jenkins for Drupal and WordPress – Your First Job

This is part 3 in a series. If you are not familiar with Jenkins, please read Part 1 and Part 2 first.

This walkthrough should give you an overview of the Freestyle Project.

Our first Jenkins job we will set up is to run Drupal Cron every hour, with a 1 hour throttle.

Jenkins — Drupal Cron

  1. From the main menu select New Item
  2. Enter a Name for the job and select Freestyle Project, click OK at the bottom of the form.

    You should now be on the config screen.
  3. Check Discard Old Builds, cleaning up older builds will help the admin interface perform better. You can choose between Days or Number of Jobs to keep.
  4. Check Build periodically, this sets the schedule for the build. Jenkins cron system adds a ‘H’ symbol. This is a hashed value based on the job name.
  • H * * * * Would once an hour
  • H H * * * Would once a day
  • H H(0-7) * * * means some time between 12:00 AM (midnight) to 7:59 AM.
  1. In the Build section choose Add build step and Execute shell
  2. On your Drupal site navigate to /admin/config/system/cron and copy the cron URL
  3. End the command text box enter the wget -O - -q -t 1 https://example.com/cron.php?cron_key=drupalkey7hSsc7QtW3lpANY9VxP2O
  4. Save and Build now. If all went well the job will be successful.

While curl works for cron, you will need ssh for more robust remote work. Using SSH Agent Pluginyou can add ssh keys per project, folder or globally. If you have access to ssh in to the server it’s best to add the trusted_hosts via sudo su -l jenkins (setup is beyond the scope of this article).

If you do not have shell access you can use -o StrictHostKeyChecking=no for the first run of the job. Removing the flag after you have verified connection will ensure the remote server has not changed.

ssh -o StrictHostKeyChecking=no -t [email protected] "/usr/local/bin/drush cron -y --root=/var/www/www.example.com"

Once you have ssh access setup and working you can perform other functions; here are a few examples.

Back up the database

ssh -t  [email protected]  "/usr/local/bin/drush sql-dump -y --root=/var/www/www.example.com  --gzip --result-file=~/dbbackup/backup.sql"
scp  [email protected]:dbbackup/backup.sql.gz /var/livedb/backup.sql.gz

Flush cache

ssh -t  [email protected]  "/usr/local/bin/drush cr -y --root=/var/www/www.example.com"

Import config

ssh -t  [email protected]  "/usr/local/bin/drush cim -y --root=/var/www/www.example.com"

WordPress Cron

ssh -t  [email protected]  "/usr/local/bin/wp cron event run --all --path=/var/www/www.example.com"

WordPress Flush Cache

ssh -t  [email protected]  "/usr/local/bin/wp cache flush --path=/var/www/www.example.com"

These examples are just the start of what Jenkins can do for you. If you have questions please leave them in the comments.

Using Jenkins for Drupal and WordPress – Navigating and Setup

Our second in the Using Jenkins for Drupal and WordPress series, we will cover the interface and a few key administration settings. ReadPart 1 – Installation

Interface

Jenkins interface out to the box can look utilitarian to some if you are not used to it. There are also elements of confusion as it can change or disappear based on the type of page you are on or plugins you have installed.

When you first start Jenkins you will be greeted with this welcome screen

On the left side of the screen, you will find the Main Menu, Build Queue and Build Executor Status

New Item – Brings up the job type selection page.
People – Listing of people Jenkins is aware of. This could be users with a password or names it has pulled from git commits.
Build History – Timeline and list view of the job build history.
Manage Jenkins – Admin and configuration settings for the system, adding plugins, managing users, and permissions.
Credentials – Quick link to the values you have setup. Here you can add ssh keys, APT tokens, and other settings. Values can be used in your Jobs and pipeline scripts.
Build Queue – Jobs waiting to be run.
Build Executor Status – Active Jobs

Manage Jenkins

For now, we will just cover the Configure System area, where you will make the bulk of the system-wide changes and the Manage Plugins.

Manage Plugins

There are four tabs on the plugin page Updates, Available, Installed and Advanced. The sections are self-explanatory, I myself have not found a need to change any settings in the Advanced tab. I encourage you to explore and read about the available plugins and try a few out on a testing environment.

Configure System

Here are a few settings I use and update as needed including Slack notification and core settings.

Maven Project Configuration Section:

\# of executors – The number of jobs Jenkins can run at one time. For what we are doing you will need at least 2. If you want to run parallel testing you will want to run 4 or more.
Labels – When running multiple Jenkins instances in master/slave Labels allow you do ‘tag’ where a process or job is run.
Quiet period – When this option is checked, newly triggered builds of this project will be added to the queue, but Jenkins will wait for the specified period of time before actually starting the build.
SCM checkout retry count – When working with git how often should Jenkins retry checkout.

Global properties

Checking on the Environment variables setting will expose the list of variables configured. Names are usually all uppercase i.e. LIVE_SERVER_PATH, USERNAME,

Using Environment variables are a good idea for security items like DeployBot API Keys or for items that could change like remote IP addresses.

Jenkins Location Section:

Jenkins URL – The URL for Jenkins server
System Admin e-mail address – Email address of the admin

Global Slack Notifier Settings

 
I recommend adding the Jenkins App
Base URL – If you are using the Slack Jenkins Bot the URL
Integration Token Credential ID – create a secret text entry using the token from Slack, the ID should lowercase no spaces, the Description is free-form text.

Is Bot User? – Checked

The next post we will cover building your first job.

Drupal / WordPress Continuous Integration (CI), testing each commit

Continuous Integration (CI) at a high level is testing every code push from each developer, every time. While this sounds like an argues task, it does not have to be. Drupal and WordPress both have tools available that can help you test your code.

PHP CodeSniffer

PHP CodeSniffer is a command line (cli) utility that will check your php, JavaScript, and CSS to verify it meets a set of coding standards. Xeno Media has a nice walkthrough, WordPress coding standards for the Drupal developer that will help you get setup for local testing. The steps are similar if you are testing on a server via bash scripts or using Jenkins. Using and testing for coding standards help when onboarding new developers or reviewing code you wrote months ago.

Behat

Behat is a Behavior-Driven Development testing tool. Wordhat and Drupal Behat Extension provide most of what you need for testing your sites.

With Behat you can “walk” through your site one scenario at a time. You can fill out forms as a user would. Check for the existence of text on a page, even limit it to a pre-defined region of your site like the #footer. You can also log in a user with a specific role to test the admin or user function.

PHPUnit

Of the three I am covering here, PHPUnit is the most involved. Each module or plugin you write needs a compatible test written. In this case, each class is proven to work with the test data provided. From the results, you can be ensured the new code has not broken the site. IMO the best way to work with PHPUnit is to use Test Driven Development (TDD); where you start with a failing test and write the code needed to get it to pass.

Summary

These three methods are only some of the tests you can run on your site but they cover the bulk of changes made by the development team. Testing your code as you go ensures a better product and allows you to act quickly if an issue arises; most times before it goes live.

Using Jenkins for Drupal and WordPress – Installation

This is the first in a series on Using Jenkins for Drupal and WordPress. Over the next few posts I will cover which plugins to use, server-side software needed, how to back up the remote database, testing each commit and more.

Jenkins is an open source automation server which enables developers around the world to reliably build, test, and deploy their software.1

The commands that I will be using have been tested using Ubuntu 16.04 2. Jenkins will run on Mac, Windows and most Unix/Linux based servers Java 8 (either JRE or JDK) If you do not have a server for the install you can get one for a low as $5 a month from Digital Ocean. Though Jenkins will run on the low-end server for testing for production I would highly recommend a multi-core setup, the $40 plan or larger.

As a first run, we are going to run Jenkins on the same host as our testing server. As you become more familiar with working with Jenkins I suggest running a master and multiple slave agents. Do not try to run Jenkins on your Drupal or WordPress production servers.

Pre-Installation setup

Remote into your server with a user that has sudo privileges, it is not advised to use the root user.

To us Jenkins for php testing we need to install a few tools first.

sudo apt-get update
sudo apt-get install -y --no-install-recommends apt-utils

Requirements

Install PHP and git

sudo apt-get install -y php php-cli php-xsl php-json php-curl php-intl php-mcrypt php-pear curl php-mbstring git ant php-mysqlnd zip
sudo apt-get clean -y

Install a webserver3

sudo apt-get apache
or 
sudo apt-get nginx

Install Composer4

php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
php -r "if (hash_file('SHA384', 'composer-setup.php') === '544e09ee996cdf60ece3804abc52599c22b1f40f4323403c44d44fdfdd586475ca9813a858088ffbc1f233e9b180f061') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"
php composer-setup.php
php -r "unlink('composer-setup.php');"
sudo mv composer.phar /usr/local/bin/composer

Install PHPUnit5

wget https://phar.phpunit.de/phpunit.phar
chmod +x phpunit.phar
sudo mv phpunit.phar /usr/local/bin/phpunit
phpunit --version

Install PHPLoc6

wget https://phar.phpunit.de/phploc.phar
chmod +x phploc.phar
sudo mv phploc.phar /usr/local/bin/phploc
phploc  --version

Install PHP CodeSniffer7

sudo pear install PHP_CodeSniffer
phpcs --version

Install PHP Depend8

wget http://static.pdepend.org/php/latest/pdepend.phar
chmod +x pdepend.phar
sudo mv pdepend.phar /usr/local/bin/pdepend
pdepend  --version

Install PHP Mess Detector9

wget -c http://static.phpmd.org/php/latest/phpmd.phar
chmod +x phpmd.phar
sudo mv phpmd.phar /usr/local/bin/phpmd
phpmd  --version

Install PHP Copy/Paste Detector (PHPCPD)10

wget https://phar.phpunit.de/phpcpd.phar
chmod +x phpcpd.phar
sudo mv phpcpd.phar /usr/local/bin/phpcpd
phppcpd  --version

Install PHP Documentation Generator11

wget http://phpdox.de/releases/phpdox.phar
chmod +x phpdox.phar
sudo mv phpdox.phar /usr/local/bin/phpdox
phpdox --version

Installation

Now that we have the requirements done we can install Jenkins and the needed plugins:

wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt-get update
sudo apt-get install jenkins

During the install a jenkins user will be created, if you are running Jenkins on a staging server I suggest adding jenkins to the www-data group

sudo usermod -g www-data jenkins

Add Drupal and WordPress Coding standards to Jenkins user

sudo su - jenkins
# Install Composer tools
# composer parallel install plugin
composer global require hirak/prestissimo
# Drupal Coder, PHP_CodeSniffer, and Drupal Coding Standards
composer global require drupal/coder
# Adds WordPress Coding Standards
composer global require wp-coding-standards/wpcs:dev-master
exit
# Sets Config for PHP_CodeSniffer
sudo phpcs --config-set installed_paths /var/lib/jenkins/.composer/vendor/drupal/coder/coder_sniffer,/var/lib/jenkins/.composer/vendor/wp-coding-standards/wpcs

Open your browser and navigate to http://localhost:8080/ substitute localhost with the server IP address if you are installing remotely. You should see a screen like the one below.

First screen you see after installing Jenkins
First screen you see after installing Jenkins
sudo more /var/lib/jenkins/secrets/initialAdminPassword

Choose the “Install Suggest Plugins” options, or you can customize your own. Then create your admin user. If successful you will see the “Welcome to Jenkins” message

Initial welcome screen for Jenkins
Initial welcome screen for Jenkins

Next, we will need to get the API Token for your admin user. Click on People, the user you created, then Configure, finally Show API Token....

Jenkins user form
Jenkins user form

Copy the token to a safe place for use in the command line.

Open the terminal again to add the required plugins. Replace the user and API token with your own.

wget http://localhost:8080/jnlpJars/jenkins-cli.jar
java -jar jenkins-cli.jar -auth [user]:[apitoken] -s http://localhost:8080/ install-plugin checkstyle \
cloverphp crap4j dry htmlpublisher jdepend plot pmd violations warnings xunit publish-over-ssh \
ansicolor bitbucket slack lockable-resources pipeline-milestone-step

java -jar jenkins-cli.jar -auth [user]:[apitoken] -s http://localhost:8080 safe-restart

That is it! You have installed Jenkins and everything required for testing PHP projects.

  1. https://jenkins.io/press/ ↩
  2. The Jenkins website has instructions for installing other systems. ↩
  3. Apache or nginx setup is beyond the scope of this article ↩
  4. https://getcomposer.org/doc/00-intro.md#globally ↩
  5. https://phpunit.de/getting-started.html ↩
  6. https://github.com/sebastianbergmann/phploc ↩
  7. https://github.com/squizlabs/PHP_CodeSniffer ↩
  8. https://pdepend.org/ ↩
  9. https://phpmd.org/download/index.html ↩
  10. https://github.com/sebastianbergmann/phpcpd ↩
  11. http://phpdox.de/ ↩

Setting up Docker4Drupal/Docker4WordPress with multiple projects on Mac – Redo

Edited 2018-01-24 to switch to .test for local

DNS name docker.for.mac.host.internal shoud be used instead of docker.for.mac.localhost (still valid) for host resolution from containers, since since there is an RFC banning the use of subdomains of localhost. See https://tools.ietf.org/html/draft-west-let-localhost-be-localhost-06.

#####

Since writing Setting up Docker4Drupal with multiple projects on Mac I have discovered a better way. Træfik allows for automatic detection, so you can leave it running in it’s own container. With a few changes to Docker4Drupal and Docker4Wordpress YAML files you can start and stop your development sites without having to reconfigure Træfik.

Træfik container

Create a working folder for Træfik:

mkdir ~/Sites/traefik
cd ~/Sites/traefik

Create wildcard cert files for *.test (If you have not setup Dnsmasq locally I recommend it)

mkdir certs
cd certs
openssl genrsa 2048 > traefik.key
openssl req -new -x509 -nodes -sha1 -days 3650 -key traefik.key > host.cert
#[enter *.test for the Common Name]
openssl x509 -noout -fingerprint -text < traefik.cert > traefik.info
cat traefik.cert traefik.key > traefik.pem
Trust cert

sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain traefik.cert
cd ~/Sites/traefik

Make the Docker-compose.yml file with https support

version: '2'
services:
    traefik:
        image: traefik:1.3.6
        restart: unless-stopped # you can use always if you choose, I perfer unless-stopped locally
        command: '-c /dev/null --web --docker --logLevel=DEBUG --defaultEntryPoints=''http'',''https'' --entryPoints="Name:https Address::443 TLS:/certs/traefik.cert,/certs/traefik.key" --entryPoints="Name:http Address::80"'
        networks:
            - webgateway
        ports:
            - '80:80'
            - '443:443'
            - '8080:8080'
        volumes:
            - '/var/run/docker.sock:/var/run/docker.sock'
            - './certs:/certs'
    portainer:
        image: portainer/portainer
        restart: unless-stopped
        networks:
            - webgateway
        command: '--no-auth'
        volumes:
            - '/var/run/docker.sock:/var/run/docker.sock'
        labels:
            - traefik.backend=portainer
            - traefik.port=9000
            - 'traefik.frontend.rule=Host:portainer.test'
networks:
    webgateway:
        driver: bridge

Start your container. Once it is running it will keep going even after reboot.

docker-compose up -d

Docker4Drupal/Docker4Wordpress

In each project add the following code for each container:

    networks:
      - web

And one global setting:

# Add networks section
networks:
  web:
    external:
      name: traefik_webgateway

Example: (Assumes the folder is called drupal i.e. /Sites/drupal

version: "2"

services:
  mariadb:
    image: wodby/mariadb:10.1-2.3.5
    networks:
      - web
    environment:
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: drupal
      MYSQL_USER: drupal
      MYSQL_PASSWORD: drupal
#    volumes:
#      - ./mariadb-init:/docker-entrypoint-initdb.d # Place init .sql file(s) here.
#      - /path/to/mariadb/data/on/host:/var/lib/mysql # I want to manage volumes manually.

  php:
# 1. Images with vanilla Drupal – wodby/drupal:[DRUPAL_VERSION]-[PHP_VERSION]-[STABILITY_TAG].
    image: wodby/drupal:8-7.1-3.0.0
    networks:
      - web
    environment:
      PHP_SENDMAIL_PATH: /usr/sbin/sendmail -t -i -S mailhog:1025
      PHP_FPM_CLEAR_ENV: "no"
      DB_HOST: mariadb
      DB_USER: drupal
      DB_PASSWORD: drupal
      DB_NAME: drupal
      DB_DRIVER: mysql
#      PHP_XDEBUG: 1
#      PHP_XDEBUG_DEFAULT_ENABLE: 1
#      PHP_XDEBUG_REMOTE_CONNECT_BACK: 0
#      PHP_XDEBUG_REMOTE_HOST: "10.254.254.254"
#      PHP_XDEBUG_PROFILER_OUTPUT_DIR: /mnt/files/xdebug/profiler
#      PHP_XDEBUG_TRACE_OUTPUT_DIR: /mnt/files/xdebug/traces
    volumes:
      - codebase:/var/www/html
## Options for macOS users (https://docker4drupal.readthedocs.io/en/latest/macos)
#      - codebase:/var/www/html:cached # User-guided caching
#      - docker-sync:/var/www/html # Docker-sync
## For Xdebug profiler files
#      - files:/mnt/files

  nginx:
# wodby/drupal-nginx:[DRUPAL_VERSION]-[NGINX_VERSION]-[STABILITY_TAG].
    image: wodby/drupal-nginx:8-1.13-3.0.1
    networks:
      - web
    depends_on:
      - php
    environment:
      NGINX_STATIC_CONTENT_OPEN_FILE_CACHE: "off"
      NGINX_ERROR_LOG_LEVEL: debug
      NGINX_BACKEND_HOST: php
      NGINX_SERVER_ROOT: /var/www/html/web
    volumes:
      - codebase:/var/www/html
# Options for macOS users (https://docker4drupal.readthedocs.io/en/latest/macos)
#      - codebase:/var/www/html:cached # User-guided caching
#      - docker-sync:/var/www/html # Docker-sync
    labels:
      - 'traefik.backend=drupal_nginx_1'
      - 'traefik.port=80'
      - 'traefik.frontend.rule=Host:drupal.test'

  mailhog:
    image: mailhog/mailhog
    networks:
      - web
    labels:
      - 'traefik.backend=drupal_mailhog_1'
      - 'traefik.port=8025'
      - 'traefik.frontend.rule=Host:mailhog.drupal.test'

#   portainer: is not needed in each project

# traefik: is not needed in each project

# Add networks section
networks:
  web:
    external:
      name: traefik_webgateway

volumes:
  codebase:
   docker-sync:
     external: true

Using DNSmasq locally will allow Safari and Firefox to work for the test sites.

EDIT: Download docker-compose.yml on GitHub

Using Dnsmasq for local development on macOS

Edited 2018-01-24 to switch to .test for local

DNS name docker.for.mac.host.internal shoud be used instead of docker.for.mac.localhost (still valid) for host resolution from containers, since since there is an RFC banning the use of subdomains of localhost. See https://tools.ietf.org/html/draft-west-let-localhost-be-localhost-06.

#####

There is a known issue with Docker for Mac, “.localhost DNS doesn’t resolve in browsers other than Chrome“. Docker4Drupal talks about it here and Træfik has a thread here. I found using Dnsmasq a better solution for me than editing the hosts file for each project I work with.

Adapted and edited from Using Dnsmasq for local development on OS X – Thomas Sutton

Most web developers will be familiar with the process of updating your /etc/hosts file to direct traffic for coolproject.test to 127.0.0.1. Most will also be familiar with the problems of this approach:

  • it requires a configuration change every time you add or remove a project; and
  • it requires administration access to make the change.

Installing a local DNS server like Dnsmasq and configuring your system to use that server can make these configuration changes a thing of the past. In this post, I’ll run through the process of:

  1. Installing Dnsmasq on OS X.
  2. Configuring Dnsmasq to respond to all .test  requests with 127.0.0.1.
  3. Configure OS X to send all .test  requests requests to Dnsmasq.

Before we get started, I should give you a warning: these instructions show you how to install new system software and change your system configuration. Like all such changes, you should not proceed unless you are confident you have understood them and that you can reverse the changes if needed.

Installing Dnsmasq

To quote the Dnsmasq project home page

Dnsmasq is a lightweight, easy to configure DNS forwarder and DHCP server … is targeted at home networks.

There are plenty of ways to install Dnsmasq but my favourite (on OS X) is to use the Homebrew package manager. Installing Homebrew is fairly simple but beyond my scope here.

Once you have Homebrew installed, using it to install Dnsmasq is easy:

# Update your homebrew installation
brew up
# Install dnsmasq
brew install dnsmasq

The installation process will output several commands that you can use to start Dnsmasq automatically with a default configuration. I used the following commands but you should use whichever commandsbrew tells you to:

# Copy the default configuration file.
cp $(brew list dnsmasq | grep /dnsmasq.conf.example$) /usr/local/etc/dnsmasq.conf
# Copy the daemon configuration file into place.
sudo cp $(brew list dnsmasq | grep /homebrew.mxcl.dnsmasq.plist$) /Library/LaunchDaemons/
# Start Dnsmasq automatically.
sudo launchctl load /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist

Configuring Dnsmasq

Now that you have Dnsmasq installed and running, it’s time to configure it! The configuration file lives at /usr/local/etc/dnsmasq.conf by default, so open this file in your favourite editor.

One the many, many things that Dnsmasq can do is compare DNS requests against a database of patterns and use these to determine the correct response. I use this functionality to match any request which ends in .test  and send 127.0.0.1 in response. The Dnsmasq configuration directive to do this is very simple:

address=/test/127.0.0.1

Insert this into your/usr/local/etc/dnsmasq.conffile (I put it near the exampleaddress=/double-click.net/127.0.0.1 entry just to keep them all together) and save the file.

You may need to restart Dnsmasq to get it to recognise this change. Restarting Dnsmasq is the same as any other service running under launchd:

sudo launchctl stop homebrew.mxcl.dnsmasq
sudo launchctl start homebrew.mxcl.dnsmasq

You can test Dnsmasq by sending it a DNS query using thedigutility. Pick a name ending intest and use dig to query your new DNS server:

dig testing.testing.one.two.three.test @127.0.0.1

You should get back a response something like:

;; ANSWER SECTION:
testing.testing.one.two.three.test. 0 IN A       127.0.0.1

Configuring OS X

Send only .test  queries to Dnsmasq.

Most UNIX-like operating systems have a configuration file called /etc/resolv.conf which controls the way DNS queries are performed, including the default server to use for DNS queries (this is the setting that gets set automatically when you connect to a network or change your DNS server/s in System Preferences).

OS X also allows you to configure additional resolvers by creating configuration files in the /etc/resolver/ directory. This directory probably won’t exist on your system, so your first step should be to create it:

sudo mkdir -p /etc/resolver

Now you should create a new file in this directory for each resolver you want to configure. Each resolver corresponds – roughly and for our purposes – to a top-level domain like ourdev. There a number of details you can configure for each resolver but I generally only bother with two:

  • the name of the resolver (which corresponds to the domain name to be resolved); and
  • the DNS server to be used.

For more information about these files, see the resolver(5) manual page:

Create a new file with the same name as your new top-level domain (I’m using test, recall) in the /etc/resolver/ directory and add a nameserver to it by running the following commands:

sudo tee /etc/resolver/test \>/dev/null \<\<EOF
nameserver 127.0.0.1
EOF

Heretestis the top-level domain name that I've configured Dnsmasq to respond to and127.0.0.1 is the IP address of the server to use.

Once you’ve created this file, OS X will automatically read it and you’re done!

Testing

Testing you new configuration is easy; just use ping check that you can now resolve some DNS names in your new top-level domain:

# Make sure you haven't broken your DNS.
ping -c 1 www.google.com
# Check that .dev names work
ping -c 1 this.is.a.test.test
ping -c 1 iam.the.walrus.test

You should see results that mention the IP address in your Dnsmasq configuration like this:

PING iam.the.walrus.test (127.0.0.1): 56 data bytes

You can now just make up new DNS names under.test whenever you please. Congratulations!

Setting up docker4drupal with multiple projects on Mac

I have recently started testing and using docker4drupal. (I also use DrupalVM and vagrant; but that’s another post.) I setup to be quick and easy.

Setup

You will need:

Docker Compose Spacely Sprockets (new site)

~/Sites/spacely-sprokets/docker-compose.yml

version: "2"

services:
  mariadb:
    image: wodby/mariadb:10.1-2.3.3
    environment:
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: drupal
      MYSQL_USER: drupal
      MYSQL_PASSWORD: drupal
 
  php:
# 1. Images with vanilla Drupal – wodby/drupal:[DRUPAL_VERSION]-[PHP_VERSION]-[STABILITY_TAG].
    image: wodby/drupal:8-7.1-2.4.4
    environment:
      PHP_SENDMAIL_PATH: /usr/sbin/sendmail -t -i -S mailhog:1025
      DB_HOST: mariadb
      DB_USER: drupal
      DB_PASSWORD: drupal
      DB_NAME: drupal
      DB_DRIVER: mysql
      # PHP_XDEBUG: 1
      # PHP_XDEBUG_DEFAULT_ENABLE: 1
      # PHP_XDEBUG_REMOTE_CONNECT_BACK: 0         # This is needed to respect remote.host setting below
      # PHP_XDEBUG_REMOTE_HOST: "10.254.254.254"  # You will also need to 'sudo ifconfig lo0 alias 10.254.254.254'
    volumes:
       - spacely-sprokets-sync:/var/www/html # Docker-sync

  nginx:
# wodby/drupal-nginx:[DRUPAL_VERSION]-[NGINX_VERSION]-[STABILITY_TAG].
    image: wodby/drupal-nginx:8-1.13-2.4.2
    depends_on:
      - php
    environment:
      NGINX_STATIC_CONTENT_OPEN_FILE_CACHE: "off"
      NGINX_ERROR_LOG_LEVEL: debug
      NGINX_BACKEND_HOST: php
      NGINX_SERVER_ROOT: /var/www/html/web
    volumes:
       - spacely-sprokets-sync:/var/www/html # Docker-sync
    labels:
      - 'traefik.backend=nginx'
      - 'traefik.port=80'
      - 'traefik.frontend.rule=Host:spacely-sprokets.docker.localhost'

   mailhog:
    image: mailhog/mailhog
    labels:
      - 'traefik.backend=mailhog'
      - 'traefik.port=8025'
      - 'traefik.frontend.rule=Host:mailhog.spacely-sprokets.docker.localhost'
 
volumes:
# Docker-sync for macOS users
   spacely-sprokets-sync:
     external: true

Docker Sync Spacely Sprockets

~/Sites/spacely-sprokets/docker-sync.yml

version: “2”

syncs:
  spacely-sprokets-sync:
src: ‘./‘
sync_userid: ‘82’
sync_excludes: [‘.gitignore’, ‘.git/‘, ‘.idea/‘]

Docker Cogswell Cogs (existing code)

~/Sites/cogswell-cogs/docker-compose.yml

version: "2"

services:
  mariadb:
    image: wodby/mariadb:10.1-2.3.3
    environment:
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: drupal
      MYSQL_USER: drupal
      MYSQL_PASSWORD: drupal
      volumes:
       - ./mariadb-init:/docker-entrypoint-initdb.d # Place init .sql file(s) here.

  php:
# 2. Images without Drupal – wodby/drupal-php:[PHP_VERSION]-[STABILITY_TAG].
     image: wodby/drupal-php:5.6-2.4.3
    environment:
      PHP_SENDMAIL_PATH: /usr/sbin/sendmail -t -i -S mailhog:1025
      DB_HOST: mariadb
      DB_USER: drupal
      DB_PASSWORD: drupal
      DB_NAME: drupal
      DB_DRIVER: mysql
      # PHP_XDEBUG: 1
      # PHP_XDEBUG_DEFAULT_ENABLE: 1
      # PHP_XDEBUG_REMOTE_CONNECT_BACK: 0         # This is needed to respect remote.host setting below
      # PHP_XDEBUG_REMOTE_HOST: "10.254.254.254"  # You will also need to 'sudo ifconfig lo0 alias 10.254.254.254'
    volumes:
       - cogswell-cogs-sync:/var/www/html # Docker-sync

   apache:
     image: wodby/drupal-apache:2.4-1.0.2
    depends_on:
      - php
     environment:
       APACHE_LOG_LEVEL: debug
       APACHE_BACKEND_HOST: php
       APACHE_SERVER_ROOT: /var/www/html/www
     volumes:
    - cogswell-cogs-sync-sync:/var/www/html # Docker-sync
     labels:
       - 'traefik.backend=apache'
       - 'traefik.port=80'
       - 'traefik.frontend.rule=Host:drupal.docker.localhost'

  mailhog:
    image: mailhog/mailhog
    labels:
      - 'traefik.backend=mailhog'
      - 'traefik.port=8025'
      - 'traefik.frontend.rule=Host:mailhog.drupal.docker.localhost'

volumes:
 # Docker-sync for macOS users
   cogswell-cogs-sync:
     external: true

Docker Sync Cogswell Cogs

~/Sites/cogswell-cogs/docker-sync.yml

version: “2”

syncs:
  cogswell-cogs-sync:
src: ‘./‘
sync_userid: ‘82’
sync_excludes: [‘.gitignore’, ‘.git/‘, ‘.idea/‘]

Traefik.yml

~/Sites/traefik.yml


version: ‘2’

services:
  traefik:
    image: traefik
    restart: unless-stopped
    command: -c /dev/null –web –docker –logLevel=DEBUG
    networks:
        - spacelysprokets
        - cogswellcogs
        - sites
     ports:
        - ‘80:80’
        - ‘8080:8080’
   volumes:
       - /var/run/docker.sock:/var/run/docker.sock
  portainer:
        image: portainer/portainer
        command: '--no-auth'
        volumes:
            - '/var/run/docker.sock:/var/run/docker.sock'
        labels:
            - traefik.backend=portainer
            - traefik.port=9000
            - 'traefik.frontend.rule=Host:portainer.docker.localhost'
networks:
  spacelysprokets:
    external: 
      name: spacelysprokets_default
  cogswellcogs:
    external:
      name: cogswellcogs_default
  sites:
    external:
      name: sites_default

Commands

Start Cogswell Cogs

cd ~/Sites/cogswell-cogs
docker-sync-stack start

Start Spacely Sprokets

cd ~/Sites/spacely-sprokets
docker-sync-stack start

Start traefik and portainer.

cd ~/Sites
docker-compose -f traefik.yml up -d

Shell access to the site

cd ~/Sites/cogswell-cogs
docker-compose exec --user=82 php sh

Stop docker

cd ~/Sites/cogswell-cogs
docker-compose -f ../traefik.yml stop
docker-compose stop
docker-sync stop