Many projects, one dev environment

This article is an excursus on few technologies I've been using in these recent years that had improved the way we develop and ship our code.

If you're a solo developer, freelancer or just the I-do-everything wo/man this might be useful to you as well, but you'll notice that a greater benefit will be for a team.

At the beginning

At the beginning there was just me, my machine and a LAMP/WAMP/whatever stack built on top of it.
It was tedious at times as things were not homogenous across OSes, not all applications were portable as we wish they were and this made part of the experience of being a developer a pain.

I was developing for the sake of reaching an objective. I was creating bespoke applications for the specific server.

I make no mystery that I've switched to GNU/Linux for this very reason: I needed something that was working as close as possible to what would have been my live environment first and later all the other environments where my code should run before hitting live.
Then I started noticing that many people are not aware that they're not alone on this planet, and shipped code that was specific for their use case, the same I was. Developers that didn't know that their directory separator was different, or that a server might have been configured in a different way, are left in the utter ignorance of other capabilities and fallacies in their code.

Then things started falling down.

On the live environment you got a 500 error and in the best tradition, you just would reinstall everything or start pressing buttons randomly to make it disappear.

Breaking news for you: that, doesn't work.

Quality of code does not just mean having a clean, documented (where needed) code, but to be usable and readable by anyone, your future self and any other server/service it might encounter.

VirtualBox and Vagrant to the rescue

When I was working in a single project / single environment, a single development machine was perfect: replicating the integration, testing and live environment the closest way possible. My own machine worked for a while then it was just a problem, changing computer, maintaining it, switching configurations, ... and then, VirtualBox to the aid, I was still developing on my machine, but under conditions what made me aware a bit better of limitations and capabilities.

A probably better solution than VirtualBox alone, started spreading in recent years: Vagrant is a wrapper to VirtualBox and incorporates a lot of nice other things for provisioning a system.
Although Vagrant works best when you have a one-off project, and it's still good enough to get you started.

For a complete noob, you start by reading the Vagrant documentation, pick a base box from the Vagrant Cloud that might satisfy your needs, customise it with custom scripts, written in almost any language, to install the needed tools, configure Apache and MySQL, provision the database and then leave you to just code and load your website.

Cool, isn't it?

You can go further on, by packaging your own base box for your team, share it with anyone and forget about wrong encodings, odd configurations, missing services, etc.

But if you're working for an agency, I'm sure you're aware that you might have a huge code base, tenths if not thousands of projects that you need to switch to whenever needed.

Vagrant does not address this specific problem.

Some might have provided some solutions for this, as detailed in this post on StackOverflow, but it turn out to be a hack rather than a solution.

I want to be able to have ONE dev machine, and be able to reuse it for all the projects that share common similarities.

Why isn't this possible with Vagrant?

Well its architecture is strongly limited by what VirtualBox (primarily) provides in terms of interface to control it.
It seems I can't simply say: If you find machine X, provision that with the current project files.

The closest solution to this problem I found it on another post on StackOverflow that details how to obtain something pretty similar to what I wanted.

The post is just a step-by-step guide on how to use the project paolooo/vagrant-lamp on GitHub.

This might be enough for you, but for a team-wide solution, it's a bit tight, as all the configuration for each single project will end up residing on the same Vagrantfile. Which also means that if you ever need to share the project with someone else, you'll end up sharing all the rest of the projects with them, and if you have lots of projects, this solution won't scale properly.

Other problems I found are related to how you access the Vagrant machine: if you need to test something through another box (e.g. IE9 on a Windows VM) it's almost impossible, e.g. making the machine reachable from within the VBox network is not possible unless you have a host-only network interface configured, or you need to configure the hostname used by the Vagrant box unique and this seems not to be an option or just way too fiddly.
(If I have missed something here, let me know.)

Other solutions?

Well there seems to be one solution, which implies a mixed use of the above technologies.
It is basically what you will get by tearing apart Vagrant, and removing Virtualbox: you are left with the provisioning system (being Chef/Ansible/Puppet/...).

The idea would be to use Vagrant for what it's been created to do: download, install, configure a Virtual Machine for the specific purpose you need, and then let the actual provisioning scripts of choice (I'm going with Ansible :wink:) deal with the actual project itself: install additional service, configure Apache and MySQL and run needed migrations.

This way you'll have:

  • A self contained list of VMs with specific services running on it
  • A set of projects with some very simple provisioning scripts that will install and setup the project with the right VM

Tweaking the scripts and adapt them to a different configuration should be trivial enough, and creating a basic set of templates should be good enough.
The only additional bit that could a bit problematic is - as usual - handling the nameserver queries.

DNS setup

In order to have all the boxes handled correctly and accessible without problems, you need to have a static IP mapping in Vagrant/VirtualBox and an associated hostname of your choice that must be reachable from your host machine.

Here you have the choice of setting up a DNS server within your LAN to keep its A record, or within your machine.
I found DNSMasq to be the preferred choice when it comes to this decisions.

With it it's also quite easy to setup a static wildcard subdomain.
This is the only compromise: all projects will end up being served under a third-level domain of your guest VM.

To be more practical:

  • Guest VM: sandbox
  • Project PATH: /home/peach/workspace/myproject
  • Project URL: http://myproject.sandbox

You can obviously use a FQDN, although I don't see the need for this purpose.

Next steps?

With this in mind I've already started sketching down a list of Ansible Playbooks and I'll post a quick guide/intro here on this blog as soon as I've got something decent.

Keep in mind that the whole Vagrant part can be easily replaced with a custom-built VM, which of course will put a bit of fun in your hands ;-)

As a side note, I'd also like to start working with Docker at the moment, and I hope I can see what would it be using it and see how much needs to be adapted for the development environment.

Until then, I hope you enjoyed the article.