WTF is “continuous integration”??

So, back in the old days, building websites was simple. You had plenty of options for how to build one but me personally, I was totally fine with Wordpress and Drupal. They didn't force me to know what was going on under the hood to be productive and they provided a way for me to learn that stuff along the way while feeding my family. I learned CSS on the job while building the first couple things I ever built.

Cascading Style Sheets (CSS) is a stylesheet language used to describe the presentation of a document written in HTML or XML (including XML dialects such as SVG, MathML or XHTML). CSS describes how elements should be rendered on screen, on paper, in speech, or on other media. – https://developer.mozilla.org/en-US/docs/Web/CSS

Being a curious and hungry developer, I soon reached for a tool that I saw discussed called SASS that was something called a “CSS Preprocessor”. Basically it was a computer language that was like CSS, and that generated CSS. It allowed you to reuse bits and pieces of styles in ways that stock CSS did not at the time.

It was a tool that made your life as a front end developer easier, but there was a catch – now you had to process that SASS into CSS at some point along the way. There was now an extra piece of gear involved in your development –> production lifecycle.

Another example

Frontend development is insanely complex these days. Plain old Javascript that browsers can digest and execute has exploded like a million suns into various dialects, innumerable frameworks, and more tooling than any sane developer can keep track of. Gone are the days of including jQuery (and my interest in frontend dev along with it).

This is (arguably) great for developers to be more productive and provide high quality experiences to their audience, but there is much, much more to think about in terms of how to deliver those experiences to the end user. Yes, my marketing speak is rather loathesome.

By “deliver” I mean the nuts and bolts – all that Javascript has to go over a network connection from a server in a rack to a user's browser in order to execute and Do Stuff. You don't want your user to have to download 500,000 JS files weighing 40MB just to load your website, so you do a bunch of stuff to optimize that:

• you smash all those files into 1 file, this cuts down on num_requests because those cost time
• you compress that file into something smaller using gzip or the like, this cuts down on the weight of that code traveling over the wire because that costs time too
• you might have to compile your chosen dialect of JS into “vanilla” JS that the browser can actually execute.

And you're not going to want to do this by hand every time, so you reach for a tool that does it all for you. There is now an extra piece of gear involved in your development –> production lifecycle.

Just one more

Package managers. They are a wonderful development, and I'm talking about Rubygems, Composer, Pip, and the like. They allow the OSS community to write, publish, and use bits of code (libraries) that other people have written so you don't have to reinvent the same wheel everytime you want to add a login form to your site. Just gem install devise and bam, your Rails site has user login forms, password resets, and a ton of other functionality that you didn't have to build yourself.

All of these package managers operate on a similar principle – they have a list of the packages required for a given project and when one of your colleagues adds a new one, it gets added to the list. All you have to do to add it to your local working copy is composer install and bam, you have that new library too.

Ideally though, you're not committing that package to your Git repo because that's “3rd party code”. It bloats your Git repo which is just a bad smell, but it's also code that you don't own and should never really touch. What should be committed to your Git repo is just your code. At deploy time, just run composer install and you're done, but you need to have a Thing that will be able to execute composer install.

There is now an extra piece of gear involved in your development –> production lifecycle.

Continuous Integration

That extra piece of gear is what's commonly referred to as a “build pipeline” or a “continuous integration layer”. It's essentially just a process that will execute your instructions for how to tranform your development codebase into your production codebase or artifact.

Plenty of players in the space, starting with a simple Bash script on the user's laptop before pushing to Git. More involved and flexible options include Jenkins, Travis, or CircleCI. Gitlab has a built in CI pipeline, it's part of why they will ultimately win over Github.

Problem with all these tools is that they're other tools, and you have to pay for them and maintain them and learn how to use them and deal with API changes etc. Platform.sh has this stuff all built in, and we're the only PaaS vendor that I know of that has this level of continuous integration built right in in the form of our build and deploy hooks.

We have shortcuts for Composer based projects, and full support for adding any number of build time dependencies (probably a whole nother post's worth of material to discuss). Most other PaaS vendors make you buy CircleCI and wire the stuff together in order to achieve the same level of cushy, modern developer experience that we offer, and that's where I've run out of steam on this post. If you got questions, ask em.

#generaldevelopment #platformsh

Problemspace

I haven't posted anything in over a year. I'm almost rusted shut, yet I lately feel the need to commit to creating a new row in the database that backs this blog.

Solutionspace

Just start writing.

I've been at this gig for over 3 years now, and I've learned a decent deal about the “real world”, the one that I wanted to exist in toward the end of my last career. I scored that tech job, I got bumped up to mgmt about 8 months ago. I actually really enjoy it and I'd kind of like to journal some of the things I've learned along the way here.

Anyway, if you knew me in my 20s and you're reading this – thank you. I'm going to just start writing.

#life

Well hey, I haven't written anything here in 2018, that's wild.

So here's what I'm doing – I'm removing the GA tracker from this blog. I don't check my stats anymore and I suspect what audience there is to this blog probably has ad blocking enabled. Hell, I have ad blocking enabled. Seems kind of hypocritical to have an ad blocker installed so that the eye of tech-Sauron can't follow me as easily around the internet, and yet to have GA tracking on my own blog. So I'm getting rid of it.

#life

The Book of Platform – 2.md

Getting started

In this chapter we'll walk through the process of setting up and deploying your very first project on Platform.sh. We'll go through the free trial and checkout process, clone a simple project from a public GitHub repository, and push it to your new Platform.sh project.

After that we'll dazzle you with how easy it is to set up development environments for various features with the click of a button. We'll make a couple of simple code changes and deploy them each separately to their own hosting environments and then merge everything back together to get it ready for a production release.

Step 1 – new project creation

The easiest way to get started is to take advantage of Platform.sh's free 1 month trial. This will allow you to step through this on boarding tutorial free of charge and get familiar with the way we make it easier for you to decide for yourself whether or not Platform.sh is a good fit for your team.

On the front page of Platform.sh's website you can find a link to a Free Trial, currently at top right of the main navigation. Clicking through that brings you to the account creation workflow. I personally prefer just creating an account on the site for whatever given service, but you are also free to take advantage of our Bitbucket, GitHub, or Google integrations for authentication.

#platformsh

The internet is hard these days.

It started simply enough – for instance, all you really needed was a Geocities account and some initiative to learn HTML and you could have your own place to put whatever you wanted and make it available to the entire world. From such simple seeds, complex structures did grow..

Geocities was permanently shut down in 2009, at once both a tragedy for the loss of so much content, so much history, and yet also a wake-up call for so many of us that we needed to have control over our own content, applications, and businesses. Many of us chose to host our own websites so that a seemingly arbitrary decision from some faceless corporate power couldn't upend overnight what we'd created over years. But that decision also made things a little more complicated.

At that point all you really needed was a web hosting account somewhere with Apache HTTPD installed as a web server. Then you could edit and upload your HTML just as before, only this time it couldn't be taken away from you because you were (more) in control of the setting.

Somewhere along the way you were likely introduced to something like Wordpress or Drupal or Ruby on Rails, which were all essentially frontends to some kind of database, and that database is where you would store your content. This was a wonderful development, not only enabling non-technical users to publish content to the web without knowing anything about HTML or FTP, but also for small businesses to be able to create their first eCommerce stores online and take advantage of an entire global market. Again, this was the march of technological progress creating new opportunities for primitive man to make use of highly advanced tools without having a Computer Science degree.

But, as the saying goes, with great power comes great responsibility. The responsibility in this case is that of having to set up and maintain your own web hosting infrastructure. Some of us like this kind of work – setting up and managing fleets of servers with all manner of different pieces of software on them to serve the world's internet needs – but some of us really just like writing code and building websites and applications.

Platform.sh is a new breed of hosting service, and was created expressly for this second group of technologists.

Platform.sh gives you an incredibly flexible set of tools with which you can build and deploy a huge range of different types of applications to the world with the click of a button or a push to a Git repo. Platform.sh currently has support for PHP, Ruby, Python, and Node.js with other runtimes like Go and Java either in public beta or in planning mode internally.

Platform.sh was also architected from the ground up to fulfill the promise of Git as a codebase management tool. No longer are your working feature branches trapped on your machine or in a remote repo with no context to make them live (and testable) for your teammates. Platform.sh will provision a fully functional, completely segregated hosting environment for each one of your Git branches, with all of the data and uploaded files that your app's codebase needs to be a fully functioning application.

Lastly, Platform.sh was designed to grow with your project as your project's needs evolve. The days of filing support tickets to have a PHP extension or a new database server installed are effectively over. Not only will we provision all of your application's software dependencies – Redis, MySQL, ElasticSearch, and many others – but you can choose from many different versions of each of these dependencies. Want to see how ruby:2.4 or python:3.6 or postgresql:9.6 or php:7.2-rc improves your app's performance? We always endeavor to provide the latest versions of each so that upgrading the underlying software on which your application runs is as simple and painless as changing a line of configuration.

If this sounds like something that might take some of the tedium out of your development day or possibly increase the velocity with which your business can bring new ideas and features to your customers then please, read on. In the coming chapters we'll walk you through getting started with Platform.sh by stepping you through setting up your first project and deploying a simple app with a few commands. After that we'll dive deep into more advanced topics such as -

• Configuring your project with YAML
  • routes.yaml
  • services.yaml
  • .platform.app.yaml or “app yaml”
• Managing and interacting with various administrative functions of your project both via the user interface as well as the Platform CLI
  • branching
  • merging
  • backups
• How all of this seeming magic works under the hood
  • containers
  • environment variables
  • copy on write

So, welcome to The Little Platform.sh Book! We sincerely hope to make internet-ing a little bit easier for you and your team.

#generaldevelopment #platformsh

So much to say, so much to say..

Some day I'll write a memoir, but that day is not this day my friends. The most outwardly memorable thing I've done so far this summer is fly out to Oregon to the String Summit to cover for Mike Devol of Greensky. Mike had a baby a few days before, and back in the winter he called me up to see if I might be interested in covering for him on some summer festival shows. Duh, yes.

So I had several months to prepare, but not prepare so much that I got actually excited about the gig because it quite likely might not happen. It was a strange mental exercise.

I can't even begin to recount the whole weekend here, though it was definitely worth thinking back on. Friday morning I got to judge the band contest and though they didn't win, this band deserved to win..

Something about the line “then I'll have my family band, make play the tunes that they don't really understand” hit the waterworks button and it got stuck. I was seriously sobbing for about 30 minutes or so, fortunately I'd bought aviators on the way to the airport.

The whole experience was just so intensely cathartic for me, it's not possible to express my thanks to those guys for letting me do it.

#life #music

I just switched this blog over to a Silex application for a few weeks, but I never got around to adding in the functionality to add new tickets, and add new tags, and link the two together, and all that other stuff that a framework like Laravel just gives you.

So we're back.

#life

Problemspace

You've got a new site on Platform.sh that is basically at the end of its development stage, and you're preparing to go live. You've decided on Cloudflare to host your DNS. Cloudflare is a good choice for smaller sites, and I recommend it often. Is has a few things going for it -

• It has a free tier, which gives you pretty much everything you really need for a personal or small business site.
• it has a very robust and modern global network.

One of the main features that a modern DNS provider needs to have in order to work with Platform.sh is somethat that's colloquially known as “CName Flattening”. This solves an age-old problem (in internet years) – being able to point your “root domain” to a domain name rather then an IP address. This post explains it better than I can, and that's not eh point of this post anyway.

Another nice feature of Cloudflare is something they call “Flexible SSL”. SSL, HTTPS, TLS, they all mean basically the same thing – that the traffic from your user's browser to your website is being encrypted. This is very important in a practical sense if you're ever on a public wifi network, for example. Setting up SSL on your website can be a bit of a headache though, involving buying a cert, generating some rather arcane crytographic keyfiles, installing them correctly on your server, etc.

Cloudflare offers a bit of a relief from this headache with “flexible SSL”. That means that your site can use Cloudflare's SSL cert to encrypt the traffic from user to Cloudflare (remember, cloudflare is sitting in between your users and your website's server). Traffic from Cloudflare to your website then travels unencrypted over plain old HTTP. This is “suboptimal”, but it does alleviate some of the attack vectors on your users.

Cloudflare's "Flexible SSL" option
 HTTPS HTTP
User ----->----> Cloudflare ----->----> "Origin" server (your website)

The other alternatives to this are either running your site unencrypted over HTTP or using “full SSL”, in which you have to install a cert at the origin in order to encrypt the traffic between Cloudflare and your website.

Cloudflare's "Full SSL" option
 HTTPS HTTPS
User ----->----> Cloudflare ----->----> "Origin" server (your website)

Getting there...

So you've been in development this whole time, you're using HTTPS and redirecting HTTP traffic to HTTPS like a good net citizen and now it's time to go live. You figure you'll just skip setting up a SSL and use Cloudflare's Flexible option. You immediately run into a problem though, of a redirect loop. Here's why..

• User requests website over HTTP, gets redirected to HTTPS by your application
• HTTPS travels from user to Cloudflare where it's decrypted and handed back to your website over HTTP
• Repeat until browser crashes and tells you you have a redirect loop.

You have two options at this point – either allow HTTP traffic as well so that traffic can flow from Cloudflare to your server without being redirected, or go full SSL.

Option A will not work very well in the end. Since your app thinks it's running HTTP, all of the in-site generated links will point to the HTTP version of the pages, which means that as soon as someone clicks a link on your site, they'll be on HTTP. No good.

Solutionspace

Get a cert, install it, and go Full SSL.

Soon come

A post about using letencrypt to set up certs on your Platform.sh project.

#devops #platformsh

I was just remembering this post from back in the early days of this blog. Social media was being used to shine a bright light on oppression and old, repressive regimes were falling.

Fast forward to now, the end of 2016. Social media is being used to organize vastly more repressive regimes, and the ease with which anyone can now spread convincing lies makes it almost impossible to tell what's true anymore.

Traditional journalism has never been more important than it is now.

#life

Problemspace

You have a decent sized project and your deployments are taking a while. Platform.sh rebuilds your entire application from scratch with each git push so in some cases the process of downloading all those 3rd party packages can take quite a while. We can and do manage local caches of some composer packages due to our PHP heritage, which helps to make composer install a pretty snappy affair, but it's simply not possible to effectively do this with Nodejs.

Compounding this problem for npm is the fact that npm's dependency graph, that is the dependencies of your dependencies, have to be worked out every time you run npm install. This can lead to developers in your org installing different versions of packages which will cause you problems.

Most other package managers overcome this with the use of a “lockfile”. A lockfile is a file that's generated when you run composer install for the first time, or bundle install if you're working Ruby. A lockfile is the result of the dependnecy graph being worked out, and then specifying the exact versions of each package. This file is checked into Git, and each dev in the project gets the exact same versions of the packages required for the project.

Solutionspace

I was listening to the most recent Laravel podcast over the weekend and they got to talking about a new quasi-npm-replacement that had just come out called Yarn.

Yarn aims to be an almost drop in replacement for npm. There are a number of ways of installing it, but the most simple is just npm install -g yarn. My coworkers thought I was trolling them with that, but it makes perfect sense if you think about it.

The only other step is run the yarn command locally in order to have Yarn traverse your node_modules directory and build up the Yarn lockfile – yarn.lock. Then commit that to git and let's rock and roll on your .platform.app.yaml. We're going to require Yarn in the global dependencies section -

dependencies:
  ruby:
    sass: "**" # not required, just assuming
    nodejs:
    gulp-cli: "**" # same here
    yarn: "**"

And then replace npm install in your hook.build with yarn install instead, like so -

hooks:
  build: |
    yarn install
    gulp default // for a Laravel project

This took my previously 6 minute builds down to about 1 minute. In other words, the time that it took out of my build phase was longer than the time it took to completely move from npm to Yarn in the first place. The reason for this speed boost is that Yarn doesn't have to generate the dependency graph every single time (like npm does) since the lockfile, and Yarn downloads the packages in parallel rather than whatever npm does, which must be one at a time.

If you're using npm install as part of your build step on Platform.sh, it's really a no-brainer. Check it out!

#javascript #platformsh