outcoldman
outcoldman Denis Gladkikh

Hosting GitHub Pages with HTTPS

jekyll, github pages, https, nginx, docker, and jenkins

Sad but true: GitHub Pages still does not support HTTPS for custom domains (see Add HTTPS support to Github Pages - this is not an official bug tracking for GitHub Pages). You basically have just two options:

I chose second option for three reasons:

At the end I got the same (and even better) workflow as I had with GitHub pages, where:

Now let’s go in details how you can do that.

Preparing sources

As I said I host my own GitLab server at home using Docker (see Using docker at home), so at first I moved my repository from GitHub to my own GitLab (this is not a requirement, I just did it because it was easier for me to do).

I use rbenv to not mess with the system installed ruby (on OSX), so in my case I have a .ruby-version file

2.0.0-p645

Also I created a Makefile which contains most important steps for me

# Install tools, requiered for building
installtools:
	@npm install bower
	@rbenv install -s
	@rbenv exec gem install bundle

# Don't need to run it often, just few client side dependencies,
# allows me to update dependency css and font-awesome.
updateclientdeps:
	@bower install
	@cp bower_components/pygments/css/monokai.css css/syntax.css
	@cp bower_components/normalize-css/normalize.css css/normalize.css
	@cp bower_components/font-awesome/css/font-awesome.min.css css/font-awesome.min.css
	@cp bower_components/font-awesome/fonts/* fonts/

# Install ruby/jekyll/github-pages dependencies
deps:
	@rbenv exec bundle install
	@rbenv rehash

# To build a local version of website (including drafts and right links)
build-local:
	@rbenv exec bundle exec jekyll build --draft --config=_config.yml,_local_config.yml

# To build staging version of website (including drafts and right links)
build-staging:
	@rbenv exec bundle exec jekyll build --draft --config=_config.yml,_staging_config.yml

# To build production version (without drafts)
build-production:
	@rbenv exec bundle exec jekyll build --config _config.yml

# Just to server local version
server-local:
	@rbenv exec bundle exec jekyll server --watch --draft --config=_config.yml,_local_config.yml

# Fix permissions before deploying to nginx servers
predeploy-fix-permissions:
	@find ./_site/ -type f -exec chmod 644 {} +
	@find ./_site/ -type d -exec chmod 755 {} +

# Deploy staging version
deploy-staging: predeploy-fix-permissions
	@rsync -r --rsh="ssh -p9022" --checksum --delete-after --delete-excluded --numeric-ids ./_site/ root@myhomeserver:

# Deploy production version
deploy-production: predeploy-fix-permissions
	@rsync -r --rsh="ssh -p9022" --checksum --delete-after --delete-excluded --numeric-ids ./_site/ root@outcoldman.com:

I tried to explain everything in my Makefile with comments, but still few words about my workflow

source 'https://rubygems.org'
gem 'github-pages'%

As you can see I have three version of configuration for jekyll, the base one _config.yml

author: Denis Gladkikh
description: Blog about software development
url: https://www.outcoldman.com
title: Blog about software development
deployment: production

markdown: redcarpet
permalink: /:categories/archive/:year/:month/:day/:title/
safe: false

disqus_short_name: outcoldman
disqus_show_comment_count: false
disqus_registered_url: https://www.outcoldman.com

google_analytics_tracking_id: UA-7023371-5

gems:
  - jekyll-sitemap

exclude:
  - .gitignore
  - .ruby-version
  - Gemfile
  - Gemfile.lock
  - Makefile
  - _config.yml
  - _local_config.yml
  - _staging_config.yml
  - bower.json
  - bower_components
  - docker-compose.yml
  - node_modules

And two overrides: _local_config.yaml

url: http://localhost:4000
deployment: local

and _staging_config.yaml

url: https://outcoldman-staging.myhomeserver.com
deployment: staging

Few things are important in these configurations:


{% if site.deployment == "production" %}
    <!-- Include disqus or Google Analytics -->
{% endif %}

Setting up nginx servers

As I mentioned in Using docker at home I use nginx-proxy, which allows me to easily host multiple websites on the same server in separate docker containers. My own nginx servers serve HTTP requests on 80 port, nginx-proxy handles HTTPS requests and forwards them to my nginx containers.

So on current moment I have two of nginx-proxy containers, one on DigitalOcean, which serves production version, and one on my local home server to serve staging version. Each is configured appropriately with certificates.

The other step is to configure nginx servers with SSH server installed, this is a Dockerfile created for my production version

FROM nginx

# Install OpenSSH Server, supervisor and rsync
RUN apt-get update \
    && apt-get install -y openssh-server supervisor rsync \
    && mkdir -p /var/run/sshd

# Disable password authentication for SSH
RUN sed -i 's/#\s*PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config

# Use rrsync on client
RUN mkdir -p /root/bin \
    && gunzip /usr/share/doc/rsync/scripts/rrsync.gz -c > /root/bin/rrsync \
    && chmod +x /root/bin/rrsync

# Jenkins public key (don't forget to replace JENKINS_SSH_PUBLIC_KEY with your
# SSH public key
RUN mkdir -p /root/.ssh \
    && echo 'command="/root/bin/rrsync /usr/share/nginx/html/",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-rsa JENKINS_SSH_PUBLIC_KEY' > /root/.ssh/authorized_keys

# Custom nginx configuration
COPY nginx.conf /etc/nginx/nginx.conf
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf

CMD ["/usr/bin/supervisord"]

Also nginx.conf

user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    gzip  on;

    server {
        listen          80;
        location / {
            root /usr/share/nginx/html;
            index index.html;
        }
        error_page 404 /404.html;
        expires 1d;
    }
}

And supervisor.conf

[supervisord]
nodaemon=true

[program:sshd]
command=/usr/sbin/sshd -D

[program:nginx]
command=nginx -g "daemon off;"

Important things from this file

The last step is to configure docker-compose.yaml file

nginx:
  build: .
  ports:
    - '9022:22'
  environment:
    - VIRTUAL_HOST=outcoldman.com
    - VIRTUAL_PORT=80
  mem_limit: 128m
  cpu_shares: 256
  restart: always
  log_opt:
    max-size: 10m

For this container I map port 9022 for SSH to deploy website. Port 80 will be used by nginx-proxy. I’m using standard logger from this image, because I collect everything I need from nginx-proxy.

For staging server I also included basic auth just to make sure that one day bots will not start to parse it. To do that I included two more lines in Dockerfile

COPY htpasswd /etc/nginx/.htpasswd
RUN chmod 0644 /etc/nginx/.htpasswd

And added few more lines in nginx.conf (see auth_basic*)

    # ...
    server {
        listen          80;
        location / {
            root /usr/share/nginx/html;
            index index.html;
            auth_basic  "Restricted";
            auth_basic_user_file /etc/nginx/.htpasswd;
        }
        error_page 404 /404.html;
        expires 1d;
    }
    # ...

To generate htpasswd file you can use htpasswd

htpasswd -c htpasswd yourusernamehere

At this point we have servers with nginx and SSH servers, you can actually try to deploy them from your local environment (don’t forget to include the right public key in the nginx containers, you can include multiple if you want).

If everything works on this step - we can go to next step to automate deployment using Jenkins.

Setting up Jenkins

I use official Jenkins image. The only one problem I saw with it - that you need to be careful with file permissions as Jenkins is using not root user.

For example when you mount /var/jenkins_home/ you need to setup right permissions, this is example of my Dockerfile which installs some dependencies for rbenv and rsync

FROM jenkins

USER root

RUN apt-get update \
    && apt-get install -y \
        autoconf bison build-essential libssl-dev libyaml-dev libreadline6-dev zlib1g-dev libncurses5-dev libffi-dev libgdbm3 libgdbm-dev \
        rsync \
    && rm -rf /var/lib/apt/lists/*

USER jenkins

As you can see I switch to root to do update steps and after that switch back to jenkins user, the same with data volume container, I need to be sure that volume which I share has right permissions

vdata:
  image: busybox
  volumes:
    - /var/jenkins_home
  command: chown -R 1000:1000 /var/jenkins_home

jenkins:
  build: .
  volumes_from:
    - vdata
  environment:
    - VIRTUAL_HOST=jenkins.myhomeserver.com
    - VIRTUAL_PORT=8080
  mem_limit: 4g
  cpu_shares: 256
  restart: always
  log_opt:
    max-size: 10m

I’m not going to describe all Jenkins configurations which you should do, just mention important (to learn more about jenkins you can always find some books, like Jenkins. The definition guide).

After that you need to generate SSH keys on Jenkins, just open bash in jenkins container and generate SSH keys following for example GitHub documentation (do not specify passphrase, as it will be hard to use this key in deployment scripts, if you want to make it more secure you can use separate scripts for GitLab and deployment)

docker exec -it jenkins_jenkins_1 bash
container$ cd /var/jenkins_home
container$ mkdir .ssh
container$ ssh-keygen -t rsa -b 4096 -C "someemail@myhomeserver.com"

The other step I did, I removed StrictHostKeyChecking on Jenkins for my servers, as I recreate them very often and I don’t want to upgrade fingerprint so often

container$ echo 'Host myhomeserver
>     StrictHostKeyChecking
> Host productionwebsite.com
>     StrictHostKeyChecking no' > ~/.ssh/config

After that you need to go to the Jenkins configuration and add generated key to Jenkins Credentials.

Setup Jenkins with GitLab by following this README.md and GitLab documentation (the last one is little out of date).

At this point we are ready to create new project in Jenkins, where you need to specify

make deps
make build-staging
make deploy-staging
make build-production
make deploy-production

You are ready to build it. First build will take a lot of time because Jenkins will need to download ruby for the first time and build it, all next builds will be much quicker.

Have feedback or questions? Looking for consultation?

My expertise: MongoDB, ElasticSearch, Splunk, and other databases. Docker, Kubernetes. Logging, Metrics. Performance, memory leaks.

Send me an email to public@denis.gladkikh.email.

The content on this site represents my own personal opinions and thoughts at the time of posting.

Content licensed under the Creative Commons CC BY 4.0.