Deployment strategy and symbolic links
Here's a quick neat idea involving using symbolic links on a production environment.
A most basic deployment strategy usually involves taking your site down, upgrading your database/files either manually or through svn.
Some more sophisticated strategies may involve tools such as phing (for php).
But there can be some problems, most of all the fact that the site might have to be down while maintenance is going on.
A better way to do this is to have your production environment sit on a symbolic link.
You can then deploy the new version to an entirely new directory and test it to make sure everything is fine, and then instantly switch the symbolic link to the new version when you are ready to release it.
If the flak hits the fan and you need to revert back, simply point the sym-link back to the original directory.
For example, lets say your site is currently on version 1.4, and you are upgrading to 1.5. The files might be here
/var/sites/mysite/version1.4/
Now copy everything from the above directory into:
/var/sites/mysite/version1.5/
Do your normal release strategy on this new directory.
Your symbolic link might looks something like:
/var/www/vhosts/mysite.com -> /var/sites/mysite/version1.4
/var/www/vhosts/staging.mysite.com -> /var/sites/mysite/version1.5
Once you are happy to do the switch, simply point the link to the same place as your staging environment
/var/www/vhosts/mysite.com -> /var/sites/mysite/version1.5
Just doing this is great for mostly read only content and no database changes.
Next up I'll write up something which deals with database changes and how to have a site that never goes down, even when complicated maintenance is required.
Linux USB drive automatic mount unmount management script
Hi All,
Just a little something I wrote to make my travelling life easier. I wrote this 3am London time while jetlagged... don't blame me :)
Preface, I run a barebones Linux GUI (blackbox). I got sick of constantly reading dmesg/blkid and manually mounting the USB drives. Here are two python scripts to make my life simple.
usbon.py - just mounts whatever USB you have attached under /mnt/usb_label_name.
usboff.py - just safely umounts the USB and cleans up the directory for you.
Use as you see fit, just give me some credit :)
I'll put the raw files on this page once I find out how to get wordpress to do it :(
usbon.py
#!/usr/bin/python
#Author: Mark Liu
#Version: 0.1
#Date: 19 August 2010
# good old imports
import commands, re, os
# global variables
total_devices = []
mount_base = '/mnt/'
# class definition to make lives a bit easier
class MountableDevices:
"""Simple Object to represent system mountable disk"""
def __init__(self, dev_name, dev_UUID, dev_label, dev_type):
self.dev_name = dev_name
self.dev_UUID = dev_UUID
self.dev_label = dev_label
self.dev_type = dev_type
self.dev_mount = False
def already_mounted(self):
self.dev_mount = True
def details(self):
print 'Device Name: %s' % (self.dev_name)
print 'Device UUID: %s' % (self.dev_UUID)
print 'Device Label: %s' % (self.dev_label)
print 'Device Type: %s' % (self.dev_type)
print 'Device Mounted?: %s' % (self.dev_mount)
print ''
# Functions to help with process blkid outputs
def process_blkid_output(device_blk_output):
global total_devices
dev_LABEL, dev_UUID, dev_TYPE = '_not_set', 'N/A', 'N/A'
dev_NAME = device_blk_output[0][:-1]
for item in device_blk_output:
if re.match('UUID', item): dev_UUID = re.search('\".*\"', item).group(0).strip('\"'); continue
elif re.match('TYPE', item): dev_TYPE = re.search('\".*\"', item).group(0).strip('\"'); continue
elif re.match('LABEL', item): dev_LABEL = re.search('\".*\"', item).group(0).strip('\"'); continue
total_devices.append(MountableDevices(dev_NAME, dev_UUID, dev_LABEL, dev_TYPE))
# Functions to help with process mount outputs
def process_mount_output(device_blk_output):
global total_devices
dev_NAME = device_blk_output[0]
if dev_NAME == 'none' or dev_NAME == 'proc' or dev_NAME == 'binfmt_misc' or dev_NAME == 'gvfs-fuse-daemon':
pass
else:
for device in total_devices:
if dev_NAME == device.dev_name:
device.already_mounted()
# Do some mounting Prep work
os_commands = ['blkid', 'mount']
process_system_output = {'blkid': process_blkid_output, 'mount': process_mount_output}
# Processing System information like blkid and mounts`
for operation in os_commands:
raw_outputs = commands.getstatusoutput('sudo ' + operation)
if raw_outputs[0] == 0:
#split output via new line
outputs = raw_outputs[1].split('\n')
for output in outputs:
clean_blk_output = output.strip().split()
process_system_output[operation](clean_blk_output)
else:
print 'sudo %s... failed' % (operation)
sys.exit(1)
# Main section compare and mount any desired devices
for device in total_devices:
if device.dev_mount == True:
print 'Ignoring device: %s (already mounted)' % (device.dev_name)
elif device.dev_type == 'swap':
print 'Ignoring device: %s (swap)' % (device.dev_name)
else:
mount_dir = mount_base + device.dev_label
mkdir = 'sudo mkdir %s' % (mount_dir)
mount = 'sudo mount -U %s %s' % (device.dev_UUID, mount_dir)
print 'Mounting device: %s (%s)' % (device.dev_name, mount_dir)
if not os.path.exists(mount_dir):
result = commands.getstatusoutput(mkdir)
if result[0] != 0:
print 'dir creation failed, aborting USB mount...'
sys.exit(1)
else:
result = commands.getstatusoutput(mount)
else:
print 'unclean mount directory, please clean up...'
usboff.py
#!/usr/bin/python
# Author: Mark Liu
# Version: 1.0
# Date: 19 Aug 2010
# Imports
import dircache, commands
# Definitions and prepwork
mount_base = '/mnt/'
mounts = dircache.listdir(mount_base)
# Umount, then safely delete directories
for mount in mounts:
mount_dir = mount_base + mount
result = commands.getstatusoutput('sudo umount %s' % (mount_dir))
result = commands.getstatusoutput('sudo umount %s' % (mount_dir))
if result[1] == 'umount: %s: not mounted' % (mount_dir):
commands.getstatusoutput('sudo rm -r %s' % (mount_dir))
Django and Google App engine
Update: prove.com.au layout has been fixed!
So Mark finally got bothered to whip up a wordpress blog for us. Nice one....
As he seems to have a list with things he'll never get around to doing, I might as well get one up as well.
- Fix layout on prove.com.au
I never said it had to be a long list...
So I started playing with Django and Google's App Engine. I'll try and explain my first thoughts individually and then a little about what i think of them combined.
Django
I genuinely like Django. If I had to describe it in a single word it would be 'concise'. They really do take the DRY (don't repeat yourself) principle seriously. It's auto admin system is incredibly powerful and can be setup with very little effort. The Model Template View architecture is clear and well implemented. The model stack is exceptionally powerful and is able to sync the database automatically making migrations a piece of cake. The template language is very simple and can be picked up in minutes (given you are already familiar with HTML). Lastly, its routing and URL handling is incredibly advanced, and has a very cool generic view system which allows you to setup simple view actions automatically.
My only concern is that it's current popularity is overshadowed by some other frameworks such as its red cousin ruby on rails. This may not sound like a valid argument to some, but it does make a massive difference when considering active discussion and plugins. Hopefully this should rapidly change over the next few years.
Google App Engine
First the good. I think app engine is revolutionary. Instead of having to worry about a distributed system with load balancing, I can now consider my hardware stack a giant auto-elastic server with virtually unlimited resources. Google is also very genrous with the free quota, for any normal app, you will not go anywhere near the limits set per day.
However with the good must alas come the bad and there are too many to list here. To keep it in one sentence, Google App Engine is good for one app, Google search. You must use either Python or Java, you must know Datastore/GQL, no GQL results beyond 1000 rows (sound familiar?). App engine currently have some limits that seriously scare me. I hope they will lift this over time.
With your powers combined, I am....
Not quite captain planet. The problem comes down to the model stack. Django uses SQL, App Engine uses Datastore/GQL which to me is a huge issue. By putting your app on App Engine, you instantly lose your django admin system, sessions and will likely have to refactor your entire model layer to accommodate. Given Django's DRY principle (Fat models, skinny views), this would probably mean ~50% of your codebase.
In conclusion, I think I'm going to keep playing with Django, but give app engine a rest.... for now.
