Higher Logic notes on screwing with tech…


Cisco Email Config Backup

ET fone home... using email config backup

This is a nice little feature that most cisco routers have to allow simple cisco config backup Since I didn't want to install a messy service like RANCID or sftp servers just to get cisco configs, this is a light weight solution for getting the cisco to mail you its own config once in a while.

contact-email-addr mark@cisco_router.com
mail-server your_mail_server priority 1
profile "ConfigBackup-1"
destination address email mark@xxx.com
subscribe-to-alert-group configuration export full periodic monthly 11 10:00

This cisco will mail you once a month on the 11th at 10am.


Innobackupex a faster MySQL re-slaving process (Ubuntu)

A shoutout to my friend Colm for putting these instructions together, I'm putting it here for easy access. The following contains simple steps for re-slaving a broken mysql slave by using innobackupex. I've cleaned it up a bit for conciseness. This procedures uses the Percona tools for a much quicker master to slave sync. MySQL dump just takes wayyy to long.

Innobackupex is a Percona tool which does a binary copy of the raw MySQL data files adjusting for transactional integrity. This process is a lot faster than the standard mysql dump + load + slave option. We have been using this in production for a while now.

Prepping using innobackupex

# if the package hasn't already been installed:
master# sudo apt-get install percona-xtrabackup
master# cd ~ && rm -rf mysqlbak/ && mkdir mysqlbak/
# To avoid issues if your connection breaks
master# screen
# reduce the I/O impact on MySQL funtioning
master# nice -n 19 ionice -c3 innobackupex mysqlbak/
# Clean up the transaction so the database is consistent
master# innobackupex --apply-log mysqlbak/*
master# ssh backupserver "rm -rf mysqlbak/" && scp -Cr mysqlbak/ backupserver:

Start up the slave

slave# service mysql stop
slave# mv /var/lib/mysql /var/lib/mysql_old
slave# mv mysqlbak/* /var/lib/mysql
slave# chown -R mysql:mysql /var/lib/mysql
# The following file has to be present in the data directory
slave# cp /var/lib/mysql_old/master.info /var/lib/mysql/
# Update Offsite slave root password from the Master DB password
slave# mysqladmin -u root -pMasterPassword -S /var/run/mysqld/mysqld.sock password 'OffSiteSlavePassword'
slave# service mysql start

# Setup Slave configuration
# This file contains the master_log_file and master_log_pos information
slave# cat /var/lib/mysql/xtrabackup_binlog_info
slave# mysql -e 'change master to master_log_file="xxx", master_log_pos=yyy;'
slave# mysql -e 'start slave;'
slave# mysql -e 'show slave status\G'

Useful Links

This procedure has brought a previous 12 hour+ restore process down to 2 hours.
Percona Toolkit


Vagrant + VirtualBox + Puppet on OSX (Rapid puppet testing)


I'm a massive fan of Puppet, its made my life so easy. At the same time Puppet is very powerful when you give it full control of your entire systems. One of the things thats been lacking in my environments has been the ability to quickly test Puppet code destined for production and see how it fares. This is where Vagrant and VirtualBox comes in :D

Quickly, quickly I need to be somewhere else

# (Optional) Homebrew which gives you access to useful packages.
Download and install Homebrew
# VirtualBox which is kinda fundamental to this.
Download and install VirtualBox
# Installs vagrant, puppet and puppet-line (puppet style checker)
sudo gem install vagrant puppet puppet-lint --no-ri --no-rdoc
# Downloads a Ubuntu 12.04 LTS image to work off (google for your poison)
sudo vagrant box add precise64 http://files.vagrantup.com/precise64.box
# It also sets the correct permissions to run vagrant
sudo chown -R localuser ~/.vagrant.d

I choose to install vagrant, puppet and co through gems because there's enough package management crap on my system as it is without me introducing more pain :)

Can I do something now

# This is the folder you want to work one set of VM's out of
mkdir vm_working_dir
# Let Vagrant do some magic
vagrant init
# Edit the VagrantFile (see below)
# Add a bash file (see below)

Add a bash file to setup basic stuff on my new 12.04 LTS

hostname xxx
apt-get update
#apt-get upgrade -y
apt-get -y install puppet

This is powerful because you can set the hostname to which ever box in production you would like to mimic/test.

Edit VagrantFile to look like this:

# This refers which base image to use, our previous downloaded precise64
Vagrant::Config.run do |config|
config.vm.box = "precise64"

# This can be any arbitary script covered above
puppet binary e.g. apt-get install puppet
config.vm.provision :shell do |sh|
sh.path = "base_setup.sh"

# I pointed module_path to my local modules folder.
# I setup a custom facter fact to set vagrant variable (useful, more later).
# I pointed the manifest file at the top level puppet file so it includes my whole tree.
config.vm.provision :puppet, :facter => { "vagrant" => "true" }, :module_path => "~/puppet/modules" do |puppet|
puppet.manifests_path = "~/puppet/manifests"
puppet.manifest_file = "site.pp"

Lets fire this baby up...

cd vm_working_dir
# starts up vagrant and new base image the very first time
vagrant up
# If your VagrantFile is correct, it will:
# Copy the precise64 to start your local image
# Setup fundamental networking
# Run the bash script on it once its up
# Run the node against my own puppet manifests based on its own hostname

# once its all fired up
vagrant ssh (this drops you onto the box)
# Shuts down the box once you are done all state is preserved for later use
vagrant halt

Explain damn you!

    I prefer running it just on my local puppet files rather than through another local puppet server + new server CA + new client cert + *vomit*...
    The reason I added my own Puppet facter fact is because you don't want your local vagrant box applying production network/security settings that might prevent "vagrant ssh" to work.
    Now that this is setup, you can just run Vagrant destroy + Vagrant up to build a box from base OS image.
    If running a full base build is too time consuming and unnecessary just shutdown using vagrant halt and vagrant up to apply the latest changes...

Useful stuff

# This will start you up from scratch and destroy anything puppet and bash script did
vagrant destroy
# This will delete the image you downloaded...
vagrant box remove precise64

    Let me know if I've missed anything.

When I get a chance i'll put up an article on puppet continuous build and release. I <3 Vagrant!


First gen Macbook Air weak link

Just spent a large part of tonight trying to figure out why my macbook air had spotty wifi connection... it was driving me insane!

I finally tore it up and found two wires broken from the hinge body of the air. Manually joining the b0rked wire changed wifi condition from 50% packet drops to 900 kb/s  rsync. Anybody know why the wifi wires are so thin? I'm thinking of soldering it back, but i'm worried about the thinness of the connections. Is there a better way?



Update: I soldiered it two lines today and full wireless strength now w00t!

Filed under: Uncategorized No Comments

Simple host-based firewall Ubuntu style

Tired of pesky port scans? Just want to deploy a firewall for the heck of it? Like that added layer of protection? Whatever your reason lets do a simple one. Here a version where it includes a persistent ruleset and integration with your Ubuntu startup.

For me personally I tend to deploy iptables firewall for simple host-based firewall, e.g. just blocking stuff and making sure no one tries too many things on your server. For something more complicated I prefer to use shorewall (thats another write up for another day :)

Show me the bacon

New Script: /etc/init.d/firewall (something simple for start stop and reload)
This is a pretty simple script all it needs to run is the actual firewall rules in "/etc/default/firewall".

#! /bin/sh
# Provides: Firewall
# Required-Start: $network
# Required-Stop:
# Default-Start:
# Default-Stop:
# Short-Description: Start/stop Firewall

test -r /etc/default/firewall || exit 0

case "$1" in
echo -n "Starting Firewall Services"
/sbin/iptables-restore < /etc/default/firewall
echo -n "Stopping Clearing Firewall Rules"
/sbin/iptables -F
/sbin/iptables -X
/sbin/iptables -t nat -F
/sbin/iptables -t nat -X
/sbin/iptables -t mangle -F
/sbin/iptables -t mangle -X
/sbin/iptables -P INPUT ACCEPT
/sbin/iptables -P FORWARD ACCEPT
/sbin/iptables -P OUTPUT ACCEPT
echo -n "Restarting Firewall Services"
/sbin/iptables-restore < /etc/default/firewall
echo "Usage: /etc/init.d/firewall {start|stop|restart|reload}"
exit 1

exit 0

The actual firewall rules, this one is up to you but here's a simple version allowing only DNS and WEB traffic on the inbound and everything on the outbound. Store it under "/etc/default/firewall"

# Generated by iptables-save on Mon Feb 7 19:05:46 2011
# Completed on Mon Feb 7 19:05:46 2011
# Generated by iptables-save on Mon Feb 7 19:05:46 2011
:INPUT ACCEPT [48:3929]
:OUTPUT ACCEPT [22:2312]
# Completed on Mon Feb 7 19:05:46 2011
# Generated by iptables-save on Mon Feb 7 19:05:46 2011
:INPUT DROP [2:513]
:FILTER - [0:0]
-A INPUT -i lo -j ACCEPT
-A INPUT -i eth0 -p icmp -j ACCEPT
-A INPUT -i eth0 -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -i eth0 -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -i eth0 -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -i eth0 -p tcp -m multiport --dports 80,443 -j ACCEPT
-A OUTPUT -o eth0 -j ACCEPT
# Completed on Mon Feb 7 19:05:46 2011

Now apply the varnish

# Make it executable
chmod +x /etc/init.d/firewall
# Add it to the startup levels
sudo update-rc.d firewall defaults

Kick the tire

# If you are at the console and aren't afraid of your firewall rules locking you out
sudo /etc/init.d/firewall reload
sudo iptables -L -n # check that its loaded
# If you are over a ssh session and you are afraid of being dropped, this way you only get dropped for 10 seconds if you stuff up :)
sudo /etc/init.d/firewall reload && sleep 10 && sudo /etc/init.d/firewall stop


Reverse DNS wall using DJBDNS

Another DNS post yay! Out of all aspects of DNS services this one is used least. So whats a reverse DNS wall? You know reverse DNS? thats where you look up an IP address and get the name associated as opposed to the other way round.

Now "some" special needs servers on the internet (FTP, SSH) like to do a reverse resolve of the incoming IP and block based upon the result. Some even rarer server would then would do a A record check on the reverse to make sure that matches the originating IP.

Lets say I have a whole bunch of computers that don't have DNS names, not now, not ever, how do they connect to these pesky services? Tell the service provide to stop be daft! Once you recover from the slap they gave you then setup our own reverse DNS wall. When queried these things will respond with the good stuff and take you to the promise land!

# Install
apt-get install daemontools daemontools-run ucspi-tcp djbdns
adduser --no-create-home --disabled-login --shell /bin/false walldns

# Config
walldns-conf walldns dnslog /etc/walldns
mkdir /etc/service ; cd /etc/service ; ln -sf /etc/walldns/

# Start and Test
initctl start svscan
# Checking status
svstat /etc/service/walldns
# Shutting down
svc -d /etc/service/walldns
# Starting up
svc -u /etc/service/walldns

Now get the upstream hosters of your IP address delegate the reverse zone to your server and you are good to go.


Useful Chrome Extensions

I thought I jot down a whole bunch of awesome chrome extensions which has made my life easy:

AdBlock - The grand daddy of all extensions :)

Chrome to Phone - If you have an Android phone this is a easy and painless way to send a link direct to your Android's browser

Instapaper - For those of you who read Instapaper, this takes all the hassle out of adding pages

1Password - Install this one through your 1Password's settings. This is a life saver, a single place for all my passwords and secret information, one click login to any website.

Official Facebook Extension - Enough said.

Ultimate Chrome Flag - I stumbled across this extension and boy is it a gem! It tells you lots of useful information about the website you are currently visiting like country, IP address, Google page rank, geo location and trust rating.

IP Address Information - Great extension for those of us that are interested in a bit of networking. Everything you want, DNS, Reverse DNS, ASN, Spam block lists you name it its just plain awesome.

Screen Capture (by Google) - take all those shiney pics of whats inside your tabs

Google Tasks (by Google) - Nice embedded way to access your tasks that sits with Google Calendar.

Google Calendar (by Google) - Nice way to see whats coming in your Google calendar. I found it a little too cumbersome and no quick way of getting to my actual calendar.

Google Calendar Checker (by Google) - Same as above but it acts as an instant jump to your calendar.

Google Translate (by Google) - If you view foreign language pages with any frequency this one will save you some hassle.

Google Share Button (by Google) - If you're a share-a-holic this is great :D

Do you have any favourite extensions? I would love to try it and put it on.