Moved transcoding process to Amazon EC2
September 26th, 2010
No comments
It was quite easy to move the transcoding process for the railscasts podcast to an EC2 instance. I installed an ubuntu instance similar to the vm I used before, and added a cronjob on my home pc to automatically start, stop the instance and run the transcoding process via ssh.
Most of the work is done through the Amazon APIs.
#!/usr/bin/env ruby
# simply run job on ec2 instance
#
# Author: gschaden at gmail dot com
# Date: August 2010
# Version: 1.0
#
# connect to amazon ec2 and start an existing instance
# after the instance has booted, run commands via ssh (public key, authentication)
# then stop the instance
# if the job takes to long, it is aborted
#
# This script requires following gems
# amazon-ec2
# net-ssh
# system_timer
#
require 'rubygems'
require 'AWS'
require 'net/ssh'
require 'system_timer'
AWS_ACCESS_KEY_ID = 'PUT KEY HERE'
AWS_SECRET_ACCESS_KEY = 'PUT SECRET KEY HERE'
# path to pem file, e.g /home/gschaden/.ssh/ec2.pem
SSH_KEY = 'PATH TO SSH KEY'
# command to execute
REMOTE_CMD = 'transcode.sh'
# instance id, eg. i-3dfb8edf
INSTANCE_ID="PUT INSTANCE ID HERE"
JOB_TIMEOUT=10*60 #seconds
exit_code = 0
begin
conn = AWS::EC2::Base.new(:access_key_id => AWS_ACCESS_KEY_ID, :secret_access_key => AWS_SECRET_ACCESS_KEY)
# start your instance
puts "----- start instance -----"
conn.start_instances(:instance_id => [INSTANCE_ID])
# wait for your instance to start up
puts "----- listing instances -----"
while true do
instances=conn.describe_instances(:instance_id => [INSTANCE_ID])
state = instances.reservationSet.item[0].instancesSet.item[0].instanceState.name
break if state == "running"
puts "waiting"
sleep 1
end
ip_address=instances.reservationSet.item[0].instancesSet.item[0].ipAddress
sleep 10
puts "----- running ssh #{ip_address} ------"
SystemTimer.timeout(JOB_TIMEOUT) do
begin
Net::SSH.start( ip_address, 'ubuntu', :keys => [SSH_KEY] ) do |session|
puts session.exec!(REMOTE_CMD)
end
rescue Timeout::Error
puts "Job timeout"
exit_code = 1
rescue SocketError
puts "Could not connect to your instance"
exit_code = 1
end
end
puts "----- stop instance -----"
puts conn.stop_instances(:instance_id => [INSTANCE_ID])
puts "----- listing instances -----"
while true do
instances=conn.describe_instances(:instance_id => [INSTANCE_ID])
state = instances.reservationSet.item[0].instancesSet.item[0].instanceState.name
break if state == "stopped"
puts "waiting"
sleep 1
end
rescue AWS::InvalidInstanceIDMalformed
puts "Invalid instance id"
exit_code = 1
rescue AWS::AuthFailure
puts "Invalid credential, check access access_key_id and secret_access_key"
exit_code = 1
rescue SocketError
puts "Could not connect to EC2: ",$!
exit_code = 1
end
exit exit_code
