Category — Development

A new Debian development server needed trac installed. I would rather use the new trac .11, but etch comes with .10 as a package so we used that.

The install is not that big of a deal. The biggest problem was that I upgraded to python 2.5 instead of stock 2.4 for debian, so clearsilver fails it - it does not know about 2.5 in it’s make files.

Following the advice here:

https://launchpad.net/ubuntu/+source/clearsilver/+bug/86685

I installed the newest clearsilver, and it stopped whinging about neo-cgi. But then it couldnt find the svn bindings because they are in python 2.4 site-packages. That is my one beef with python - I’m sure it’s site-packages is a good idea, but I have had a hard time wrapping my head around them pretty much anytime I have to deal with a Python stack.

Oh well. On this devel box, python 2.4 is fine. I’ll unlink python 2.5 and let it rot. The clearsilver hack was probably not needed anyway, and worrying about this anymore takes away from time actually using Trac.

Project had a model without a timestamp field, but we needed to figure out (roughly) how old each one was. They needed a report of their age to make them pay up - the model was for paid advertisers.

Best fit was to find when their logo was last updated. That day will be at least since they were entered online, and in general pretty close to the original date.

Advertiser.find(:all).each do |a|
if a.logo
puts "#{a.market.name} - #{a.name}:" + File.mtime(a.logo.full_filename).to_s
end
end

Capistrano 2.1 did not agree with me. I am a huge Capistrano fan, but the newest version just doesn’t do it. My install of version 1 isn’t broken, I’m not going to fix it.

Version 2.1 has 37 signals only code in it and doesnt work with mongrel_cluster 1.0.3. palmtree 0.0.6 will give it a new mongrel cluster recipe, but why do you have to use that too? Not really interested in installing another gem to get Capistrano working like it did before.

Project need a screenscraper to pull out the google maps portion of a page and embed on our own.

Problem is Radiant cms likes to take over all your urls, so how to make a custom ruby page that lives within a Radiant powered site?

Radiant by default is a systemwide gem that would prefer you didnt do things like this, and going to the trouble of adding a custom plugin to manage one page was overkill. I don’t remember if there was a static content plugin - I think I found one, but it wasn’t able to run custom Ruby code.

What we needed was really really simple, no edibility or site layout on the page so here’s what I made.

Here we go:

* start by adding the app/controllers and app/views folders.
* add in the controller you want with these specifics:

skip_before_filter :authenticate

up_path = File.join(File.dirname(__FILE__), '..')
render_file up_path + '/views/location/index.rhtml' and return true


The skip filter removes Radiant’s checking for admin, and the render_file explicitly loads the correct template. Radiant is using the system gem path for views, so it’s necessary to run relative to it’s install.

* routes.rb hacking. at the bottom:
builder = ActionController::Routing::RouteBuilder.new
route = builder.build('YOURURL', {:controller => 'NEWCONTROLLER'})
custom_routes = Array.new
custom_routes << route

# put it in first place
ActionController::Routing::Routes.routes = custom_routes + ActionController::Routing::Routes.routes

Pretty cool! The new url as just a controller will exist on the Radiant powered domain, but not barf over Radiant. I tried this with 0.6.4. YMMV

The actual screenscraping was done hpricot/open-uri, and was pretty straightforward.

http://www.igvita.com/blog/2007/02/04/ruby-screen-scraper-in-60-seconds/
http://weblog.jamisbuck.org/2006/10/2/under-the-hood-rails-routing-dsl

The end result was a big hack, but I got in running in about an hour.

sudo gem upgrade radiant
Bulk updating Gem source index for: http://gems.rubyforge.org
Killed

Turns out the server was out of ram (!) vmstat showed 150MB free, but the bulk update used too much ram and the kernel killed it. Wow.

To fix it, I dropped the production site down to 8 mongrels instead of 20. vmstat shows 500+MB free, update worked.

The original plan to have 20 mongrels was when that project powered the static content too, but now radiant manages that (with it’s own 4 mongrels). With site traffic being much lower than expected, and 20 being too much, dropping to 8 seemed right.

Chalk this win up to Debian being stable enough to run a few months with a Sarge/Etch kernel/userland mismatch.

What happened was one server that got ordered right around the Etch release would run Etch fine, but the hosting company still had to officially run Sarge because of the RAID controller. The upgrades were left to us, and all they would restore to is Sarge.

Anyway, cut to later this year - I noticed something wonky when aptitide was installing linux-image-2.6.18-*, but uname said I was running sarge’s 2.6.8 kernel. The real fun was that the uptime was 155 days on the old Sarge based system. What had happened was GRUB never got updated, so it kept using the old kernel.

travis@debian/etc $ cat debian_version
4.0

travis@debian~ $ uname -mrs
Linux debian 2.6.18-5-k7 #1 SMP Wed Sep 26 18:29:20 UTC 2007 i686 GNU/Linux


Awesome. I was not happy to think about what would have happened if it didn’t come back online. I

If we had console access (which we didn’t at that host), I’m sure GRUB could have booted the old kernel from that, and then fix whatever happened. I have a feeling their would have been some something wrong rebooting a Sarge kernel into an Etch userland though. Would make an interesting experiment to see if it’s possible, or if some libc error botches it all.

rake test:plugins PLUGIN=attachment_fu TESTOPTS=”-v”

attachment_fu’s tests hate me, or at least hate my MacBook.

Problems:
1. always whines about s3 even if i don’t want to use it
2. ImageWithThumbsClassFileAttachment fails it - says full_filename doesn’t exist
3. test_should_create_file_from_uploaded_file is reading a text file as a png
4. makes a sqlite database file in my root project directoy!
5. gives a weird error: unknown IO error whilst running rake

1 and 2 are fixable. http://beast.caboo.se/forums/2/topics/1074

3: I really did not understand this test failing. Opening a text file, checking if nil height - okay. Maybe a bug in my processing since it returns 612px width! Who knows.

4. Can be fixed by hardcoded sqlite to use an in memory database, but I could not get erb to run in the yaml file to give it a dynamic location. Doh!

5: This was a real head scratcher:

class MinimalAttachment < ActiveRecord::Base
has_attachment :path_prefix => ‘vendor/plugins/attachment_fu/test/files’, :processor => :rmagick
validates_as_attachment

def filename
“#{id}_file”
end
end

That fixes it. For some reason Tempfile gives an unknown IO error if the tempfile ends in .file

Can verify by editing attachment_fu.rb:

def random_tempfile_filename
"#{rand Time.now.to_i}#{filename || 'attachment'}.file"
end

and then get the errors. What?? I dug around some but just didn’t have time to really fix that.

I figure as long as I understand why the tests fails I’m okay. The only issue I found with any changes I made was using backward compatibility :file_system_path. I don’t use that.

2008 Update:
Rick does seem to be updating the code base to include these fixes. http://pastie.textmate.org/67051 Yay.

The project needed a minimum payload to sent over HTTP to test the VOIP service with. I ended up boiling it down to:

'username' => username,
'password' => password,
'alarm_type' => 'immediate',
'tts_or_wav' => 'tts',
'phone_recips' => '5555555',
'tts_msg' => 'this is an automated test from a ruby script. so cool.',
'deliv_type' => 'phonecall'


Sure enough, the creepy text to speech system called me up. This made testing this project pretty cool. For full testing it will send sample request to the VOIP server, and my phone rings a few seconds later.

The full extent of that project was to play an arbitrary WAV file to the receiving phone number, so there were some extra tests involving verifying the WAV, etc. If I were really lazy, I would set up a speech recognition engine over a VOIP receiver and match the test WAV to what comes in over the phone, and make sure it’s understandable.

It would be one of the strangest rake tasks I can think of.

Project required periodic posting to HTTPS via http-access/httpclient.

The first issue was how to set the certficate to use. Safari does not export SSL certificates in a way useful to openssl. The .cer or .cert files give an exception of “OpenSSL::X509::StoreError”

To track that down I had to dig around the curl docs of all places - they have better documentation than http-access.

The way to do this is open with OS X keychain, export as .pem (base64 encoded version), and save .pem file in the config dir of rails.

This led to the next problem - the HTTPS server had a bum certificate, it was from a different host. I think their setup was done on the cheap. http-access kept failing with OpenSSL::SSL::SSLError - hostname not match.

The way to get around this is to set verify_mode = nil


require 'httpclient'
client = HTTPClient.new()
client.ssl_config.verify_mode = nil


http://ghouston.blogspot.com/2006/03/using-ssl-with-ruby-http-access2.html
http://curl.haxx.se/docs/faq.html#What_certificates_do_I_need_when
http://dev.ctor.org/svn/http-access2/trunk/lib/httpclient.rb
http://groups.google.co.uk/group/soap4r/browse_thread/thread/872f2f5c61ce20c6

rails 1.2 app was using a db backed session store and was kind of “neglected”:


cupid_production=> select count(*) from sessions;
count
--------
380754
(1 row)


http://www.realityforge.org/articles/2006/03/01/removing-stale-rails-sessions
http://robsanheim.com/2007/04/
http://groups.google.com/group/melbourne-ruby/browse_thread/thread/c5eecde3fb6a0b08

destroy_all is un-needed since we don’t have callbacks on the session table. use delete_all instead.

save into lib/tasks as session_cleanup.rake


desc "Clear database-stored sessions older than a day old"
task :session_cleanup => :environment do
CGI::Session::ActiveRecordStore::Session.delete_all ["updated_at < ?", 1.day.ago ]
end


setup to run daily via rake/cron


30 1 * * * cd /var/www/appname/current && rake session_cleanup RAILS_ENV=production