Gravatar

Geoff Evason

Speeding up Slow Specs and Features on Ubuntu 10 with Postgres

I’ve recently been working on a Rails 2.3.10 project using Ubuntu 10 and Postgres 8.3 on my dev machine. The specs and features were running really slow (>30min) and I wanted to speed them up.

After some profiling and searching I found 2 quick fixes.

1. Tell postgres not to wait for confirmation that changes are committed to disk:

Add the following lines to your postgres configuration (/etc/postgresql/8.3/main/postgresql.conf) and then restart postgres. These have a very noticable affect on Ubuntu 10 using the ext4 filesystem. They tell postgres not to wait for confirmation from the filesystem that changes have been committed to disk, which results in table truncates/deletes being faster but also increases the chance of a corrupt db if your machine crashes. (that shouldn’t matter for dev/test)

  fsync = off
  synchronous_commit = off

2 – Use Ruby Enterprise Edition

If you haven’t seen the ‘Grease your Suite’ presention – you should watch it. If you are using rvm and can use REE on your project, you should do so. It is a lot faster (for my tests at least). It reduced my test time by 40%.

If you do use REE – add the following to your ~/.bashrc file

  export RUBY_HEAP_MIN_SLOTS=1000000
  export RUBY_HEAP_SLOTS_INCREMENT=1000000
  export RUBY_HEAP_SLOTS_GROWTH_FACTOR=1
  export RUBY_GC_MALLOC_LIMIT=1000000000
  export RUBY_HEAP_FREE_MIN=500000

These exports optimize memory allocation and garbage collection for REE.

After these 2 changes the entire test suite took 9min. That is still pretty long, but much better than >30min.

Testing Illegal UTF-8 Characters in Rails

I wanted to make a User model given an error it the password contained an illegal UTF8 character.

Here is the spec


it 'should give an error on illegal characters' do
  # The hex code FF is illegal as a UTF8 character
  illegal_utf8_string = ("pass%c" %0xff)  

  user = User.new(:email => 'test@test.com',
      :password => illegal_utf8_string,
      :password_confirmation => illegal_utf8_string)
  user.should_not be_valid
  user.errors_on(:password).should include("has invalid characters.")
end

Here is the implementation


def validate
  if @password.toutf8 != @password
    errors.add(:password, "has invalid characters.")
  end
end

Cross domain workaround for @font-face and Firefox

All major browsers (even ie4+) now support the @font-face css property, which is great news for designers.  Unfortunately there are still some kinks:

  • Different browsers support different formats.  You can specify multiple formats within a css file, and as long as you can provide .eot, .ttf, and .svg you’ll be okay.
  • Firefox (which supports @font-face from v3.5) does not allow cross-domain fonts by default.  This means the font must be served up from the same domain (and sub-domain) unless you can add an “Access-Control-Allow-Origin” header to the font.
  • At the time of writing, you cannot set the Access-Control-Allow-Origin” header on S3

I wanted to use @font-face served up from cloudfront – so here is what I did:

  1. Go to fontsquirrel.com and download the font-face kit you want to use.
  2. Go to the fontsquirrel font-face generator and upload the .ttf file from the kit you just downloaded.
  3. Select the ‘Expert’ option.
  4. For format select ‘TrueType’, ‘EOT’, and ‘SVG’.  (Woff is a compressed format only supported by firefox, but firefox also support .ttf, so woff is extraneous)
  5. Under the CSS options select ‘Base64′ Encoding.
  6. Download and use the files provided.

What does this do?  It actually embeds the TTF font within the CSS file, so it can be served up directly from S3/cloudfront and still work on Firefox.  This solution works, but is sub-optimal.  In particular, it bloats your css for other browsers (like mobile safari and IE) that can’t use the TTF format. :-(

Awkward ad after earthquake in Chili

I was cleaning through my files and ran across this funny screen capture I took shortly after the earthquake in Chili.

I was looking at CNN to check out the latest news and couldn’t help but laugh at the awkward video ad that was in rotation. Have a look for yourself.

Nested has_many: through in Rails (or how to do a 3 table join)

Today I came across a problem that I would have expected would work in Rails (2.3.5), but doesn’t. I wanted to use a has_many :through relationship on a association that was itself a has_many :through.

Here is an example. Let’s say I have Group class, which has many users through memberships. Further, a user has many comments. What I want is a simple way to get all the comments that were made by users in a group.


class Group < ActiveRecord::Base
  has_many :memberships
  has_many :users, :through => :memberships

  # This is what I would like to do, but this does not work!
  # has_many :comments, :through => :users
end

class User < ActiveRecord::Base
  has_many :comments
  has_many :memberships
  has_many :groups, :through => :memberships
end

class Comment <  ActiveRecord::Base
  belongs_to :user
  has_many :likes

  named_scope :approved, :conditions => {:approved => true}
end

What I want is to be able to call


  @group = Group.find(params[:id])
  @comments = @group.comments.approved.find(:all, :include => :likes)

Option #1 : Includes : use includes when loading the group so I can eager load the comments. Then I can collect them:


  @group = Group.find(params[:id], :include => {:users, :comments})
  @comments = @group.users.map{|u| u.comments}.flatten

Problems with Option #1 : First, it is inefficient. If I don’t use the user objects anywhere, I’m loading them for nothing. Second, I can’t use pagination or other filters easily on the comments association.


Option #2 : find_by_sql : use a method on Group to load up comments, like so:


class Group < ActiveRecord::Base
  has_many :memberships
  has_many :users, :through => :memberships

  # This is what I would like to do, but this does not work!
  # has_many :comments, :through => :users

  def comments
    # Use a 3 table sql join to load the comments for all users in this group.
    Comment.find_by_sql("
             SELECT c.* FROM comments c
               INNER JOIN users u ON u.id = c.user_id
                 INNER JOIN memberships m ON m.user_id = u.id
             WHERE m.group_id = #{id}")
  end
end

Problems with Option #2 : You can’t do eager loading, pagination, or use any named scopes on the comments class. So, if I wanted to load on ‘approved’ comments I’d have to write another method. boo.


Option #3 : named_scope + method : use a named_scope on Comment and a method on Group so that i can make the calls that I want to…


class Group < ActiveRecord::Base
  has_many :memberships
  has_many :users, :through => :memberships

  # This is what I would like to do, but this does not work!
  # has_many :comments, :through => :users

  # This lets us call the code in a nice looking way:
  # e.g. group.comments.approved
  def comments
    Comment.all_for_group(self)
  end
end

class Comment <  ActiveRecord::Base
  belongs_to :user
  has_many :likes

  named_scope :approved, :conditions => {:approved => true}

  # perform the 3 table join in a way that will
  # let us also call include and other filters.
  named_scope :all_for_group, lambda{ |group|
      {
        :joins      => {:users, :memberships},
        :conditions => {:memberships => {:group_id => group.id},
        :select     => "DISTINCT `comments`.*"
      }
    }
end

Option #3 is definitely the cleanest, and what I would recommend.

I’m not sure if Rails 3 supports nested has_many :through, but 2.3.5 does not. There is a (very old) ticket for Rails. There is also a nested has_many :through plugin that is has an experimental branch for 2.3.x. I don’t like using things that are ‘experimental’.

The number of times you need to do multiple has_many :through associations should be fairly small. If you are doing it a lot, you should probably reconsider your data model. In most cases, a simple named_scope and method ought to do the trick for you like it did for me.

IE Error : File Download on Rails Form Post

If you are using Rails, you many notice that some form posts from IE (6/7) result in IE asking you where you want to save a file, even if your response is supposed to be a redirect. It took me a while to figure out what was happening here so thought I’d share it and save someone else the trouble.

It seems that IE doesn’t always send the correct accepts headers for a post. Rails (v2.3.5 at least) if it can’t find a matching format in a responds_to block, will just render the first response.

So, make sure you always order your responds_to blocks so that html is first.

def action
  # my action code
  respond_to do |format|
    # Always make sure HTML is first otherwise
    # you'll send a js response to IE!
    format.html { redirect_to '/' }
    format.js { }
  end
end

Otherwise, when you submit a form, Rails will render the js result, which IE sees as a file download.

Wedding Pictures and Social Wedding Planning

We’ve been pretty busy recently working on some great new wedding related features. In particular we launched wedding pictures and wedding forums.

We’re pretty excited about this launch. We did a lot of really great things in the code, so that users can comment/like/save pretty much any of the wedding pictures, or forum posts, and it’s all extremely DRY and RESTful.

We may try to extract some plugins from the latest launch… More to follow.

Cool new wedding planning tools

We had our biggest launch yesterday since starting our wedding website a couple years ago.  If you are planning a wedding, check out the new tools, including a wedding checklist, wedding budget and wedding vendor search and comparison.

We’re pretty excited about these new features – and hope they’ll help a lot of people plan their weddings (or other big events)

Set Cookie in HTML Body for Rails 2.3.2

I recently upgraded an app to rails 2.3 and on many pages on my local machine the top of the HTML body included something like this:


Set-Cookie: _myapp_session_id=BAh7B...; path=/; HttpOnly

I couldn’t figure out what was happening until I found this forum post which explained that the problem was related to passenger 2.0.6. To fix the problem I just needed to update the version of passenger I was running.

To update passenger (on a mac) was pretty simple. Just run the following 2 commands:


sudo gem install passenger
sudo passenger-install-apache2-module

Depending on how you set up your local apache instance, you may also need to update the config. I needed to update /etc/apache2/httpd.conf to


LoadModule passenger_module /Library/Ruby/Gems/1.8/gems/passenger-2.1.2/ext/apache2/mod_passenger.so
PassengerRoot /Library/Ruby/Gems/1.8/gems/passenger-2.1.2

Swiff Uploader argumentsToXML Error

I was working on getting the Swiff uploader working with a wedding vendors application and was getting the following javascript error:


__flash__argumentsToXML is not defined

After banging my head for a while (the exact same code was working fine on another website) I discovered the problem. It was a really silly oversight on my part. I didn’t have the file Swiff.Uploader.swf in the right location!!!

Make sure that when create a swiffy object you have a the above swiff file where you set it in the path:


    $('upload-fallback').removeClass('hide');
  	var swiffy = new Fancyuploader2($('upload-status'), $('upload-list'), {
  	...
        /* This is the important bit re: this blog post */
  		'path': '/swf/Swiff.Uploader.swf',
  	...
  		'target': 'upload-browse-images', /* This is for flash 10 */
  		'onLoad': function() {
  			$('upload-status').removeClass('hide');
  			$('upload-fallback').destroy();
  		}

  	});