Apr 23, 2010

Nested has_many: through in Rails (or how to do a 3 table join)

Today I came across a problem that I would have expected would work in Rails (2.3.5), but doesn’t. I wanted to use a has_many :through relationship on a association that was itself a has_many :through.

Here is an example. Let’s say I have Group class, which has many users through memberships. Further, a user has many comments. What I want is a simple way to get all the comments that were made by users in a group.


class Group < ActiveRecord::Base
  has_many :memberships
  has_many :users, :through => :memberships

  # This is what I would like to do, but this does not work!
  # has_many :comments, :through => :users
end

class User < ActiveRecord::Base
  has_many :comments
  has_many :memberships
  has_many :groups, :through => :memberships
end

class Comment <  ActiveRecord::Base
  belongs_to :user
  has_many :likes

  named_scope :approved, :conditions => {:approved => true}
end

What I want is to be able to call


  @group = Group.find(params[:id])
  @comments = @group.comments.approved.find(:all, :include => :likes)

Option #1 : Includes : use includes when loading the group so I can eager load the comments. Then I can collect them:


  @group = Group.find(params[:id], :include => {:users, :comments})
  @comments = @group.users.map{|u| u.comments}.flatten

Problems with Option #1 : First, it is inefficient. If I don’t use the user objects anywhere, I’m loading them for nothing. Second, I can’t use pagination or other filters easily on the comments association.


Option #2 : find_by_sql : use a method on Group to load up comments, like so:


class Group < ActiveRecord::Base
  has_many :memberships
  has_many :users, :through => :memberships

  # This is what I would like to do, but this does not work!
  # has_many :comments, :through => :users

  def comments
    # Use a 3 table sql join to load the comments for all users in this group.
    Comment.find_by_sql("
             SELECT c.* FROM comments c
               INNER JOIN users u ON u.id = c.user_id
                 INNER JOIN memberships m ON m.user_id = u.id
             WHERE m.group_id = #{id}")
  end
end

Problems with Option #2 : You can’t do eager loading, pagination, or use any named scopes on the comments class. So, if I wanted to load on ‘approved’ comments I’d have to write another method. boo.


Option #3 : named_scope + method : use a named_scope on Comment and a method on Group so that i can make the calls that I want to…


class Group < ActiveRecord::Base
  has_many :memberships
  has_many :users, :through => :memberships

  # This is what I would like to do, but this does not work!
  # has_many :comments, :through => :users

  # This lets us call the code in a nice looking way:
  # e.g. group.comments.approved
  def comments
    Comment.all_for_group(self)
  end
end

class Comment <  ActiveRecord::Base
  belongs_to :user
  has_many :likes

  named_scope :approved, :conditions => {:approved => true}

  # perform the 3 table join in a way that will
  # let us also call include and other filters.
  named_scope :all_for_group, lambda{ |group|
      {
        :joins      => {:users, :memberships},
        :conditions => {:memberships => {:group_id => group.id},
        :select     => "DISTINCT `comments`.*"
      }
    }
end

Option #3 is definitely the cleanest, and what I would recommend.

I’m not sure if Rails 3 supports nested has_many :through, but 2.3.5 does not. There is a (very old) ticket for Rails. There is also a nested has_many :through plugin that is has an experimental branch for 2.3.x. I don’t like using things that are ‘experimental’.

The number of times you need to do multiple has_many :through associations should be fairly small. If you are doing it a lot, you should probably reconsider your data model. In most cases, a simple named_scope and method ought to do the trick for you like it did for me.

Mar 22, 2010

IE Error : File Download on Rails Form Post

If you are using Rails, you many notice that some form posts from IE (6/7) result in IE asking you where you want to save a file, even if your response is supposed to be a redirect. It took me a while to figure out what was happening here so thought I’d share it and save someone else the trouble.

It seems that IE doesn’t always send the correct accepts headers for a post. Rails (v2.3.5 at least) if it can’t find a matching format in a responds_to block, will just render the first response.

So, make sure you always order your responds_to blocks so that html is first.

def action_name
  # my action code
  respond_to do |format|
    # Always make sure HTML is first otherwise
    # you'll send a js response to IE!
    format.html { redirect_to '/' }
    format.js { }
  end
end

Otherwise, when you submit a form, Rails will render the js result, which IE sees as a file download.

Mar 27, 2009

Setting a Capistrano Variable from the Command Line

It took me a little while to find a solution for this, so I thought I’d post it.

I was cleaning some deployment dirs and wanted, just for this instance, to only leave 1 release as opposed to the 5 releases that capistrano leaves by default. Keep in mind this was across about 6 apps and 2 stages for each.

Option 1: Add the following to the deploy.rb files:


set :keep_releases, 1

That would require changing them all back afterwards.

Options 2: Set the capistrano variable from the command line:


cap deploy:cleanup -s keep_releases=1
Feb 26, 2009

Domain Name Registration API Plugin for Rails

If you’ve ever had an app where you want to allow users to purchase a domain name, you’ve probably felt the pain of trying to interface to a registrar.  Although some have APIs, my search found that most were hard to interface to or poorly documented.  Many even required signing up as a partner (and paying a big fee) before you could even get documenation.

After much searching and experimenting I decided to go with Register.com’s XML api for my app. They offered the best API, and the easiest signup path.

I bundled the main part of the interface into a rails plugin.  The plugin is stored on github: http://github.com/geoffevason/register-api/tree/master

To install the plugin do this:


script/plugin install git://github.com/geoffevason/register-api.git

The plugin is of little value unless you have spoken to register.com and have received their API documentation. You need to register as a partner (it’s free) and have the IP of your dev machine whitelisted for testing.

Most of the info on use is in the readme in the plugin. You can call any of the Register.com API methods by calling Register::API.


# A call to the API looks like this
# Register::Api::Call(params)

# Example to check if the domain name google.com is available
Register::Api::Check(:tld => 'com', :sld => 'google')

The plugin also contains a few helper methods and classes. If all you want to do is let people search for an available domain, and purchase it, then everything you need is in these helpers. Some important logic remains in my controllers, but if you have any questions, let me know.

Oct 27, 2008

Simple Currency Conversion Rate API Consumption For Ruby / Rails

I had the need to consume some exchange rate data for an internal project so I began looking for an about. My searching found no api for Google Finance, Yahoo Finance, XE, or Oanda. :-(

Fortunately I found the currency converter from Xavier Media and their simple currency exchange rate xml API, which includes historical data too.

I put together an absolutely minimal lib to get the data I need.  I just needed the AU/US rate.  The xml provides all data as base to EUR, but with some simple math I can find the rate I need with reasonable accuracy.  In this case ‘accuracy’ is based on spot checking it against the yahoo rates.

I thought it worth sharing in case others are looking for something similar.

UPDATE: I changed the xml string below to better handle single digit date months. Xavier needs '01' instead of '1'

require "cgi"
require "uri"
require "net/https"
require "rexml/document"

module XavierMedia
  # Returns the exchange rate (AUD/USD) on the given date.
  def self.exchange_rate_on(date)
    url = URI.parse("http://api.finance.xaviermedia.com/api/#{date.year}/#{date.strftime("%m")}/#{date.strftime("%d")}.xml")

    resp = Net::HTTP.get(url)
    xml  = REXML::Document.new(resp)

    us_to_eur = 1.0
    au_to_eur = 1.0
    xml.elements.each("//exchange_rates/fx") { |el|
      if el.elements[1].text == "USD"
        us_to_eur = el.elements[2].text.to_f rescue 1.0
      end
      if el.elements[1].text == "AUD"
        au_to_eur = el.elements[2].text.to_f rescue 1.0
      end
    }

    return us_to_eur/au_to_eur
  end
end

 

Oct 26, 2008

Rails Fragment Caching Slowness With Regex Expiry

Fragment caching in rails works great.  We use it a lot for MomentVille and easily get most of our responses in under 150ms because it dramatically reduces teh number of queries we need to run for our most common actions.

I did notice some slowness recently though, and was quite confused.  The slowness only occurred on our production servers. Our dev, test, and pre-production servers were all still fast.  I tried using new relic rpm service to help pinpoint the problem, and while it does a great job in helping you track things, it didn’t help me narrow down the problem.

Ultimately I discovered that the problem had to do with how we clear the cache. For some actions we have to expire multiple fragments, so we used regex expiration.  Unfortunately, that is very slow.  It seems that regex expiry compares your regex to each fragment stored, even if you think your regex looks like it’s targeting a directory.

Alexander Dymo had a post about regex expiry of fragment caches in rails that outlined a solution for him.  It helped guide me to a solution that works well for us.  OUr fragment caches are actually structured around the data as opposed to the actions, so related fragments are stored within sub directories.  When we need to clear a bunch at once, we just want to wipe out the whole directory.  So, I created a file called fragment_dir_expiration.rb and put it into my /config/initializers folder.  It looks like this:

# For rails 2.0 and lower
module ActionController
  module Caching
    module Fragments

      #dir is the cache path relative to the cache root
      def expire_fragment_dir(dir, options = nil)
        return unless perform_caching
        self.class.benchmark("Expired fragments in dir : #{dir}") do
          ActionController::Base.cache_store.delete_fragment_dir(dir, options)
        end
      end

      class UnthreadedFileStore

        def delete_fragment_dir(dir, options = nil)
          path = @cache_path + dir
          return unless File.exist?(path) #it's ok to not have the cache dir
          search_dir(path) do |f|
            begin
              File.delete(f)
            rescue SystemCallError => e
              # If there's no cache, then there's nothing to complain about
            end
          end
        end

      end
    end
  end
end

When you want to call this you can call it from an observer with a call like this

class WidgetSweeper < ActionController::Caching::Sweeper
  observe Widget
  def after_save(widget)
    # Expire all the fragments for the updated widget
    expire_fragment_dir("/widget/#{widget.id}/")
  end
end

UPDATE: For rails 2.1 and above use the following:

module ActionController
  module Caching
    module Fragments

      #dir is the cache path relative to the cache root
      def expire_fragment_dir(dir, options = nil)
        return unless perform_caching
        self.class.benchmark("Expired fragments in dir : #{dir}") do
          ActionController::Base.cache_store.delete_fragment_dir(dir, options)
        end
      end

    end
  end
end

module ActiveSupport
  module Cache
    class FileStore

      def delete_fragment_dir(dir, options = nil)
        path = @cache_path + dir
        return unless File.exist?(path) #it's ok to not have the cache dir
        search_dir(path) do |f|
          begin
            File.delete(f)
          rescue SystemCallError => e
            # If there's no cache, then there's nothing to complain about
          end
        end
      end
    end
  end
end