Using RSpec with BackgrounDRb Workers

Posted by david

A Rails app I'm working on performs some expensive operations that should be offloaded to another process, so I'm using this as an opportunity to try out BackgrounDRb. Because I'm doing BDD with RSpec, my first instinct, after installing and generating a worker, was to write a spec for my worker. However, Googling for the best way to do this turned up nothing, so I'm posting what I did. If you have a better way to do this, I'd love to hear it. If not, I hope this saves someone some time.

First, I created a new directory under my spec directory:

svn mkdir spec/workers

Then, I wrote the following in a file called collecting_worker_spec.rb in the newly-created workers directory (my worker is called CollectingWorker)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
 
require File.dirname(__FILE__) + '/../spec_helper'

describe CollectingWorker, "with feeds needing collection" do
  
  before(:each) do
    @worker = CollectingWorker.new
  end       
  
  it "should pull a single feed" do                        
    mock_collector = returning mock('collector') do |m|
      m.should_receive(:collect)
    end
    Collector.should_receive(:pop).and_return(mock_collector)
    @worker.do_work(true)
  end
end

For this spec to run correctly, though, I needed to add some code to my spec_helper.rb. This isn't pretty, but it is working for me:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

module BackgrounDRb
  module Worker
    class RailsBase 
      def self.register; end
    end
  end
end 

worker_path = File.dirname(__FILE__) + "/../lib/workers"
spec_files = Dir.entries(worker_path).select {|x| /\.rb\z/ =~ x}
spec_files -= [ File.basename(__FILE__) ]
spec_files.each do |path|
  require(File.join(worker_path, path))
end

All of the necessary RailsBase methods get mocked or stubbed in the spec itself. The class declaration is there so the necessary constants can be found when they're needed. There's probably a better place for that declaration, but the fact that so little code is needed for me to begin speccing my workers is a testament to the power of RSpec and its mocking facilities.

Development Database Maintenance

Posted by david

When working with a software development team, you typically want each developer to have their own database schema. That way, each developer is free to modify their schema as part of their development without getting in the way of other developers. When doing this, however, it's crucial to have the ability to trivially do the following:

  1. Rebuild the database from scratch
  2. Distribute changes to the schema

(1) is important because it's possible during development to put your database into a bad state that's difficult to back out of. If you can easily rebuild the database, you don't need to waste time figuring out how to back out changes that you've decided against.

(2) is important to keep the developers in synch. If developers need to rebuild their database to incorporate every schema change, they will do so less often than if they can run a script that brings their database up to date with the lasted schema in version control. And more importantly, by doing (2) as part of your development process, you have a way to test the changes that will ultimately be applied to your production database.

I don't if these are obvious, but the majority of projects I've seen do not have these processes in place. While I would place these processes among those that the best teams are following (such as automated deployments, continuous integration, test-driven development, etc.) they are not talked about as much among programmers.

The main thing that drew me to Ruby on Rails was that it incorporates so many good development practices - not only making them possible to use, but encouraging you to use them. Databases are no exception. Rails's method of defining database schemas in terms of migrations is the best way to accomplish practice (2) that I've seen. It also makes rebuilding the schema from scratch very easy, going a long way to accomplish practice (1).

However, rebuilding the database from migrations gives you an empty database. For your test database, this is what you probably want (using fixtures to populate the tables). However, you probably do not want an empty development database. Among other things, user interface problems (both bugs and usability issues) that are obvious when lots of data is on the screen may remain hidden when the only data present is what the developer has created in the process of trying out their own changes.

If you have a demo environment, used either by your sales force or as part of communicating with your customers, it's helpful for your development database to be populated with the same data as your demo database. Since your demo environment, by definition, is used to show off features of your product, it can also give you good data to work with during development. And by working with the data that will be used to demo your product, you may be able to avoid some nasty surprises during demo.

One way to copy data from your demo environment to your development enviroment is to just copy the database, DDL and all. However, this approach does not play well with migrations. Fortunately, if you're working with Rails, you have Ruby, Rails, and Rake to help you out. I use the following code to accomplish this task. Just copy it into a Rake file in your tasks directory and create a directory in the top level of your project called "gold" and another within that called "data". Then, running "rake gold:export" will pull all the data from within your database into a bunch of YAML files, one for each table. These files are structured like test fixtures. You can import the data using "rake gold:import". This approach has the advantage that your development/demo data can easily be stored in source control.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34


  namespace :gold do

    task :export do
      require RAILS_ROOT + '/config/environment'
      conn = ActiveRecord::Base.connection
      tables = conn.tables.reject {|i| i == 'schema_info'}
      tables.each do |table|
        filename = RAILS_ROOT + '/gold/data/' + table.pluralize + '.yml'
        open(filename, 'w') do |f|
          rows = conn.select_all("SELECT * FROM #{table}")
          rows.each do |row|
            f.puts("gold_#{row["id"]}:")
            row.keys.each do |key|
              f.puts "  #{key}: #{row[key]}"
            end
          end
        end
      end

    end

    task :import do
      require RAILS_ROOT + '/config/environment'
      require 'test_help'
      Dir.glob(RAILS_ROOT + '/gold/data/*.yml').each do |file|
        Fixtures.create_fixtures(
          RAILS_ROOT + '/gold/data', 
          File.basename(file, '.yml'))
      end
    end

  end

Vacation Reading

Posted by david

I'm leaving for a much-needed vacation in a couple days. In the weeks leading up to a trip, I tend to spend as much time thinking about what I'm going to read on that trip as I do thinking about what we're going to do when we get there. Since vacation is one of the few times I can spend long stretches of time reading, I tend to save books I've been wanting to read for those times.

As a rule, I don't bring my laptop on vacations (my wife appreciates this), so I try to avoid reading anything that will make me want to sit down and start coding right away. This rules out many technical books. I also tend to bring more than I think I will need (especially when travelling to a country where I don't speak the language) because of the fear that one of them will be a stinker.

On my last vacation (which happened to be my honeymoon), I read the following:

  • Beautiful Evidence, Edward Tufte

    I'm a huge Edward Tufte fan, and this is his latest book. It was difficult to wait until vacation to start reading this. While it's not my favorite of his books (The Visual Display of Quantitative Information remains that), it was still a pleasure to read, and it made the flight over the Atlantic go by quickly.

  • A Madman Dreams of Turing Machines, Janna Levin

    The subjects of this book about the lives of Alan Turing and Kurt Gödel are fascinating men. Unfortunately, this book was a little disappointing - it was easier to put down than I'd hoped it would be. The focus of it was on the mental struggles of these men with very little description of the mathematics involved. I'd love to find biographies of these two geniuses written by authors who did not assume their readers were afraid of mathematical detail.

  • Special Topics in Calamity Physics, Marisha Pessl

    This novel drew me in early. While it doesn't have the depth that the reviews of it suggest, Marish Pessl's use of language is entertaining, and the story moves quickly enough that the book seemed much shorter than its 500 pages.

After an embarassingly large amount of deliberating, I've decided to bring the following on my upcoming vacation:

  • Against the Day -- Thomas Pynchon's latest.

  • Compilers: Principles, Techniques, and Tools -- You know, the dragon book. I'm embarassed to have not yet read this and am looking forward to finally doing so. This may break my rule about bringing books that make me want to code, but that's a risk I'm willing to take.

  • Dreaming in Code -- I'm a sucker for stories about software projects.

After I get back from my vacation, I'm in Chicago just long enough to shower, catch a nap, and grab my laptop before heading to Portland for RailsConf. I can't wait.