FLPR

Setting default boolean values in migrations

Posted by Pat Tue, 07 Feb 2006 14:47:00 GMT

 def self.up
    add_column :restaurants, :visa, :boolean, :default => false, :null => false
 end

 def self.up
    add_column :restaurants, :visa, :boolean, :default => 0, :null => false
 end

I imagine this is because MySQL doesn’t provide actual boolean support. Not quite sure what the problem is, and I don’t have time to look at the code right now, but if you run into that problem, that’s how you solve it.

Posted in PostgreSQL, Rails | Tags migrations, postgresql, rails | no comments

Database-less Testing

Posted by Pat Tue, 24 Jan 2006 10:08:00 GMT

Rails is awesome for building database-driven applications, and has a very nice testing framework built right in. Unfortunately it’s a bit too built in. I’ve recently been writing a number of small applications that don’t use a database. I started off writing my tests as usual..then rake blew up on me. I honestly don’t remember the exact errors…I asked on the Rails list, and made a frustrated post after my first question didn’t lead to satisfactory results. No luck still.

Okay enough about that..I think I’ve found the answer. Basically you need to strip AR entirely out of your Rails project, and redefine one of the rake tasks.

First of all, in your app’s environment.rb file, make sure you don’t load AR:

config.frameworks -= [ :active_record ]

You should of course exclude any other frameworks you don’t want.

The next thing to do is make a stripped down version of the test_help.rb file. This isn’t the same as test_helper.rb which lives in the test/ dir, it’s a file named test_help.rb I found in the bowels of Railties, but without the AR and fixture loading stuff. This should go in the lib/ dir.

require 'application'

# Make double-sure the RAILS_ENV is set to test, 
# so fixtures are loaded to the right database
silence_warnings { RAILS_ENV = "test" }

require 'test/unit'
require 'action_controller/test_process'
require 'action_web_service/test_invoke'
require 'breakpoint'

Here’s the original code in railties, so you can see what I took out.

require 'application'

# Make double-sure the RAILS_ENV is set to test, 
# so fixtures are loaded to the right database
silence_warnings { RAILS_ENV = "test" }

require 'test/unit'
require 'active_record/fixtures'
require 'action_controller/test_process'
require 'action_web_service/test_invoke'
require 'breakpoint'

Test::Unit::TestCase.fixture_path = RAILS_ROOT + "/test/fixtures/"

def create_fixtures(*table_names)
  Fixtures.create_fixtures(RAILS_ROOT + "/test/fixtures", table_names)
end

Now go back and edit test/test_helper.rb to use this test_help file instead of the default rails one. It’s a simple matter of loading our custom test_help file:

ENV["RAILS_ENV"] = "test"
require File.expand_path(File.dirname(__FILE__) + "/../config/environment")
require 'test_help_without_ar'

class Test::Unit::TestCase
  # Add whatever specific functionality you want
end

After some more digging through railties, I came across the prepare_test_database rake task. Redefine it to do nothing, and everything works fine. I found some code that Blair Zajac wrote to provide a method for redefining a rake task. I’m only showing the code here, but his full comments and credit appear in the attached file.

# The following code is by Blair Zajac and appeared in the Ruby on Rails mailing list.
# See archived message here:  http://article.gmane.org/gmane.comp.lang.ruby.rails/27426

module Rake
   class Task
     # Clear all existing actions for the given task and then set the action for the task to the given block.
     def self.redefine_task(args, &block)
       task_name, deps = resolve_args(args)
       TASKS.delete(task_name.to_s)
       define_task(args, &block)
     end
   end
end

# Clear all existing actions for the given task and then set the action for the task to the given block.
def redefine_task(args, &block)
   Rake::Task.redefine_task(args, &block)
end

# The following code is by (obvious) master hacker Pat Maddox
desc "Prepare the test database"
redefine_task :prepare_test_database do |t|
  # Don't do anything, there's no database to prepare
end

Final Thoughts

This gave me a real headache back when I first tried to do it, I couldn’t find anything to help me out. However now I can happily write all my tests and use rake to automatically run the tests. I’ve noticed that it seems to be a bit slow..I’m really not sure why, but if it’s working then I’m happy. Hopefully someone else will find this useful.

Posted in Rails | Tags rails, testing | 6 comments

Testing with database constraints

Posted by Pat Mon, 12 Dec 2005 12:38:00 GMT

PostgreSQL calls itself “the world’s most advanced open source database.” I want to use some of the advanced features, dammit! The first, not-so-spectacular-but-still-nice feature that I want to use is foreign key constraints. If you don’t know what they are (read the links!), the gist of it is that you can specify that a field in one table refers to a field in another table. That’s the foreign key part. The constraint is that the record in the other table must exist. Something like:

Table Companies
id integer
name varchar(80)

Table Products
id integer
name varchar(80)
company_id integer  # This is a foreign key

company_id is a foreign key to the id field in the companies table. We haven’t specified it yet, and I’ll show how to do that later, but for now just understand that concept. Any value insert into products(company_id) must exist in companies(id). So for example, if you make products(company_id) = 2982475, and there’s no record in ‘companies’ with id = 2982475, postgresql will give you an error and no insert is made.

Similarly, if you try to delete a record in ‘companies’, and there are records in ‘products’ that reference it, postgresql will again give you an error. <anecdote style=”useless”>This would have been really nice the time I deleted a couple records I “didn’t need” anymore, and then my whole site stopped working. Turns out that there were still a few records somewhere else that referenced these. I should have used console to make the changes…but I didn’t know about it.</anecdote>

This is a good thing because it ensures you maintain referential integrity. Rails is nice and all, but a simple fact of business apps is that the data is far more valuable and has a longer life than the actual app you use to access it. You want to make sure that the database has no chance of having broken references. Bad data at best causes you lots of headaches, and at worst is absolutely worthless.

TDD with fkey constraints

I’m a huge fan of test-driven development. It’s just fun for some reason, and I quickly make lots of progress. Anyway, I wanted to be able to get foreign key constraints into my apps, so I needed to fit them into my process. Basically here’s what I came up with:

1. Write tests 2. Write code to make tests pass 3. Add foreign key constraints 4. Make sure tests still pass

I didn’t know the best way to add foreign key constraints, but then found out about migrations and thought I could give that a shot. I had a tough time getting started, so I made a post to the Rails list and then Robby Russell showed me how. Go read that article to learn how to use constraints in your migrations.

At first my development process went very smoothly. Then it blew up when I created two tables that referenced a single table. The problem is that when you run tests, rake doesn’t actually wipe the db clean like you might expect. Instead it just rolls it back to its initial state, which is to have the fixtures loaded. This means you end up with dirty tables, and then postgresql complains about you violating foreign key constraints. I haven’t explained it well, honestly, but the entire issue is covered here. There are a couple solutions listed there, and I’ll explain what I’ve managed to get working so far.

First solution – delete records in teardown()

def teardown
  self.class.fixture_table_names.reverse.each do |table_name|
    klass_name = Inflector.classify(table_name.to_s)
    if Object.const_defined?(klass_name)
      klass = Object.const_get(klass_name)
      klass.connection.delete("DELETE FROM #{table_name}", 'Fixture Delete')
    else
      flunk("Cannot find class for table '#{table_name}' to delete fixtures")
    end
  end
end

This overrides the teardown method so that when a test is done completes, all the fixtures will be deleted from the db. The only problem with this solution is that I couldn’t use transactional fixtures with it. Testing worked, it was just a bit slow.

Second solution – Make your models paranoid

# neuter fixture deletions and multiple fixture insertions to work with database constraints
class Fixture
  attr_reader :class_name
end 

class Fixtures
  @@inserted_fixture_list ||= {}
  alias :original_insert_fixtures :insert_fixtures

  def insert_fixtures
    return if @@inserted_fixture_list[values[0].class_name]
    @@inserted_fixture_list[values[0].class_name] = true
    unless ActiveRecord::Base.connection.select_one("select 1 from #{fixture_class_to_table_name(values[0].class_name)}")
      original_insert_fixtures
    end
  end

  def delete_existing_fixtures() end

  # compute a fixture's table name from its (known) class name
  def fixture_class_to_table_name(class_name)
    class_name.gsub(/([a-z])([A-Z])/) { |match| "#{$1}_#{$2}" }.downcase.pluralize
  end
end

I did run into a problem here because it turns out that this code doesn’t delete any fixtures, which becomes obvious once you see that delete_existing_fixtures is empty. If I commented out the redefinition, I got a bunch of warnings about a transaction not being in progress. At this point I realized that I wouldn’t really want to delete any objects that are referenced anyway…I might not want them accessible to the user under normal circumstances, but I definitely don’t want them deleted from the database. So I installed Rick Olson’s acts_as_paranoid module and added that to the models that I wanted to be paranoid in the db. Paranoid models don’t actually get deleted when you call destroy(), they just get a deleted_at field set to the timestamp. Pretty slick.

Go with either of these solutions and you should be able to add foreign key constraints and test away to your heart’s content.

Posted in PostgreSQL, Rails | Tags constraints, postgresql, rails, testing | no comments

script/console is the coolest thing ever

Posted by Pat Thu, 17 Nov 2005 08:48:00 GMT

If you’re not using script/console, you really need to be. For those who don’t know what it is, it’s basically irb with the Rails environment loaded in. In fact, that might be precisely what it is, though I’m not sure.

Why is it so cool? Well you can access your entire Rails application from the command-line, which is cool if you’re a geek. A very nice use is to access your model and make database updates through there, rather than messing with your database directly. I remember a while ago I spent about an hour debugging some problem because I had used the psql client to delete some records…had I use script/console, my application logic would have taken care of it for me.

What I did that showed me how cool it is

I upgraded Typo to the current version. I found that they’ve got a sweet tag, <typo:code>, which you wrap around a code block and it formats it nicely. Adds scroll bars so that it doesn’t break your layout, puts a nice blue background, and you can probably specify more in CSS if you’re good with that.

In my previous posts, I was using the Textile @ to specify code formatting, and sometimes using <code> blocks. They didn’t look nearly as good as Typo’s cool <typo:code> tag, so I wanted to convert all my posts to use them. I barfed at the thought of having to go through all my posts and manually edit them…so I wondered if I could use script/console to load up all the posts and make the changes I wanted. I looked at a few and figured out all the cases that I had code blocks in, and wrote a little function to do all the substitutions. Then I loaded up script/console and made it happen.

def fix_code(article)
  article.body = article.body.gsub("<br/>\r\n@", "\r\n<typo:code>")
  article.body = article.body.gsub("@\r\n", "</ typo:code>")
  article.body = article.body.gsub("</ typo:code><typo:code>", "\r\n")
  article.body = article.body.gsub("<code><pre>", "<typo:code>")
  article.body = article.body.gsub("</pre></code>", "</ typo:code>")
  article.body = article.body.gsub("<pre><code>", "<typo:code>")
  article.body = article.body.gsub("</code></pre>", "</ typo:code>")
  article.save
end

articles = Content.find_all_by_type("Article")
articles.each { |a| fix_code(a) }

Unfortunately the </typo:code> in the code I posted above messes things up, so I made it ”< /typo:code>”. Anyway, there’s obviously not supposed to be a space.

Using script/console saved me a lot of time, and turned a boring task into a pretty fun challenge/learning experience. I suggest you start using it.

Posted in Rails | Tags console, rails, ruby | 2 comments

Tip: Future-proofing your app

Posted by Pat Fri, 14 Oct 2005 08:18:00 GMT

I noticed that the Rails book has references to session, flash, params, etc while a lot of the apps you’ll find online use @session, @flash, @params, etc. I went on #rubyonrails to ask what the difference was (other than instance variable vs method call), and it turns out that it’s actually no more complex than instance variable vs method call. I was told that they both work fine for now, but the preferred way of doing this is using the method, and to change the instance variables to method calls to future-proof your app.

Well there are going to be a bajillion references if you’ve got even a moderatly-sized application…so it’s tough. We’ll use the power of FreeBSD’s command line utilities to make it really, really easy.

To save you the trouble of typing all these commands, I created a shell script that you can use. Just download it, drop it in your rails root dir, chmod+x and run it. A description of what it does is below:

$find lib app \( -name "*.rb" -o -name "*.rhtml" \) -print0 | xargs -0 sed -i '' -e 's/\@session/session/'
$find lib app \( -name "*.rb" -o -name "*.rhtml" \) -print0 | xargs -0 sed -i '' -e 's/\@flash/flash/'
$find lib app \( -name "*.rb" -o -name "*.rhtml" \) -print0 | xargs -0 sed -i '' -e 's/\@params/params/'
$find lib app \( -name "*.rb" -o -name "*.rhtml" \) -print0 | xargs -0 sed -i '' -e 's/\@request/request/'
$find lib app \( -name "*.rb" -o -name "*.rhtml" \) -print0 | xargs -0 sed -i '' -e 's/\@response/response/'

Okay I don’t want to explain each individual part…but here’s how it goes. First we find all the files in the current directory and all the subdirectories that have a .rb or .rhtml extension, turn them into command line arguments and pass them to sed, which replaces each instance of @session with session.

This only performs the operations on the lib and app directories…those are the only ones that have any of this code besides the tests, I think. Making these changes to the tests will break them, thus the find command only looks for the files in lib/ and app/

Your app ought to work fine after that…I haven’t had any problems so far. Now you’ve got all the nice new method calls instead of the outdated instance variables.

Posted in Rails | Tags rails, tips | no comments

Putting it all together - Running your app on lighttpd

Posted by Pat Thu, 25 Aug 2005 07:22:00 GMT

Almost there!

The last thing we have to do to get the basics of this sweet environment set up is lighttpd. (I say basics because we’ll also go over SVN, mail servers, and more over time). Right now we’re going to install and configure lighttpd, and get it set up serving a real Rails app – Typo (that’s the software that runs this sweet blog).

Installing lighttpd

The mailing list and irc channel is often filled with people asking how to set up lighttpd with fastcgi and Rails. Apparently a lot of people find it really hard…in fact, I may have found it just as difficult until I learned how. I’m going to make it really easy for you.

# cd /usr/ports/www/lighttpd/ && make install clean

lighttpd_enable="YES"

# cd /usr/local/etc && cp lighttpd.conf.sample lighttpd.conf

# cd /var/log/ && touch lighttpd.access.log lighttpd.error.log && chown www:www lighttpd.*.log

Set up directory structure

I like to set up my sites so that I know exactly what sites belong to which client. I keep their site data and log files in their home directory. For the fastcgi processes, I like them to all be grouped together so I don’t have to go hunting for them. You’ll see how to set this up in the section on configuring lighttpd.

# cd /home/username && mkdir logs www

# mkdir www/site && mkdir logs/site

# pw groupadd clients
# pw groupmod clients -m www,username

# chown -R username:clients logs www
# chmod -R 775 logs www

# cd /var && mkdir lighttpd
# chown www:www lighttpd && chmod 755 lighttpd

Typo is very nice blogging software, written by Tobias Luetke. We’re setting this up now because it’s a pretty easy app to set up, and a lot of people like blogs.

% cd www/site
% fetch http://rubyforge.org/frs/download.php/5602/typo-2.5.5.tgz && tar zxf typo-2.5.5.tgz

% cd typo-2.5.5 && psql typo typo < db/schema.psql.sql

production:
  adapter: postgresql
  database: typo
  host: localhost
  username: typo
  password: typ0

% chmod -R 775 log/ public/

% touch log/production.log && chmod 666 log/production.log

%./script/server -e production

Making Typo run on lighttpd

$HTTP["host"] =~ "(www\.)?mysite.com" {
        server.document-root = "/home/username/www/site/typo-2.5.5/public/"
        server.errorlog = "/home/username/logs/site/lighttpd-error.log"
        accesslog.filename = "/home/username/logs/site/lighttpd-access.log"

        url.rewrite = ( "^/$" => "index.html", "^([^.]+)$" => "$1.html" )
        server.error-handler-404 = "/dispatch.fcgi"

        fastcgi.server = ( ".fcgi" =>
          ( "localhost" =>
            (
              "socket" => "/var/lighttpd/site.socket",
              "bin-path" => "/home/username/www/site/typo-2.5.5/public/dispatch.fcgi",
              "bin-environment" => ( "RAILS_ENV" => "production" ),
              "min-procs" => 5,
              "max-procs" => 5,
              "idle-timeout" => 60
            )
          )
        )
}

Not sure if that looks strange to most of you, or if it looks normal. I’ll try to break it down.

• $HTTP[“host”] – That sets a virtual host. It can be an IP address, a domain name, or a regular expression (like above). If you just want to set it to a single address, use == instead of =~, and obviously use your address as the value.

• server.document-root, server.errorlog, accesslog.filename – All self-explanatory
• url.rewrite – This is a rewrite rule that basically just makes Rails work. I can’t explain it, because I didn’t write it.
• server.error-handler-404 – /dispatch.fcgi is the script that converts web requests into Rails action requests. This setting just says to use it for any 404 errors.
• fastcgi.server – This code block signifies the use of fastcgi, saying it should act on files ending in .fcgi on the server localhost.
• socket – The name of the fastcgi socket that gets created
• bin-path – The script to use to create the fastcgi socket
• bin-environment – This is where you can set environment variables for the fastcgi processes. Just like when we used the -e flag in Webrick earlier, we set RAILS_ENV to production so it uses our DB, along with just making general production optimizations such as caching and less logging.
• min-procs – This is the minimum number of fastcgi processes to run
• max-procs – The max number of fastcgi processes to run

You run multiple fastcgi processes to improve the performance of your application. If there’s only one process running, it can only handle one request at a time. Create 5 processes, and now there are five little fastcgis waiting to handle a request. Some people have reported zombie processes if the min and max aren’t the same, so that’s why we made them the same. Plus your app will run faster because as soon as one process gets killed, another one starts up immediately, instead of waiting for a request.

full lighttpd.conf

# /usr/local/etc/rc.d/lighttpd start

Well, that’s the end of this “tutorial” on setting up FLPR. I’ll keep posting tips and tricks on setting up your machine to host all your Rails projects. I hope you enjoyed reading this, you learned a lot, and most of all now have a nice production server for your Rails applications.

Posted in lighttpd, Rails | Tags lighttpd, rails | 4 comments

Finally - Setting up Rails!

Posted by Pat Thu, 25 Aug 2005 04:43:00 GMT

Getting started with Rails on FreeBSD

As I said in my Intro to FLPR, I think Rails was made to be on FreeBSD. It just works so well, and as you’ll see installation is a snap. Right now we’re going to install RubyGems, and then install a few of the gems you’ll need to run Rails. Some we’ll install from gems, stome we’ll install from ports. Enough talk.

Installing RubyGems

# cd /usr/ports/devel/ruby-gems && make install clean

# gem --version
0.8.11

# gem install rails

note: I had previously showed how to install the ruby-postgres adapter through ports. Now you should install the postgres adapter as a gem .

# cd /usr/ports/www/ruby-fcgi/ && make install clean
# cd /usr/ports/converters/ruby-iconv/ && make install clean

Get a Rails app going

$ rails myapp
$ ./myapp/script/server

That’s all we’re going to do as far as setting up Rails. Time to move on to lighttpd and finish up the qua…diad?

Posted in Rails | Tags rails | 4 comments