FLPR

Typo-ing with slightly less reckless abandon

Posted by Pat Mon, 12 Dec 2005 08:33:00 GMT

I want to keep on top of the latest typo version, because that’s what the cool kids do. In the post I made yesterday, I told you the problems I ran into when I svn updated and the migrations failed. Now I’ll show you how to avoid that problem in the future. This takes advantage of svn being super sweet.

The first thing to do is make a copy of your typo database. This is just so you have some good working data to test migrations on. Do this by dumping your production dn, then creating a new db and loading the data in. I named mine ‘flpr_typo_dev’, because by default typo uses the development database to do migrations.

$ pg_dump -U typo typo > db_data.sql
$ createdb -U pgsql -O typo typo_dev
$ psql typo_dev typo < db_data.sql
$ svn update
$ rake migrate

// Now if your migrations fail..
$ rake migrate VERSION=28
$ svn update -r 761

Tags typo | 2 comments

Typo-ing with reckless abandon

Posted by Pat Sun, 11 Dec 2005 02:11:00 GMT

I installed Typo from trunk a while back (that was a pain in the ass, converting from a stable install), and periodically update to get the latest version. Apparently none of the Typo devs are using PostgreSQL as a backend, or if they are, they weren’t too careful with the latest round of migrations I think. One of the migrations complains nastily when I run it, and rendered my database unusable. I had fun restoring from the nightly backups. Bleh.

Anyway before a Typo dev flips out on me for calling them out, I’d like to make it clear that I don’t know for a fact whether there’s actually an issue with the Typo migrations. I’m going to take a look at the code and see what the deal is, and then hopefully I can be back using the latest and greatest version of Typo.

Quick note: I just looked at Typo’s trac and it looks like this issue has already been brought up. So I guess I’ll be waiting on a fix.

Also, I’m going to be smart in the future…copy my production db over to a dev db and then run the migrations on that. If all is good, I can migrate the production db.

Tags typo | no comments

portsnap has become part of FreeBSD's base (6.0 only)

Posted by Pat Tue, 06 Dec 2005 23:16:00 GMT

A while ago we used portsnap to upgrade the ports tree when first installing FreeBSD. If you’ve made the upgrade to 6.0, portsnap is a part of the base system, so you should remove the package and any configuration.

# cd /usr/ports/sysutils/portsnap && make deinstall
# rm -rf /var/db/portsnap/* && rm -rf /usr/local/portsnap
# rm /usr/local/etc/portsnap.conf
# portsnap fetch && portsnap extract

portsnap works because it uses the config at /etc/portsnap.conf now. The new portsnap binary is located in /usr/sbin, so you’ll need to update your crontab entries to reflect that.

Posted in FreeBSD | Tags freebsd, portsnap | no comments

Installing the postgres adapter

Posted by Pat Mon, 05 Dec 2005 08:58:00 GMT

In a previous article, we installed the ruby-postgres adapter using the ports system. I’ve spoken with a few people, and they’ve suggested using the postgres gem instead.

Unfortunately you can’t just do a “gem install postgres” to install it, you have to go through a few extra steps in order to get it to build cleanly on FreeBSD. Fortunately, I’ve done it and have all the steps here :)

# gem install postgres

You’ll get a bunch of errors and it’ll say it can’t create the Makefile. Don’t worry about that. We have to manually configure and install the gem.

For whatever reason, the Postgres installation doesn’t put the libpq-fe.h file in its include dir, so we need to symlink it. Then we’ll configure and install the gem.

# ln -s /usr/local/include/libpq-fe.h /usr/local/include/postgresql/server/
# cd /usr/local/lib/ruby/gems/1.8/gems/postgres-0.7.1/
# ruby extconf.rb --with-pgsql-include-dir=/usr/local/include/postgresql/server --with-pgsql-lib-dir=/usr/local/lib/postgresql
# make install

Should be all good to go. You’ll want to remove the ruby18-postgres package if you have it installed.

Posted in FreeBSD, PostgreSQL | Tags freebsd, postgresql, ruby | no comments

script/console is the coolest thing ever

Posted by Pat Thu, 17 Nov 2005 08:48:00 GMT

If you’re not using script/console, you really need to be. For those who don’t know what it is, it’s basically irb with the Rails environment loaded in. In fact, that might be precisely what it is, though I’m not sure.

Why is it so cool? Well you can access your entire Rails application from the command-line, which is cool if you’re a geek. A very nice use is to access your model and make database updates through there, rather than messing with your database directly. I remember a while ago I spent about an hour debugging some problem because I had used the psql client to delete some records…had I use script/console, my application logic would have taken care of it for me.

What I did that showed me how cool it is

I upgraded Typo to the current version. I found that they’ve got a sweet tag, <typo:code>, which you wrap around a code block and it formats it nicely. Adds scroll bars so that it doesn’t break your layout, puts a nice blue background, and you can probably specify more in CSS if you’re good with that.

In my previous posts, I was using the Textile @ to specify code formatting, and sometimes using <code> blocks. They didn’t look nearly as good as Typo’s cool <typo:code> tag, so I wanted to convert all my posts to use them. I barfed at the thought of having to go through all my posts and manually edit them…so I wondered if I could use script/console to load up all the posts and make the changes I wanted. I looked at a few and figured out all the cases that I had code blocks in, and wrote a little function to do all the substitutions. Then I loaded up script/console and made it happen.

def fix_code(article)
  article.body = article.body.gsub("<br/>\r\n@", "\r\n<typo:code>")
  article.body = article.body.gsub("@\r\n", "</ typo:code>")
  article.body = article.body.gsub("</ typo:code><typo:code>", "\r\n")
  article.body = article.body.gsub("<code><pre>", "<typo:code>")
  article.body = article.body.gsub("</pre></code>", "</ typo:code>")
  article.body = article.body.gsub("<pre><code>", "<typo:code>")
  article.body = article.body.gsub("</code></pre>", "</ typo:code>")
  article.save
end

articles = Content.find_all_by_type("Article")
articles.each { |a| fix_code(a) }

Unfortunately the </typo:code> in the code I posted above messes things up, so I made it ”< /typo:code>”. Anyway, there’s obviously not supposed to be a space.

Using script/console saved me a lot of time, and turned a boring task into a pretty fun challenge/learning experience. I suggest you start using it.

Posted in Rails | Tags console, rails, ruby | 2 comments

Upgrading to PostgreSQL 8.1

Posted by Pat Wed, 16 Nov 2005 13:14:00 GMT

The awesome guys over at PostgreSQL released 8.1, and it’s got a bunch of cool new features/enhancements. After spending a bit of time playing with things, I figured out how to get 8.1 going, and here I’m going to take you through the task of upgrading your FreeBSD system to use PostgreSQL 8.1.

I’m going to list the basic steps so you have an idea of what we’re going to do, then I’ll give you detailed instructions.
1. BACKUP YOUR DATABASES!!
2. Install the new ports
3. Remove old packages and fix dependencies
4. Wipe the old data dir and initialize the new one
5. Create your users, databases, and import your old data

Backup Databases

You should back up your databases often, at least nightly. You should especially back up your databases whenever you’re upgrading Postgres to the latest version. I’ve got a dirty little secret though…I never do. This upgrade is a bit different, and you HAVE to back up your databases, otherwise you lose them. This is because the 8.1 server and client are totally incompatible with the 8.0 data structure, so you’re actually going to delete the entire psql data directory and start all over. Yeah, you might want to have a way of getting your data back in :)

#!/usr/local/bin/ruby

# This is used to set the user when you run pg commands.  Change
# 'pgsql' to your main postgres user, or make it blank if you
# don't need to specify
usrcmd = "-U pgsql"

# Vacuum all databases
`vacuumdb #{usrcmd} -a -f -z`

# This lists all the dbs, but there's some header info to get rid of
dboutput = `psql #{usrcmd} -l`
databases = dboutput.split("\n")
databases.slice!(0..2)
databases.pop

# Loop through all the databases
databases.each do |db|
  dbname = db.split[0] # Gets the database name, first text before a space
  next if dbname == "template0" or dbname == "template1" # Don't back these up
  `pg_dump #{usrcmd} #{dbname} > #{dbname}.sql`
end

The comments should explain everything. Basically we’re just taking all the databases and exporting their contents to a file in the current directory. All the databases will have separate backup files.

A few important notes. The backup script doesn’t do the create database statements, so you’ll have to do that manually for each database. Take a look at the pg_dump help to see how to include the create database statements. Also, this doesn’t backup any of your database users, so you’ll have to do that manually as well. I’m going to enhance this script to include all that stuff, but for now I just wanted to have something to help me quickly upgrade to 8.1.

Making the upgrade

# cd /usr/ports/databases/postgresql81-client && make install clean

# cd /usr/ports/databases/postgresql81-server && make install clean

Now we’re going to have to delete the old 8.0 packages. You probably have a number of packages on your system that depend on these, so we’ll have to force the deletion. Then we’ll fix the package database to make those packages depend on the new 8.1 package.

# pkg_deinstall --force postgresql-server-8.0.4
# pkg_deinstall --force postgresql-client-8.0.4

# pkgdb -F

This is going to show you a bunch of stale dependencies, because we removed the postgresql-client.8.0.x package. Most of the time it’s smart enough to suggest postgresql-client-8.1.0, and you can just hit yes, but one time it suggested the server package when it should use the client. If the current package depends on postgresql-client-8.0.x, then obviously you want to update it to use client-8.1.0. So just watch closely, and if it suggests the wrong one, manually enter it.

# portupgrade -fr postgresql-client-8.1.0

Set up your databases

# rm -rf /usr/local/pgsql/data/

# su -l pgsql -c initdb

# /usr/local/etc/rc.d/010.pgsql.sh start

# createuser -U pgsql -P newuser

# createdb -U pgsql -O newuser newdb

# psql newdb pgsql < newdb.sql

Upgrade finished

Your system should be all good to go now. If you have any services that rely on PostgreSQL, you should restart them now. For example, I’d restart lighttpd obviously. I’ve also got my mail server configured to use PGSQL, so I’d need to restart that as well. Maybe that’s the subject of a future article :)

Posted in PostgreSQL | Tags postgresql | 7 comments

What's been going on

Posted by Pat Wed, 16 Nov 2005 11:21:37 GMT

I haven’t updated this thing in a while, as you can see. I started this blog because I was setting up a new server, and figured I could post all the steps I went through to get everything going. Since then, I really haven’t done a whole lot of coding, life’s just gotten in the way. Now I’m happy to say that I’m getting back into the swing of things, and I’ve got a bunch of ideas already. Looking forward to sharing some stuff with you.

I really want to upgrade to PostgreSQL 8.1… I’ve got a test machine that did the upgrade fine, but the ruby-postgresql bindings won’t rebuild. It looks like it’s got a dependency on 7.4.9, even though it should build with 8.1.0 installed. I shot off an email to the FreeBSD Ports guys, because according to FreshPorts, there’s no active maintainer. Hopefully they’ll get this fixed up, and maybe I can be the maintainer so the ruby-postgres bindings port stays up to date.

That’s it for now, just gonna hack for the rest of the night.

Posted in General | no comments

Upgraded to 6.0

Posted by Pat Wed, 16 Nov 2005 11:10:00 GMT

I finally got around to upgrading the server to FreeBSD 6.0 tonight. Sweet. I forgot to write down all the steps I took and turn it into a guide, but there’s really nothing to do. Just change your supfile to use RELENG_6_0, follow the basic upgrade procedures and you’re good to go. mergemaster is a pain…lots of stuff to do. Be sure to merge the group and master.passwd files, otherwise you’re in for a world of hurt.

Remember after you’ve upgraded to run ‘portupgrade -af’ to rebuild all your ports. You need to do this because the new libraries and interfaces in 6.0 can break your ports compiled against 5.x.

Posted in General, FreeBSD | Tags freebsd | no comments

Tip: Future-proofing your app

Posted by Pat Fri, 14 Oct 2005 08:18:00 GMT

I noticed that the Rails book has references to session, flash, params, etc while a lot of the apps you’ll find online use @session, @flash, @params, etc. I went on #rubyonrails to ask what the difference was (other than instance variable vs method call), and it turns out that it’s actually no more complex than instance variable vs method call. I was told that they both work fine for now, but the preferred way of doing this is using the method, and to change the instance variables to method calls to future-proof your app.

Well there are going to be a bajillion references if you’ve got even a moderatly-sized application…so it’s tough. We’ll use the power of FreeBSD’s command line utilities to make it really, really easy.

To save you the trouble of typing all these commands, I created a shell script that you can use. Just download it, drop it in your rails root dir, chmod+x and run it. A description of what it does is below:

$find lib app \( -name "*.rb" -o -name "*.rhtml" \) -print0 | xargs -0 sed -i '' -e 's/\@session/session/'
$find lib app \( -name "*.rb" -o -name "*.rhtml" \) -print0 | xargs -0 sed -i '' -e 's/\@flash/flash/'
$find lib app \( -name "*.rb" -o -name "*.rhtml" \) -print0 | xargs -0 sed -i '' -e 's/\@params/params/'
$find lib app \( -name "*.rb" -o -name "*.rhtml" \) -print0 | xargs -0 sed -i '' -e 's/\@request/request/'
$find lib app \( -name "*.rb" -o -name "*.rhtml" \) -print0 | xargs -0 sed -i '' -e 's/\@response/response/'

Okay I don’t want to explain each individual part…but here’s how it goes. First we find all the files in the current directory and all the subdirectories that have a .rb or .rhtml extension, turn them into command line arguments and pass them to sed, which replaces each instance of @session with session.

This only performs the operations on the lib and app directories…those are the only ones that have any of this code besides the tests, I think. Making these changes to the tests will break them, thus the find command only looks for the files in lib/ and app/

Your app ought to work fine after that…I haven’t had any problems so far. Now you’ve got all the nice new method calls instead of the outdated instance variables.

Posted in Rails | Tags rails, tips | no comments

Using Subversion

Posted by Pat Tue, 30 Aug 2005 03:23:00 GMT

Why the need for version control?

Because I screw up all the time. Seriously. I break my code so much, thrash up my system all the time…I need to be able to go back to a time when things worked. So I use version control.

There are also some pretty nifty things you can do with it to help deploy your apps. I’ll cover some of those in a future article.

There are a couple different pieces of software for version control. CVS, RCS (which I had to use when I was doing Software Engineering at RIT – bleh), and Subversion. Subversion seems to be the tool of choice for Rails developers, so that’s why I started using it. Again, not really based on good technical reasons, but it works really well for me. I’m not sure many people would notice a lot of difference in version control systems because most of us have simple needs, so it’s not that big of a deal.

Getting it set up.

# cd /usr/ports/devel/subversion && make install clean

Next I like to create a subversion user, and store all my repositories under that user’s home dir. It’s pretty simple, just use FreeBSD’s adduser command to create the user. I always set it up as username ‘svn’, random password, rest are defaults. Only time I ever become svn is when I su from root. Alter the setup to match your preferences.

I had initially started instructions on how to create a repository, import a project, etc but that’s all covered very well in the SVN Quickstart Guide. Just so you guys know, I like to store all my repositories in /home/svn/repositories, and I put a bunch of scripts in /home/svn.

Running svnserve to serve up your repository

For people to be able to access your repositories, you have to actually serve it somehow. I do it with svnserve, but you can also use apache+mod_dav_svn. That’s all covered in the excellent SVN Book.

#!/bin/sh
svnserve -d -r /home/svn/repositories --listen-host machine.hostname

%svn checkout svn://hostname/projectname

SVN runs on port 3690 by default, so be sure to open that in your firewall, or specify a different port using the—listen-port directive.

Setting up authentication for users

[general]
password-db = passwd
anon-access = none
realm = My Projects Repository

[users]
pergesu=ilovefreebsd

% svn checkout --username pergesu svn://hostname/myproject

Wrapping up

So you should have a working SVN repository going, maybe even with authentication. Check out the SVN Book to learn how to do practically everything with SVN. It’s pretty cool. I’ll show you my neat little deployment set up in a couple days. Have fun for now

Posted in FreeBSD | Tags freebsd, subversion | no comments