Posted by Pat
Thu, 25 Aug 2005 07:22:00 GMT
Almost there!
The last thing we have to do to get the basics of this sweet environment set up is lighttpd. (I say basics because we’ll also go over SVN, mail servers, and more over time). Right now we’re going to install and configure lighttpd, and get it set up serving a real Rails app – Typo (that’s the software that runs this sweet blog).
Installing lighttpd
The mailing list and irc channel is often filled with people asking how to set up lighttpd with fastcgi and Rails. Apparently a lot of people find it really hard…in fact, I may have found it just as difficult until I learned how. I’m going to make it really easy for you.
All we’ll do right now is install lighttpd itself. I go with the default options (OPENSSL only).
# cd /usr/ports/www/lighttpd/ && make install clean
Now let’s start up lighttpd just to make sure it’s running. As with Postgres, we’ll have to add an entry to the rc.conf file. So open /etc/rc.conf and add the line:
Now we need to copy the sample configuration file, so that we can make changes to it.
# cd /usr/local/etc && cp lighttpd.conf.sample lighttpd.conf
Before we start the server, we need to create the log files it will use. The defaults are /var/log/lighttpd.access.log and /var/log/lighttpd.error.log. We need to make sure these are writable by www.
# cd /var/log/ && touch lighttpd.access.log lighttpd.error.log && chown www:www lighttpd.*.log
Now start up lighttpd, and navigate to your server’s IP in your web browser. The default document root is /usr/local/www/data, but since there’s nothing in there, you’ll get a 404. It’s nothing pretty, but we know that lighttpd is working.
Set up directory structure
I like to set up my sites so that I know exactly what sites belong to which client. I keep their site data and log files in their home directory. For the fastcgi processes, I like them to all be grouped together so I don’t have to go hunting for them. You’ll see how to set this up in the section on configuring lighttpd.
The first thing to do is create the www and logs files in your user’s home directory. Set these up as root so you can easily change permissions.
# cd /home/username && mkdir logs www
Next we create a site directory in the user’s www dir. I like to keep all the sites grouped in that directory, but they should all be in their own dirs to make things easy. What you call the dir doesn’t matter (‘site’ in the example- definitely change that!). I name them after the site that will be running the app. Do the same for the logs directory, to keep all the log files separate.
# mkdir www/site && mkdir logs/site
Because I have a number of clients on my server, I need a way to separate them all from each other, but still allow lighttpd to serve their files. The first step in setting this up is by creating different user accounts on the server for each client, and then creating the www and logs directories for each one, as we saw above. But in order to allow lighttpd to view the necessary files, we can take advantage of FreeBSD’s permission system. I add all my client accounts, as well as the www user, to a group named clients. So first we’ll create the clients group, then add the necessary users to it.
# pw groupadd clients
# pw groupmod clients -m www,username
Now that those are added, we’ll set the proper permissions so that lighttpd can read and write to the www and logs directories.
# chown -R username:clients logs www
# chmod -R 775 logs www
This configuration allows lighttpd to write to the logs and www directories, but it also allows
ANY user in the clients group to write to those. The clients that I host don’t have
SSH access to the machine, so it’s not really that big of a deal. If you need to set things up a bit differently, by all means go ahead.
The last part of setting this up is creating a place for all the fastcgi processes to live. I put these in /var/lighttpd/, but you can put them wherever you want.
# cd /var && mkdir lighttpd
# chown www:www lighttpd && chmod 755 lighttpd
h3. Setting up Typo
Typo is very nice blogging software, written by Tobias Luetke. We’re setting this up now because it’s a pretty easy app to set up, and a lot of people like blogs.
Download typo and extract it to the site directory. Since this is the user’s app, su back to the user and the privileges will come out fine. To get the latest stable release, go to
Typo Stable Downloads. At the time of this writing, the latest release was 2.5.5, but you should check for the latest one and use that in the following commands.
% cd www/site
% fetch http://rubyforge.org/frs/download.php/5602/typo-2.5.5.tgz && tar zxf typo-2.5.5.tgz
Check out the
README file for instructions on how to set it up…or follow these instructions. First we’ll set up the database tables.
% cd typo-2.5.5 && psql typo typo < db/schema.psql.sql
Now edit the config/database.yml file so that Rails can connect to our typo db. The only important one at this point is the production settings, so let’s edit that. It should look like the following block (of course your database/username/password may be different if you chose anything different earlier):
production:
adapter: postgresql
database: typo
host: localhost
username: typo
password: typ0
The last bit of command-line setup work for typo is to give write access to lighttpd on two directories – log and public.
% chmod -R 775 log/ public/
We also have to create the production log file. You’ll understand more about what this is later. For now, just know that we need to create this for Typo to be happy.
% touch log/production.log && chmod 666 log/production.log
To see that typo is working (and to set up basic admin stuff), start up the rails server, and then visit the index page in your browser. We specify that Webrick should run in the production environment, so Rails knows to use the database settings we set earlier..
%./script/server -e production
Now visit the index page in your browser. It will let you register an account, and then you can set up your blog. Play around for a while…change the title, make a few posts, whatever you want.
Making Typo run on lighttpd
Now for the meat of it, the final step in setting up
FLPR. Here we’ll edit the lighttpd config file so that it knows to run our Rails app using fastcgi. You can either edit /usr/local/etc/lighttpd.config, or you may decide to wipe it out and just paste this one in (changing a couple names for the user and site). You need to make sure you enable (by uncommenting) mod_rewrite, mod_alias, mod_fastcgi, and mod_accesslog. You can comment all the other server modules if you want. Below is the meat of the config – it’s the virtual host config for our Typo app.
$HTTP["host"] =~ "(www\.)?mysite.com" {
server.document-root = "/home/username/www/site/typo-2.5.5/public/"
server.errorlog = "/home/username/logs/site/lighttpd-error.log"
accesslog.filename = "/home/username/logs/site/lighttpd-access.log"
url.rewrite = ( "^/$" => "index.html", "^([^.]+)$" => "$1.html" )
server.error-handler-404 = "/dispatch.fcgi"
fastcgi.server = ( ".fcgi" =>
( "localhost" =>
(
"socket" => "/var/lighttpd/site.socket",
"bin-path" => "/home/username/www/site/typo-2.5.5/public/dispatch.fcgi",
"bin-environment" => ( "RAILS_ENV" => "production" ),
"min-procs" => 5,
"max-procs" => 5,
"idle-timeout" => 60
)
)
)
}
Not sure if that looks strange to most of you, or if it looks normal. I’ll try to break it down.
- $HTTP[“host”] – That sets a virtual host. It can be an IP address, a domain name, or a regular expression (like above). If you just want to set it to a single address, use == instead of =~, and obviously use your address as the value.
- server.document-root, server.errorlog, accesslog.filename – All self-explanatory
- url.rewrite – This is a rewrite rule that basically just makes Rails work. I can’t explain it, because I didn’t write it.
- server.error-handler-404 – /dispatch.fcgi is the script that converts web requests into Rails action requests. This setting just says to use it for any 404 errors.
- fastcgi.server – This code block signifies the use of fastcgi, saying it should act on files ending in .fcgi on the server localhost.
- socket – The name of the fastcgi socket that gets created
- bin-path – The script to use to create the fastcgi socket
- bin-environment – This is where you can set environment variables for the fastcgi processes. Just like when we used the -e flag in Webrick earlier, we set RAILS_ENV to production so it uses our DB, along with just making general production optimizations such as caching and less logging.
- min-procs – This is the minimum number of fastcgi processes to run
- max-procs – The max number of fastcgi processes to run
You run multiple fastcgi processes to improve the performance of your application. If there’s only one process running, it can only handle one request at a time. Create 5 processes, and now there are five little fastcgis waiting to handle a request. Some people have reported zombie processes if the min and max aren’t the same, so that’s why we made them the same. Plus your app will run faster because as soon as one process gets killed, another one starts up immediately, instead of waiting for a request.
full lighttpd.conf
Now (re)start lighttpd and you’ve got Typo running on lighttpd!
# /usr/local/etc/rc.d/lighttpd start
h3. All done (for now)
Well, that’s the end of this “tutorial” on setting up FLPR. I’ll keep posting tips and tricks on setting up your machine to host all your Rails projects. I hope you enjoyed reading this, you learned a lot, and most of all now have a nice production server for your Rails applications.
Posted in lighttpd, Rails | Tags lighttpd, rails | 4 comments
Posted by Pat
Thu, 25 Aug 2005 04:43:00 GMT
Getting started with Rails on FreeBSD
As I said in my Intro to FLPR, I think Rails was made to be on FreeBSD. It just works so well, and as you’ll see installation is a snap. Right now we’re going to install RubyGems, and then install a few of the gems you’ll need to run Rails. Some we’ll install from gems, stome we’ll install from ports. Enough talk.
Installing RubyGems
This is a snap. Just install the port.
# cd /usr/ports/devel/ruby-gems && make install clean
Boom. Gems is installed.
Now we’ll install Rails. This is simple, because you just follow the basic Rails installation, using gem. Say yes to all the dependencies.
Now we’re going to install a couple things from ports. The two important ones are the postgresql adapter, and the ruby fcgi module. The adapter is used to let your Rails apps connect to a Postgres database, and the fcgi module is to use fastcgi with lighttpd. I’ll also show how to install iconv, in case you’re using something that requires it (
Salted Hash Login Generator, for example)
note: I had previously showed how to install the ruby-postgres adapter through ports. Now you should install the postgres adapter as a gem .
# cd /usr/ports/www/ruby-fcgi/ && make install clean
# cd /usr/ports/converters/ruby-iconv/ && make install clean
Now install whatever gems you want. I’ve got all the rails ones, builder, localization_generator, rails_product, RedCloth, rubygems-update, salted_login_generator, shipping, and site_generator). Check out
RubyForge for a bunch of different gems you can install.
Get a Rails app going
Just to see that we got Rails installed, let’s start up a Rails app. We’re just going to create a new app and start the included Webrick server to see that it runs.
$ rails myapp
$ ./myapp/script/server
Then check out your server’s IP address at port 3000, and you should see the “Congratulations, you’ve put Ruby on Rails!” page.
That’s all we’re going to do as far as setting up Rails. Time to move on to lighttpd and finish up the qua…diad?
Posted in Rails | Tags rails | 4 comments
Posted by Pat
Thu, 25 Aug 2005 04:12:00 GMT
Done with the hard stuff
So far things have taken quite a bit of time…it’s been a bit of pain, you guys are itching to start developing and deploying your Rails apps. I know, because I’m going through all these steps on a server of my own! From here on out though, things will be a lot easier, and a lot quicker. We can take advantage of the fact that we got the ports system set up nicely, and easily install whatever apps we want.
Now we’re going to set up the database. I use PostgreSQL 8 in all my projects, so that’s what we’ll be installing. Let’s jump right in.
Install Postgres from ports
This is pretty simple stuff. Basically just go into the port dir and start the install.
# cd /usr/ports/databases/postgresql80-server/ && make install clean
Select any of the options you want. I just go with the default selections (only
NLS enabled when I installed it).
Okay you’re done. Seriously.
Initialize and start database
This step is pretty simple as well. Before you can use postgres, you have to initialize the database system. You do this by using the initdb command. However, you need to run it as the pgsql user. pgsql is Postgres’s equivalent of MySQL’s root/admin user.
The easiest way to start and stop the database server is using an RC script (we’ll use these a lot). The Postgres installation actually places one in /usr/local/etc/rc.d for you, so you don’t have to do anything there. First though, you have to enable it in your rc.conf file. This will also make sure that the database starts up whenever your server boots.
Add the following line to /etc/rc.conf:
Save rc.conf, and then start postgresql:
# /usr/local/etc/rc.d/010.pgsql.sh start
Now connect to postgres just to see that everything worked. psql is the client application, template1 is the database you’re connecting to, and pgsql is the username you’re using.
You’ll see the postgres prompt, it looks like ‘template1=#’
Creating users, databases, and all that fun stuff
Postgres has, from what I’ve read, the best implementation of ANSI-SQL of any database, open source or otherwise. So if you have a decent grasp on SQL, you should be able to figure most things out.
The best thing to do is just get familiar with the Postgres documentation.
Database Users and Privileges
Managing Databases
We’ll set up a user and a database right now so that you can see how it’s done. It also may give you a little hint into what we’ll be doing in the next article. The user and database name will both be ‘typo’, and we’ll set up a password for the user as well. You can use whatever you like, of course.
# createuser -P -U pgsql typo
# createdb -O typo -U pgsql typo
The -P option of createuser prompts you for a password that will authenticate the new user, -U is the username you’re connecting as (NOT the user you’re creating), and then typo is the name of the new user. I made the password ‘typ0’ and opted not to let the typo user create databases or users.
The -O option of createdb means that the owner will be set to ‘typo’, -U is the user you connect as to create the db, and then typo is the name of the database.
That’s it…on to setting up Rails
Posted in PostgreSQL | Tags postgresql | no comments
Posted by Pat
Thu, 25 Aug 2005 00:21:00 GMT
This is the nitty-gritty
Now we’re going to upgrade the base system. FreeBSD is different from linux in that “FreeBSD” isn’t simply a kernel like linux is – it’s a combination of the kernel and all the basic system utilities you’re used to. Those are called the “userland apps” in FreeBSD-speak.
Some of you may be wondering when we’re getting to the Rails stuff…that’s a bit later. As I said in the first post, this series of guides is to help you set up a deployment platform. A key component of that is making sure that your system is up-to-date, so you don’t have to worry about any security issues.
Upgrading the kernel isn’t all that hard…though it can be kind of scary at first. I’ll be honest, the first few times I upgraded my kernel I was scared shitless, hoping my machine would boot up. But don’t worry, it’s pretty easy stuff.
Most importantly, be sure to read the FreeBSD Handbook section on Configuring the FreeBSD Kernel. Failure do so will lead to you being lost, and may even screw up your machine. I’ll take you through the steps necessary, but you should still read up on it. Don’t say I didn’t warn you.
Install cvsup and update your sources
cvsup is a nifty little utility that updates your system sources, so you can upgrade your base system whenever you need. It can actually be used for updating anything basically, including the ports tree, but I just use it for the base. portsnap is excellent for managing the ports tree.
# cd /usr/ports/net/cvsup-without-gui/ && make install clean
Once you’ve got it installed, you need to tell cvsup what sources to update, where to get them from, and what versions. These are all read in by default from a file of your choosing, sometimes called a supfile. The following is my supfile, which updates the base system for FreeBSD 5.4, and only does security updates.
# vi /usr/local/etc/security-supfile
*default host=cvsup7.freebsd.org
*default base=/usr
*default prefix=/usr
*default release=cvs tag=RELENG_5_4
*default delete use-rel-suffix
src-all
Now update the base system sources using cvsup. It’ll connect to the cvs servers and download all the latest code:
# cvsup -L 2 /usr/local/etc/security-supfile
h3. Create a kernel config file
The first thing I do now is create a custom kernel config file. I’ll show you where to go, but I won’t give my example config file because you need to enable different settings based on your machine. Take a look at
The Configuration File to find out what all the options are.
# cd /usr/src/sys/i386/conf/
# cp GENERIC HOSTNAME
# vi HOSTNAME
What I will recommend you put in the kernel config is PF support. PF is a firewall, though you can use a couple others if you don’t want it. For more info, check out the info on
The OpenBSD Packet Filter.
# pf support
device pf
device pflog
device pfsync
# ALTQ support
options ALTQ
options ALTQ_CBQ # Class Bases Queuing (CBQ)
options ALTQ_RED # Random Early Detection (RED)
options ALTQ_RIO # RED In/Out
options ALTQ_HFSC # Hierarchical Packet Scheduler (HFSC)
options ALTQ_PRIQ # Priority Queuing (PRIQ)
I like to copy my config file to /root as well, just because when I do backups, I don’t backup the /usr/src directory and its subdirectories.
Rebuild the kernel and system binaries
Very first thing to do is check to see if you need to do anything special when upgrading…to do so, read the /usr/src/UPDATING file
Note: If you’ve got /tmp set to noexec, you’ll run into problems here. The system updating process requires that /tmp be executable, so just temporarily enable it.
Now go to the sources directory and start the build. The -j4 flag just makes it run faster. This will take a while, depending on the speed of your machine.
# cd /usr/src
# make -j4 buildworld
Now build the kernel. This will only take a few minutes. Substitute
HOSTNAME with whatever you named your kernel config file earlier.
# make buildkernel KERNCONF=HOSTNAME
Once this is done, install the kernel. This should be pretty quick.
# make installkernel KERNCONF=HOSTNAME
Then install the world (userland apps).
The last step is to run mergemaster. This takes all your existing configuration files and merges necessary changes into them. You’ll want to back up your existing config files, in case something goes wrong.
# cp -Rp /etc /home/backupetc
Now run mergemaster
This will prompt you to make changes to your files. For the most part, you can just install the changes. However,
DO NOT INSTALL A NEW master.passwd, passwd, or group files. If you do, you’ll have to recover them from the backup you just made. Also, don’t overwrite any custom files you may have like firewall config files.
If you’re prompted to make a change, enter ‘i’ to install the change, and ‘d’ to discard it. Sometimes the file may be longer than the screen, in which case you can scroll down to see it in its entirety, or just hit ‘q’.
When the last file is done, you’ll be asked to delete the temp root directory. Choose ‘yes’.
Now reboot your machine and pray (just kidding! sorta…)
Rejoice or recover
In all likelihood your machine booted just fine. If not, you can recover it by loading the old kernel. Do this by doing a soft reboot (ctrl+alt+delete). When the machine is booting up, it will ask if you want to boot a different kernel by pressing any key other than ‘Enter’. So press something other than ‘Enter’ and type
# unload
# boot kernel.GENERIC
If your server is hosted at a datacenter, and you don’t have physical access to it, submit a support ticket and have one of their techs boot to the old kernel.
Posted in FreeBSD | Tags cvsup, freebsd, kernel | 2 comments
Posted by Pat
Wed, 24 Aug 2005 17:03:00 GMT
Assumptions
I assume that you already have FreeBSD installed…a pretty big assumption, I realize. However, I just got a new server, and I have to set it up anyway, so it’s a good opportunity for me to take you through the steps I take when configuring a server for Rails hosting. I’ll make another article at a later date going over FreeBSD installation…but for the mean time, read the Installing FreeBSD section of the FreeBSD Handbook
Also, I assume that you did the Developer install, which includes the base system sources. You should also install the ports tree, which the handbook covers.
Setting up the ports utilities
The first thing I ever do is upgrade the ports system, so that any software I install is at the latest version. We’ll also set up cron jobs to check all your software for security flaws (these are checked against a database created by the FreeBSD Security Team).
We’ll be installing three ports.
portaudit – Checks your installed ports against a database for any known security flaws
portsnap – Updates the ports tree to the latest versions
portupgrade – Amazingly, upgrades your installed ports
su to root and execute the following commands
# cd /usr/ports/security/portaudit && make install clean
# cd /usr/ports/sysutils/portsnap && make install clean
# cd /usr/ports/sysutils/portupgrade && make install clean
Note: Depending on your ports tree version, the installation of portupgrade may fail. That’s because portaudit notices a vulnerability in a dependency and doesn’t let you install it. If it fails, skip to the “Upgrade ports tree” and then install portupgrade.
After you install the ports utilities type “rehash” at the command prompt so that you can run the newly installed programs.
Update the ports tree
We want to install the latest version of any software we install, so you’ll need to update the ports tree. First thing to do is copy your portsnap.conf file to its proper location:
# cd /usr/local/etc/ && cp portsnap.conf.sample portsnap.conf
Next thing to do is fetch and extract the updated ports tree.
# portsnap fetch
# portsnap extract
This is optional, but at this point I set up portsnap to fetch the latest ports tree in a cron job, so I don’t manually have to, or at least it will be mostly updated when I do run fetch/extract, thus take less time. I have this entry in my /etc/crontab file:
0 2 * * * root /usr/local/sbin/portsnap cron
That runs portsnap at 2 am every night.
Note: If you haven’t installed portupgrade because it failed the first time around, do so now
Upgrade your installed ports
First get the absolute latest snapshot of the ports three. You’ll have this since we just installed it, so this is mostly just a reference for the future. Keep in mind you should run it even if you have a cron job set up, so that you get any changes made between the time the cron job ran and the time you’re updating your ports.
# portsnap fetch && portsnap update
There are a number of ways to check out what ports are up-to-date and which ones need updating. Personally, I use pkg_version, which will list all the packages you have installed, along with whether they’re up-to-date with, ahead or behind of the ports tree. To just see which ones need updating, do
You’ll want to read the upgrade notes before upgrading. The vast majority of the time you can upgrade your ports without worrying about anything, but occasionally one will need some steps taken before you can upgrade.
Upgrade a single port using portupgrade:
Or to upgrade all the ports on your system, use the -a flag:
h3. Congratulations (and a sweet little treat!)
You’re all done getting the ports tree updated, installed a few nice utilities to help check the security of your ports, as well as update them.
I’ve got a nice little surprise for you at this point…when you installed portupgrade, FreeBSD installed ruby along with it because it’s listed as a dependency. Think I’m kidding?
# ruby --version
ruby 1.8.2 (2004-12-25) [i386-freebsd5]
You’ve got the latest version of Ruby and didn’t even have to do anything – sweet! Go ahead, play around a bit.
Next we’ll learn how to bring the base system up to date.
Posted in FreeBSD | Tags freebsd, ports, portsnap, portupgrade | 2 comments
Posted by Pat
Sat, 20 Aug 2005 07:57:00 GMT
What is FLPR?
If LAMP is the Open Source Web Platform, then FLPR is the Most Excellent Open Source Web Platform. Party on dudes.
FLPR stands for FreeBSD Lighttpd PostgreSQL Rails
Why should I use it?
This is just the opinion of one tiny web developer (but don’t worry, with FLPR, I’m going to take over the world!). There’s tons of debate all over the net on MySQL vs Postgres, FreeBSD vs Linux, Rails vs PHP/ASP/Java/FlavorOfTheWeek.
Here are my thoughts on it. I’m actually going to explain it all in reverse order. As I mentioned, there’s already a ton of exiting content debating the merits of all the various technologies. I’m just going to discuss why I use this particular setup (besides the obvious fact that it makes a cool acronym).
- Rails – This is just a sweet, sweet web framework. That’s basically my reasoning for using it. Development is easy and just makes sense. I can code a lot faster in it than Java (my previous lang of choice), and it’s (gasp!) fun.
- PostgreSQL – This is about as enterprise a database as you can get and still be free. Free as in beer, free as in speech. Can’t beat that. Views, stored procedures, and it’s been totally solid and lightning quick for me. I feel in general it’s superior to MySQL, and Rails has awesome support for it, so it just makes sense. Plus it’s true open source, vs MySQL’s pseudo-open source ;)
- lighttpd – Apache is the de facto for web hosting…except in Rails apps. I’m not even sure why, really. Basically it’s taht Rails has to run as a CGI app, Rails in CGI is slooooow so you have to use FastCGI, and lighttpd + FastCGI is super quick. I’ve just played around with it a bit and it seems pretty nice so far. It’s been very stable (no crashes yet), very fast, and uses less resources than Apache. Most excellent.
- FreeBSD – This, as they say on MTV’s Cribs, is where the magic happens. I think this is the best server OS around right now. Solid, fast, secure…and maintaining it is super easy. Best of all, I think Rails was written to run on it, because of how easy it is. Have problems installing Ruby and Rails on another OS? Not on FreeBSD! Installing the pure Postgres adapter is a cakewalk when you install from ports, compared to being a pain in every other OS I’ve tried.
Okay so as I look at that list, there isn’t really a compelling reason to use FLPR over LAMP, or any combination of OS, httpd, database, and language. I’ve used a lot of web languages and a lot of OSes, and it really comes down to feel for me. The combination I discuss here just feels right when developing, and so far it’s been a winner in production.
I think the FLPR combo is absolutely killer for developing and deploying web apps. If you like it, awesome, if not, that’s cool too. Over the next week, I’ll be posting instructions on how to build a FLPR machine from scratch. The point of this will be to build a machine that can host your web applications online, and eventually I hope to incorporate all aspects of a standard internet server (web, ftp, etc).
Why can’t I use Linux?
Cause then you’re a LLPR, and that means your apps fall apart. (I’ll be here all week, don’t forget to tip your waitress)
Posted in General | Tags flpr, freebsd, lighttpd, postgresql, rails | 6 comments
