Posted by Pat
Thu, 09 Feb 2006 22:10:00 GMT
I’ve been using Subversion for my source code control for a few months now. For the server, I’ve just been using the built-in svnserve, which has worked perfectly fine for me. However lately I’ve been doing more projects for outside companies, and dealing with SVN access has been a pain. I have access control on my repository, but svnserve doesn’t allow any fine-grained access…any user has access to the entire repository. I also want to make some of my code publicly available over SVN. Naturally I don’t want everyone in the world to have access to all my business code.
One way to have more fine-grained access control over the repository is to use Apache and mod_dav_svn. Unfortunately we can’t use lighttpd for the task because it doesn’t support mod_dav_svn. I’m not too pleased about having to install Apache, but it’s the right tool for the job. Also, since it will only be a front end to my Subversion repository, I can keep it pretty small.
Installing and Configuring Apache 2
The Subversion documentation says you need to use Apache 2.0, not 1.3. Checking out /usr/ports/www, you’ll see there are three versions of Apache 2 – 2.0, 2.1, and 2.2. I figure it makes sense to just go with the latest one, so that’s what I’ll install. Should work fine though if you decide to go with 2.0 or 2.1 instead.
There are a couple make options that we need to set for the Apache install. You can do these on the command line, but it’s probably a good idea to throw them in /etc/make.conf so that when you upgrade Apache, those options are remembered. So put the following lines in /etc/make.conf:
# Apache config
.if ${.CURDIR:M*/www/apache2*}
WITH_AUTH_MODULES=yes
WITH_DAV_MODULES=yes
WITH_SSL_MODULES=yes
WITH_BERKELEYDB=db42
.endif
# Subversion config
.if ${.CURDIR:M*/devel/subversion}
WITH_APACHE2=true
WITH_MOD_DAV_SVN=yes
.endif
Now go ahead and build Apache:
# cd /usr/ports/www/apache22/ && make install clean
Now you’ll need to rebuild subversion:
# cd /usr/ports/devel/subversion && make deinstall && make install clean
For Apache to be able to use the repository, it must have read/write access to it. Initially I set up my repository at /home/svn/repos, but since I’ll no longer be using the ‘svn’ user, I wanted to move it under /usr/local. So just copy the whole repository over and change the ownership.
# cp -Rp /home/svn/repos /usr/local/repos
# chown -R www:www /usr/local/repos
Finally you need to configure Apache to expose your
SVN repository. Go ahead and edit /usr/local/etc/apache22/httpd.conf. I commented out all the modules I didn’t need, so that I could keep Apache as small as possible. Here’s what my module list looks like, along with an example Subversion config:
LoadModule authn_file_module libexec/apache22/mod_authn_file.so
LoadModule authz_host_module libexec/apache22/mod_authz_host.so
LoadModule authz_groupfile_module libexec/apache22/mod_authz_groupfile.so
LoadModule authz_user_module libexec/apache22/mod_authz_user.so
LoadModule authz_dbm_module libexec/apache22/mod_authz_dbm.so
LoadModule authz_owner_module libexec/apache22/mod_authz_owner.so
LoadModule authz_default_module libexec/apache22/mod_authz_default.so
LoadModule auth_basic_module libexec/apache22/mod_auth_basic.so
LoadModule auth_digest_module libexec/apache22/mod_auth_digest.so
LoadModule include_module libexec/apache22/mod_include.so
LoadModule log_config_module libexec/apache22/mod_log_config.so
LoadModule logio_module libexec/apache22/mod_logio.so
LoadModule env_module libexec/apache22/mod_env.so
LoadModule dav_module libexec/apache22/mod_dav.so
LoadModule dav_svn_module libexec/apache22/mod_dav_svn.so
LoadModule authz_svn_module libexec/apache22/mod_authz_svn.so
LoadModule dav_fs_module libexec/apache22/mod_dav_fs.so
<Location />
DAV svn
SVNPath /usr/local/repos
</Location>
Now put the following line in /etc/rc.conf:
And start up apache:
# /usr/local/etc/rc.d/apache22.sh start
Congrats, you should now be able to go to http://ip/ in your web browser to view your repository. For more detailed config info, check out the Apache section of the SVN Book.
Switching your working copies to use the new URL
You have two options at this point – either just check out your projects again using the new
URL, or use the
svn switch command to switch each working copy to the new
URL. Either one works fine, and the first is probably easier in general. Just ‘svn co http://12.34.56.78/project’. However in case you want to immediately switch it, run the following commands (you need to have svnserve running for this to work):
$ cd working_copy
$ svn switch --relocate svn://12.34.56.78/project http://12.34.56.78/project
Once you’ve done all of this, you don’t need to run svnserve anymore, so you should kill it off. If you forgot to switch a working copy to the new URL, you’ll get an error as soon as you try to execute a command like svn update, so you’ll immediately know to just check out the project again.
Final thoughts
I spent about 15 minutes configuring Apache last night, have a nice little setup. Now my SVN repository is available to the public, and my private projects are all password protected. In addition, I can create users for each private project I’m working on, so the companies I’m developing for have easy SVN access to their own projects, without exposing other companies’ projects. Pretty sweet.
Posted in FreeBSD | Tags apache, subversion | 5 comments
Posted by Pat
Tue, 24 Jan 2006 06:30:00 GMT
There was a post on the Rails list the other day about installing awstats on lighttpd. If you don’t know, awstats is a log analyzer tool that runs through your web server logs and generates cool stats. Since I want to see how much traffic I’m (not) getting, I figured I’d install it. Should be a breeze, because I’m using FreeBSD :)
cantona# cd /usr/ports/www/awstats/
cantona# make install clean
Now awstats is installed to /usr/local/www/awstats/. Follow the instructions on the link above to finish..took me about 5 minutes. I’d like to see if somehow I can make stats.mydomain.org automatically append the config for the incoming hostname. That sure would be sweet..might have to hack the script up a little bit.
Tip: In your lighttpd config file, you may have virtual hosts set up with something like:
$HTTP["host"] =~ "(www\.)?mydomain\.com"
I was stumped for a minute because it kept matching “stats.mydomain.com” when I didn’t think it should. Anyway, since it’s using Perl pattern matching, the key is to make sure that you check to see that the domain starts with that, which you do using ^:
$HTTP["host"] =~ "^(www\.)?mydomain\.com"
Pretty simple..regular expressions sure are fun.
Posted in FreeBSD, lighttpd | Tags awstats, lighttpd | 2 comments
Posted by Pat
Tue, 06 Dec 2005 23:16:00 GMT
A while ago we used portsnap to upgrade the ports tree when first installing FreeBSD. If you’ve made the upgrade to 6.0, portsnap is a part of the base system, so you should remove the package and any configuration.
It’s pretty easy. Just delete the package get rid of the existing portsnap database and regenerate it.
# cd /usr/ports/sysutils/portsnap && make deinstall
# rm -rf /var/db/portsnap/* && rm -rf /usr/local/portsnap
# rm /usr/local/etc/portsnap.conf
# portsnap fetch && portsnap extract
portsnap works because it uses the config at /etc/portsnap.conf now. The new portsnap binary is located in /usr/sbin, so you’ll need to update your crontab entries to reflect that.
Posted in FreeBSD | Tags freebsd, portsnap | no comments
Posted by Pat
Mon, 05 Dec 2005 08:58:00 GMT
In a previous article, we installed the ruby-postgres adapter using the ports system. I’ve spoken with a few people, and they’ve suggested using the postgres gem instead.
Unfortunately you can’t just do a “gem install postgres” to install it, you have to go through a few extra steps in order to get it to build cleanly on FreeBSD. Fortunately, I’ve done it and have all the steps here :)
You’ll get a bunch of errors and it’ll say it can’t create the Makefile. Don’t worry about that. We have to manually configure and install the gem.
For whatever reason, the Postgres installation doesn’t put the libpq-fe.h file in its include dir, so we need to symlink it. Then we’ll configure and install the gem.
# ln -s /usr/local/include/libpq-fe.h /usr/local/include/postgresql/server/
# cd /usr/local/lib/ruby/gems/1.8/gems/postgres-0.7.1/
# ruby extconf.rb --with-pgsql-include-dir=/usr/local/include/postgresql/server --with-pgsql-lib-dir=/usr/local/lib/postgresql
# make install
Should be all good to go. You’ll want to remove the ruby18-postgres package if you have it installed.
Posted in FreeBSD, PostgreSQL | Tags freebsd, postgresql, ruby | no comments
Posted by Pat
Wed, 16 Nov 2005 11:10:00 GMT
I finally got around to upgrading the server to FreeBSD 6.0 tonight. Sweet. I forgot to write down all the steps I took and turn it into a guide, but there’s really nothing to do. Just change your supfile to use RELENG_6_0, follow the basic upgrade procedures and you’re good to go. mergemaster is a pain…lots of stuff to do. Be sure to merge the group and master.passwd files, otherwise you’re in for a world of hurt.
Remember after you’ve upgraded to run ‘portupgrade -af’ to rebuild all your ports. You need to do this because the new libraries and interfaces in 6.0 can break your ports compiled against 5.x.
Posted in General, FreeBSD | Tags freebsd | no comments
Posted by Pat
Tue, 30 Aug 2005 03:23:00 GMT
Why the need for version control?
Because I screw up all the time. Seriously. I break my code so much, thrash up my system all the time…I need to be able to go back to a time when things worked. So I use version control.
There are also some pretty nifty things you can do with it to help deploy your apps. I’ll cover some of those in a future article.
There are a couple different pieces of software for version control. CVS, RCS (which I had to use when I was doing Software Engineering at RIT – bleh), and Subversion. Subversion seems to be the tool of choice for Rails developers, so that’s why I started using it. Again, not really based on good technical reasons, but it works really well for me. I’m not sure many people would notice a lot of difference in version control systems because most of us have simple needs, so it’s not that big of a deal.
Getting it set up.
I almost feel bad making a post, because subversion is so easy to set up on FreeBSD.
# cd /usr/ports/devel/subversion && make install clean
And that’s it. I’ll admit this is the first article I’ve written where I’m not typing in the steps as I do them, but I’m pretty sure that’s right.
Next I like to create a subversion user, and store all my repositories under that user’s home dir. It’s pretty simple, just use FreeBSD’s adduser command to create the user. I always set it up as username ‘svn’, random password, rest are defaults. Only time I ever become svn is when I su from root. Alter the setup to match your preferences.
I had initially started instructions on how to create a repository, import a project, etc but that’s all covered very well in the SVN Quickstart Guide. Just so you guys know, I like to store all my repositories in /home/svn/repositories, and I put a bunch of scripts in /home/svn.
Running svnserve to serve up your repository
For people to be able to access your repositories, you have to actually serve it somehow. I do it with svnserve, but you can also use apache+mod_dav_svn. That’s all covered in the excellent SVN Book.
I put the following command in a shell script that I execute as the svn user:
#!/bin/sh
svnserve -d -r /home/svn/repositories --listen-host machine.hostname
Just chmod+x it and run it whenever you want to start up your
SVN server. The command means svnserve will run in daemon mode (basically transparent, runs as a background process), and the root is at /home/svn/repositories, so if someone accesses svn://yourserver/projectname it serves up the repository named projectname under /home/svn/repositories – good for security so people can’t just access any file on your system. The last thing to mention is the—listen-host directive. I have to put this in because by default, svnserve runs with IPv6 enabled, so I’m unable to actually make a connection to my server. You may or may not need it – I’d just leave it in.
Last thing to do is connect to your svn server. This is pretty simple. From a remote machine (whatever you develop on, usually), type:
%svn checkout svn://hostname/projectname
And you’ve got an up-to-date copy of your project. Check out the quickstart and boook for more info.
SVN runs on port 3690 by default, so be sure to open that in your firewall, or specify a different port using the—listen-port directive.
Setting up authentication for users
If you want to make sure that only authorized users ever take a look at or modify your code, you’ll need to set up basic authentication. After you’ve created a repository, you should have a svnserve.conf file in the conf directory. Edit the contents to contain:
[general]
password-db = passwd
anon-access = none
realm = My Projects Repository
This will prevent anyone from checking out or making modifications to your code without authenticating. If you want people to check it out, set anon-access to read.
The password-db command means that the password database will be stored in a file named passwd. So create a passwd file in the conf directory that looks like
[users]
pergesu=ilovefreebsd
That means I can authenticate with username pergesu and password ‘ilovefreebsd’. That’s not my actual password, by the way :)
Now that you’ve set up authentication, you’ll need to be sure to use it when checking out a project:
% svn checkout --username pergesu svn://hostname/myproject
Enter your password at the prompt, good to go. Once you’ve authenticated once, you never need to use—username in there again. Not sure how – must be magic.
Wrapping up
So you should have a working SVN repository going, maybe even with authentication. Check out the SVN Book to learn how to do practically everything with SVN. It’s pretty cool. I’ll show you my neat little deployment set up in a couple days. Have fun for now
Posted in FreeBSD | Tags freebsd, subversion | no comments
Posted by Pat
Thu, 25 Aug 2005 00:21:00 GMT
This is the nitty-gritty
Now we’re going to upgrade the base system. FreeBSD is different from linux in that “FreeBSD” isn’t simply a kernel like linux is – it’s a combination of the kernel and all the basic system utilities you’re used to. Those are called the “userland apps” in FreeBSD-speak.
Some of you may be wondering when we’re getting to the Rails stuff…that’s a bit later. As I said in the first post, this series of guides is to help you set up a deployment platform. A key component of that is making sure that your system is up-to-date, so you don’t have to worry about any security issues.
Upgrading the kernel isn’t all that hard…though it can be kind of scary at first. I’ll be honest, the first few times I upgraded my kernel I was scared shitless, hoping my machine would boot up. But don’t worry, it’s pretty easy stuff.
Most importantly, be sure to read the FreeBSD Handbook section on Configuring the FreeBSD Kernel. Failure do so will lead to you being lost, and may even screw up your machine. I’ll take you through the steps necessary, but you should still read up on it. Don’t say I didn’t warn you.
Install cvsup and update your sources
cvsup is a nifty little utility that updates your system sources, so you can upgrade your base system whenever you need. It can actually be used for updating anything basically, including the ports tree, but I just use it for the base. portsnap is excellent for managing the ports tree.
# cd /usr/ports/net/cvsup-without-gui/ && make install clean
Once you’ve got it installed, you need to tell cvsup what sources to update, where to get them from, and what versions. These are all read in by default from a file of your choosing, sometimes called a supfile. The following is my supfile, which updates the base system for FreeBSD 5.4, and only does security updates.
# vi /usr/local/etc/security-supfile
*default host=cvsup7.freebsd.org
*default base=/usr
*default prefix=/usr
*default release=cvs tag=RELENG_5_4
*default delete use-rel-suffix
src-all
Now update the base system sources using cvsup. It’ll connect to the cvs servers and download all the latest code:
# cvsup -L 2 /usr/local/etc/security-supfile
h3. Create a kernel config file
The first thing I do now is create a custom kernel config file. I’ll show you where to go, but I won’t give my example config file because you need to enable different settings based on your machine. Take a look at
The Configuration File to find out what all the options are.
# cd /usr/src/sys/i386/conf/
# cp GENERIC HOSTNAME
# vi HOSTNAME
What I will recommend you put in the kernel config is PF support. PF is a firewall, though you can use a couple others if you don’t want it. For more info, check out the info on
The OpenBSD Packet Filter.
# pf support
device pf
device pflog
device pfsync
# ALTQ support
options ALTQ
options ALTQ_CBQ # Class Bases Queuing (CBQ)
options ALTQ_RED # Random Early Detection (RED)
options ALTQ_RIO # RED In/Out
options ALTQ_HFSC # Hierarchical Packet Scheduler (HFSC)
options ALTQ_PRIQ # Priority Queuing (PRIQ)
I like to copy my config file to /root as well, just because when I do backups, I don’t backup the /usr/src directory and its subdirectories.
Rebuild the kernel and system binaries
Very first thing to do is check to see if you need to do anything special when upgrading…to do so, read the /usr/src/UPDATING file
Note: If you’ve got /tmp set to noexec, you’ll run into problems here. The system updating process requires that /tmp be executable, so just temporarily enable it.
Now go to the sources directory and start the build. The -j4 flag just makes it run faster. This will take a while, depending on the speed of your machine.
# cd /usr/src
# make -j4 buildworld
Now build the kernel. This will only take a few minutes. Substitute
HOSTNAME with whatever you named your kernel config file earlier.
# make buildkernel KERNCONF=HOSTNAME
Once this is done, install the kernel. This should be pretty quick.
# make installkernel KERNCONF=HOSTNAME
Then install the world (userland apps).
The last step is to run mergemaster. This takes all your existing configuration files and merges necessary changes into them. You’ll want to back up your existing config files, in case something goes wrong.
# cp -Rp /etc /home/backupetc
Now run mergemaster
This will prompt you to make changes to your files. For the most part, you can just install the changes. However,
DO NOT INSTALL A NEW master.passwd, passwd, or group files. If you do, you’ll have to recover them from the backup you just made. Also, don’t overwrite any custom files you may have like firewall config files.
If you’re prompted to make a change, enter ‘i’ to install the change, and ‘d’ to discard it. Sometimes the file may be longer than the screen, in which case you can scroll down to see it in its entirety, or just hit ‘q’.
When the last file is done, you’ll be asked to delete the temp root directory. Choose ‘yes’.
Now reboot your machine and pray (just kidding! sorta…)
Rejoice or recover
In all likelihood your machine booted just fine. If not, you can recover it by loading the old kernel. Do this by doing a soft reboot (ctrl+alt+delete). When the machine is booting up, it will ask if you want to boot a different kernel by pressing any key other than ‘Enter’. So press something other than ‘Enter’ and type
# unload
# boot kernel.GENERIC
If your server is hosted at a datacenter, and you don’t have physical access to it, submit a support ticket and have one of their techs boot to the old kernel.
Posted in FreeBSD | Tags cvsup, freebsd, kernel | 2 comments
Posted by Pat
Wed, 24 Aug 2005 17:03:00 GMT
Assumptions
I assume that you already have FreeBSD installed…a pretty big assumption, I realize. However, I just got a new server, and I have to set it up anyway, so it’s a good opportunity for me to take you through the steps I take when configuring a server for Rails hosting. I’ll make another article at a later date going over FreeBSD installation…but for the mean time, read the Installing FreeBSD section of the FreeBSD Handbook
Also, I assume that you did the Developer install, which includes the base system sources. You should also install the ports tree, which the handbook covers.
Setting up the ports utilities
The first thing I ever do is upgrade the ports system, so that any software I install is at the latest version. We’ll also set up cron jobs to check all your software for security flaws (these are checked against a database created by the FreeBSD Security Team).
We’ll be installing three ports.
portaudit – Checks your installed ports against a database for any known security flaws
portsnap – Updates the ports tree to the latest versions
portupgrade – Amazingly, upgrades your installed ports
su to root and execute the following commands
# cd /usr/ports/security/portaudit && make install clean
# cd /usr/ports/sysutils/portsnap && make install clean
# cd /usr/ports/sysutils/portupgrade && make install clean
Note: Depending on your ports tree version, the installation of portupgrade may fail. That’s because portaudit notices a vulnerability in a dependency and doesn’t let you install it. If it fails, skip to the “Upgrade ports tree” and then install portupgrade.
After you install the ports utilities type “rehash” at the command prompt so that you can run the newly installed programs.
Update the ports tree
We want to install the latest version of any software we install, so you’ll need to update the ports tree. First thing to do is copy your portsnap.conf file to its proper location:
# cd /usr/local/etc/ && cp portsnap.conf.sample portsnap.conf
Next thing to do is fetch and extract the updated ports tree.
# portsnap fetch
# portsnap extract
This is optional, but at this point I set up portsnap to fetch the latest ports tree in a cron job, so I don’t manually have to, or at least it will be mostly updated when I do run fetch/extract, thus take less time. I have this entry in my /etc/crontab file:
0 2 * * * root /usr/local/sbin/portsnap cron
That runs portsnap at 2 am every night.
Note: If you haven’t installed portupgrade because it failed the first time around, do so now
Upgrade your installed ports
First get the absolute latest snapshot of the ports three. You’ll have this since we just installed it, so this is mostly just a reference for the future. Keep in mind you should run it even if you have a cron job set up, so that you get any changes made between the time the cron job ran and the time you’re updating your ports.
# portsnap fetch && portsnap update
There are a number of ways to check out what ports are up-to-date and which ones need updating. Personally, I use pkg_version, which will list all the packages you have installed, along with whether they’re up-to-date with, ahead or behind of the ports tree. To just see which ones need updating, do
You’ll want to read the upgrade notes before upgrading. The vast majority of the time you can upgrade your ports without worrying about anything, but occasionally one will need some steps taken before you can upgrade.
Upgrade a single port using portupgrade:
Or to upgrade all the ports on your system, use the -a flag:
h3. Congratulations (and a sweet little treat!)
You’re all done getting the ports tree updated, installed a few nice utilities to help check the security of your ports, as well as update them.
I’ve got a nice little surprise for you at this point…when you installed portupgrade, FreeBSD installed ruby along with it because it’s listed as a dependency. Think I’m kidding?
# ruby --version
ruby 1.8.2 (2004-12-25) [i386-freebsd5]
You’ve got the latest version of Ruby and didn’t even have to do anything – sweet! Go ahead, play around a bit.
Next we’ll learn how to bring the base system up to date.
Posted in FreeBSD | Tags freebsd, ports, portsnap, portupgrade | 2 comments
