These are some brief notes on how you can use a personal domain as a way of maintaining an online identity
Establishing an online identity (Basic)
The simplest way to use a personal domain is simply to register one as a way of "staking a claim" to your chosen online identity. Even if you don't use the domain for anything else, you can use the part of the domain name that's unique to you (e.g., the "foo" part or "foo.com", as a userid for other services such as Twitter where having a convenient "handle" is useful.
The hardest part here is finding an online identifier that is not already in use either in domain names or with popular online services. I personally advocate finding a handle you can use as a .com domain name, as opposed to get domain names like "foo.net", "foo.us" or "foo.name". (I broke this rule myself, since I don't own "hecker.com" and "hecker" is already in use by other people on some services.) You can either attempt to use your name directly if you have an unusual one (e.g., "John Doespx"), use a variant of it, or use a handle unrelated to your name.
Whatever you choose, you should use Google or other search engines to see if and where your chosen handle is already on use on the net. You should also check to see if the name is in use with Twitter and other online services. (For example, if you've chosen "johndoespx" as your handle, you can check the URL <http://twitter.com/johndoespx> to see if someone already has a Twitter account under that handle.
Once you've chosen a handle, you should register it as a .com domain, e.g., "johndoespx.com". You can also register it in other top-level domains, e.g., .org or .net, but if you want or need to save money then just .com is probably sufficient.
You can and should also use your handle to register yourself with popular online services. For example, you could sign up for a Twitter account, sign up with popular blogging services like Wordpress, Blogger, establish a MySpace account, etc. Even if you have no plans to use the services in the near term (or ever, for that matter), you'll at least ensure that no one else can claim your handle for their own use.
There are lots of places with information on how to register domain names; if anyone finds one that's especially good please add it here. It would also be nice to add some more general references on using a domain name to establish an online identity. This also connects with the ideas of personal branding and online identity management, although much on the online material on those topics is excessively marketing-oriented.
Using your personal domain for email and blogging (Intermediate)
Once you have a unique handle and an associated domain and have reserved your userid at the most popular online services, the next most useful things to do are to establish a personal email account at your own domain, and to establish a personal blog at your domain name. The blog can then also serve as your own personal web site -- a much simpler approach than trying to create a web site by hand using HTML or some other means.
Google Apps supports the same interface and features as Gmail, and like using regular Gmail, setting up a mail account using Google Apps is no charge (as long as you don't mind seeing ads). Note that what you want to use is Google Apps Standard Edition, not Google Apps Premier Edition (which costs $50/year and is marketed to businesses) or Google Apps Education Edition (which is for academic institutions).
Google Apps allows multiple people can have email accounts under a single domain, or for a single person to have multiple email addresses under that domain. (For example, you might use a different address for personal email vs. business email.) It's also possible to migrate email from an existing Gmail account to Google Apps, as well as to use Thunderbird and other IMAP mail clients (e.g. Mail.app for OS X) with Google Apps. For more information see Scott Hanselman's description of migrating his family to Google Apps.