Monday, September 29, 2008

Emacs Hidden Save

One thing about emacs that annoys me is its default behavior for backing up files. I don't have anything against generally backing up files, but I am annoyed that any directory I work heavily in doubles in size because of the backup files.

I've seen a number of different solutions for this little annoyance, ranging from eliminating backups entirely to creating an entire shadow directory structure for backups. The closest thing I've seen to what I like, though, is passing an option to ls that excludes backup files.

Although excluding files ending in ~ works fairly well, if someone else is browsing through my filesystem that doesn't know to exclude these files will still be annoyed by them. A more general solution would be to prepend . to all of the file names so that they'd be hidden for everybody. Now, this isn't a perfect solution as backups of hidden files are still visible, but I feel this is a more acceptable degree of loss than visitors having problems. (Note: this program is covered under the GPL)

;;; Copyright (c) 2008, Colin Williams
;;; All rights reserved

;;; This program is free software: you can redistribute it and/or modify
;;; it under the terms of the GNU General Public License as published by
;;; the Free Software Foundation, either version 3 of the License, or
;;; (at your option) any later version.

;;; This program is distributed in the hope that it will be useful,
;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;;; GNU General Public License for more details.

;;; You should have received a copy of the GNU General Public License
;;; along with this program.  If not, see .

(setq make-backup-file-name-function (lambda (file)
           (concat (file-name-directory file)
            (file-name-nondirectory file)

(defun backup-file-name-p (file)
  "Return non-nil if FILE is a backup file name (numeric or not).
This is a separate function so you can redefine it for customization.
You may need to redefine `file-name-sans-versions' as well."
  (string-match "^\\..*~$" (file-name-nondirectory file)))

;; Much of this function was based off of file-name-sans-version
;; in files.el, the source and credit for this package is located
;; at 
(defun file-name-sans-versions (name &optional keep-backup-version)
  "Return file NAME sans backup versions or strings.
This is a separate procedure so your site-init or startup file can
redefine it.
If the optional argument KEEP-BACKUP-VERSION is non-nil,
we do not remove backup version numbers, only true file version numbers."
  (let ((handler (find-file-name-handler name 'file-name-sans-versions)))
    (if handler
 (funcall handler 'file-name-sans-versions name keep-backup-version)
      (if (eq system-type 'vax-vms)
   ;; VMS version number is (a) semicolon, optional
   ;; sign, zero or more digits or (b) period, option
   ;; sign, zero or more digits, provided this is the
   ;; second period encountered outside of the
   ;; device/directory part of the file name.
   (substring name 0
       (or (string-match ";[-+]?[0-9]*\\'" name)
    (if (string-match "\\.[^]>:]*\\(\\.[-+]?[0-9]*\\)\\'"
        (match-beginning 1))
    (length name)))
 (if keep-backup-version
   (let ((file-name (file-name-nondirectory name))
  (directory-name (file-name-directory name)))
     (concat directory-name (let* ((file-name-end
        (or (string-match "\\.~[0-9.]+~\\'"
            (string-match "~\\'"
            (length file-name)))
        (if (= file-name-end
        (length file-name))
          (string-match "^\\." file-name))))

         (substring file-name
      (if (= file-name-start 0)
          (length file-name)

Ironically, the code for actually creating the filename was the easy part. The tricky bits were the support functions backup-file-name-p and file-name-sans-versions. I've been using this scheme on my desktop for a while without any problems, if anybody has suggestions for improving this scheme please let me know.

Wednesday, January 16, 2008

Six Degrees of Wikipedia Automation

Well, it's been a while since my last post, been waiting till I got the six degrees of wikipedia stuff put together. There are basically 5 things that I had to do to get the automation in working order.

  • Create a blog that could be updated via email
  • Create an email address to administer the game through
  • Setup mutt
  • Write a new revision of the generation script
  • Add a line to my crontab

The first step was one of the easier ones. Blogger fits the bill pretty well, letting me create an email address that anything mailed to it will be posted.

The second step was even easier, gmail fit the bill perfectly. The only real parameters I had were that it supported pop3 or imap and smtp.

Mutt is a text-based mail client which can send emails automatically when text is received from STDIN. Although simple in concept, I had a bitch of a time getting it set up correctly. First I found a simple configuration file and made a minor tweak.

set imap_user = ""
set imap_pass = "*******"

set smtp_url = "smtp://"
set smtp_pass = "*******"
set from = ""
set realname = "Six Degrees of Wikipedia"
set content_type = "text/html"

set folder = "imaps://"
set spoolfile = "+INBOX"
set record="+[Gmail]/Sent Mail"
set postponed="+[Gmail]/Drafts"

set header_cache="~/.mutt/cache/headers"
set message_cachedir="~/.mutt/cache/bodies"
set certificate_file=~/.mutt/certificates

set move = no

set sort = 'threads'
set sort_aux = 'last-date-received'

ignore "Authentication-Results:"
ignore "DomainKey-Signature:"
ignore "DKIM-Signature:"
hdr_order Date From To Cc Content-Type

You'll note that the only real tweak I had to make was set content_type = "text/html". This was necessary because if the content_type of the email were "text/plain" then blogger would replace all of my angle brackets with < and > and change the urls into hyperlinks. Figuring out how to set the content-type was very poorly documented and I only really found out because I went onto the irc channel and asked the developers directly. It is a little annoying and not at all obvious that the way to set 'Content-Type' is to set the variable 'content_type'.

Once I got the content-type set correctly, I found out that revision 1.5.17-1 had an incredibly annoying bug where it segfaulted whenever you try to send an email using stdin. I was able to sync to revision 1.5.17-2, but until that's put into the debian testing branch I have to keep an eye on that file in my apt repository.

There were two problems with the wikipedia script:

  • wget creates a file whenever it's run, which was polluting my file system
  • the name in the anchor tag isn't a true title

The first problem was easy, a simple rm took care of it fairly directly. Before I removed the file, though, I was able to extract the title tag from the file that was created.


echo "This round's challenge:<br>"
for ((i=0;i<2;++i))
    wgetout=($(wget -nv 2>&1 \
      | sed 's/^.* URL:\([^ ]*\) .* "\([^\"]*\)".*$/\1 \2/'<br>))
    echo -n "<a href='${wgetout[0]}'>"
    sed -n "/<title>.*<\/title>/s/.*<title>\(.*\) - Wikipedia, the free encyclopedia<\/title>.*/\1/p" ${wgetout[1]}
    echo "</a><br>"
    rm ${wgetout[1]}

One final piece that I needed was to keep track of the round number. To this end, the file ~/round holds the number of the next round, and is incremented in the crontab line.

0 0 * * 1,3,5 round=$(cat ~/round);/home/sixdegreesofwikipedia/bin/generategame | mutt ******* -s "Round $round";echo $(($round + 1)) > ~/round

Although this represents the work required to get a new game generated and posted every couple days, posting the victor from the previous day isn't complete. My friend Mike has written a script to verify that a list of links is correct, and I need to write a script that collects all of the mail for a single game together and passes that information into his script. In the meantime, you can play the generated games at

Wednesday, November 14, 2007

Improvements on the Wikipedia Script

(Edit: Blogspot is lame, if I set nowrap on an element then it will cut off the text at the preconfigured length.)

(Edit: apparently blogspot's preview of the code tag behaves differently from the actual post, so I had to insert the newlines manually.)

In my previous post I introduced a script that would create a start and end point for the six degrees of wikipedia game. In retrospect, I shouldn't have worked so hard to keep it on one line when it sacrificed so much readability. The following code is a little more DRY (Don't Repeat Yourself (Pragmatic Programmer)), and I hope at least slightly more readable.

for ((i=0;i < 2;++i))
        echo "$(wget -nv 2>&1 | sed "s/^.* URL:\([^ ]*\) .* \"\([^\"]*\)\".*$/<a href='\\1'>\\2<\\/a>/")

There are still a couple outstanding issues, but they're fairly minor:

  • Escaped double quote in page title breaks regex.
  • Underscores in link text should be replaced with spaces.

Tuesday, November 13, 2007

Six Degrees of Wikipedia Game Generator

(Edit: Blogspot is lame, if I set nowrap on an element it cuts off the text at the preconfigured length.)

(Edit: I seem to have made up the codeblock tag, so I'm replacing it with the code tag, which actually exists.)

My friend Mike made a blog posting last month and I thought I'd write some code to generate games. I was originally going to write it in lisp, but since I've been studying bash lately I wrote a one-line command.

This generates links to two wikipedia articles, it's not the prettiest script, but it gets the job done. There are still a few bugs so if anybody has an improvement they should let me know. (I'd say post it in the comments, but I believe code tags aren't accepted, so just email it to me)

echo -e "$(wget -nv 2>&1 | sed "s/^.* URL:\([^ ]*\) .* \"\([^\"]*\)\".*$/<a href='\\1'>\\2<\\/a>/")\n$(wget -nv 2>&1 | sed "s/^.* URL:\([^ ]*\) .* \"\([^\"]*\)\".*$/<a href='\\1'>\\2<\\/a>/")"

Wednesday, November 7, 2007

Additional thoughts on Hyperproductivity and Useful Computer Tools

After another week of trying to adhere to a structured life, I'm finding that a couple things are a little more difficult to maintain. The first is my calendar, although I've created a rather strict schedule, sticking to that schedule is a little more difficult then I anticipated, if I get even slightly off I stop following it altogether. This blog post is a perfect example of that, as I stopped checking my calendar and thus forgot until the last minute that I intended to write a post today.

One solution I began adopting earlier this week was to move anything that didn't have to happen at a specific time to my to do list. This was greatly simplified when my girlfriend introduced me to the recurring task functionality, which I did not realize existed.

Although task organization is an important part of hyperproductivity, it's not the only place where one can increase the amount of tasks accomplished each day. As I spend most of my day on the computer, any tool that allows me to shorten the amount of time I spend doing a repetitive task quickly overwhelms the time lost to the learning curve. The rest of this post will be devoted to tools I've found useful in increasing my productivity.

The first tool I want to talk about is Launchy. This little program is a really handy keyboard launcher, allowing me to skip crawling through the start menu. The way it works is no matter what application I'm in, I can hit Alt-space and start typing in my program name. Once the desired program is auto-completed, you can hit enter to start the program. I'm also obligated to mention that it has a number of plugins and skins you can use to customize it, but I honestly only use it for the primary function.

Next, I want to talk about my sidebar. This is a rather new addition to my productivity suite, and so I haven't fully customized it yet. I actually had a bit of trouble determining who to go with, but ultimately decided on Desktop Sidebar. I chose this one mainly because it allowed me to import an opml file, which neither Google Desktop nor SideSlide did. Although it still doesn't have all of the functionality I'd like, it does most of what I need right now, and I plan on creating a few panels to extend its functionality.

I can't really talk about tools that increase my productivity without mentioning emacs. I know there's a lot of controversy over text editors, but this one's really powerful and I'm significantly more productive when working in it. The Pragmatic Programmer talks about how you should pick one text editor and use it for everything, which I've taken to heart. I've invested a significant amount of time learning all of its various features, and it has made a noticeable difference in my development speed.

These are just a few of the tools I've adopted to improve my productivity. I was introduced to most of them through Lifehacker, which I highly recommend for anybody interested in tips on how to improve their productivity.

Wednesday, October 31, 2007


A state in which the rate of completion exceeds the rate of assignment where the rate of assignment is equal to or greater than that of the average person.

If you're like me, you always have a long list of projects that you never seem to have time for. The tips on Lifehacker have motivated me to approach my personal work queue more proactively.

The first thing I did was create a to-do list, I prefer the one built into my Palm Treo. This allows me prioritize and schedule the work that I never get around to doing.

Second, I'm constantly evaluating the following algorithm:

  1. Is my current activity on the list?
    1. If so, is it currently the highest priority?
      1. If so, continue working on this task.
      2. If not, why am I not working on the highest priority?
        1. If you are immediately able to overcome the impediment, overcome it now.
        2. If this is not the highest priority, but is the highest priority you are currently able to work on, continue working on it.
    2. If not, should it be?
      1. If so, I'll add it.
      2. If not, why am I doing it?
        1. If the task is unavoidable, continue doing it.
        2. Otherwise, stop doing it.
There are a couple major implications to this algorithm. First, it forces me to constantly think about the worth of my current task, hence weeding out superfluous activities like getting sucked into video games. Second, all tasks have a natural tendency to bubble to the top of my list. Once at the top, all other tasks take second priority to the current one, forcing me to actually get those goals accomplished.

Although this efficiently allocates my free time, determining what is actually free time still remains a tricky endeavor. Fortunately, my phone also has a calendar built in, which lets me map every bit of occupied time. I've explicitly written out every bit of my routine, so that regardless of how mundane an activity is, I realize that it occupies free time I could otherwise be using to reduce my to-do list. Now we've left the realm of successful approaches and entered what is still experimental in my life, because although I've drafted a schedule for my day, I haven't successfully kept to that schedule for more than a day.

Once I successfully force my life into a rigid schedule, I'll begin scheduling the tasks that currently exist in my to-do list into my free time on the calendar. This also requires a decent talent at estimating the amount of time a task can take, this is a talent in which I am woefully inadequate.

To summarize, I'm using a number of organization tools to replace the unproductive aspects of my life with work towards long term goals I've set for myself. I've also prioritized those long term goals so that all goals will eventually bubble to the top where they will get the most attention.