Using **kwargs with CherryPy and WTForms

June 23rd, 2013

I just ran into a problem with using **kwargs as a catchall parameter in cherrypy while sending the data into a form built with WTForms. Like this:


def index(self, **kwargs):
sf = forms.SearchForm(kwargs)

This leads to an error:

TypeError: formdata should be a multidict-type wrapper that supports the 'getlist' method

The reason is that WTForms expects something that supports the ‘getlist’ method while kwargs is a plain dictionary and does not have any ‘getlist’ method. The solution i found was to subclass dict and just add the getlist method. As far as I know, getlist refers to a method in the official cgi module and the cgi module documentation says that

This method always returns a list of values associated with form field name. The method returns an empty list if no such form field or value exists for name. It returns a list consisting of one item if only one such value exists.

So, a simple implementation of a subclass of dict that just adds the getlist method looks like this:


class InputDict(dict):
def getlist(self, arg):
for key in self.keys():
if self.has_key(arg):
if isinstance(self[arg], list):
return self[arg]
else:
return [self[arg]]
else:
return []

And we can use it to fill our form:


def index(self, **kwargs):
sf = forms.SearchForm(InputDict(kwargs))

Highlighting WordPress posts

November 11th, 2012

A trick I just discovered while trying to spice up CSS Viking is to use a category as a flag for highlighting an article. This is very easy because every article gets category-categoryname, tag-tagname etc. as classnames on its’ article element. This is theme-dependent but it works in the current default theme (WordPress 2012), Blaskan and probably most other modern themes.

I don’t display categories on CSS Viking (just tags) so I made a category named highlight that will be used for highlighting posts. Since it is not displayed the only effect is that articles are given the class category-highlight.

There are endless possibilities to visually highlight an article, but the currently used code for this is the following:


/* Highlighted articles on frontpage */
body.home article.category-highlight {
background-color: #FFFFFF;
color: #000000;
}
body.home article.category-highlight * {
color: #000000;
}
body.home article.category-highlight footer {
background-color: #0088FF;
color: #FFFFFF;
}
body.home article.category-highlight footer * {
color: #FFFFFF;
}
body.home article.category-highlight:hover {
background-color: #FFFFFF;
color: #000000;
}

And it looks like this at the monent (the white/blue one is the higlighted):

A part of the CSS Viking frontpage with a highlighted article

The current code higlights an article only on the frontpage but the class category-highlight is applied to the article in most other views too so it can be highlighted wherever it is displayed. If you don’t want to use a category for this, consider using a tag (and make sure tags are not displayed in your theme) or install a plugin that provides something usable for this. I’m sure they exist but a two-minute search didn’t uncover any of them.

A simpler way to right-align a block-level element

December 3rd, 2011

The standard way to right-align a block-level element is to float it. It works well and is simple if you don’t mind that the following content creeps up beside your element. If you want that, fine. If not, you need to add a clearfix hack or make sure that the following content content clears your element.

A different and simple way to achieve the right-alignment that also makes sure that following content doesn’t creep up can be achieved with some margin-trickery.

As a start, we take a look at the html and css that is used to center a block-level element within its parent:
<div class="parent">
<div class="block">
</div>
<p>Some other content</p>
</div>

.parent {
border: 1px solid black;
width: 500px;
}
.block {
background-color: blue;
margin-left: auto;
margin-right: auto;
height: 100px;
width: 250px;
}

background-color, height and border are only included to visualize the elements.

What we want to achieve is simply done by setting the right margin to zero:
.block {
background-color: blue;
margin-left: auto;
margin-right: 0;
height: 100px;
width: 250px;
}

This also has the added benefit of introducing less side effects for older versions of Internet Explorer than the use of floating. Handy if you or your clients care about that.

View sample page

A cache module for python CGI scripts

October 14th, 2008

After an earlier failed attempt at writing a cache module for python CGI scripts, a not-so-nice email from one of my web hosts made me try again after they mentioned they have no plans for enabling Apache’s mod_cache module. I suspect that the pickle module somehow messed up my previous attempt, leaving me with no other choice than to disable it and burden the server more than needed.

The building blocks for a flat-file cache module is a unique mapping from url to filename and a place to store the files. An md5 hash creates a sufficiently unique mapping and a directory is a nice place to store files. We also need a time limit (in seconds) so the web pages are not stored forever.

The complete module


"""A module that writes a webpage to a file so it can be restored at a later time
Interface:
filecache.write(...)
filecache.read(...)
"""

import time
import os
import md5

def key(url):
k = md5.new()
k.update(url)
return k.hexdigest()

def filename(basedir, url):
return "%s/%s.txt"%(basedir, key(url))

def write(url, basedir, content):
""" Write content to cache file in basedir for url"""
fh = file(filename(basedir, url), mode="w")
fh.write(content)
fh.close()

def read(url, basedir, timeout):
"""Read cached content for url in basedir if it is fresher than timeout (in seconds)"""
fname = filename(basedir, url)
content = ""
if os.path.exists(fname) and (os.stat(fname).st_mtime > time.time() - timeout):
fh = open(fname, "r")
content = fh.read()
fh.close()
return content

A minimal example, including time measurement

Instead of explaining what the functions are doing, I hope they are fairly understandable and that a usage example is sufficient for understanding how it works. As a bonus, the example includes timing so you can see how long it takes to build your pages from scratch as opposed to reading from cache.


import time
startTime = time.clock()

import sys
import os
import filecache

cache_timeout = 10
cache_basedir = "cache"

cache = filecache.read(os.environ.get("REQUEST_URI", ""), cache_basedir, cache_timeout)
if cache:
print cache
print ""%(time.clock() - startTime)
sys.exit()

# generete output
output ="stuff"

#Write output to cache
filecache.write(os.environ.get("REQUEST_URI", ""), cache_basedir, output)
print output
print ""%(time.clock() - startTime)

Store the example as example.py, the cache module as filecache.py and create a directory named cache. Run as


python example.py

Note that the timeout is set very low, at 10 seconds. This is fine for testing but not much more.

While this very minimal example is slower when the output is fetched from cache, I can assure you that this is not the case with more realistic web pages. In my case, I have experienced speedups from around 0.7 seconds to hardly measurable time (0.00 to 0.01 seconds). This does not include the time needed to start the Python interpreter and importing the time module so a very popular site might still get you in trouble with your web host. I think the mod_cache module for Apache would take care of that too, but that wasn’t available in my case.

There is no way to remove the cache other than a rm * or similar in the cache directory. It works for me but probably not for a very dynamic site.

This cache module is used in production at Good Web Hosting Info with a timeout of one hour. The time measurement is shown at the bottom of the source code. It’s not a very busy site so it’s quite likely to get a page built from scratch if you look beyond the front page. The time precision is 1/100 second so cached pages normally have 0.000 of 0.010 seconds.

The python files are also available from here.

Using SBCL for Common Lisp shell scripting

October 5th, 2008

I have previously developed some Commom Lisp shell scripts with Emacs/Slime/SBCL and used Clisp for running the scripts. But after running into a compatibility problem between SBCL and Clisp while developing a script for maintaing an automatic mirror of my music collection where flac files are converted to much smaller ogg files, I decided I might as well do what has to be done for using SBCL directly for running the script.

Prepping SBCL

As was mentioned in a comment in my previous post about using Common Lisp for shell scripting, the SBCL manual outlines a piece of code that must be added to an initialization file (I have added it to my $HOME/.sbclrc file).

After adding that to .sbclrc, the next step is to add a shebang line to my script


#!/usr/bin/sbcl --noinform

Adapting for development

The problem now is that the code that is needed for running the script will also be executed while compiling the file inside Slime. I found that this can be fixed by inspecting the *posix-argv* variable. This is a list, that inside Slime has one entry (“/usr/bin/sbcl”), and when the script is executed from the command line has two entries (path to the SBCL interpreter and path to your script in addition to possible command line arguments to the scripts). So if *posix-argv* is longer than 1, we can execute the script. One caveat here, if you are inside a package, *posix-argv* is not directly available. We must use sb-ext:*posix-argv* instead. This leads us to the following file structure:


#!/usr/bin/sbcl --noinform

;; Lisp code (defuns, defclasses, whatever)

;; If run from command line, the following if-test will succeed
(if (> (length sb-ext:*posix-argv*) 1)
;; Code that is executed from the command line only goes here
)

Now, the script can be developed as usual in Slime and be executed from the command line without changing anything. This also means that the command line arguments to the script is found from the third position and onwards in ext:*posix-argv*.

Script details

The complete script.

This loops through my music directory and creates a parallell directory structure where flac files are converted to ogg files and other file types are just hard linked to the original file. Soft links does not work, because when you try to copy the files to your portable music player, you get a soft link/permission error instead of getting your actual music file.

The flac to ogg conversion is done with sox, so you need to install sox in addition to sbcl and cl-asdf to run this script.

Script usage

General:
./converter.lisp basedir targetdir quality
Example:
./converter.lisp /media/sda4/musikk/ /media/sda4/ogg-musikk/ 5
Note: You need the ending / in the directory paths, it does not work otherwise.

Portability hints for Clisp

The main problem I ran into with Clisp is different behaviour in the directory function. This can probably be fixed by using the complete pathname library from PCL chapter 15. The shebang line must be changed to f.ex.
#!/usr/bin/env clisp
and command line arguments are available in ext:*args*

More hints for porting shell scripts to other CL implementations are available in the Common Lisp Cookbook.