Ever wonder what the heck is going on in your django cache? Ever wish you could clear an entry or two or all? Well I have, and CacheMgr is my answer:

This does not negate the need for proper cache use, but it can be a life saver when you want to update your sites templates, but don’t want to restart the server because people are say, using it to submit proposals before a deadline. This app is not feature complete. Currently only the following cache backends are supported:

• dummy
• simple
• localmem

The remaining backends need to be implemented. The database and file based backends are simple to do, but memcached will take a bit more work. The system is extensible, so you can extend it to work with your own cache backends the same way you can with django’s cache system. All you have to do is extend the base class the same way you would with the django cache system:

class BaseCacheMgr(object):
    """No description has been set.
    """
    has_cull = False
    scheme = 'unknown'

    def __init__(self, cache, host, params):
        self.cache = cache
    def __iter__(self):
        """Should be overridden to iterate over the cache and return
        a dict with the form {'key': key, 'short_key': short_key,
                                     'repr': cache_value,
                                     'expires': datetime_expires, 'expired': False}
        """
        raise NotImplementedError
    def info(self):
        default = (str(self.cache.default_timeout)
                   if hasattr(self.cache, 'default_timeout') else 'None')
        return [{'name': 'Scheme', 'value': self.scheme},
                {'name': 'Description', 'value': self.__doc__},
                {'name': 'Default Timeout', 'value': default},
                {'name': 'Has Cull', 'value': repr(self.has_cull)}]
    def clear(self):
        raise NotImplementedError
    def cull(self):
        if not self.has_cull: return
        raise NotImplementedError
    def delete(self, key):
        self.cache.delete(key)

Fairly self explanatory when you look at the image above. Take a look at the simple backend implementation for more details.

Other features that need to be implemented:

• Clear Expired Button
• Paged View (like the admin)
• Sort Table heading links (like the admin)
• Search Fields (like the admin)
• Move the repr/short_key/expires helper code into the base class.

PyCon-Tech (the python behind pycon) is an open source project for the management of community run conferences. It is actually a framework for developing conference websites, and doing collaborative management, written on top of the django web framework. The project is broken into multiple application, most of which can be used independently. The issue is, not many people even know this resource exists.

One of the stated goals of PyCon-Tech is to give back to the python community which makes the conference possible. This series is my attempt to shine a light on some of the general use applications under the hood and how you can use them for other projects. The first app in this series is the feedutil, a lightweight generic RSS/Atom feed pull. (it is also the only app in PyCon-Tech being used for other projects that I know of).

Overview

Feedutil is a lightweight app for pulling RSS/Atom feeds onto your site with django template tags, or custom views. This is not a full feed aggregator like feedjack, but you could write one with it. This is more along the lines of the blogger plugin which allows you to have the latest 5 entries from an RSS feed appear on your sidebar. We use it on the PyCon website for the main about page which has summaries of the latest PyCon Blog entries via Atom, and on an organizer page which replicates a Trac RSS issue feed for open website bugs. This does not use any django models, and there is no database interaction. You could create your own models for managing your feeds, but that is not the purpose of feedutil.

The feedutil provides two primary template tags {% feed feed_url [posts_to_show] [cache_expires] %} and {% get_feed feed_url [posts_to_show] [cache_expires] as var %}. There is also a higher level interface to feedparser which includes caching pull_feed(feed_url, posts_to_show=None, cache_expires=None) => posts_dict.

Read the rest of this entry »

Well I have been rather busy lately and have not had time to post about anything. There has been a lot going on with ODF, and Py3K, but I have nothing of value to add on those fronts. There is a lot going on with PyCon 2008 Chicago, and I have been neck deep in Django for about six weeks now, but that will wait for another day. The big thing I want to mention is the Django World Wide Sprint! I have been on the fence about attending this as I am already overwhelmed with work, and this weekend is going to be particularly busy with stART on the Street. But the draw is proving too much.

So I will be taking part in the sprint. I have no axes to grind, so I will most likely focus on bug reports. I live in western Mass, and work in the Burlington area, so my ‘Boston’ location is a bit disingenuous. I know there are a number of Django people in the Mass area, and I would love to meet up somewhere to coordinate. My current plan is to commute into Burlington and camp out at Borders at WaySide Commons (google map). I picked this location for many reasons, but am not committed. (Reasons include: away from home, coffee, free internet, coffee, public transportation, coffee, across the street from work, coffee.) So if anyone is interested in meeting up, let me know!

or staring at grains of salt.

I am currently very very frustrated as I am trying to create a brine for making a nice smoked shoulder for a party Saturday and for the life of me, I have been unable to find pickling salt! Pickling salt is very very fine and does not contain the anti-clump agents (usually corn starch and potassium nitrate), and iodine (that which makes the ocean smell like the ocean). It also dissolves in cold water. Regular table salt gives pickles or brined meats a brown-orange color, can add a somewhat ‘fishy’ smell to them, and does not readily dissolve in cold water. Defiantly not what I want for smoked pork. I am heading to a local ethnic butcher in the morning to pick up my meat order, and I am hoping they have some. All this reminded me I wanted to make a post about the recent discussions around the default template system in Django.

There are a number of posts about logic in Django templates. The short version for those who do not want to do a bunch of reading:

• Django template logic contains a minimal set of functional operators, but no real logic operations (i.e. you can’t write {% if len(items) < 3 %})
• Why not just use python as the template language?
• You can use other template engines in Django, the doc is very poor on this, or how such integrations would work.
• Django tries to strike a balance and meet the 80/20 rule for template use cases or ‘hit the big bulgy part’ of the bell curve of users.
• Complex logic in templates is bad. The language should be minimal, and designed for UI writers.
• Complex logic in templates is good. UI writers are not stupid, give them power!
• Design is now in CSS, and HTML is the structure, thus an integral part of the programming producing that structure, not design.

Ok, that is the extreme cliff notes version, and I apologize if I am misrepresenting anyone here. There are many good points being made and some differences of opinion. I have my own opinions here, but first lets see what a very experienced PHP expert has to say on the matter circa early 2006:

Hopefully you found some simple ways to improve your template design, but also recognised that some complexities can arise when doing this and sometimes comprimises must be made.
One (simplified) way to look at it is like this:

You should be able to completely change templates, including all image files and CSS files, without having to touch any of your PHP files.

All fundamental data required in your templates should be fetched or determined ahead of time and assigned to your templates (this doesn’t include meta data, such as the length of a string, etc.).

That bolding is the authors emphasis, not mine. I believe the Django template system helps make this type of separation between templates and logic. I agree with Jacob that it strikes the right balance and hits that sweet spot. There was another blog post which summed up my feelings quite nicely, but of course, I cannot find it for the life of me. For PyCon-Tech I added a lot of logic to the templates. I felt I could make up the difference in the CSS. For some of the CSS, I templetized that as well! I made custom template tags for modifying the local context (variable evaluation and assignment no less), and went so far as to have a filter which modified and saved the model! The Idea there was if you included or extended the template, you were viewing that data, and you wanted to update the ‘last_viewed’ record.

PyCon UK is using PyCon-Tech, for the proposal system at least. But the templates are so complex, and the logic for the structure is very difficult to integrate into another site which is not based around having the navigation style of us.pycon.org. The end result is the coolest applications will not be used. The templates are code, and thus the structure is part of the code. The end result is, the templates are not templates, they are code. It might as well be python, and thus to use the conference framework for your conference of choice, you must change the code. This is with old Django functionality even. YUCK! My main mistake here is that I have much of the page structure determined by user permissions which are determined in the template. Once you make that decision, it kind of cascades to everything else.

Now some people have no problem with this, and are smart enough to keep their html structure generation code clean enough that CSS is all you need to change (Plone comes to mind here). For those people, I guess there is nothing wrong with python as the template language. Coincidentally I was talking with a friend about implementing PHP in python recently. We came to the conclusion that this would not be too hard to do. It would require a bit of encoding magic, a custom import loader, a module level meta class, some minor monkey patching to __builtins__. Now there is no need to do all this if what you want is python as the template language. For that, a custom encoding, and a module level metaclass should be enough:

# -*- encoding: python-template -*-
<%python
import pytpl
__metaclass__ = pytpl.metaclass
python%>
<head>
<title>foo</title>
</head>
<body>
<%python
for i in xrange(30):
    echo(i) # importing pytpl, and the metaclass adds echo to the module global namespace,
          # maybe it does a magic_escape? wait, thats another post…
python%>
</body>

This is basicly a pybraces on steroids, and adding an ‘echo’ and maybe some other minor additional helper functions. The encoder code builds a template document model for the local contexts and replaces them with ‘echo’ calls for that text. All echo does is add the text to a document or string queue. the metaclass deals with things like adding a __main__ block which if run from the command line will print the string queue to stdout. If a imported as a module, well there are other features added to the module like functions for adding variables (context) to the template, dealing with other template imports, getting the rendered string, etc. I have no real desire to work on such a project, but some times an idea grabs a hold of you. Why not use python as the template language? Well I have my opinions on why I would never use such a system, but that is my choice.

WOW! I mean WOW! James Tauber has done it again, and no doubt I will be wasting enjoying my time on his latest site Potter Predictions. I love his cats or dogs site and reveling in the predictive Potterness. Of course it’s built on his Quisition framework written in Django. I added one prediction, and have a few others to add. I need to create a group and invite friends…. and to pick a name for the group before all the good ones are chosen. Gah, so much for working on my project tonight!

Once the book is released the stats will be closed and the results posted. I can’t wait… I soo want to know what people think. I need to e-mail the penny-arcade folks, tyco will go nuts for this!!!

[UPDATE: I got group Gryffindor!]

Peter did another fantastic job putting together this months meetup. We decided to stick with Wednesdays so we would not collide with the Plone meetup which is on Thursdays, but um… oh well…

There were two guest speakers:

1: George Lambert, Goldenware Technology
2: Mike Pittaro, SnapLogic open source data integration Project implemented in 100% Python

I decided to try something new and record the event on my little sensa mp3 player. The audio is bad at best, but it is mostly audible. We were in Somerville so at some points an airplane goes overhead. If these prove useful to people I will bring better recording equipment next time. My A/V production equipment is tied up on another project so all I did was split the audio into multiple tracks and do a lame re-encode. I tried to keep the files to under 25Meg while splitting based on topics. Unfortunately meetup.com has a 10Meg per file limit and a 100Meg per group max, so that was out of the question. The first file is under 8Meg, so please check that out first and only if you can withstand the audio quality, check out the others. I and my bandwidth will appreciate it.

NOTE: The audio is extremely soft at points and at the beginning, so you will need to crank the volume up.

• Introductions and Django.June recap (mp3, ogg)
• Mass TLC recap, and an extensive discussion on GPLv3, Licensing, Patents, and Python (mp3, ogg)
• Lightning rounds with George Lambert and Mike Pittaro. (mp3, ogg)
• Open Discussion (mp3, ogg)

The software George Lambert mentions which is used to view changes in the GPLv3 draft is Plone! Though there is talk of converting the FSF web site over to a Django based one. I sent an e-mail to the lists giving better information on OLPC for those interested as well. Noah Kantrowitz responded offering to help anyone in the group get started with development.

Paul Bissex proposed, organized, hosted, and MC’d a Django unconference (Django.June) over the weekend. Paul has a wrapup on his blog and Jay Graves put some flickr pictures up. There were over 20 people in attendance, and by all accounts it was a huge success. I was only able to catch the second half, and was not able to stay for the music festival. It was fantastic seeing people whom I primarily know from reading mailing lists, blogs, and irc chats. There were so many things I wanted to talk to specific people about that never happened. The conversations were just too interesting. Adrian’s impromptu talk on contributing to Django, the development process and his experiences on contributing to large OSS projects was illuminating. I almost didn’t go out to feed the meter because of it; there was a two hour max, and the car next to me got ticketed.

Read the rest of this entry »

I mentioned earlier that June is fully booked (kids birthday parties, and more events that there are days; not to mention work deadlines). Unfortunately one event I might need to miss is Django.June which has me very upset. It is occuring at the same time as the Plone training I also want to go to. These are two encredible events. A Chance to learn from arguably the top Zope 3 instructor in the world, and a chance to attend a Django unconference at a Django festival (Adrian Holovaty will be less than two hours from my house!)

Django.June

An informal meeting of Django web framework people
Northampton, Massachusetts, USA – June 16, 2007

Here in Northampton, there’s a fun annual music festival called Django in June. It’s all about gypsy jazz — performances, music clinics, and jam sessions.

During the festival a small group of Django (web framework) developers are planning to get together. We’ll do it during the day on June 16, with the evening free for gypsy jazz!

Details

This is an informal event in the “unconference” vein. I’m hoping to have several short talks and maybe even some collaborative coding. I’m perfectly happy to have the content be driven by the desires of the attendees.

We’ll be meeting right in the heart of town, in the public meeting room of the Media Education Foundation. There’s a projector, a big meeting table, and plenty of chairs. The MEF shares the building (which used to be the town firehouse) with one of the best cafes in Northampton. As long as we’re buying some of their excellent food and drink it shouldn’t be a problem to use their free wifi. The room is reserved for us from 9am to 5pm.

See the NorthamptonArea page for a little more info about the locality, lodging options, etc.

People

Check out the DjangoDotJuneAttendees page. Please add your name and info there if you plan on coming!

The problem is that it is also ‘girls camping weekend’, so my wife will be away, and I need a babysitter. Not any babysitter, one available all day, and which I trust enough to be two hours away. I want this to happen, but there are other events days later that I will be attending. June is going to be fun!

Forget the cute dictionary definition tonight, we all know what being backlogged is like. Too many projects, tasks, bugs, deadlines, and a sever lack of time. A co-worker once called it suffering from ‘an acute temporal deficiency and a terminal case of deadline-itous.’ Terminal in the sense that you either the item gets killed off, or you do. You can’t do it all and something has to give. June and July have been fully booked for months, and I will be making a few announcements of some interesting things soon. It looks like some plone classes, and two sprints are going to be casualties. A few of the things I have been able to get done (NOTE: these were all team efforts):

1. 15 new Yew shrubs planted in the back yard (NEE!)
2. All the beds cut, weeded, trimmed, pruned, and a huge new bed one roughed out
3. First half of the border stones are in, awaiting four yards of mulch
4. New feature productized in the engine after four years of research and development
5. Re-re-re-re-re-re-re-learned how complex SWIG typemaps can get
6. Learned how roundup differs from trac, and how it does not (conference paper submission and review, NICE!)
7. Created a detailed list of new features and code snippits to incorporate into the PyCon-Tech code base (dynamic schema creation rocks!)
8. Was present at two very successful python meetups (can’t really take credit for their success, but I actually made it to them on time!)
9. Wrapping my brain around multi-threading (ipython + Stackless + scipy + matplotlib = my new shell; too bad matplotlib is not thread safe on win32)
10. Got the LightScribe CD art done for a set of Sugar Live CD’s I am planning on burning

I have 6 other blog posts started (most very short). All last week I was sleep deprived. This weekend improved things, mainly because my body shut down. I promised myself I would not write e-mails or these posts late at night or when I was over tired, as I tend to ramble and go off on tangents. oops….