Dougma (dŭg·mə) n.

This year we tried something new to get feedback from attendees this year. After a talk you would put little plastic bingo chips into one of three buckets to vote. We also have a talk interest system on the schedule where we get a count on what talks people are selecting. I captured the data just before and after the conference along with other metrics. All this data I have compiled into a massive raw data dump on google docs. It’s a firehose of information, you have been warned. Check the second and third sheets for details on the data. I would like to boil this data down into something useful. More on that below.

History

It was started by a suggestion by Bruce Eckle a few days before the conference. He mentioned a system from another conference where attendees could put slips of colored paper into a basket at the end of the talk to indicate what they thought (red=bad, yellow=ok, green=good). Yarko suggested using marbles, and then Ted and I ordered had found and ordered 108 tubs of colored bingo chips (clearing out the supply). It was at this point that Yarko pointed out that 3 colored buckets and a scale would be simpler, faster, and much, much easier. It is taking me longer to type this than it took for all that to happen… In retrospect we moved a bit too fast on the idea.

Boiling it Down

There are a number of issues with the data. There were two occurances of ballat stuffing which are marked with comments on the spreadsheet. Ignoring those, and some cancellations. The biggest issue is that people vote with their feet at PyCon. If you dont like a talk, you go to a different talk. The IRC channel is filled with people more than willing to inform you of how the other talks are going. Some celebrity speakers can draw people from other talks. There is open space and plenty of other things to do. The raw data includes talk interest calculated before and after the conference from the schedule app. These correlate (@ about 97%) with actual attendance. How that works out is a post by its self as there are finer grained details.

I am thinking of finding corrilations within the data and then using that to compute weights to be used against the voting counts. Then using that to discount the green votes by the attendance and yellow and red votes. The end result will be quite noisy and most of the talks will fall within the delta error. Many of the talks will fall above the delta error and we can use those for determining which speakers to give invited talks and to influence the program commitee decisions next year. This information would also be valuable in conjunction with the online and printes survey information (to come later) for speakers.

If someone would like to help me with this I would be ever grateful. I love working on stuff like this but we need to start working on the 2010 stuff yesturday (actually some people have been working on PyCon 2010 Atlanta for months).

PyCon is less than a week away. Where has the time gone?

Registration

Online registration is closed. The full count is not determined yet (due to sponsor passes, financial aid, and a few minor issues) but attendance will be around 850, but could be as high as 900. We had 400 people in ‘06, 600 in ‘07, and bearly broke 1K in ‘08. With the dramatic downturn in the economy and corporations removing all travel and conference expenses, this is astounding! Initially we were planning on about 1200 (plan made back in April last year), and some believed halving that number would be a better estimate not long ago (well I made statements to that effect anyway).

The new registration system this year is fantastic (Granted anything would be better than last years mess). Massimo, Yarkot and Carl really out did them selves. A. M. K., Yarkot, Massimo, Carl, David, and Kurt have done an outstanding job managing all the issues and fielding attendee requests. Early on I was keeping track of issues which poped up on the reg list to make sure someone was taking care of them (but insanely thankful I was not doing it this year). I stopped after 3 weeks as I was just wasting my time. There were a few kinks here and there, but they were minor.

The Schedule

I never did a follow-up on the program committee work, mainly due to time and general program burnout. I think the schedule is fantastic. If you don’t like the way the talks are scheduled, you can blame me. I stole the job from the person whom was doing it and unilaterally put together the schedule you now see. It was a beast of a job. I tried to get the talks placed into tracks. A testing track, a core python track, etc. I also tried to make sure no two talks occuring at the same time would appeal to the same people. I also tried to balance the talk levels to ensure there was something for every level at a given time and that tracks progressed across the conference. That way you could go to a beginner talk on a subject on the morning and be ready for an intermediate talk on teh same subject in the evening or the next day. Against that backdrop there was also the requirement that anyone giving multiple talks would have those talks given on different days. Some speakers have 2 talks, none have more than that, and it is key to only give one core talk a day if you want the quality to be maintained. Also there was the problem of the talk lengths….. My first draft has some glaring errors in it which thankfully other noticed.

Last night I actually made it to the Boston Python Meetup. There 3 PyCon presentations were given and they were… encredible! I am sooo glad I was able to make it. At first I went because I wanted to see the talks so I could see other talks at Pycon during those times. I will be attending Jesse Noller’s talk again at PyCon. I know I will learn more by seeing it a second time, and I learned quite a bit seeing it the first time. The other two talks have already spawned some changes to plans at PyCon which I will hold off on mentioning until I hear back from people.

Audio Visual

I did not mention this is the volunteer post because I did not want it to get lost there. This year Carl has put together an absolutely insane proposal for AV. He has a dedicated staff for managing the AV complete with capture systems (to capture the raw feed going to the projectors), camera operators, a mixer, and an editor doing real-time production!!! If we wanted to put in some extra effort (and a fair amount of cash) we could broadcast live! I would not expect TED level quality, but the samples produced so far are on par with google talks.

Misc

We gave out a mamoth amount in financial aid (don’t have the number from Ted yet). Open Space use is booming. Sprints are picking up speed and people quite fast. It is going to be a very enjoyable PyCon this year!

Question

The PyCon Lab: Solve This! is most likely going to be held as an Open Space this year. How much interest do people have in this? It has not been properly advertised yet. I am not running it this year, but I will try to poke my head in.

Ok, I just put up a post on the PyCon blog asking for help, and I have to say I dropped the ball here. I could come up with a number of excuses, but the truth is I should have gotten out a call for volunteers months ago. So if you are going to PyCon please consider volunteering as a Session Runner. Below is a rambling laundry list of changes at PyCon this year. These come directly from feedback from previous year attendees, speakers and volunteers. We can not make it happen without more help. Wether that help be as part of the Session Staff or as part of the general Volunteer Mob, we need the community to step up to make it a much better experience for all.

Session Runners

We need Session Runners desperatly. The job is easy; much easier than being a Session Chair (which we also need). You volunteer for a set of 2-3 talks in a session, so it’s not for even a full day. You meet with presenters in the Green Room before thier talk. Help them test their laptop and get thier slides ready. Maybe you get a copy of the slides on a USB drive. You then go with them to the talk and help them hook the machine up if they need help. If there are people interested in an Open Space continuation from the previous talk, you help them get a room and get it posted on the board. If not, help the Session Chair move people out from the provious talk if needed. After that, you do whatever you want until 15min before the next talk. This means that you might miss the beginning or ending of a few talks, but beyond that you get to talk to cool presenters and keep things running smoothly.

The Perks

You get to have a cool radio with headset (either boom, or secret-service style). You get access to Green Room food (just the normal snacks but available durring non-break times). You get to meet and work with the speekers one-on-one. You get to help the conference run much more smoothly, allowing the talk time to be the talk, and not setup or breakdown. You get a ribbon for your badge (if you want it)! By helping out for one session, you are ensuring we have enough volunteers to cover the other sessions so that you are not having talk time wasted watching a presenter set up his laptop instead of presenting. That last one seems like reason enough to me.

A Green Room/OPS for PyCon

Here is the deal. In past years I have noticed a number of small problems which crop up at PyCon, each in and of themselves are not really a big deal. Together they amount to a fair amount of lost time and wasted effort. Much of that wasted effort is in trying to solve specific problems! This year I want to try an experiment to see if we can put in place a framework for dealing with problems in general. The number one way you can improve the ability to deal with a problem at a conference: communication. With that said I will still try to target some specific issues raised in the past.

Session Staff and many department heads will have radios with headsets. We will be able to communicate with key people very quickly and be able to get information out FAST. We could have prevented some rather public issues last year if we had this simple tool alone. These are not toys and we have a limited supply. They are public band FRS radios, so the world will hear us. I won’t bother with more ‘radio’ details here, but the point is between the radios and onside ‘general volunteer mob’ we should be able to get communication to and from key organizers solved.

The Green Room/OPS (behind the registration desk) will be a communication hub with white boards and a fixed radio. Session Chairs whom are stuck in a room will have a means of communicating and getting help at the push of a button. Knowledge of which speakers have arrived already and which are MIA will be known (without having to search stacks of badges). When a speaker, volunteer, or sponsor has a question, they will have a central place from which they can be directed to someone whom can answer thier question. We might not have the answer at OPS, but we will know who does. General attendee questions will be handled at the volunteer center next to registration, or at registration its self.

One of the main things I want to achieve with the Green Room is to give back to the speakers. Those who present at PyCon pay their own way, just like attendees, yet we historically have not treated them very well. These are people whom we are asking to come and present after a grueling review process. They often miss out on the break food. They have to bring their own equipment and set it up them selves (cutting into their presentation time). They are just expected to be where they are supposed to be with no real support or communication. On the other side the Session Chairs and organizers are also volunteers whom pay their own way. They have no clue where the speakers are, what problems are going to come up, and have little forewarning, if any, about special needs of presenters. In the end we have a mass of people at the podium after every talk, no clue if the next speaker is even in the building and the first 5min of every talk is watching the presenter hook up their laptop. In theatre, we call this a farce.

Massive Laundry List O Changes

See.. we do listen

• Public Volunteer Center (next to registration)
• Large board for people to post stuff (‘Hey Bob, we are at Uno’s!’)
• Tons more Lightning talk sessions (two a day!) all via signup sheet at the conference
• Financial Aid forms are dropped off in Volunteer Center (Ted will collect, make checks, and distribute at a set time each day in the green room. Exact details will be sent to those whom received aid via e-mail)
• Session Chairs are provided stop watches
• Session Chairs are provided largeish signs on sticks with 15, 10, 5, and STOP for clearly alerting speakers
• Session Staff are not all alone in a room, but can call on volunteers and will have help for the transitions
• Radios for Session Staff and Dept Heads (mine can make a duck call!)
• Speakers go to the Green Room after registration to check in and get a personalized handout
• Speakers go to the Green Room before talks to get prepped so they are ready to go immediatly when their talk starts
• Speakers and Session Staff are introduced to each other earlier whenever possible
• Printers, a duplicate projection setup, and display adapters in the Green Room will allow problems and talks to be prepped BEFORE the podium
• Speakers and Session Staff have access to snacks in the green room (nothing special, but just this small thing will have a huge impact)
• If presenters or attendees wish to have an Open Space followup, Session Runners will have do the leg work to get the room assigned. Session Runners and Chairs can help prompt people and even make announcements. The Open Space need not occur immediately
• Session Runners will have USB drives for getting the talk slides just before the talk, and get them up on the website for the presenters
• Session Runners and Chairs will be able to edit talk descriptions on the website for adding links and other information promised during Q&A
• Most transitions between talks are now 10min instead of 5! (This will make a huge difference in the talk transitions and overages which always occur)
• Badge Ribbons (for those who want them) The rainbow ‘VOLUNTEER’ ribbon is pretty…. shiny…. you know you want one… you know what you must do to get one…. precious….
• A brief orientation for speakers and session staff on Friday evening to get everyone on the same page. Then anyone whom can not make that session will have plenty of people who can fill them in!
• They key is to get as much information to as many people in a timely manner. If everyone is up to date on the information, then we can work together to solve the problems that arise instead of against each other due to lack of information and co-ordination.

There is a bunch more minor things as well which may or may not occur depending. The Idea is to see what works and what doesn’t, try to only do the things which work, and share that knowledge with everyone else. If something in the above list is causing more problems than helping, then we will stop doing it. Simple as that.

Right now (as I am typing this) Ned Batchelder is giving his “A Whirlwind Excursion through C Extensions” talk at the Boston Python Meetup. This is a talk which he will be giving at PyCon; a talk I argued for as part of the program committee (with full conflict of interest information given). It is scheduled against some other very good talks (I know, I did the schedule, mores the fool me). I did want to have a chance to see this talk before PyCon in the small atmosphere that is Beta House, and have a chance to give feedback and ask more questions than I ever will later on. I feel like I am failing here. The snow was just the last straw on top of an already insane schedule.

Besides, why travel 30min to see and help a friend when we can both travel half a continent in a few weeks and not talk to each other there!

So here is a public apology of sorts to Ned. I hope we can connect at PyCon and have a chance to chat for more than 5min.

With luck the video of the presentation will be put up on ustream soon like the James Tauber Pinax one was (James is 15min away from me, same story).

1 hour per frame, 24 frames per second, 1 second per day, 40 days.

High res avi is here (14Meg).

Just wanted to give a status update to all those who are wondering what is going on with the PyCon US 2009 program committee.

We have recieved 109 proposals. (NOTE: there is now a dynamic image URL so you can embed the above graph in your blog! You need to right click as it uses  redirect.)

This number is lower than last year, but that is to be expected. Why? Because:

1. We did not advertise like last year (where we actually paid for professional help to get the call for proposals announced all over the place).
2. We did not announce it in popular IRC channels like in past years
3. We started collecting proposals later than last year
4. We had a much shorter submission period this year (even before last years extension)
5. We did not extend the submission period like we did last year
6. We did not put out regular announcements of the deadline approaching like we did last year
7. There have been a TON of new regional and international python conferences this year
8. The economy?

Seriously, I am shocked that given all that, we still got the number of proposals we did. I am also ecstatic to say that we have 39 reviewers from a broad cross section of the python community. We also have a number of invited speakers. These speakers were selected from the feedback forms from previous years and from general feedback from the greater community. The organizers and the Program Committee reviewed feedback from 08 (the surveys, blog posts, reddit, and mailing lists, IRC logs, etc) in an attempt to discover what things worked last year, the things which did not, and how to do better.

So what is happening now?

The program committee is activly commenting on and reviewing the proposals. At the time of my writting this, there have been 170 comments. We use comments to communicate with the authors for clarifications and to suggest changes. Authors can still edit their proposals and we are working towards reaching a critical mass. As of this post there are only 132 reviews written. That number will grow to well over 400 before we are done. That is a monumantal effort for the program committee, and it is just the tip of the iceburg.

What happens after that?

Once the comment period is over, the PC will will head into the home stretch for reviewing. The proposals will be made read only and the PC will make final review edits, and ensure that there are no proposals which are under reviewed. That is where the fun begins; if you consider trying to get 40 people to all agree on anything fun. I won’t bother rehashing how that happens without thing degrading into chaos except to say that should not given the processes used.

What do you think so far?

There has been quite a bit of discussion on the pycon-pc list on how reviews should be written, how best to communicate issues with the authors via comments, and how to manage the PC effort. Even though the core of the process is stable, every year it is different. One complaint last year was that there was not enough transparency with this process. I have tried to address some of this in blog posts, etc. and this is part of said effort. One missing element is early public discussion on the PC from submitters, reviewers, and the general community. So lets start some of that right now.

If you are so inclined please leave a comment on this post on your thoughts about the process thus far. Please mention if you are a reviewer or proposal author for this year, for a previous year, a past PyCon US attendee, or just someone in the general community.

There are some very strong opinions out there, so please do not get upset just because you don’t agree with someone elses opinion and please keep it civil.

UPDATE: Yes the registration system had issues, we are talking about the program committee. No, authors were not hidden from the reviewers last year, that is a myth I have attempted to debunk time and again. Yes, we all know the sponsor lightning talks were a mistake, that horse is pulp; I will delete any comments brining that up. Same goes for any comments with profanity. *sigh*

The talk proposal deadline is Monday. We are on track to get 100 proposals, given previous years trends. Most people do not submit their talk until the deadline. We have more space and more time in the schedule this year so we will be able to accept more proposals, and more 45min proposals. We are also going to have invited speakers this year; well known speakers on topics people have asked for.

Once the deadline closes, proposals will still be editable, and edits will be accepted until around the 15th. So get those proposals in!

I just posted yet more information on the review process for those interested.

Ok, I meant to do this post about 3 months ago, a writeup 2 months ago and ended up just posting the video below over a month ago. I simply have not had time, and for that I apologize. I made the video because I could simply talk out what was in my head. I used a presentation I was still in the middle of writing. It was also 3 am after a long day. Not the best recipe for success.

I will try to get an additional post on other aspects of the PyCon 2009 program selection, but for now we will focus on the submitted proposals, and the process used to select the best talks from those proposals.

Identify the Champion

We are using a process based around the ‘Identify the Champion‘ paper by Oscar Nierstrasz. I highly recommend reading this document in detail. It does not dictate how a program committee will do its work, but rather sets a framework from which many different systems can be developed to meet the specific needs of the conference or group in question. The central theme is breaking down the problem by ensuring the proper people (the ‘champions’) drive the process. I do want to hilight one fact before we get into the process:

Reviewer Votes do NOT determine if a Proposal is accepted or declined

The program committee as a whole decides which proposals will be accepted. We will get back to this later, but I want to dispel this idea of ’scores’ on proposals up front. With that said, lets start…

Read the rest of this entry »

Only private because I cannot afford the bandwidth of it being public until next month. (Used almost all of my 100GB limit this month…)

Program Committee Software (90mb flv file)

Writeup TBD.

I will be in Montana again in early July. This year I will have a rental car and thus more freedom than in the past. Are there any pythonistas in the Missoula/Lolo area? Or coding in general? Security?

It has become tradition that I start getting my personal game plan for PyCon together. I am hoping to do less coding and more organizational work. It would be nice to talk to anyone who has been to a recent conferences (technical or not, professional or not, python or not, does not matter).