I got this cool LCD from Adafruit. It has a tutorial for connecting it to an Arduino.

I wrote a tiny driver with web.py and spawning. Spawning has the cool feature of being able to use eventlet to handle multiple simultaneous requests in one process, in one thread. This helps me share my single USB port with all the requests; I didn't have to write any special queuing stuff. I would have gotten that property for free with twisted, too, but twisted.web and nevow are more complicated to use.

My code is essentially this:

import web, serial

ser = serial.Serial(port='/dev/ttyUSB1', baudrate=9600, timeout=1,

                    xonxoff=0, rtscts=0)

class index(object):

    def POST(self):

        ser.write("\xff" + web.data() + "\x00") # I made up this protocol and implemented it on the arduino side

        return "ok"

urls = (r'/', 'index')

app = web.application(urls, globals())

application = app.wsgifunc()

# spawn --port=9017 --threads=0 host.application

This is probably more than one needs just to connect an LCD to a computer, but I intend to run several other sensors and light controls off the same Arduino, so my web server will be getting more elaborate.

In the meantime, I can run unix shell commands onto the LCD:

% forever { date | curl -d @- http://dash:9017/; sleep 1 }

Or I can do a live search on twitter and display its results on the LCD:

import sys, restkit, jsonlib, urllib

login = "...", "..."

keyword = sys.argv[1]

response = restkit.request(

    "http://stream.twitter.com/1/statuses/filter.json?track=%s" %

    urllib.quote(keyword), filters=[restkit.BasicAuth(*login)])

for out in response.body_stream():

    text = jsonlib.read(out)['text']

    print text

    restkit.request(method="post", url="http://dash:9017/", body=text)

Video

I have two ShiftBrite modules behind the monitors on my desk, so I tried setting their colors according the colors on my screen. Python-xlib has bindings to XGetImage, which was good enough for this project. (It's very slow, though, so I can't watch the whole screen with this technique. XCB might be better.) 

I read two rectangles at the top of my monitors, have PIL calculate their average colors, and send those colors to the RGB LEDs.

Here's the essence of the code. 

The whole program is here.

I started a new project using jabber and SIOC. It's a jabber bot that records sioc:Post entries in an RDF store.

I also used wokkel, twisted, and rdflib.

The idea is that you make a new bot account and have some users subscribe to it. Anything they say to the bot is logged (the diary part), and the bot will send out nag messages if it hasn't heard from anyone in a while. Each users' posts are broadcast to all the other users.

Kelsi and I tried it out, and I noticed an unintended usage pattern: when I posted something, she followed up with an elaboration. This is natural since she gets my message as a broadcast from the bot, and answering it makes a new post. Someday when I get around to finding or writing a SIOC viewer for this stuff, it would be nice to connect the entries that were made very close together.

Here's what the log looks like in n3:

      
@prefix : <http://rdfs.org/sioc/ns#> .
@prefix XML: <http://www.w3.org/2001/XMLSchema#> .
@prefix dc: <http://purl.org/dc/terms/> .

<http://example.com/forum>     :container_of  [
         a :Post;
         dc:created "2009-09-19T20:59:33.33-07:00"^^XML:dateTime;
         dc:creator "drewp@jabber.bigasterisk.com/Coccinella@dash";
         :content "good evening diary" ],
             [
         a :Post;
         dc:created "2009-09-19T22:18:00.12-07:00"^^XML:dateTime;
         dc:creator "drewp@jabber.bigasterisk.com/Coccinella@dash";
         :content "weather is looking fine" ] .

SIOC says to use sioc:has_creator which points to a sioc:User, but I haven't gotten all those connections done yet. The code also doesn't know what forum URI to use for the bot. The log store is only appended to, but the code currently rewrites the whole thing every time, which is a waste (and a risk!).

I'm thinking about adding a separate mode where each person writes his own local diary with one bot, and there's no broadcasting or sharing of messages between the users. In other words, it would be like you made separate bot accounts bot_user1, bot_user2, etc, except you wouldn't have to actually make all those accounts.

Source code

I replaced the LED nightlight in our bathroom with a MegaBrite.

I got some plastic domes on eBay. Do you want one? I have several extras.

I attached the dome to the ceiling by tying some bicycle tire around pushpins that press out against the inside of the dome. Here are a few other uses for old bike tubes.

Here are some unretouched photos of the MB with its dome lighting up the bathroom. Shutter times are 1/2 and 1 sec.

When I was testing the MegaBrite, it was hanging down from the ceiling and I happened to have a glass of water nearby:

The first two pictures are from different positions of the MB in the glass; in the third one I am sloshing around the water:

I'm controlling the MegaBrite with a parallel port, which is very simple to do. I'm getting LED power from the 5V rail of an old computer power supply. The power is going over perhaps 50 feet of cat5 which probably adds a lot of resistance. I measure 4.7V at the MB end (with the MB turned on). The parport control code is here. It has no attempt at optimization yet, and I think it takes just under 1ms to do a single update. If I add more MBs to this chain I may need to fix the inner loop in that code.

I tried doing an openid login in python, and it was really easy. Here's how to do it. This uses a python openid library that ubuntu users can get as 'python-openid'.

Paste this into python (or ipython), with your openid on the second-to-last line and some working URL on the last line:

import openid.consumer.consumer
import openid.store.filestore
store = openid.store.filestore.FileOpenIDStore('/tmp/openid')
sess = {}
c = openid.consumer.consumer.Consumer(sess, store)
info = c.begin('drewp.myopenid.com')
info.redirectURL('http://example.com/', 'http://example.com/')

Browse to the URL you get from the last command. After possibly some confirmations on the openid server site, you'll come back to your working URL plus a long query string. Copy the query string starting right after the question mark and put it in place of the '...' in these lines. Replace example.com with your working URL again:

import cgi
d = dict(cgi.parse_qsl('...'))
resp = c.complete(d, 'http://example.com/')

Now resp should be a SuccessResponse object with an identity_url attribute that has been authenticated.

Something was wrong with the atom feed for one of my newsbruiser instances; and I think the working feed had some problems under bloglines. I'm trying to get off newsbruiser anyway, so I finally wrote an RDF exporter program for it. With an RDF version of the entries, it's really easy to make an atom feed (and hopefully I got it right).

My exporter program imports the actual newsbruiser code, fakes enough startup and config material to get going, and then adds statements about the notebook and entries to an rdflib graph. Man, does newsbruiser spend a ton of code on config and option nonsense.

My blog is currently 963 statements; and Kelsi's blog is 1011 statements. I just store them as NT files. They're about 300kB each.

The atom generator code uses rdflib but not newsbruiser. It's this SPARQL query:

  SELECT ?e WHERE {
          ?blog sioc:links_to ?e .
          ?e td:modified ?mod .
        } ORDER BY desc(?mod) LIMIT 5

with one more pattern if I'm trying to make a feed for a particular category. Then I use lxml's ElementMaker like the slides told me to.

winpdb is cool when it works because you get a tree view of the locals at your breakpoint. But it's hard to memorize the invocation. Somehow I can remember "import pdb; pdb.set_trace()" after all these years, but I keep having to look up the winpdb one.

So here's the tip: run 'pydoc winpdb' and edit whatever file that displays. On my ubuntu box, it's /var/lib/python-support/python2.5/winpdb.py. Just sudo vi that file and add these lines at the start:

#! /usr/bin/env python

"""
embedded line:
import rpdb2; rpdb2.start_embedded_debugger('x')


    winpdb.py

    A GUI for rpdb2.py

Now "pydoc winpdb" will show the critical line for embedded debugging. I think they should just make "import rpdb.embed" do the right thing and launch the gui with a connection to the current code.

Suppose your firefox session keeps crashing AND the new sessionrestore page is corrupted so the table is too small to show anything. The data you want is in .mozilla/firefox/<profile>/sessionstore.js but it's hard to read.

Here is a program that pretty-prints that file. PyPI/easy_install would be a good way to distribute this tool, but that's several minutes harder than pasting the code here:

#!/usr/bin/python

"""
pretty-prints a .mozilla/firefox/<profile>/sessionstore.js file
"""

import sys, simplejson

js = simplejson.loads(open(sys.argv[1]).read().strip('()'))

# if you already ran firefox, you'll have one tab with sessionrestore showing
if len(js['windows']) >= 1:
    entry = js['windows'][0]['tabs'][0]['entries'][0]
    if entry['url'] == 'about:sessionrestore':
        js = simplejson.loads(entry['formdata']['#sessionData'].strip('()'))

for window in js['windows']:
    print "Window:"
    for tab in window['tabs']:
        entry = tab['entries'][tab['index'] - 1]
        print "  %s" % entry['url']

# enable this to see the full js contents
#print simplejson.dumps(js, sort_keys=False, indent=2)

I went to SHDH at Sandbox Suites last night. Here I am talking with Joel about the signin system:

photo

I am trying to get the SHDH crew to use FOAF and RDF as the way to keep track of guests and their interests. FOAF includes one nice idea which is to treat email addresses sort of like passwords and only store their hashes. I wrote a demo of how to take an email address, hash it, use an RDF search engine to find documents with that hash, and search those documents for a full name to go with the email.

getNameFromEmail code and docs

Then I looked at CruiseControl for a while. It might be cool, but it unfortunately falls in the class of tools where the configuration is so hard, you might be able to use standard unix tools to get most of what you want a lot faster. Nagios is also like this (but munin does ok).

Here's a silly http proxy server: httpauthproxy

If you have a lame client, say google gadgets rss reader, which can't send http auth headers, and a reasonable http server that wants them, say twitter.com, this proxy can provide the glue.

I started a followup to my wildly successful Twisted python interface to MPD: a web version. The idea is that if you could GET and POST the various MPD commands, you could write cool music player user interfaces completely in javascript. And you could write lots of them, because they'd be so easy to make. You could make a mac desktop widget, an ipod touch version, and even players for use on non-apple products! The player UI would use AJAX to send commands back to the web server, which interfaces to mpd.

Need to control many mpd instances from one UI? Don't want to run php (like all the heavyweight mpd web clients use)? Like zero page reloads in your web UIs? Want to implement some fine-grained security scheme, where selected users can use selected mpd commands? Want to make mpd calls from unix, but your network only allows http (maybe through a proxy)? It's plain to see that there are between 'five' and 'countless' reasons to use mpd over http.

Progress appears here (because I haven't migrated this project to darcs yet): viewcvs

Someday, I hope to do (or find) the same thing for email. Why can't I write a new special-purpose webmail client every weekend? Why isn't my email box abstracted as a bunch of URLs I can GET (including folders, indexes, searches, etc), and why aren't my contacts represented as a bunch of URLs I can POST new email to?

This patch hopefully works on the new newsbruiser release 2.6.2. It lets us use the excellent TinyMCE editor widget to write blog posts. Along with this patch, you have to actually put tinymce such that it shows up at /resources/tiny_mce/tiny_mce.js.

(Coming soon: another newsbruiser patch and accompanying tools that let us send picture posts from our cell phones.)

Index: nb/CoreCGIs.py
===================================================================
RCS file: /cvs/newsbruiser/nb/CoreCGIs.py,v
retrieving revision 1.100
diff -u -r1.100 CoreCGIs.py
--- nb/CoreCGIs.py	27 Apr 2008 11:52:33 -0000	1.100
+++ nb/CoreCGIs.py	29 Apr 2008 08:16:41 -0000
@@ -271,7 +267,7 @@
         self.printDisplaySnippets('aboveEntryText')
         print '<table><tr><td valign="top">'
         self.textArea(self.notebook.getEntryBoxLength(), 70,
-                      const.ENTRY_CGI_KEY)
+                      const.ENTRY_CGI_KEY, html=True)
         self.printDisplaySnippets('rightOfEntryText')
 
         if showHelp:
@@ -819,7 +815,7 @@
         print '<table><tr><td valign="top">'
 
         self.textArea(self.notebook.getEntryBoxLength(), 70,
-                      const.ENTRY_CGI_KEY, entry.text)
+                      const.ENTRY_CGI_KEY, entry.text, html=True)
         self.printDisplaySnippets('rightOfEntryText')
         if names:
             print '</td><td valign="top">'
Index: nb/lib/LOMP.py
===================================================================
RCS file: /cvs/newsbruiser/nb/lib/LOMP.py,v
retrieving revision 1.16
diff -u -r1.16 LOMP.py
--- nb/lib/LOMP.py	5 Dec 2004 20:20:51 -0000	1.16
+++ nb/lib/LOMP.py	29 Apr 2008 08:16:59 -0000
@@ -183,9 +183,41 @@
             maxlength = ''
         print '<input type="text" name="%s" value="%s" size="%s"%s />' % (name, cgi.escape(str(value), 1), size, maxlength)
 
-    def textArea(self, rows, cols, name, content=''):
-        "Prints a standard multiline text box."        
-        print '<p><textarea wrap="virtual" rows="%s" cols="%s" name="%s">%s</textarea></p>' % (rows, cols, name, cgi.escape(str(content)))
+    def textArea(self, rows, cols, name, content='', html=False):
+        """Prints a standard multiline text box.
+
+        drewp adds html mode using tinymce
+        """
+        escContent = cgi.escape(str(content))
+        if html:
+            # it would be cool to grab recent images and put them in
+            # the external_image_list
+            #  see http://wiki.moxiecode.com/index.php/TinyMCE:Configuration/external_image_list_url
+
+            # the way i include this js probably only works if there's
+            # one html textarea on the page
+            print """
+<script type="text/javascript" src="/resources/tiny_mce/tiny_mce.js"></script>
+<script type="text/javascript">
+	tinyMCE.init({
+		mode : "exact",
+                elements : "elm1",
+		theme : "advanced",
+                table_inline_editing : true,
+                plugins : "table,emotions,style",
+                theme_advanced_blockformats : "p,pre,div,h1,h2,h3,h4,h5,h6,blockquote,dt,dd,code,samp",
+                theme_advanced_toolbar_location : "top",
+                theme_advanced_statusbar_location : "bottom",
+                theme_advanced_resizing : true,
+                theme_advanced_buttons1 : "bold,italic,underline,strikethrough,|,justifyleft,justifycenter,justifyright,justifyfull,|,bullist,numlist,|,outdent,indent,blockquote,|,formatselect,fontselect,fontsizeselect",
+                theme_advanced_buttons2 : "undo,redo,|,link,unlink,anchor,image,cleanup,help,code,|,insertdate,inserttime,preview,|,forecolor,backcolor,|,hr,removeformat,visualaid,|,sub,sup,|,charmap,emotions,styleprops",
+                theme_advanced_buttons3 : "tablecontrols"
+	});
+</script>
+
+            <p><textarea id="elm1" name="%(name)s" rows="20" cols="60">%(escContent)s</textarea></p>""" % vars()
+            return
+        print '<p><textarea wrap="virtual" rows="%s" cols="%s" name="%s">%s</textarea></p>' % (rows, cols, name, escContent)
 
     def listBox(self, boxName, names, values, defaults=None, size=1,
                 allowMultiple=0):

Blogger wouldn't format this right, so I'm putting my response here.

Original post

Here's a version using the file iterator capabilities on stdin, enumerate for the line number, all the logic combined into one if-statement, and a different strategy for printing the highlights.

#!/usr/bin/python -u

import sys

try:
   pat = sys.argv[1]
except IndexError:
   pat = None

for lineno, line in enumerate(sys.stdin):
   if (pat and pat in line) or (not pat and lineno % 2 == 0):
      print "\x1b[7m%s\x1b[0m" % line.rstrip()
   else:
      print line,

Here are some things I learned about at pycon that I'm going to talk about tonight at baypiggies.

pep 3101 formatting

pyke

trestle

eggserver

codecs.open

pyglet

resolver one

chandler

tahoe

slide

roundup

bug.gd

lawrencejournal

freebase (query)

supervisor

Here's a tiny demo of using the opencv face finder on a PIL image and displaying the result with pygame:

Source code

That's using the SWIG opencv interface (available on ubuntu from the python-opencv package), not the recently-released pycv (which I know nothing about).

My headtrack project is an attempt at something like Johnny Chung Lee's head tracking project. I tried opencv since I don't have a wiimote yet. My goal is to move my desktop windows according to my head position, so I can peek around a window or lean in to look at windows that are outside the normally-visible screen.

Meanwhile, the compiz hackers started a similar project.

I made a web app to help people with date/time format conversions in python. The only one I especially checked was the one I needed at the moment:

email ("Wed, 26 Dec 2007 11:42:54 -0800")

to

iso8601 ("2007-12-26T11:42:54-08:00").

Got an enttec opendmx box, which works great with these instructions. The dmx4linux project claims to support opendmx, but I couldn't get it to build easily on a modern kernel. I only had a little time in a real theater today, so I borrowed an oscilloscope to see if I was even getting any DMX output at all.

That's a successful test of turning some channel from 0 to 255. You can see the bits in that byte getting set over the four photos. (Lower voltage means 'set' here)

I actually debugged some python/pyrex code with the oscilloscope. I was testing multiple channels and at first, only the first channel seemed to work. Or, only the first N channels worked if they were all nonzero. It turned out that I was using a strncpy somewhere on the DMX output packet, which was stopping at the first null byte. I didn't need a scope to find that bug, but I happened to be watching the output with one already.

In the theater, the opendmx worked perfectly. Audio from the laptop to the theater sound system also worked perfectly. This should be a very nice show if I don't have to spend any time debugging flickering lights and humming audio.

Meanwhile, I seem not to be the only one doing theater lighting with python and twisted.

Update: talk video is here http://video.google.com/videoplay?docid=-7786085282361681216 (my part starts at 37min)

Tonight I spoke at the baypiggies event at Google. My onscreen notes are repeated below.

The things I recommended in my talk were:

• hotshot or cProfile
• JSON
• exhibit
• twisted
• nevow
• kcachegrind

The code is at http://projects.bigasterisk.com/silhouette/pstats2web and http://projects.bigasterisk.com/silhouette/profileResults.html (the exhibit viewer). http://projects.bigasterisk.com/silhouette/ is a darcs repository too.

 >>> import hotshot, pstats
 >>> p = hotshot.Profile().runcall(main)
 >>> pstats.Stats(p).sort_stats('time').print_stats()
        58666 function calls (56532 primitive calls) in 0.980 CPU seconds
  Ordered by: internal time
  ncalls  tottime  percall  cumtime  percall filename:lineno(function)
       1    0.240    0.240    0.240    0.240 :0(create)
     472    0.220    0.000    0.220    0.000 :0(call)
    1696    0.050    0.000    0.050    0.000 sre_parse.py:182(__next)
    6022    0.030    0.000    0.030    0.000 :0(append)
  181/27    0.030    0.000    0.060    0.002 sre_compile.py:27(_compile)
     216    0.020    0.000    0.030    0.000 interface.py:850(fromFunction)
      26    0.020    0.001    0.020    0.001 advice.py:69(addClassAdvisor)
      23    0.020    0.001    0.220    0.010 netconnect:125(__init__)
1381/201    0.020    0.000    0.040    0.000 ro.py:58(_flatten)
       1    0.020    0.020    0.980    0.980 netconnect:23(?)
       1    0.020    0.020    0.020    0.020 Tkinter.py:31(?)
       1    0.010    0.010    0.010    0.010 urllib.py:1086(quote)
    2590    0.010    0.000    0.010    0.000 :0(get)
       1    0.010    0.010    0.010    0.010 compat.py:115(Connection)
       1    0.010    0.010    0.010    0.010 tksupport.py:24(?)

As fun as those are to look at, I want to quickly find out where the bottlenecks are; whether they're in my code or not; and any other clues that will help me fix them. I couldn't find anything that performs further visualization on python profile output.

So I wrote a quick one with output like this. It parses profiler output back in for easiest use with "twistd -p", which means it only has base names of .py files. I attempted to color local files by their top dir, which helps a little.

The next big step is to use the profile output directly.

I also might make the report viewer into a full web server so it can do interesting sorts and filters. Or I might tag up the data well enough and do the UI in javascript. That would be harder to write, but it would mean that the report could be a standalone file to archive, share, etc. An example of "interesting filter" would be to show one function and all the deeper calls (as much as we can tell with the profile data).

http://members.cox.net/apoco/interval/

This is an excellent model of how to write and publish doctests. Try http://members.cox.net/apoco/interval/docs/public/interval.Interval-class.html and click the 'between' method. You get a longer docstring and doctest examples of the two interesting modes.

The module itself seems to be doing a good job of cleaning up a lot of min/max junk in my code, too, which is just what I wanted.