drewp

Using zc.recipe.cmmi with zc.recipe.egg

2012-02-02T21:26:48.395777-08:00

gp.recipe.node is nice, and it depends on zc.recipe.cmmi and zc.recipe.egg at the same time. If those two got installed into dirs like these:

eggs/zc.recipe.egg-1.3.2-py2.7.egg/zc/recipe/

eggs/zc.recipe.cmmi-1.3.5-py2.7.egg/zc/recipe/

. . you'll get an error like this:

% bin/node
Traceback (most recent call last):
File "bin/node", line 17, in <module>
import gp.recipe.node.script
File "eggs/gp.recipe.node-0.2-py2.7.egg/gp/recipe/node/__init__.py", line 5, in <module>
from zc.recipe.egg import Scripts
ImportError: No module named egg

A fix is to replace eggs/zc.recipe.cmmi-1.3.4-py2.7.egg/zc/recipe/__init__.py with these contents:

__import__('pkg_resources').declare_namespace(__name__)
import pkgutil
__path__ = pkgutil.extend_path(__path__, __name__)

Next, I wish gp.recipe.node would run node in a way that replaces its own process, instead of running a child. Then supervisord would have an easier time killing the node process.

How to run graphite 0.9.9 without installing any code

2012-01-08T23:07:29.626119-08:00

Here's how to start from a linux box with python 2.7 and virtualenv, and end up with a running graphite web server and data collector daemon (carbon-cache). I use supervisord and nginx as well, but you don't need exactly those to make the setup work.

mkdir /new/dir

cd /new/dir

Fetch software

virtualenv .

bin/easy_install "gunicorn==0.13.4"
bin/easy_install "Django==1.3.1"
bin/easy_install "graphite_web==0.9.9"
bin/easy_install "carbon==0.9.9"

bin/easy_install "whisper==0.9.9"

bin/easy_install http://django-tagging.googlecode.com/files/django-tagging-0.3.1.tar.gz

Setup configs

cp ./lib/python2.7/site-packages/carbon-0.9.9-py2.7.egg/conf/carbon.conf.example ./carbon.conf
echo "TIME_ZONE = 'America/Los_Angeles'" > local_settings.py
ln -s ../../../../../local_settings.py ./lib/python2.7/site-packages/graphite_web-0.9.9-py2.7.egg/graphite/local_settings.py

mkdir -p storage/log/webapp

Run this once to init a sqlite database

GRAPHITE_STORAGE_DIR=storage/ DJANGO_SETTINGS_MODULE=graphite.settings bin/django-admin.py syncdb

It will say "you don't have any superusers defined. Would you like to create one now?" Say yes and put add minimal information. I'm not sure where this information ever shows up again.

Run these two daemons

GRAPHITE_CONF_DIR=. GRAPHITE_ROOT=. bin/carbon-cache.py --debug start

GRAPHITE_STORAGE_DIR=storage/ bin/gunicorn_django -b 0.0.0.0:9037 ./lib/python2.7/site-packages/graphite_web-0.9.9-py2.7.egg/graphite/settings.py

Here's a config for supervisord that runs those lines:

[program:graphite_carbon_2003]
directory=/opt/graphite/0.9.9
environment=GRAPHITE_CONF_DIR=.,GRAPHITE_ROOT=.
command=/opt/graphite/0.9.9/bin/carbon-cache.py --debug start
# --debug needed for fg mode, but now it logs every datapoint operation
stdout_logfile=/dev/null

[program:graphite_9037]
directory=/opt/graphite/0.9.9
environment=GRAPHITE_STORAGE_DIR=storage/
command=/opt/graphite/0.9.9/bin/gunicorn_django -b 0.0.0.0:9037 ./lib/python2.7/site-packages/graphite_web-0.9.9-py2.7.egg/graphite/settings.py

Serve the web page

Set your web server to proxy http://something.example.com/ to localhost:9037. The graphite pages are full of /root addresses, so it's very hard to serve graphite from anywhere but the top of its own (sub)domain. Serve http://something.example.com/content from the static files at ./lib/python2.7/site-packages/graphite_web-0.9.9-py2.7.egg/webapp/content

Here's an example config for nginx:

server {
server_name graphite.example.com ;
location / {
proxy_pass http://localhost:9037/;
}
location /content {
root /opt/graphite/0.9.9/lib/python2.7/site-packages/graphite_web-0.9.9-py2.7.egg/webapp;
}

}

Add data

To track the number of files in your home dir over time, run this in a loop or cron:

echo demo.numFiles `ls -1 ~ | wc -l` `date +%s` | nc -q 0 localhost 2003

Galena is a python module that does the same thing.

Released comment system

2011-07-16T22:54:28.859486-07:00

I just released the code for the comment system I'm using on this blog, another secret blog, and my photo site which is also secret. The comment system is implemented as a web service you run internally. Your interesting web site calls over to commentServe when it wants to show existing comments or render an add-comment form.

It's a poor man's self-hosted Disqus, but it still has spam filtering, notifications, HTML support, RDF integration, and some other things you wouldn't normally have if you just hacked one of these together for a day. (You'd probably exceed my system if you hacked for two days, however.)

Source: http://bigasterisk.com/darcs/?r=commentserve;a=tree

Readme: http://bigasterisk.com/darcs/?r=commentserve;a=headblob;f=/readme

Github mirror: https://github.com/drewp/commentserve

Fix for a nevow/formless/annotate/zope.interface bug

2011-07-08T15:32:19.730245-07:00

    from nevow import tags as T
  File "/usr/lib/python2.6/dist-packages/nevow/__init__.py", line 143, in 
    load(basic_adapters)
  File "/usr/lib/python2.6/dist-packages/nevow/__init__.py", line 29, in load
    registerAdapter(_namedAnyWithBuiltinTranslation(a),
  File "/usr/lib/python2.6/dist-packages/nevow/util.py", line 183, in _namedAnyWithBuiltinTranslation
    return namedAny(name)
  File "/home/drewp/projects-local/ffg/eggs/Twisted-10.0.0-py2.6-linux-x86_64.egg/twisted/python/reflect.py", line 464, in namedAny
    topLevelPackage = _importAndCheckStack(trialname)
  File "/home/drewp/projects-local/ffg/eggs/Twisted-10.0.0-py2.6-linux-x86_64.egg/twisted/python/reflect.py", line 400, in _importAndCheckStack
    return __import__(importName)
  File "/usr/lib/python2.6/dist-packages/formless/__init__.py", line 9, in 
    from formless.annotate import *
  File "/usr/lib/python2.6/dist-packages/formless/annotate.py", line 851, in 
    TypedInterface = MetaTypedInterface('TypedInterface', (InterfaceClass('TypedInterface'), ), {})
  File "/usr/lib/python2.6/dist-packages/formless/annotate.py", line 731, in __new__
    _typedInterfaceMetadata[cls, '__id__'] = nextId()
  File "/home/drewp/projects-local/ffg/eggs/zope.interface-3.6.4-py2.6-linux-x86_64.egg/zope/interface/interface.py", line 685, in __hash__
    return hash((self.__name__, self.__module__))
AttributeError: 'MetaTypedInterface' object has no attribute '__name__'

That's an annoying error I just started getting, maybe with my Ubuntu 11.04 upgrade, or maybe from something else. Here is a workaround:

from zope.interface.interface import InterfaceClass
def __hash__(self):
    return hash((getattr(self, __name__, ''), self.__module__))
InterfaceClass.__hash__ = __hash__

House power measurement plan

2011-06-13T01:07:40.573919-07:00

This article taught me how new home power meters often have an IR LED that pulses with power usage. blalor checked his meter and found a 10ms pulse.

I took a camcorder out to my own meter (GE kv2cs) and it does seem to be pulsing out the front. My plan is to follow the guides on irmetermon, probably just schematic.adc, and connect my phototransistor to a pin on the closest parallel port which I'm already interfacing several other things to.

I hope I'll be able to read the parport fast enough to not miss pulses, and I hope I don't get too much extra IR light coming from the sun. That parport will eventually change to an arduino, and then I'll be able more easily to do the analog sensing and calibration that the irmetermon project is mainly about.

The goal is to get instantaneous power usage data into my graphite server, where I will be able to plot it along many other measurements, including solar panel generation, temperature, ISP bandwidth, and computer clock speeds.