Search this site


Metadata

Articles

Projects

Presentations

When all you have is a hammer, make your own tools?

Clarifying my position from this post:
The "ops folks need coding skills" groupthink is lame. Software requires extra coding because it is shitty, not because people are unskilled

I will lead with this: I want more people who use technology to grow and learn better skills for bending that technology to their needs. An ops guy with programming skills is, to me, more valuable than one who cannot - programming in any language or platform lets you extend an otherwise static system.

Anyway, back at the post in question, I'm not trying to say people (ops or otherwise) shouldn't want stronger programming skills. I'm saying the equipment we use is pretty shitty.

I am part of the generation raised near devices ever blinking "12:00". Devices which have no business caring what time it is, nor any sane reason to make the state of "I don't know what time it is" a high priority alert worth blinking forever.

It's 2012, and this problem persists - my microwave refuses to cook food unless it has the time *and* date from user input. Now I have to program it every time it has a power disruption (which is has frequently due to some bug in the hardware causing it to power off randomly with certain dishes at home).

Now I have to learn to program or configure these devices before they'll stop irritating me. And, damn it I hate that. If, instead, this were enterprise software, I could report these irritations to the vendor who would kindly offer me training and consulting for extortionate piles of money.

I love coding. It's fun, and many times lets me solve problems I couldn't otherwise. Allowing me to abuse an analogy, "When all you have is a hammer, you can sit down and build whatever tool you need to repair the delusion that everything is a nail."

But despite being able to solve my own problems in software, I don't think this is a great pattern of work. I write code, most of the time, because the solutions available are terrible or don't meet my requirements. With a new software popping up every day, I see a strong correlation between software availability and people asking for more programmers.

So, the more software we have, the more programmers we need to work around limitations in the available body of software. I think that's pretty lame :(

And regarding my microwave problems, I want some confidence that the problems being solved are meaningful problems, not programming learned for the sake of working around bugs and misfeatures in software we're suffering with.

Getting your python as rpms

I was working on a new python 2.6 rpm to push at work and started wondering about how to get python eggs to become rpms. Ruby has a gem package called gem2rpm that aids in generating rpms from ruby gems, but there's not really an egg2rpm project.

We're in luck, though. Python's setuptools supports generating rpms by default, it seems. Those 'python setup.py' invocations you may be accustomed to can trivially generate rpms.

The secret sauce is the 'bdist_rpm' command given to setup.py:

% wget -q http://boto.googlecode.com/files/boto-1.8d.tar.gz
% tar -zxf boto-1.8d.tar.gz
% cd cd boto-1.8d
% python setup.py bdist_rpm
% find ./ -name '*.rpm'
./dist/boto-1.8d-1.noarch.rpm
./dist/boto-1.8d-1.src.rpm
Piece of cake. I've tried this on a handful of python packages (boto, simplejson, etc), and they all seem to produce happy rpms.

However, if you have multiple versions of python available, you'll want to explicitly hardcode the path to python:

% python setup.py bdist_rpm --python /usr/bin/python2.6
% rpm2cpio dist/boto-1.8d-1.noarch.rpm | cpio -it | grep lib | head -3
2745 blocks
./usr/lib/python2.6/site-packages/boto-1.8d-py2.6.egg-info
./usr/lib/python2.6/site-packages/boto/__init__.py
./usr/lib/python2.6/site-packages/boto/__init__.pyc
The default python on this system is python 2.4. Doing the above forces a build against python2.6 - excellent, but maybe we're not quite there yet. What if you need this package for both python 2.4 and 2.6? For this, you'll need separate package names. However, the bdist_rpm command doesn't have a way of setting the rpm package name. One way is to hack setup.py with the new name:
% grep name setup.py
setup(name = "boto",
% sed -re 's/name *= *"([^"]+)"/name = "python24-\1"/'  setup.py > setup24.py
% grep name setup24.py
setup(name = "python24-boto",

# Now build the new rpm with the new package name, python24-boto
% python setup24.py bdist_rpm --python /usr/bin/python2.4
For our boto package, this creates an rpm with a new name: python24-boto. This method is good (hack the setup.py script) because the command to build the rpm stays basically the same. The alternative would be to use 'python setup.py bdist_rpm --spec-only' and edit the spec file, then craft whatever rpmbuild command was necessary. The method above is less effort and trivially automatable with no knowledge of rpmbuild or specfiles. :)

Repeat this process for python26, and now we have two boto rpms for both pythons.

% rpm -Uvh python2?-boto-*noarch.rpm
Preparing...                ########################################### [100%]
   1:python26-boto          ########################################### [ 50%]
   2:python24-boto          ########################################### [100%]

% python2.4 -c 'import boto; print True'
True
% python2.6 -c 'import boto; print True'
True
Excellent.

Find that lost screen session: Episode 3.

Previous posts about screen have shown a few new tools for searching your list of open screen sessions.o

Today, I finally sat down and worked on the next installment: Being able to query any screen window and the window list. The difference between the previous script is that we can now grep screen windows other than the 0th one. Additionally, we can now grep the screen window list (which, by the way, has some excellent information).

To that end, I present now two scripts:

You need both for this to work optimally, but they exist separately because the functionality is somewhat distinct.

The 'hardcopy' script takes a single argument, a screen session. It will hardcopy all windows in that screen session including the window list. If you specify OUTDIR in your environment, the screen hardcopies will be put in that directory; otherwise, the output directory is printed to stdout for consumption by another script.

The 'search' script runs the hardcopy script on all active screen sessions (in parallel, yay xargs). Once it has all of the copies, it will grep through the output for your query string (regular expression). It supports 3 flags:

  • -t - only search 'window titles' (ie; only window list output)
  • -w - only search window contents (ie; exclude window list output)
  • -l - only search the 'location' field of the window list
Now, with a single command, I can find out where that ssh session to 'foo' disappeared to. Here's an example screen window list capture (accessed with Ctrl+A " (doublequote))
Num Name                                                              Location Flags

  0 zsh                                                                        syn $
  1 zsh                                                                      scorn $
Now, I want to find all sessions open to 'scorn':
% screen-session-search.sh -t 'scorn'
sty 18210.pts-8.snack window 1
sty 18556.pts-0.snack window 0
It found 2 sessions. I can attach to the first one with:
screen -x 18210.pts-8.snack -p 1
          ^ screen session     ^ window
caveat: I've been hacking on things all night, so the code may or may not be very readable. Apologies if you go blind while trying to read it ;)

Find that lost screen session, episode 2.

Like I said, I run screen in all of my xterms...

xterm sets an environment variable in child processes: WINDOWID. This is the X window id of the xterm window. Using this, we can extend upon my last post and come up with a much neater solution. Knowing what screen session you want to bring forward (assuming it's running in an xterm), we can run a command inside that session that grabs the $WINDOWID variable in the shell and uses xdotool to activate the window.

session=$(sh screen-find.sh "#freebsdhelp")
STY=$session screen -X screen sh -c 'xdotool windowactivate $WINDOWID'
Running this causes my IRC window to be activated by xdotool, which means it is now active, focused, and on top.

This isn't entirely optimal, because it assumes the xterm attached to that screen session is the xterm that launched it. If you run 'xterm -e screen -RR' and close the xterm (don't log out of the shell), then rerun 'xterm -e screen -RR' it will attach to that previous screen session, but the WINDOWID will understandably not be correct any longer.

So what do we do? Using the screen session given, we create a new window in that session and set the title of that window to a generated string. We then use xdotool to search for that string and activate the window. Once we get there, we can kill that new screen window we created and we are left with the terminal holding our screen session sitting in front of us.

I wrote a script to do just that tonight: screen-activate.sh. Example usage: screen-activate.sh 24072.pts-25.snack

This has a great benefit of supporting every terminal program that understands how to set the terminal window title when screen changes it's title. I have tested my .screenrc in Eterm, Konsole, gnome-terminal, and xterm - all know when screen changes it's title if you put this in your .screenrc:

hardstatus string "[%n] %h - %t"
termcapinfo xterm 'hs:ts=\E]2;:fs=\007:ds=\E]2;screen (not title yet)\007'

# Might need this:
termcapinfo  * '' 'hs:ts=\E_:fs=\E\\:ds=\E_\E\\'

Find that lost screen session

Scenario: I run lots of xterms. Each xterm runs a single screen session(*). At any given time, I can only see some of the xterm windows (the others are hidden).

(*) All my xterms run with: 'xterm -e screen -RR'. This causes them to attach to the first-found detached screen, and if none exist creates a new screen session. See run-xterm.sh for my pleasant, random-colored xterm script.

Problem: I forget where I put things. I can't find that terminal where I'm editing foo.c!

Possible Solutions:

  1. Bad: Kill the vim session that's editing the file, and rerun vim somewhere else.
  2. Good: Use xdotool to search window titles for 'foo.c'
  3. Great: Find the screen STY variable for the process 'vim foo.c'
  4. Great: Ask each open screen session about what it is on screen
Today, we'll cover the two 'great' solutions. I wrote both of these a while ago, but I totally forgot to post about them. Here you go :)

Find a screen by it's child processes
Tool: screenps.sh

This tool takes a regexp pattern as the only argument and will output a list of screen sessions having child process commands that match that pattern. This is useful for finding what screen is running 'vim foo.c'

% ./screenps.sh 'vim foo.c'
23464.pts-0.snack
Find a screen by what is being displayed
Tool: screen-find.sh

This tool takes a regexp pattern as the only argument. It uses screen's hardcopy command to save the on-screen buffer and then applies the regexp given to the buffer. If it matches, the screen session is output. There is special behavior if only one screen session is found: If the screen session is currently attached, it will flash that screen session giving you a visual clue about where it is; if it is not attached, it will attach to it.

% ./screen-find.sh "keynav"
28504.pts-27.snack
In case you still aren't clear, the two tools help you find your lost screen sessions. Maybe they aren't lost, but certainly it's easier to search for them by text than by eyeballs if you know what's in them.

A short summary: screenps.sh will search for commands running in a screen session and screen-find.sh will search for literal text displayed in a screen session. Both are super useful.

Note: Currently, screen-find.sh can only capture the contents of the 0th screen window (screen sessions can have multiple windows). I worked for a while on solving this, but for whatever reason I couldn't get it working properly.

Wakeup call using /bin/sh and at(8)

Using at(8), I can schedule jobs to occur, say, when I need to wake up.
nightfall(~) % atq
Date                            Owner           Queue   Job#
Thu Jul 13 07:30:00 PDT 2006    jls             c       14
Thu Jul 13 08:00:00 PDT 2006    jls             c       15
Thu Jul 13 08:20:00 PDT 2006    jls             c       16
All of those jobs run my 'wakeup.sh' script which is somewhat primitive, but it does the job.

Using this script: scripts/wakeup.sh