Search this site


Metadata

Articles

Projects

Presentations

Introducing FPM - Effing Package Management

Having become fed up with dealing with rpmbuild, spec files, debian control files, dh_make, debuild, and the whole lot, I automated my way back to sanity.

The result is a tool I call "fpm" which aims to help you make and mangle packages however you choose, all (ideally) without having to care about the internals of your particular native package format.

The goal of this project is not to undermine upstream packaging but to grant everyone the ability to trivially build and edit packages. Why? Not all software is packaged. Not all software of the version you want is packaged. And further, not all users are willing or able to take the time to learn all the ins and outs of their package build tools.

For example, you can package up your /etc/init.d directory as an RPM by doing simply this:

% fpm -s dir -t rpm -n myinitfiles -v 1.0 /etc/init.d
...
Created /home/jls/rpm/myinitfiles-1.0.x86_64.rpm
fpm will create a simple package for you and put it in your current directory. The result:
% rpm -qp myinitfiles-1.0.x86_64.rpm -l
/etc/init.d
/etc/init.d/.legacy-bootordering
/etc/init.d/NetworkManager.dpkg-backup
...

% rpm -qp myinitfiles-1.0.x86_64.rpm --provides
myinitfiles = 1.0-1
% rpm -qp myinitfiles-1.0.x86_64.rpm --requires
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(CompressedFileNames) <= 3.0.4-1
You can package up any directory. But there's more.

Above, I didn't specify a package summary, so how about fixing the rpm to include the description? You can use RPMs as the source (-s flag) in fpm. There's also a helpful '-e' (--edit) flag that'll let you edit the rpm spec (or debian control) file before building.

% rpm -qp myinitfiles-1.0.x86_64.rpm --info | grep Summary
Summary     : no summary given

% fpm -s rpm -t rpm -e myinitfiles-1.0.x86_64.rpm
... this opens up $EDITOR so you can edit the spec file it generated ...
... make some changes to the spec, including adding a proper 'Summary' ...
Created /home/jls/rpm/myinitfiles-1.0-1.x86_64.rpm

% rpm -qp myinitfiles-1.0-1.x86_64.rpm --info | grep Summary
Summary     : my /etc/init.d directory
The '-s dir' flag says the source of the package is a directory. There's also support for other package sources like rubygems, other rpms, debs, and more on the way.

With FPM, you can specify dependencies, architecture, maintainer, etc. All from a simple command line, and never forcing you to learn the pain and suffering that can come with rpm spec files or debian package building.

You can install fpm with: gem install fpm

The project page is here: https://github.com/jordansissel/fpm

The wiki is here (has more examples): https://github.com/jordansissel/fpm/wiki

Getting your python as rpms

I was working on a new python 2.6 rpm to push at work and started wondering about how to get python eggs to become rpms. Ruby has a gem package called gem2rpm that aids in generating rpms from ruby gems, but there's not really an egg2rpm project.

We're in luck, though. Python's setuptools supports generating rpms by default, it seems. Those 'python setup.py' invocations you may be accustomed to can trivially generate rpms.

The secret sauce is the 'bdist_rpm' command given to setup.py:

% wget -q http://boto.googlecode.com/files/boto-1.8d.tar.gz
% tar -zxf boto-1.8d.tar.gz
% cd cd boto-1.8d
% python setup.py bdist_rpm
% find ./ -name '*.rpm'
./dist/boto-1.8d-1.noarch.rpm
./dist/boto-1.8d-1.src.rpm
Piece of cake. I've tried this on a handful of python packages (boto, simplejson, etc), and they all seem to produce happy rpms.

However, if you have multiple versions of python available, you'll want to explicitly hardcode the path to python:

% python setup.py bdist_rpm --python /usr/bin/python2.6
% rpm2cpio dist/boto-1.8d-1.noarch.rpm | cpio -it | grep lib | head -3
2745 blocks
./usr/lib/python2.6/site-packages/boto-1.8d-py2.6.egg-info
./usr/lib/python2.6/site-packages/boto/__init__.py
./usr/lib/python2.6/site-packages/boto/__init__.pyc
The default python on this system is python 2.4. Doing the above forces a build against python2.6 - excellent, but maybe we're not quite there yet. What if you need this package for both python 2.4 and 2.6? For this, you'll need separate package names. However, the bdist_rpm command doesn't have a way of setting the rpm package name. One way is to hack setup.py with the new name:
% grep name setup.py
setup(name = "boto",
% sed -re 's/name *= *"([^"]+)"/name = "python24-\1"/'  setup.py > setup24.py
% grep name setup24.py
setup(name = "python24-boto",

# Now build the new rpm with the new package name, python24-boto
% python setup24.py bdist_rpm --python /usr/bin/python2.4
For our boto package, this creates an rpm with a new name: python24-boto. This method is good (hack the setup.py script) because the command to build the rpm stays basically the same. The alternative would be to use 'python setup.py bdist_rpm --spec-only' and edit the spec file, then craft whatever rpmbuild command was necessary. The method above is less effort and trivially automatable with no knowledge of rpmbuild or specfiles. :)

Repeat this process for python26, and now we have two boto rpms for both pythons.

% rpm -Uvh python2?-boto-*noarch.rpm
Preparing...                ########################################### [100%]
   1:python26-boto          ########################################### [ 50%]
   2:python24-boto          ########################################### [100%]

% python2.4 -c 'import boto; print True'
True
% python2.6 -c 'import boto; print True'
True
Excellent.