Search this site


Page 1 of 53  [next]

Metadata

Articles

Projects

Presentations

Installing Windows 8 Consumer Preview

I have a fresh workstation and am running through the windows 8 installer on USB. When choosing the drive to install to, I get an error:
We couldn't create a new partition or locate an existing one
Lots of googling and I didn't find any hints for windows 8, but windows 7 has a similar error and folks pointed at diskpart nonsense to fix it. So let's do that -
  • At the installer, choose "Repair your computer"
  • Choose "troubleshoot"
  • Choose "advanced options"
  • Choose "command prompt"
  • Run diskpart.
In diskpart, you'll want to make sure your target drive is formatted and active.
list disk

# now pick your disk
select disk 0
clear
create partition
format fs=ntfs compress quick
active
Now reboot and try the installer again, it worked for me.

When all you have is a hammer, make your own tools?

Clarifying my position from this post:
The "ops folks need coding skills" groupthink is lame. Software requires extra coding because it is shitty, not because people are unskilled

I will lead with this: I want more people who use technology to grow and learn better skills for bending that technology to their needs. An ops guy with programming skills is, to me, more valuable than one who cannot - programming in any language or platform lets you extend an otherwise static system.

Anyway, back at the post in question, I'm not trying to say people (ops or otherwise) shouldn't want stronger programming skills. I'm saying the equipment we use is pretty shitty.

I am part of the generation raised near devices ever blinking "12:00". Devices which have no business caring what time it is, nor any sane reason to make the state of "I don't know what time it is" a high priority alert worth blinking forever.

It's 2012, and this problem persists - my microwave refuses to cook food unless it has the time *and* date from user input. Now I have to program it every time it has a power disruption (which is has frequently due to some bug in the hardware causing it to power off randomly with certain dishes at home).

Now I have to learn to program or configure these devices before they'll stop irritating me. And, damn it I hate that. If, instead, this were enterprise software, I could report these irritations to the vendor who would kindly offer me training and consulting for extortionate piles of money.

I love coding. It's fun, and many times lets me solve problems I couldn't otherwise. Allowing me to abuse an analogy, "When all you have is a hammer, you can sit down and build whatever tool you need to repair the delusion that everything is a nail."

But despite being able to solve my own problems in software, I don't think this is a great pattern of work. I write code, most of the time, because the solutions available are terrible or don't meet my requirements. With a new software popping up every day, I see a strong correlation between software availability and people asking for more programmers.

So, the more software we have, the more programmers we need to work around limitations in the available body of software. I think that's pretty lame :(

And regarding my microwave problems, I want some confidence that the problems being solved are meaningful problems, not programming learned for the sake of working around bugs and misfeatures in software we're suffering with.

Goodbye, 2011.

This year's been pretty good, but the last two months were pretty lame.

In the last six weeks, I found out Caramel has lymphoma, got unemployed, and had emergency surgery to remove my appendix on Christmas Day. The unemployment caused me to lose an in-progress mortgage refinance.

I'll pick up the mortgage thing once I remedy the employment problem, but I'm staying quite happily unemployed until after my kid is born - should be any day now!

Most of my career-growing moves were outside of work: at meetups, in open source efforts, or in networking with folks on IRC or twitter. Lots of awesome folks out there, so go introduce yourself. Don't be a dick. :)

I didn't write much on this site, but mainly, that was due to an increase in my activities on IRC and twitter. Most of what I published this year was code and was less writing about said code. I'd like to fix that, though.

This years successes were topped by two new major projects, fpm and logstash. I also released some major improvements to xdotool and other tools.

The current implementation of logstash isn't very old, but prototypes, hacks, and other incarnations of pretty much the same thing date back to at least 2005 and probably earlier. This project has been a long-time-coming, and Pete Fritchman and I have been talking about logstash for years, so it's nice to finally have some code shipped and a community building around it.

FPM had a crazy positive response. I wrote it as a hack, and it's used all over the place now. Bonus that people are contributing patches and other improvements as well.

Sysadvent was another excellent success, the end of which marked the 4th year and 100th article posted to the project. It is awesome seeing such community involvement from so many different authors.

This year also cemented my move to git from svn. Why? Github, mostly, and not really the features of git itself. Sharing code and patches is so much easier on github than it is with other services.

I went to CarolinaCon and OSCON to talk about logstash. I also went to DevOps Days Mountain View and gave a lightning talk on logstash.

My OSCON talk was overflowing with people standing at the back of the room, etc; it went awesomely. I've also been able to do lunchtime logstash presentations at places like Square and others. I also gave talks at BayLISA meetings. It was a good year for getting out of the house and talking about code.

I tried to get a count of how much code I'd written this year, but I had lots of web-based projects that included third-party stuff like jquery, and I'm too lazy to pick through the results and trim that stuff out. I'm up to about 70 different projects on github now, some useful; some not; all fun!

Looking forward to 2012 :)

Insist on better asserts

I never really liked C's assert() feature. If an assertion is violated, it'll tell you what assertion failed but completely lacks any context:
example: example.c:9: main: Assertion `i == 3' failed.
This is better:
Assertion failed insist.c:7 in main(), insist(i == 3): Something went wrong, wanted i == 3, got 4
The main difference here is that there's context about what failed. A message for humans looking to debug this. This is especially important on Linux these days because every distro I've used recently hates sysadmins and hates debugging - all libraries are stripped of debug symbols and coredumps are disabled by default.

What's the usage look like?

#include 
#include 
#include "insist.h"

int main() {
  int i = 4;
  //assert(i == 3);
  insist(i == 3, "Something went wrong, wanted i == 3, got %d", i);
  return 0;
} 
I also added a special 'return' version of this, 'insist_return' that lets you do error checking and early aborting like this:
insist_return(fd >= 0, START_FAILURE,
              "socket() returned %d, an error: %s", fd, strerror(errno));
Works just like insist() except returns START_FAILURE if 'fd > 0' is false and additionally logs the error formatted above.

Code here: insist.h

xdotool 2.20110530release

It's been about 8 months since the last xdotool release, and I think it's long overdue! This release has a ton of new feature and fixes.

Download: xdotool-2.20110530.1.tar.gz

As usual, if you find problems or have feature requests, please file bugs or send an email to the list.

Changelist since previous announcement:

2.20110530.*
  - New set_window feature: --urgency. This lets you set the urgency flag on a
    window Window managers will interpret this as something about your window
    needing attention. It might flash in the taskbar, pop up, or other.
    Original patch and suggestion by ervandew.
    Issue: http://code.google.com/p/semicomplete/issues/detail?id=39
  - New function: xdo_window_seturgency (see above)
  - Hack in OS X support as it is missing a proper clock_gettime.
    Should fix http://code.google.com/p/semicomplete/issues/detail?id=37
    Reported by ryandesign.com
  - Add support for typing UTF-8 characters. Patch from Joseph Krahn.
  - Make all output call fflush to send data immediately (for pipes). Reported
    by Andreas Wagner on the mailing list.
  - Make 'get_desktop_viewport' output usable with 'set_desktop_viewport'
    http://code.google.com/p/semicomplete/issues/detail?id=47
  - You can now make 'libxdo.a' for embedding libxdo into your binary
    (Requested by psc on the mailing list).
  - Fixed a typing bug where the keymap changes unnecessarily 
  - Should now build cleanly in C++ environments (Reported by psc on the
    mailing list)
  - bugfix: xdotool should use command names first before trying file scripts. 
    See https://bbs.archlinux.org/viewtopic.php?pid=938309 for original report.
  - Add a 'sleep' command. (Requested by Joseph Krahn via mailing list)
  - Add --relative flag to windowmve. (Requested by Anthony Thyssen via mailing
  - Add --desktop flag to the search command. This lets you search for windows
    on specific desktop. Requires a window manager that supports multiple
    desktops in a way that EWMH supports.
    Fixes http://code.google.com/p/semicomplete/issues/detail?id=38
  - Add --limit flag to search. This allows you to break the search early after
    a certain number of matches. (Requested by Anthony Thyssen)
  - New command 'getwindowgeometry' for fetching window position and size
    (Requested by Anthony Thyssen via mailing list)
  - Add --sync flag to search command; blocks until results are found.
    xdotool will search every 0.5 seconds for results.
    http://code.google.com/p/semicomplete/issues/detail?id=42
  - windowmove can now move windows along an axis. Give literal 'x' or 'y'
    instead of a coordinate and it uses the current position. (Requested by
    etnlIcarus via mailing list)
  - Add '--args N' and '--terminator TERMINATOR' to the 'exec' command.
    Default terminator unless specified (or --args is) is ':' (Requested by
    Joseph Krahn and Henning Bekel via mailing list)
  - set_desktop now supports --relative flag (+N or -N to move relative)
    (Requested by Anthony Thyssen)
  - The mouse cursor now changes during 'xdotool selectwindow' (Requested by
    Anthony Thyssen via mailing list)
  - Added '--args N' and '--terminator TERMINATOR' to the 'type' command.
  - Add 'getdisplaygeometry' command for querying the size of your screen.
    (Requested by @rrwo via twitter)
  - Add xdo_get_viewport_dimensions function.

logstash's first major release - 1.0.0

Ready for log and event management that doesn't suck or drain your budget? It's time to logstash.

After lots of refactoring and improvements to logstash since the first minor release last November, logstash is ready for wider usage now.

Read my announcement here.

The logstash site is also online and has docs, intros, slides, and videos.

http://logstash.net

Happy logstashing!

EC2 reserved vs on-demand costs (and R graphs!)

I'm sure this is covered well elsewhere online, but that's never the point of these things ;)

I was helping with some capacity planning and run-rate math today at work and found that ec2 reserved instances are much cheaper compared to on-demand - If this is obvious to you, chill out, I have historically never really used EC2 nor have I ever been close to budgeting. ;)

I proved this conclusion with some math, but frankly I like visualizations better, so I decided to learn R. I wrote an R script that will graph an on-demand vs reserved pricing for one m1.large instance (code at end of the post).

The result is this graph:

The graph says it all, and definitely tells me that we need to be reserving all of our instances at Loggly - and it gives me a rule-of-thumb:

  • If we're going to use one instance unit for at least 9 months, reserve for 3 years.
  • If we're going to use one instance unit for at least 6 months, reserve for 1 year.
  • Otherwise, stick with on-demand.
The "reserved instances" pay structure is you pay a one-time fee for access to a reduced hourly rate.

This also means that our random "debug something" deployments that are shutdown much of the time are probably best off being reserved instances as well- at least for a 1-year thing - since we are likely to use those deployments for more than half of a year.

A 3-year on-demand price for m1.large is just shy of $9000, which is twice as expensive as the 3-year reserve. Capaticy plan and maybe start buying reserved instances. Make your CFO happy.

And in case you were going to ask, I ran the same plot with data from EC2 "quaduple extra large" instances and the savings and break-even points were the same. I bet the rest of the prices flow similarly.

The R script is follows, run it with 'R --save < yourscript.r':

# Values taken from http://aws.amazon.com/ec2/pricing/
# for an m1.large ("Large") instance
on_demand_hourly = 0.34
reserve_hourly = 0.12
reserve_1year = 910       
reserve_3year = 1400

# quadruple extra large instances
#on_demand_hourly = 1.60
#reserve_hourly = 0.56
#reserve_1year = 4290
#reserve_3year = 6590

on_demand_daily = on_demand_hourly * 24
reserve_daily = reserve_hourly * 24
x <- c(0, 365)
y <- on_demand_daily * x

# Calculate day of break-even point reserve vs on-demand rates
break_1year_x = reserve_1year / (on_demand_daily - reserve_daily)
break_3year_x = reserve_3year / (on_demand_daily - reserve_daily)

png(filename = "ec2_m1large_cost.png", width = 500, height=375)
plot(x,y, type="l", col='red', xlab="", ylab="cost ($USD)")
title("EC2 cost analysis for m1.large", 
      sprintf("(days)\n1-year is cheaper than on-demand after %.0f days of usage,\n 3-year is cheaper after %.0f days", break_1year_x, break_3year_x))
text(60, 0, sprintf("on-demand=$%.2f/hour", on_demand_hourly), pos=3)

abline(reserve_1year, reserve_daily, col='green')
text(60, reserve_1year, sprintf("1-year=$%.0f+$%.2f/hour", reserve_1year, reserve_hourly), pos=3)

abline(reserve_3year, reserve_daily, col='blue')
text(60, reserve_3year, sprintf("3-year=$%.0f+$%.2f/hour", reserve_3year, reserve_hourly), pos=3)

point_y = reserve_1year + reserve_daily * break_1year_x
points(break_1year_x, point_y)
text(break_1year_x, point_y, labels = sprintf("%.0f days", break_1year_x), pos=1)

point_y = reserve_3year + reserve_daily * break_3year_x
points(break_3year_x, point_y)
text(break_3year_x, point_y, labels = sprintf("%.0f days", break_3year_x), pos=1)

dev.off()
quit()

Introducing FPM - Effing Package Management

Having become fed up with dealing with rpmbuild, spec files, debian control files, dh_make, debuild, and the whole lot, I automated my way back to sanity.

The result is a tool I call "fpm" which aims to help you make and mangle packages however you choose, all (ideally) without having to care about the internals of your particular native package format.

The goal of this project is not to undermine upstream packaging but to grant everyone the ability to trivially build and edit packages. Why? Not all software is packaged. Not all software of the version you want is packaged. And further, not all users are willing or able to take the time to learn all the ins and outs of their package build tools.

For example, you can package up your /etc/init.d directory as an RPM by doing simply this:

% fpm -s dir -t rpm -n myinitfiles -v 1.0 /etc/init.d
...
Created /home/jls/rpm/myinitfiles-1.0.x86_64.rpm
fpm will create a simple package for you and put it in your current directory. The result:
% rpm -qp myinitfiles-1.0.x86_64.rpm -l
/etc/init.d
/etc/init.d/.legacy-bootordering
/etc/init.d/NetworkManager.dpkg-backup
...

% rpm -qp myinitfiles-1.0.x86_64.rpm --provides
myinitfiles = 1.0-1
% rpm -qp myinitfiles-1.0.x86_64.rpm --requires
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(CompressedFileNames) <= 3.0.4-1
You can package up any directory. But there's more.

Above, I didn't specify a package summary, so how about fixing the rpm to include the description? You can use RPMs as the source (-s flag) in fpm. There's also a helpful '-e' (--edit) flag that'll let you edit the rpm spec (or debian control) file before building.

% rpm -qp myinitfiles-1.0.x86_64.rpm --info | grep Summary
Summary     : no summary given

% fpm -s rpm -t rpm -e myinitfiles-1.0.x86_64.rpm
... this opens up $EDITOR so you can edit the spec file it generated ...
... make some changes to the spec, including adding a proper 'Summary' ...
Created /home/jls/rpm/myinitfiles-1.0-1.x86_64.rpm

% rpm -qp myinitfiles-1.0-1.x86_64.rpm --info | grep Summary
Summary     : my /etc/init.d directory
The '-s dir' flag says the source of the package is a directory. There's also support for other package sources like rubygems, other rpms, debs, and more on the way.

With FPM, you can specify dependencies, architecture, maintainer, etc. All from a simple command line, and never forcing you to learn the pain and suffering that can come with rpm spec files or debian package building.

You can install fpm with: gem install fpm

The project page is here: https://github.com/jordansissel/fpm

The wiki is here (has more examples): https://github.com/jordansissel/fpm/wiki

jQuery Mobile - Full Height content area

I was working on integrating jQuery Mobile stuff into fingerpoken and needed a way to make the content area of pages full-screen. By 'full screen' I mean still showing the header and footer, but otherwise the content needs to fill the rest.

I couldn't find an easy way to do this while googling, and even the jQuery Mobile demos didn't do it.

So here's a demo of what I came up with here: fullheight jQuery Mobile demo Javascript:

  var fixgeometry = function() {
    /* Some orientation changes leave the scroll position at something
     * that isn't 0,0. This is annoying for user experience. */
    scroll(0, 0);

    /* Calculate the geometry that our content area should take */
    var header = $(".header:visible");
    var footer = $(".footer:visible");
    var content = $(".content:visible");
    var viewport_height = $(window).height();
    
    var content_height = viewport_height - header.outerHeight() - footer.outerHeight();
    
    /* Trim margin/border/padding height */
    content_height -= (content.outerHeight() - content.height());
    content.height(content_height);
  }; /* fixgeometry */

  $(document).ready(function() {
    $(window).bind("orientationchange resize pageshow", fixgeometry);
  });

Introducing: fingerpoken - a mobile device as a touchpad/remote/keyboard

I'm giving a presentation this week at the Puppet Bay Area user meetup and while working on slides, I wanted to be able to present while not being attached to my laptop.

Enter: fingerpoken

Fingerpoken lets you turn your iphone/ipad/itouch into a touchpad, keyboard, and remote for another computer. The only required piece on your iphone is Safari. No appstore stuff to download!

Under the hood, it uses websockets and touch events and sends JSON-encoded requests to your workstation and will move the mouse, type, scroll, and more.

Project page: fingerpoken on github.

A short demonstration of this project in action: