photo
Jordan Sissel
geek

Mon, 13 Aug 2007

Fedora 6, utmp growth, Amazon EC2

% ls -l /var/run/[wu]tmp
-rw-rw-r-- 1 root  utmp  364366464 Aug 13 22:00 utmp
-rw-rw-r-- 1 root  utmp  1743665280 Aug 13 22:10 wtmp
That's 350 megs and 1.7 gigs. Cute. Performance sucks for anything needing utmp (w, uptime, top, etc). The 'init' process is spending tons of time chewing through cpu. System %cpu usage says 38% and is holding there on a mostly idle machine.

Lots of these in /var/log/messages:

Aug 13 22:10:27 domU-XX-XX-XX-XX-XX-XX /sbin/mingetty[6843]: tty3: No such file or d
irectory
Aug 13 22:10:27 domU-XX-XX-XX-XX-XX-XX /sbin/mingetty[6844]: tty4: No such file or d
irectory
Aug 13 22:10:27 domU-XX-XX-XX-XX-XX-XX /sbin/mingetty[6845]: tty5: No such file or d
irectory
Aug 13 22:10:27 domU-XX-XX-XX-XX-XX-XX /sbin/mingetty[6846]: tty6: No such file or d
irectory
Aug 13 22:10:32 domU-XX-XX-XX-XX-XX-XX /sbin/mingetty[6847]: tty2: No such file or d
irectory
I'm not sure why /dev/tty1 is the only /dev/ttyN device, but whatever. Either way, mingetty flapping will flood /var/run/[ubw]tmp over the span of weeks and eventually you end up with a system that spends most of its time parsing that file and/or restarting mingetty.

I fixed this by commenting out all tty entries in /etc/inittab and running "init q":

# Run gettys in standard runlevels
#1:2345:respawn:/sbin/mingetty tty1
#2:2345:respawn:/sbin/mingetty tty2
#3:2345:respawn:/sbin/mingetty tty3
#4:2345:respawn:/sbin/mingetty tty4
#5:2345:respawn:/sbin/mingetty tty5
#6:2345:respawn:/sbin/mingetty tty6

Comments: 1 (view comments)
Tags: , , , ,
Permalink: /geekery/utmp-growth-kills-performance
posted at: 22:12


1 responses to 'Fedora 6, utmp growth, Amazon EC2'

Mike Brittain posted at Fri May 16 14:45:35 2008...
Beautiful.  I've been running into this same problem recently with machines we have at EC2.  After 30 days or so, the CPU would all go away and load would eventually spike.  Thanks for posting this.


Leave a reply

You need javascript enabled to use this form. Anti-spam efforts ongoing. Also, if the comment doesn't show up, it's because the form expired. Go back and copy your comment, reload the form, and resubmit. Apologies if this is a hassle, I'm just playing with antispam methods right now. If this insists on not working, please email me about it.

Name (required)
E-mail (optional, if you want me to be able to email you back)
URL (also optional)
Comment:


Search this site

Navigation

Metadata

Home About Resume My Code (SVN)

Articles

ARP Security Dynamic DNS with DHCP OpenLDAP+Kerberos+SASL PPP over SSH SSH Security: /bin/false Week of Unix Tools Work Efficiency

Projects

fex firefox tabsearch firefox urledit grok keynav liboverride newpsm (FreeBSD) nis2ldap pam_captcha poor man's backup Solaris audio utility xboxproxy xdotool xmlpresenter xpathtool misc scripts

Presentations

Yahoo! Hack Day '06 Unix Essentials Vi/Vim Essentials

Tag Cloud

Calendar

< August 2007 >
SuMoTuWeThFrSa
    1 2 3 4
5 6 7 8 91011
12131415161718
19202122232425
262728293031 

Friends

BarCamp Kent Brewster Tantek Çelik John Resig Wesley Shields Tyler Shields

Technorati