Search this site


Metadata

Articles

Projects

Presentations

Xvfb + Firefox

Resig has a bunch of unit tests he does to make sure jQuery works properly on whatever browser. Manually running and checking unit test results is annoying and time consuming. Let's automate this.

Combine something simple like Firefox and Xvfb (X Virtual Frame Buffer), and you've got a simple way to run Firefox without a visible display.

Let's start Xvfb:

startx -- `which Xvfb` :1 -screen 0 1024x768x24

# Or with Xephyr
startx -- `which Xephyr` :1 -screen 1024x768x24
This starts Xvfb running on :1 with a screen size of 1024x768 and 24bits/pixel color depth. Now, let's run firefox:
DISPLAY=:1 firefox

# Or, if you run csh or tcsh
env DISPLAY=:1 firefox
Seems simple enough. What now? We want to tell firefox to go to google.com, perhaps.
DISPLAY=:1 firefox-remote http://www.google.com/
Now, let's take a screenshot (requires ImageMagick's import command):
DISPLAY=:1 import -window root googledotcom.png
Lets see what that looks like: googledotcom.png

While this isn't complicated, we could very easily automate lots of magic using something like the Selenium extension, all without requiring the use of a visual display (Monitor). Hopefully I'll find time to work on something cool using this soon.

Problems with screen scraping and other website interaction automation is that it almost always needs to be done without a browser. For instance, all of my screen scraping adventures have been using Perl. Browsers already know how to speak to the web, so why reinvent the wheel?

Firefox has lots of javascript-magic extensions such as greasemonkey and Selenium to let you execute browser-side javascript and activity automatically. Combine these together with Xvfb, and you can automate lots of things behind the scenes.

Tie this back to unit tests. Instead of simply displaying results of unit tests, have the page also report the results to a cgi script on the webserver. This will let you automatically test websites using a web browser and have it automatically report the results back to a server.


31 responses to 'Xvfb + Firefox'

Showing last 15 comments... (Click here to view all comments)

Ankit Singh wrote at Wed Nov 5 04:44:07 2008...
Hey I am not able to do it. I am getting this

root@hover2:/home/ankit# startx -- `which Xvfb` :2 -screen 0 1024x768x24

expected keysym, got XF86KbdLightOnOff: line 70 of pc
expected keysym, got XF86KbdBrightnessDown: line 71 of pc
expected keysym, got XF86KbdBrightnessUp: line 72 of pc
Could not init font path element /usr/share/fonts/X11/cyrillic, removing from list!

what to do???
please suggest

Jordan Sissel wrote at Wed Nov 5 04:47:42 2008...
Sounds like you don't have all the packages installed that you need or that something else is wrong.

Consult your distribution's forums or mailing lists.

Albert wrote at Wed Nov 26 09:35:29 2008...
i try to run firefox with DISPLAY=:0 /usr/bin/firefox http://google.com
And the output is below and cursor is not returning to prompt. I waited for 5 minutes but nothing happens. Please help

UA: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050524 Fedora/1.0.4-4 Firefox/1.0.4:, extra:  Firefox/1.0.4

Jordan Sissel wrote at Fri Nov 28 19:53:50 2008...
The 'firefox' command will only return you to a prompt if:
1) firefox is already running
2) when firefox exits

If firefox is not running, and you do 'firefox www.google.com' the command will wait until firefox closes.

XyZ wrote at Sun Dec 21 08:56:33 2008...
Referring to comment #6:

Didn't you mean "Xvfb -ac" (disable access control restrictions)?

XyZ wrote at Sun Dec 21 09:20:33 2008...
Referring to comment #6 and my own comment:

A more secure way than -ac (all hosts) is described at:
http://www.linuxquestions.org/questions/linux-distributions-5/rh-8.0-xvfb-connection-refused-problem.-34284/#post184558
[for only allowing localhost]

Brad wrote at Tue Feb 3 13:11:59 2009...
Is there anyway to setup the configuration to scroll up and down the whole content of the web page screen shot. Thanks.

michael wrote at Tue Mar 17 14:59:08 2009...
Hi. I began using selenium RC and Xvfb on a scientific linux box. My tests are in perl but our
team is writing tests in both perl and java. We want our tests to run concurrently on a selenium server and frame buffer that is setup in the bg and always running. I just thought I would include the info on the setup we use:

We first startup Xvfb in a terminal window this way:

{your path to xvfb here} :4 -screen -ac 1 1280x1024x8 > & /dev/null &

Then, in the same window we start up selenium this way:

env DISPLAY=:4 {path to java}  -jar {path to selenium server jar file) > & /dev/null &


Then, for a perl test:
-we point to the firefox2 executable. I use the perl module WWW::Selenium and when creating a new selenium instance, we include the line:

browser => "*firefox  {your path to firefox 2 executable here}

We will probably get rid of the "-ac" switch at some point but we are still testing things. Also,we needed to make sure that all of our individual tests reference firefox2.

We had some trouble running the above with firefox3...perhaps there is a work-around for this?

zcat wrote at Fri Apr 3 17:24:29 2009...
Setting up xvfb (2:1.4.2-10) ...
slug:~# startx -- `which Xvfb` :1 -screen 0 1024x768x24
Looks promising- but I get errors.  I am trying to run on debiian NSLU2 that has no display (I want to use a GTK app called deluge so that I can use its web interface but I get these inet6 errors.  I have no clue what to do.  Please help..
_XSERVTransSocketOpenCOTSServer: Unable to open socket for inet6
_XSERVTransOpen: transport open failed for inet6/slug:1
_XSERVTransMakeAllCOTSServerListeners: failed to open listener for inet6
Could not init font path element /usr/share/fonts/X11/cyrillic, removing from list!
[config/hal] couldn't initialise context: (null) ((null))

jart wrote at Fri Apr 10 10:44:59 2009...
for those who care about such things, i've created a couple of scripts for automating screen capture of web pages.  the first one, xffhandler.sh, handles the starting/stopping of xfvb and firefox.  while the other, ffcapture.sh, uses xwd and imagemagick's convert to take the screen shot.

the cpu usage checks in ffcapture are certainly cpu dependendant, so you'll probably have to tweak them considerably (or just use arbitrary sleep values) to make it work on your machine.  the little linux box that i have running these scripts (a fitpc) is pretty cpu-weak, so i can reliably assume that loading a page seriously affects cpu usage.  that being said, running ffcapture "http://someurl.com" gives me a valid screenshot 95% of the time; sometimes it prematurely shoots...heh.  that's okay, though, cuz i actually run this from a perl script like so:

$i = $done = 0;
$ffexec = "/path/to/ffcapture.sh \"$url\" $file";
while (!$done) { $i++;
  $ffcapture = `$ffexec`;
  $fsize = `/usr/bin/stat -c%s $file`;
  if ($ffcapture !~ /Error/is && $fsize >= 200000) { $done = 1; }
  else {
  print "SCREEN CAPTURE FILE SIZE IS TOO SMALL!\n" if $fsize < 200000;
  print "AN ERRORED OCCURED DURING SCREEN CAPTURE!!\n" if $ffcapture =~ /Error/is;
  if ($i >= 5) {
  print "I give up, I tried to take a screen capture 5 times\n";
  $done = 1;
  }
  }
}

and it hasn't failed yet.  enjoy.

#################################
## xffhandler.sh
#!/bin/bash
SLEEP_INT=10

case "$1" in
start)
  DISPLAY=:2.0
  export DISPLAY

  if [[ "$(top -bn1 | grep Xvfb)" == "" ]];
  then
  echo "Starting Xvfb (and waiting $SLEEP_INT secs)..."
  #Xvfb :2 -screen 0 1024x1400x24 >/dev/null 2>&1 &
  Xvfb :2 -screen 0 1024x2000x24 >/dev/null 2>&1 &
  while [[ "$(top -bn1 | grep Xvfb)" == "" ]];
  do
  sleep $SLEEP_INT;
  done
  else
  echo "Xvfb ALREADY LOADED"
  fi

  if [[ "$(top -bn1 | grep firefox-bin)" == "" ]];
  then
  echo "Starting firefox (and waiting $SLEEP_INT secs)..."
  firefox >/dev/null 2>&1 &
  sleep $SLEEP_INT;
  while [[ "$(top -bn1 | grep firefox-bin)" == "" ]];
  do
  sleep $SLEEP_INT;
  done
  else
  echo "firefox ALREADY LOADED"
  fi
;;

stop)
  echo "Stopping firefox/Xvfb..."
  if [[ "$(top -bn1 | grep firefox-bin)" != "" ]];
  then
  killall firefox-bin
  echo "  firefox has been killed"
  else
  echo "  firefox NOT LOADED"
  fi

  if [[ "$(top -bn1 | grep Xvfb)" != "" ]];
  then
  killall Xvfb
  echo "  Xvfb has been killed"
  else
  echo "  Xvfb NOT LOADED"
  fi
;;

status)
  if [[ "$(top -bn1 | grep Xvfb)" != "" ]];
  then
  echo "Xvfb is running"
  else
  echo "Xvfb NOT running"
  fi

  if [[ "$(top -bn1 | grep firefox-bin)" != "" ]];
  then
  echo "Firefox is running"
  else
  echo "Firefox is NOT running"
  fi
;;
esac
exit 0
#################################

#################################
## ffcapture.sh
#!/bin/bash
URL=$1
FILE=$2
CPU_LIMIT=15
SLEEP_INT=15
DISPLAY=:2.0

echo "performing a screen capture on:"
echo "  $URL"
echo "  and saving result to: $FILE"

/path/to/XFFhandler.sh start
export DISPLAY
firefox -remote "openurl($URL,new-window)"
sleep $SLEEP_INT

echo "  waiting for page to load (pid's cpu usage < $CPU_LIMIT)"
while [[ "$(top -bn1 | grep firefox-bin | awk '{print $9}')" > "$CPU_LIMIT" ]];
do
  echo "  sleeping for $SLEEP_INT secs...";
  sleep $SLEEP_INT;
done
xwd -root -silent | convert - $FILE  ## wait until page is loaded

echo "success! ...i think :)";
/path/to/XFFhandler.sh stop
exit 0
#################################

jart wrote at Fri Apr 10 10:48:29 2009...
oh yeah, i forgot to mention that the firefox addon 'full fullscreen' is awesomely useful.  it can force ff to load completely fullscreen on startup.

Prashant Shah wrote at Mon Jun 22 07:03:38 2009...
I am getting an error with the following command:

[root@domU-12-31-39-01-C0-42 automation]# DISPLAY=:99 firefox  http://google.com
No fonts found; this probably means that the fontconfig
library is not correctly configured. You may need to
edit the fonts.conf configuration file. More information
about fontconfig can be found in the fontconfig(3) manual
page and on http://fontconfig.org

Jordan Sissel wrote at Mon Jun 22 13:02:09 2009...
I would guess that you are missing some X packages. The error message mentions fonts, so that's probably a good place to start looking at what you should install.

ivan wrote at Fri Jun 26 05:18:25 2009...
I have problem - return screen image with "Your last session closed unexpectecly".

I add user.js file
user_pref("browser.sessionstore.enabled", false);
user_pref("browser.sessionstore.resume_from_crash", false);

but it did not help still firefox run with this stupid screen

Do any solve this problem?

Tennis Smith wrote at Mon Jun 29 17:31:23 2009...
Is it possible to leave Xvfb running permanently rather than start it for every test?

Also x11vnc is a great way to monitor a test's progress remotely.

-T


Leave a reply

You need javascript enabled to use this form. Anti-spam efforts ongoing. Also, if the comment doesn't show up, it's because the form expired. Go back and copy your comment, reload the form, and resubmit. Apologies if this is a hassle, I'm just playing with antispam methods right now. If this insists on not working, please email me about it.

Name (required)
E-mail (optional, if you want me to be able to email you back)
URL (also optional)
Comment: