Tuesday, December 11, 2012

virt-manager and xquartz

'ssh -X'ing to a Linux KVM machine, running virt-manager, and getting a VNC display to an OS X/xquartz box was problematic until I found http://blog.loftninjas.org/2010/11/17/virt-manager-keymaps-on-os-x/  Turns out, the keymap isn't set and falls back to the OS X mapping, which I guess isn't compatible.  On OS X:
$setxkbmap -print
xkb_keymap {
        xkb_keycodes  { include "empty+aliases(qwerty)" };
        xkb_types     { include "complete"      };
        xkb_compat    { include "complete"      };
        xkb_symbols   { include "empty(basic)"  };
        xkb_geometry  { include "pc(pc104)"     };
On an Ubuntu machine that worked:
~$ setxkbmap -print
xkb_keymap {
    xkb_keycodes  { include "evdev+aliases(qwerty)"    };
    xkb_types     { include "complete"    };
    xkb_compat    { include "complete"    };
    xkb_symbols   { include "pc+us+inet(evdev)+ctrl(nocaps)"    };
    xkb_geometry  { include "pc(pc105)"    };
I didn't take the time to figure out which differences are important.  Instead, I did the suggested "virsh edit IMAGEID" and add "keymap='en-us'" to the "graphics" line.  This edits the "/var/run/libvirt/qemu/IMAGEID.xml" file.  Once this was done, I did have to destroy to the running instance and then start it; rebooting the guest didn't work.  I had also downloaded the newest xquartz from http://xquartz.macosforge.org/ so the alt key remapping can be done via the preferences.

OpenBSD on KVM

Previous versions of OpenBSD didn't work and play well with some hypervisors, Xen and KVM included.  One had to worry about using the right virtual NIC, fixing some MP BIOS settings, etc., and even then it often wouldn't work.  With 5.3, things seem to have evened out.  One still has to set the NIC to be a e1000, but that's it.  Here are two screen shots from virt-manager.  Set things up as desired, but on the last screen, tick "Customize configuration before install" before clicking Finish.  Then, specify the NIC, and click Apply.

Some hypervisors like having "Virtio" set for disks:

And while you're at it, change the keyboard from the default so it will work with xquartz:

Tuesday, November 27, 2012

IPMI SOL, newer AMI BIOS, and CentOS 6

SOL means "serial over LAN", but there are some similarities to the more popular interpretation. Here's the combination to get everything working with newer American Megatrends Inc. BIOSs.  First, set up the BIOS by enabling the "Serial Port for Out-of-Band Management/Windows Emergency Management Services (EMS)":
You don't need to customize anything in the Settings, but here's a screenshot anyway:
CentOS (well, and Redhat) 6 changed the configuration of serial ports from using /etc/inittab to using files in /etc/init.  So, create a /etc/init/ttyS1.conf that contains:
# ttyS1 - agetty
# This service maintains a agetty on ttyS1.

stop on runlevel [S016]
start on runlevel [23]

exec agetty /dev/ttyS1 19200 vt102
Do a "initctl start ttyS1" to start the getty.  Add ttyS1 to /etc/sercuretty if you want to allow root logins on that port (that bit didn't change).  You should then be able to use ipmitool for all your SOL needs.

Device br0 does not seem to be present, delaying initialization.

I recently received this error on a CentOS Linux machine handed off to me. A bit of a head scratcher. Getting on the console and running ifup by hand looked like:
# bash -x ./ifup-eth ifcfg-br0
+ . /etc/init.d/functions
++ TEXTDOMAIN=initscripts
++ umask 022
++ PATH=/sbin:/usr/sbin:/bin:/usr/bin
++ export PATH
++ '[' -z '' ']'
++ '[' -z '' ']'
+++ /sbin/consoletype
[...] The interesting part showed up a little later:
+ CONFIG=ifcfg-ifcfg-br0
+ '[' -f ifcfg-ifcfg-br0 ']'
+ CONFIG=ifcfg-br0
+ '[' -f ifcfg-br0 ']'
+ return
+ source_config
+ CONFIG=ifcfg-br0
+ . /etc/sysconfig/network-scripts/ifcfg-br0
++ DEVICE=br0
++ ONBOOT=yes
++ TYPE=bridge
++ BOOTPROTO=static
+ '[' -r keys-br0 ']'
The eagle-eyed will have already spotted the error, the following helps the rest of us:
+ alias=
+ '[' 1 -eq 0 ']'
+ return 2
+ '[' -n '' ']'
+ '[' bridge = Bridge ']'
+ '[' bridge = Tap ']'
+ is_available br0
+ '[' -z br0 ']'
+ '[' -d /sys/class/net/br0 ']'
+ '[' -n '' ']'
++ modprobe -c
++ awk 'BEGIN { alias = ""; }
$1 == "alias" && $2 == "br0" { alias = $3; }
$1 == "install" { install[$2] = $3; }
    cmd = install[alias];
    print alias;
    if (alias == "" || alias == "off" || cmd == "/bin/true" || cmd == ":")
        exit 1;
    exit 0;
+ alias=
+ '[' 1 -eq 0 ']'
+ return 2
+ '[' -n '' ']'
+ net_log 'Device br0 does not seem to be present, delaying initialization.'
+ local 'message=Device br0 does not seem to be present, delaying initialization.'
+ local level=
+ local name=
+ '[' -z 'Device br0 does not seem to be present, delaying initialization.' ']'
+ '[' -z '' ']'
+ level=err
+ '[' -z '' ']'
+ name=./ifup-eth
+ echo Device br0 does not seem to be present, delaying initialization.
Device br0 does not seem to be present, delaying initialization.
+ '[' -x /usr/bin/logger ']'
+ /usr/bin/logger -p daemon.err -t ./ifup-eth 'Device br0 does not seem to be present, delaying initialization.'
+ return 0
+ exit 1
See the obvious "bridge = Bridge"? Well, maybe not if you're not at one with such things. Nonetheless, it takes a capital 'B' Bridge in /etc/sysconfig/network-scripts/ifcfg-br0

"Abyss" watch instructions translation

I saw a co-worker's "Abyss" watch from ThinkGeek and had to have one.  I ordered from Amazon, and here are the instructions for setting the time:
Note the completely made-up words.  Happily, the reverse side had instructions in Mandarin, and I have easy access to a native speaker/reader.  The instructions really are: tap face eight times and hold for three seconds to set the hours.  Hope this helps someone and hope that more companies employ better translators, or at least use google translate.

Friday, November 9, 2012

dd and the power of programming

I recently needed to extract part of a large (~8GB) file. The canonical method is to use dd. On a Linux system, the performance was abysmal. Here is the straight dd result.
# dd if=one.img of=two.img bs=1 skip=32768 count=8587192319
8587192319 bytes (8.6 GB) copied, 12442.3 s, 690 kB/s
That's 3.45 hours for those keeping track at home. Using /dev/shm helped.
# dd if=/dev/shm/one.img of=/dev/shm/two.img bs=1 skip=32768 count=8587192319
8587192319 bytes (8.6 GB) copied, 6232.88 s, 1.4 MB/s
That helped, but it's still 1.73 hours. Okay, so we all know the 1 byte block size is killing the performance. We also all know 8587192319 = 41809 205391, so let's bump up the block size.
# dd if=one.img of=two.img bs=205391 skip=32768 count=41809
9054+1 records in
9054+1 records out
1859682304 bytes (1.9 GB) copied, 1.59914 s, 1.2 GB/s
Clearly, much faster but also wrong: it should copy 8.6 GB. Let's reverse the numbers and try again.
# dd if=one.img of=two.img bs=41809 skip=32768 count=205391
172688+1 records in
172688+1 records out
7219937280 bytes (7.2 GB) copied, 29.4616 s, 245 MB/s
Closer, but still wrong. Something is wrong with dd's math. So, I wrote a Java program that takes an input file name, an output file name, and pairs of skip/write values. Here is the result.
# time java Chopper one.img two.img 32768 8587192319
Skipped 32768 bytes.
Writing 8587192319 bytes.

real    0m10.656s
user    0m1.744s
sys     0m8.933s
So, 10.7 seconds versus 3.45 hours. Here's the program.
** Argrments: input file, output file, pairs of skip/write bytes

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;

public class Chopper
    private static void readAndWrite (FileInputStream in, FileOutputStream out,
        long length)
        ** Keep this sized to fit in an int for the read below.  Besides, 8K
        ** is a typical disk block size.
        final long BUFFSIZE = 8192;
        byte buffer[] = new byte[(int) BUFFSIZE];

        for (;;)
            int read = 0;
            int count = (int) (BUFFSIZE > length ? length : BUFFSIZE);

                read = in.read (buffer, 0, count);
                out.write (buffer, 0, read);
            catch (IOException IOE)
                System.out.println (IOE);

            length -= read;

            if (length <= 0)

    public static void main (String args[])
        FileInputStream in = null;
        FileOutputStream out = null;

            in = new FileInputStream (args[0]);
            out = new FileOutputStream (args[1]);
        catch (IOException IOE)
            System.out.println (IOE);

        int i = 2;

        while (i < args.length)
                long skipped = in.skip (Long.parseLong (args[i++]));
                System.out.println ("Skipped " + skipped + " bytes.");
            catch (IOException IOE)
                System.out.println (IOE);

            long write = Long.parseLong (args[i++]);

            System.out.println ("Writing " + write + " bytes.");
            readAndWrite (in, out, write);

Monday, October 15, 2012

Reverse chronological package installs in OS X

pkgutil --pkgs | xargs -n 1 pkgutil --pkg-info | gawk '{printf("%s%s",$0,(NR%5==0)?"\n":"\0")}' | sort -n -k 6 | tr "\0" "\n" | gawk '/install-time/ {print $0 " " strftime("(%c)",$2); next}{print}'

Wednesday, October 3, 2012

Time(less) machine

Geez, I hate upgrading OSs.  No one gets it right, and we all collectively spend inordinate amounts of time dealing with the fallout.  Think of all the time (read: money) wasted on such headaches.  Most recently for me, the Time Machine backup program broke on an upgrade to 10.7.5.  Backups were estimated to have MAX_INT hours remaining.  It appears there is a conflict between Spotlight and Time Machine -- two programs from the same vendor.  One has to disable Spotlight via
sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist
and the run Time Machine. One might also have to reboot between the two.  After Time Machine runs, re-enable with
sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist
Running a dtruss on the backupd process shows a lot of
psynch_cvwait(0x7FE29969CE98, 0x28D0100028E00, 0x0)   = -1 Err#316
psynch_cvwait(0x7FE29969CE98, 0x28F0100029000, 0x0)   = -1 Err#316
psynch_cvwait(0x7FE29969CE98, 0x2900100029100, 0x0)   = -1 Err#316
psynch_cvwait(0x7FE29969CE98, 0x2910100029200, 0x0)   = -1 Err#316
psynch_cvwait(0x7FE29969CE98, 0x2920100029300, 0x0)   = -1 Err#316
psynch_cvwait(0x7FE29969CE98, 0x2930100029400, 0x0)   = -1 Err#316
psynch_cvwait(0x7FE29969CE98, 0x2950100029600, 0x0)   = -1 Err#316
psynch_cvwait(0x7FE29969CE98, 0x2960100029700, 0x0)   = -1 Err#316
psynch_cvwait(0x7FE29969CE98, 0x2980100029900, 0x0)   = -1 Err#316
psynch_cvwait(0x7FE29969CE98, 0x29A0100029B00, 0x0)   = -1 Err#316
psynch_cvwait(0x7FE29969CE98, 0x29B0100029C00, 0x0)   = -1 Err#316
psynch_cvwait(0x7FE29969CE98, 0x2940100029500, 0x0)   = -1 Err#316
psynch_cvwait(0x7FE29969CE98, 0x2970100029800, 0x0)   = -1 Err#316
psynch_cvwait(0x7FE29969CE98, 0x2990100029A00, 0x0)   = -1 Err#316
So, there appears to be a threading issue here somewhere, but without the Time Machine source, it's difficult to track down.

Updates.  Doing a "sudo mdutil -i off /" appears to do approximately the same thing as the launchctl, but immediately, though I'm not convinced it's exactly the same thing.  And it appears one of Apple's recent updates fixed the problem -- let's hope they added a regression test for this.

Thursday, September 20, 2012

Mail unread menu and 5.3

Another thing that broke on an OSX upgrade was mail unread menu (http://loganrockmore.com/mailunreadmenu/) which I really like.  So with a little help from http://stib.posterous.com/how-to-fix-unsupported-plugins-after-upgradin I was able to write the following script (though you can cut and paste them directly into a Terminal (Applications -> Utilities -> Terminal) window).
#! /bin/sh

a=`defaults read /Applications/Mail.app/Contents/Info PluginCompatibilityUUID`
b=`defaults read /System/Library/Frameworks/Message.framework/Resources/Info PluginCompatibilityUUID`

for i in ~/Library/Mail/Bundles*/MailUnreadMenu.mailbundle/Contents/Info.plist
    echo "$i"
    echo "${i%%.plist}"

    defaults write "${i%%.plist}" SupportedPluginCompatibilityUUIDs -array-add "$a";
    defaults write "${i%%.plist}" SupportedPluginCompatibilityUUIDs -array-add "$b";
    plutil -convert xml1 "$i"
Then a
mv ~/Library/Mail/Bundles\ \(Disabled\)/MailUnreadMenu.mailbundle/ ~/Library/Mail/Bundles
Made things right again.

Wednesday, September 19, 2012

gpgmail for 5.3

Updated to Apple Mail 5.3, and gpgmail stopped working.  Uninstalled gpgmail, gpgtools, and reinstalled both.  Had to do "chown -R beaty-admin /Library/Mail/Bundles/GPGMail.mailbundle" and run the System Preferences, GPGPreferences, About, Fix GPGtools to get everything working again.

ubuntu 12


Wednesday, September 12, 2012

Is consistency overrated?

I have a Firefox extension: https://addons.mozilla.org/en-us/firefox/addon/lightweight-theme-switcher/  I've been playing with adding a new menu, and was struggling to find all the default menu names.  Here they are: file-menu, edit-menu, view-menu, history-menu, bookmarksMenu, tools-menu, windowMenu, helpMenu.  Am I the only one that this bugs?

Sunday, August 26, 2012

Disappearing SD MMC slot on MacBook Pro

Some upgrade caused the SD MMC slot on my MacBook Pro to no longer be recognized.  Zapping the PRAM (option-command-p-r during boot) brought it back.

Friday, August 3, 2012

Incremental/interruptable Unix/OSX file copy

I wanted to copy a large file to a Unix (OSX) system, and I knew it wouldn't complete before I took my laptop home.  Indeed, it would take a couple of days, so a couple of detaches would be needed.  I couldn't find anything that seemed to do the job I wanted, so I wrote the following script.  It takes two arguments: the from file and the to file.  It examines how large the to file is, and starts the copy from there, using 1 megabyte blocks. It won't work if the file changes during the overall copy operation of course.
#! /bin/bash

output_size=`du -m "$2" | cut -f 1`

if [[ $output_size = "" ]]

dd bs=1m seek=$output_size skip=$output_size if="$1" of="$2"

Thursday, June 28, 2012

OSX keyboard macros

OSX doesn't exactly come with keyboard macros, but it's not difficult to create them.  First, write an AppleScript that simply writes keystrokes to "System Events":
Save it to your "Library/Scripts" folder.

Then, add the key mapping you want to "System Preferences, Keyboard, Keyboard Shortcuts, Application Shortcuts".  Click the '+' to add one, use the name of the script you created, and choose a shortcut.  If you want to change the shortcut later, double click right over the characters for the shortcut.

I used this to create a shortcut when editing an html pre element, so it looks similar to http://formatmysourcecode.blogspot.com/

Wednesday, June 27, 2012

OSX, mail.app, and dovecot

OSX's mail.app can't seem to read files in /var/mail sent by, you guessed it: OSX itself. So, one can install dovecot from MacPorts and do a little configuration. First, set up a self-signed cert.
sudo mkdir -p /opt/local/etc/ssl/certs
sudo mkdir -p /opt/local/etc/ssl/private
sudo openssl req -new -x509 -nodes -config dovecot-openssl.cnf -out /opt/local/etc/ssl/certs/dovecot.pem -keyout /opt/local/etc/ssl/private/dovecot.pem -days 3650
Then, copy the example configuration
sudo cp /opt/local/etc/dovecot/dovecot-example.conf /opt/local/etc/dovecot/dovecot.conf
and make minor mods to dovecot.conf
sudo cp /opt/local/etc/dovecot/dovecot-example.conf /opt/local/etc/dovecot/dovecot.conf
sudo rcsdiff /opt/local/etc/dovecot/dovecot.conf
RCS file: /opt/local/etc/dovecot/dovecot.conf,v
retrieving revision 1.1
diff -r1.1 /opt/local/etc/dovecot/dovecot.conf
< #mail_location = 
> mail_location = /var/mail/%u
<     #args = dovecot
>     args = login
<     #args = 
>     args = 
And fire dovecot up
sudo port load dovecot

Monday, June 25, 2012

pulledpork lameness

All over the snort.org site it says "In order to manage rules we officially recommend and depend on the user using PulledPork." Well they must assume everyone is registered instead of subscribed as pulledpork doesn't work for subscribers. Lame. Here are the patches to make it work.
# rcsdiff ./pulledpork.pl 
RCS file: ./RCS/pulledpork.pl,v
retrieving revision 1.1
diff -r1.1 ./pulledpork.pl
<           getstore( "https://www.snort.org/reg-rules/$rule_file/$oinkcode",
>           getstore( "https://www.snort.org/sub-rules/$rule_file/$oinkcode",
<           getstore( "https://www.snort.org/reg-rules/$rule_file.md5/$oinkcode",
>           getstore( "https://www.snort.org/sub-rules/$rule_file.md5/$oinkcode",

Wednesday, June 20, 2012

Eucalyptus 3 notes

Changes made to /etc/eucalyptus/eucalyptus.conf
# rcsdiff -r1.1 eucalyptus.conf
RCS file: RCS/eucalyptus.conf,v
retrieving revision 1.1
diff -r1.1 eucalyptus.conf
< NODES=""
> NODES=""
< VNET_BRIDGE="xenbr0"
> VNET_BRIDGE="virbr0"
< #VNET_PUBLICIPS="your-free-public-ip-1 your-free-public-ip-2 ..."
> VNET_PUBLICIPS="my public ips"
< #VNET_DNS="your-dns-server-ip"
> VNET_DNS="my dns server"

I preferred to use a real class B private IP address as with a subnet mask of isn't a proper private network address.  I set up a simple system: a front end with the CC, SC, etc., and one NC via a crossover cable.

Note: as of this writing, one cannot create bundles on OS X boxen and have them work correctly.  You'll get a 'bad superblock' error...

Also note that that on page 28, there is essentially a typo.  For a MANAGED-NOVLAN setup, there is no need to create a bridge of any sort; xen will do that automatically.  And the "TYPE=Bridge" does not work under CentOS.

To find the latest release of euca2ools, look at http://downloads.eucalyptus.com/software/euca2ools/
For Eucalyptus itself: http://downloads.eucalyptus.com/software/eucalyptus/

Thursday, May 31, 2012

Scrollbars in Ubuntu 11++

Here's how one really gets back to the original scrollbars:
sudo bash -c "echo export LIBOVERLAY_SCROLLBAR=0 > /etc/X11/Xsession.d/80overlayscrollbars"
Logout, log back in.

Sunday, April 22, 2012

Linux and OS X OCR

Mostly for my notes:
pdftoppm -f 2 -gray AmericanLegion.pdf AL

for i in *.pgm
    pnmtotiff $i > ${i%%.pgm}.tif
    rm $i

for i in *.tif
    tesseract $i ${i%%.tif}
Tesseract needs images of decent resolution; e.g.: in PowerPoint it's better to "Save as Pictures" at higher than the default resolution:

Friday, January 20, 2012

iCal alarms for recurring subscribed events

iCal won't alarm on events in subscribed to calendars, such as Facebook.  I guess the originator of the event has to set whether to alarm or not.  So I spent a little time leaning AppleScript, and here's a script that does it.
on run the_calendars
        length of calendars
    on error
        set the_calendars to {"Facebook"}
    end try
    set today to (current date)
    set this_day to day of today
    set this_month to month of today
    set to_display to ""
    tell application "Calendar"
        repeat with the_calendar in the_calendars
            log the_calendar
            tell calendar the_calendar
                repeat with the_event in every event
                    set the_date to start date of the_event
                    set the_day to day of the_date
                    set the_month to month of the_date
                    if (the_month = this_month and the_day = this_day) then
                        set to_display to (to_display & summary of the_event as string) & return
                    end if
                end repeat
            end tell
        end repeat
    end tell
    if (length of to_display > 0) then
        tell application "Finder"
            with timeout of 86400 seconds
                display dialog to_display
            end timeout
        end tell
    end if
end run

Open up you AppleScript Editor, copy and paste, and save somewhere. Now, you'll need to have it run every day. So I learned a little about launchctl. Save the following script to ~/Library/LaunchAgents. I called mine 'edu.ucar.icalsubscriptions.plist'. You'll need to modify the Program path to where you saved your script, and possibly the time it runs if you don't like 0900. You might also need to change the name of the calendar to match the name of the calendar in iCal -- mine is called Facebook -- and you can have a list of them if you want.
<?xml version="1.0" encoding="UTF-8"?>
     "-//Apple//DTD PLIST 1.0//EN"
<plist version="1.0">





Wednesday, January 18, 2012


People ask me about security, and much of it is based on having good passwords.  In general, one-time passwords are best.  These change each time one logs in; yubikeys and RSA ids are examples.  Two factor -- typically one reusable and one ephemeral (e.g.: SMS text your cell phone) -- are good too.  When we're stuck with reusable (using the same one again and again), then picking good ones is critical.  Good ones are both long and difficult to guess.  Random is best, as we humans aren't as random as we first appear.  As long and random passwords aren't memorable, we need password keepers.  Firefox will remember passwords for you; I recommend you allow it to do this, but: set a master password!  I also recommend using keepass from http://keepass.info/ for whichever platform you have.

So, let's do a little math.  26 lower case letters, 26 upper case, 10 digits, and 10 symbols equals 72 characters.  Let's say you have 8 character random passwords using each of the character classes and no repeated characters.  So, 72*71*70*69*68*67*66*65 -= 482,590,739,030,400 possible passwords.  Not bad.  Now let's go to 20.  72*71*70*69*68*67*66*65*64*63*62*61*60*59*58*57*56*55*54*53 = 759,184,772,617,383,139,127,116,820,643,840,000 possible passwords.  Good enough for the time being.

Below is a password generator written in JavaScript. This assures the password is generated on you machine, in your browser. Choose how long you want your password to be, and what type of characters it needs to contain. Click "Generate!" and then you can cut and paste it into a password field. I recommend against sites that generate passwords on their server -- they have your address and a password of yours. And remember: you don't have to remember it, have your computer do that for you.

Number of characters
Include lowercase
Include uppercase
Include numbers
Include symbols