diff --git a/TODO b/TODO index 3399d5e..b1a3bba 100644 --- a/TODO +++ b/TODO @@ -8,11 +8,64 @@ Bugs to fix and Features to add before a 4.2 release - ambiguous option list with regards to parameters -- talks about Sarge and Etch +* Interactive Password Handling (maybe also the passwd role) + + Shouldn't give up after only one password missmatch. This is just + a mater of wrapping the passwd call in a loop. + +* Fix 70-install-ssh + + Currently 70-install-ssh fails to finish installing the ssh-server + because it does not have a random seed to generate the host keys. + Mounting /proc and /dev in chroot prior to calling the install + function should fix this. If ssh-server is successfully installed + in the domU, it's key fingerprint should be echo'd in a way that + can easily be grep'ed into other scripts. + + +* Test and support more file system types. + + Actually this should be pretty simple now that the parameters are + stored in the configuration hash. The only issue is that trailing + whitespace is missing from the "make_fs_foo" option. + +* xen-delete-image doesn't remove all logical volumes if --partitions is used. + + See the link below for details how to reproduce. Reproducable at + least with --lvm. Thanks to Antoine Benkemoun for reporting. + + http://xen-tools.org/pipermail/xen-tools-discuss/2010-May/000757.html + +* xen-delete-image ignores extension setting + +* xen-create-image should check all integer options on non-digits. + +* Test suite should pass + + Currently failing: + + t/getopt.t + t/xen-delete-image.t + t/xen-tools.t + t/xt-create-xen-config.t + +* Clean up setup-hostname - copying /etc/hosts isn't reliable + + One could argue that that this gives "extra" information to the domU, since + the /etc/hosts file may contain more than just hosts on the same subnet. + This should atleast be disabled via a flag. + +* Setup locales in the hooks? + + Currently no locales are set and this causes several domU errors which appear + in the domU's logs. + +Maybe for a 4.3 or 5.0 release +------------------------------ * xen-create-image --dist=... / sources.list generation should be more fine-grained - xen-tools should offer the possibility to enabled/disable + xen-tools should offer the possibility to enable/disable security/volatile/backports as well as contrib/non-free/universe/restricted/multiverse for each of them not only based on defaults plus the Dom0's sources.list @@ -38,56 +91,31 @@ Bugs to fix and Features to add before a 4.2 release /etc/xen-tools/sources.list.d/ because that directory is no runparts-like directory. -* Test and support more file system types. - - Actually this should be pretty simple now that the parameters are - stored in the configuration hash. The only issue is that trailing - whitespace is missing from the "make_fs_foo" option. - -* Should check all integer expection options on other characters than digits. - -* Test suite should pass - - Currently failing: - - t/getopt.t - t/xen-delete-image.t - t/xen-tools.t - t/xt-create-xen-config.t - -* Add distro name and release number to pygrub screen - -* Hooks -- --vcpus - This one is trivial - -- --bridge - This one is also trivial - -- --passwd - shouldn't give up after only one password missmatch - -- Clean up setup-hostname - copying the dom0s /etc/hosts isn't reliable - -- Add a flag to not copy /etc/hosts file - -- Setup locales in the hooks? - - -Maybe for a 4.3 or 5.0 release ------------------------------- - * LVM snapshot support as an install source. * Clean up mounts on Ctrl-C, causes error while installing otherwise: Removing /dev/vg0/acromantula-domu1-disk - since we're forcing the install - Can't remove open logical volume "acromantula-domu1-disk" + Can't remove open logical volume "acromantula-domu1-disk" this should be a + matter of unmounting the mounted volume from /tmp. -* btrfs support +* Non-Interactive Password Handling + + Option to generate passwords (check pwgen, apg, makepasswd, and + gpw which of them are best suited for generating complete + /etc/passwd and /etc/shadow lines). The password should be echo'ed + after installation in a way that can be easily grep'ed into other + scripts. + +* Generic grub support + + This will generate a much nicer menu.lst as a side effect, as its currently + generated once at install, and is never updated. Installing a full grub into + the domU should update the menu.lst every time a new kernel is installed and + will also use the domU distro's menu.lst conform. * Move the hooks directory to /etc/xen-tools/ to officially allow - added and modified hooks. + added and modified hooks. * Clean up the hooks directory @@ -97,12 +125,14 @@ Maybe for a 4.3 or 5.0 release I'm sure this won't scale forever. So to minimise code duplication I'd like to have one common directory per distribution (e.g. called - ubuntu-common, fedora-common, etc.) with generic hooks valid for - all or most of the releases of one distribution and then one hook - directory per release (e.g. called ubuntu-10.04 or ubuntu-lucid or - so) which has symbolic links to everything which can be used - unchanged from the common directory and new files for everything - which has to be different or only there. + ubuntu-common, fedora-common, debian-common, maybe even deb-common, + rpm-common or yum-common for dpkg/apt-, rpm/yum-based + distributions, etc.) with generic hooks valid for all or most of + the releases of one distribution and then one hook directory per + release (e.g. called ubuntu-10.04 or ubuntu-lucid or so) which has + symbolic links to everything which can be used unchanged from the + common directory and new files for everything which has to be + different or only there. * Create users, add ssh pubkeys to .ssh/authorized_keys @@ -113,7 +143,8 @@ Maybe for a 4.3 or 5.0 release * More generic hooks Deploy a web server or setup ssmtp directly via flag when setting up - the machine. Open to suggestions, should just be simple applications + the machine. Open to suggestions, should just be some general use-cases + that are fairly common. * Sections for the xen-tools.conf file diff --git a/bin/xen-create-image b/bin/xen-create-image index 4c1952f..7c99949 100755 --- a/bin/xen-create-image +++ b/bin/xen-create-image @@ -39,7 +39,9 @@ xen-create-image - Easily create new Xen instances with networking and OpenSSH. --copy-cmd NOP: Ignored. --debootstrap-cmd - NOP: Ignored. + Specify which debootstrap command is used. Defaults to + debootstrap if both, debootstrap and cdebootstrap are + installed. --force Force overwriting existing images. This will remove existing images or LVM volumes which match those which @@ -327,7 +329,7 @@ Create a Xen configuration file in so that xm can start the new domain. memory = 128Mb # Memory size swap = 128Mb # Swap size fs = ext3 # use EXT3 filesystems - dist = sarge # Default distribution to install. + dist = stable # Default distribution to install. # # Kernel options. @@ -438,9 +440,9 @@ Create a Xen configuration file in so that xm can start the new domain. =head1 LOOPBACK EXAMPLES The following will create a 2Gb disk image, along with a 128Mb - swap file with Debian Sarge setup and running via DHCP. + swap file with Debian Stable setup and running via DHCP. - xen-create-image --size=2Gb --swap=128Mb --dhcp \ + xen-create-image --size=2Gb --swap=128Mb --dhcp --dist=stable \ --dir=/home/xen --hostname=vm01.my.flat This next example sets up a host which has the name 'vm02.my.flat' and @@ -590,7 +592,7 @@ Create a Xen configuration file in so that xm can start the new domain. =head1 ROLES Currently there are some roles scripts included which work for - the Debian Sarge and Etch distrubtions only. They are included + the Debian and Ubuntu distrubtions only. They are included primarily as examples of the kind of things you could accomplish. The supplied scripts are: @@ -601,12 +603,43 @@ Create a Xen configuration file in so that xm can start the new domain. Setup the new virtual images with commonly used packages for rebuilding Debian packages from their source. +=item cfengine +Install cfengine2 on the virtual image and copy the cfengine +configuration from Dom0. + +=item editor +Allows generalised editing of files for guests. + +This script works via a skeleton directory containing small sed files +which will contain edits to be applied to an arbitrary tree of files +upon the new domU. + +For example if we have the following sed file: + + /etc/xen-tools/sed.d/etc/ssh/sshd_config.sed + +this will be applied to /etc/ssh/sshd_config upon the new guest *if* +it exists. If the file encoded in the name doesn't exist then it will +be ignored. + =item gdm Install an X11 server, using VNC and GDM =item minimal Customise the generated images to remove some packages. +=item puppet +Install puppet on the virtual image and copy the cfengine +configuration from Dom0. + +=item tmpfs +Sets up /tmp, /var/run and /var/lock as tmpfs in the DomU. + +=item udev +Install udev in the DomU. Most distributions install udev by default +nowadays, so this role is probably only interesting for legacy systems +which need udev anyway. + =item xdm Install an X11 server, using VNC and XDM @@ -1272,9 +1305,11 @@ sub setupDefaultOptions $CONFIG{ 'extension' } = '.cfg'; # - # Installation method defaults to "debootstrap". + # Installation method defaults to "debootstrap" using + # "debootstrap" (instead of cdebootstrap). # $CONFIG{ 'install-method' } = 'debootstrap'; + $CONFIG{ 'debootstrap-cmd' } = ''; # # The program to run to create a filesystem. @@ -1465,8 +1500,9 @@ sub parseCommandLineArguments # # NOTE: We set the local variable here, not the global. # - "install-method=s", \$CONFIG{ 'install-method' }, - "install-source=s", \$CONFIG{ 'install-source' }, + "install-method=s", \$CONFIG{ 'install-method' }, + "install-source=s", \$CONFIG{ 'install-source' }, + "debootstrap-cmd=s", \$CONFIG{ 'debootstrap-cmd' }, # Misc. options "accounts", \$CONFIG{ 'accounts' }, @@ -1920,7 +1956,7 @@ EOF The output directory for creating the xen configuration file within doesn\'t exist: - $CONFIG{'dir'} + $CONFIG{ 'output' } Aborting. @@ -3229,6 +3265,16 @@ sub installSystem } + # + # Propagate --debootstrap-cmd if install-method is debootstrap + # + if ( $CONFIG{ 'install-method' } eq 'debootstrap' and + $CONFIG{ 'debootstrap-cmd' } ) + { + $cmd .= " --debootstrap-cmd=$CONFIG{'debootstrap-cmd'}"; + } + + # # Show the user what they are installing # diff --git a/bin/xen-delete-image b/bin/xen-delete-image index c153869..546e13e 100755 --- a/bin/xen-delete-image +++ b/bin/xen-delete-image @@ -340,10 +340,12 @@ sub parseCommandLineArguments my $HELP = 0; my $MANUAL = 0; my $VERSION = 0; + $CONFIG{ 'dry-run' } = 0; # Parse options. # GetOptions( "dir=s", \$CONFIG{ 'dir' }, + "dry-run", \$CONFIG{ 'dry-run' }, "lvm=s", \$CONFIG{ 'lvm' }, "evms=s", \$CONFIG{ 'evms' }, "hostname=s@", \$CONFIG{ 'hostname' }, @@ -488,8 +490,12 @@ sub deleteXenImage { if ( -e $file ) { - print "Deleting: $file\n"; - unlink($file); + if ($CONFIG{ 'dry-run' }) { + print "Would delete: $file\n"; + } else { + print "Deleting: $file\n"; + unlink($file); + } } else { @@ -507,8 +513,12 @@ sub deleteXenImage # if ( -d $prefix . $hostname ) { - print "Removing: " . $prefix . $hostname . "\n"; - rmdir( $prefix . $hostname ); + if ($CONFIG{ 'dry-run' }) { + print "Would delete: $prefix$hostname\n"; + } else { + print "Removing: " . $prefix . $hostname . "\n"; + rmdir( $prefix . $hostname ); + } } } elsif ( defined( $CONFIG{ 'lvm' } ) ) @@ -523,14 +533,22 @@ sub deleteXenImage if ( -e "/dev/$CONFIG{'lvm'}/$hostname-swap" ) { - print "Removing swap volume\n"; - runCommand("lvremove /dev/$CONFIG{'lvm'}/$hostname-swap --force"); + if ($CONFIG{ 'dry-run' }) { + print "Would remove LVM swap volume /dev/$CONFIG{'lvm'}/$hostname-swap\n"; + } else { + print "Removing swap volume\n"; + runCommand("lvremove /dev/$CONFIG{'lvm'}/$hostname-swap --force"); + } } if ( -e "/dev/$CONFIG{'lvm'}/$hostname-disk" ) { - print "Removing LVM disk volume\n"; - runCommand("lvremove /dev/$CONFIG{'lvm'}/$hostname-disk --force"); + if ($CONFIG{ 'dry-run' }) { + print "Would remove LVM disk volume /dev/$CONFIG{'lvm'}/$hostname-disk\n"; + } else { + print "Removing LVM disk volume\n"; + runCommand("lvremove /dev/$CONFIG{'lvm'}/$hostname-disk --force"); + } } } @@ -546,16 +564,26 @@ sub deleteXenImage if ( -e "/dev/evms/$hostname-swap" ) { - print "Removing EVMS swap volume\n"; - runCommand("echo Delete : /dev/evms/$hostname-swap | evms"); - runCommand("echo Delete : $CONFIG{'evms'}/$hostname-swap | evms"); + if ($CONFIG{ 'dry-run' }) { + print "Would remove EVMS swap volume: /dev/evms/$hostname-swap\n"; + print "Would remove EVMS swap volume: $CONFIG{'evms'}/$hostname-swap\n"; + } else { + print "Removing EVMS swap volume\n"; + runCommand("echo Delete : /dev/evms/$hostname-swap | evms"); + runCommand("echo Delete : $CONFIG{'evms'}/$hostname-swap | evms"); + } } if ( -e "/dev/evms/$hostname-disk" ) { - print "Removing EVMS disk volume\n"; - runCommand("echo Delete : /dev/evms/$hostname-disk | evms"); - runCommand("echo Delete : $CONFIG{'evms'}/$hostname-disk | evms"); + if ($CONFIG{ 'dry-run' }) { + print "Would remove EVMS disk volume: /dev/evms/$hostname-swap\n"; + print "Would remove EVMS disk volume: $CONFIG{'evms'}/$hostname-swap\n"; + } else { + print "Removing EVMS disk volume\n"; + runCommand("echo Delete : /dev/evms/$hostname-disk | evms"); + runCommand("echo Delete : $CONFIG{'evms'}/$hostname-disk | evms"); + } } } diff --git a/bin/xen-update-image b/bin/xen-update-image index 6cbef5b..bd5f97c 100755 --- a/bin/xen-update-image +++ b/bin/xen-update-image @@ -242,19 +242,24 @@ sub updateXenImage elsif ( $CONFIG{ 'evms' } ) { - # The EVMS volume -- note, unlike LVM, you don't need the $CONFIG{'evms'} - # to see it and mount the volume. $CONFIG{'evms'} is only used for manipulating - # the underlying object. Still, I don't want to mess with the parse code and - # make it confusing - otherwise --evms takes an argument everywhere but here, - # which will confuse users. The better solution is to make it so that --evms can - # take a following container, but doesn't require it. For the moment, it is - # better to leave it as it is, take a container, and then ignore it. + # The EVMS volume -- note, unlike LVM, you don't need the + # $CONFIG{'evms'} to see it and mount the + # volume. $CONFIG{'evms'} is only used for manipulating the + # underlying object. Still, I don't want to mess with the + # parse code and make it confusing - otherwise --evms takes an + # argument everywhere but here, which will confuse users. The + # better solution is to make it so that --evms can take a + # following container, but doesn't require it. For the + # moment, it is better to leave it as it is, take a container, + # and then ignore it. - # The best way to do it is to just read it out of the configuration file, - # tell the user what you got and where you got it from, and not bother the user - # with picking --dir or --lvm or --evms at all, but infer it from the config - # file's disk = parameter. xen-delete-image might work the same way, but - # it could be *slightly* more dangerous in the context of deleting. + # The best way to do it is to just read it out of the + # configuration file, tell the user what you got and where you + # got it from, and not bother the user with picking --dir or + # --lvm or --evms at all, but infer it from the config file's + # disk = parameter. xen-delete-image might work the same way, + # but it could be *slightly* more dangerous in the context of + # deleting. $img = "/dev/evms/$name-disk"; # make sure it exists. diff --git a/bin/xt-install-image b/bin/xt-install-image index caed1be..537ac2b 100755 --- a/bin/xt-install-image +++ b/bin/xt-install-image @@ -29,8 +29,11 @@ xt-install-image - Install a fresh copy of GNU/Linux into a directory --mirror The mirror to use when installing with 'debootstrap'. Installation Options: - --install-method Specify the installation method to use. - --install-source Specify the installation source to use. + --install-method Specify the installation method to use. + --install-source Specify the installation source to use. + --debootstrap-cmd Specify which debootstrap command to + use. Defaults to debootstrap if both, debootstrap + and cdebootstrap are installed. All other options from xen-create-image will be passed as environmental variables. @@ -401,6 +404,7 @@ sub parseCommandLineArguments # Installation method "install-method=s", \$CONFIG{ 'install-method' }, "install-source=s", \$CONFIG{ 'install-source' }, + "debootstrap-cmd=s", \$CONFIG{ 'debootstrap-cmd' }, # Misc "arch=s", \$CONFIG{ 'arch' }, @@ -686,10 +690,17 @@ sub do_debootstrap # to use cdebootstrap. # my $cmd = $CONFIG{ 'debootstrap-cmd' }; - if ( !defined($cmd) ) + if ( !$cmd ) { - print "Falling back to default debootstrap command\n"; - $cmd = '/usr/sbin/debootstrap'; + if (-x '/usr/sbin/debootstrap') { + $cmd = '/usr/sbin/debootstrap'; + } elsif (-x '/usr/sbin/cdebootstrap') { + $cmd = '/usr/sbin/cdebootstrap'; + } else { + print STDERR "Found neither debootstrap nor cdebootstrap and no --debootstrap-cmd given\n"; + exit 1; + } + print "Using $cmd as debootstrap command\n"; } diff --git a/debian/changelog b/debian/changelog index 1fed193..5c32fa5 100644 --- a/debian/changelog +++ b/debian/changelog @@ -34,11 +34,25 @@ xen-tools (4.2~rc1-1) UNRELEASED; urgency=low - xt-install-image now exits with return value 127 instead of 0 to indicate errors on running the command given in --install-method. (Closes: #534290) + - The debootstrap command now also can be configured on the + commandline with --debootstrap-cmd in xen-create-image and + xt-install-image (Enhances fix for #436480 which added the + debootstrap-cmd config file option) + - Checks for debootstrap and cdebootstrap, uses debootstrap if both + are installed (Changed "Depends: debootstrap" to "Depends: + debootstrap | cdebootstrap" in debian/control) * Removal of /etc/bash_completion.d/xm from the package since bash-completion ships a more elaborate version of that file. (Closes: #566683, #550590, LP: #538917, #484098) * Downgrade reiserfsprogs and xfsprogs to Suggests. (Closes: #561618, LP: #80233) + * Added evms-cli to Suggests. It has been removed from Debian before + Lenny, but it is necessary for some optional functionality of + xen-tools. And since some Debian derived distributions (e.g. Ubuntu + LTS and grml) still support it, it's included for the sake of + completeness and correctness. + * Added cfengine2 to Suggests. It is helpful to have it installed when + using the cfengine2 role. * Bump Standards-Version to 3.8.4 (no changes necessary) * Bump Debhelper Compatibility to 7 - Replace "dh_clean -k" by "dh_prep" diff --git a/debian/control b/debian/control index 76bfcbb..86b7836 100644 --- a/debian/control +++ b/debian/control @@ -10,9 +10,9 @@ Vcs-Git: git://gitorious.org/xen-tools/xen-tools.git Package: xen-tools Architecture: all -Depends: debootstrap, perl-modules, libtext-template-perl, libconfig-inifiles-perl, libfile-slurp-perl, ${misc:Depends} +Depends: debootstrap | cdebootstrap, perl-modules, libtext-template-perl, libconfig-inifiles-perl, libfile-slurp-perl, ${misc:Depends} Recommends: xen-hypervisor-amd64 | xen-hypervisor-i386 | xen-hypervisor-i386-pae, rinse, xen-shell, libexpect-perl -Suggests: reiserfsprogs, xfsprogs, xen-utils +Suggests: reiserfsprogs, xfsprogs, xen-utils, cfengine2, evms-cli Description: Tools to manage Xen virtual servers This package contains tools to manage Debian based Xen virtual servers. . diff --git a/hooks/debian/20-setup-apt b/hooks/debian/20-setup-apt index e008d39..9c8189b 100755 --- a/hooks/debian/20-setup-apt +++ b/hooks/debian/20-setup-apt @@ -76,9 +76,11 @@ E_O_APT # -# If the host system has security support then enable that here too. +# If the host system has security support then enable that here, too, +# except if we're installing Debian Unstable. # -if ( test '!' -e /etc/apt/sources.list || \ +if ( test "${dist}" '!=' 'sid' && test "${dist}" '!=' 'unstable' && \ + test -e /etc/apt/sources.list && \ grep ^deb.*security -r /etc/apt/sources.list /etc/apt/sources.list.d >/dev/null 2>/dev/null ) ; then cat <> ${prefix}/etc/apt/sources.list diff --git a/roles/README b/roles/README new file mode 100644 index 0000000..850324d --- /dev/null +++ b/roles/README @@ -0,0 +1,6 @@ +These files are roles which you can use with xen-create-image(1) with +the --role= option. + +If you add new roles to the xen-tools distribution in this directory, +please document them in the POD code of bin/xen-create-image under +"ROLES". diff --git a/roles/cfengine b/roles/cfengine index 382ad4c..e071f53 100755 --- a/roles/cfengine +++ b/roles/cfengine @@ -26,14 +26,14 @@ logMessage Script $0 starting # -# Install ssh +# Install CFengine # installDebianPackage ${prefix} cfengine2 # -# Make sure sshd isn't running, this will cause our unmounting of the -# disk image to fail.. +# Make sure the CFengine server isn't running, this will cause our +# unmounting of the disk image to fail.. # chroot ${prefix} /etc/init.d/cfengine2 stop diff --git a/roles/editor b/roles/editor index 8c0b74c..67ada89 100755 --- a/roles/editor +++ b/roles/editor @@ -2,8 +2,8 @@ # # Role-script for the generalised editing of files for guests. # -# This script works via a skelington directory containing small -# .sed files which will contain edits to be applied to an arbitary +# This script works via a skeleton directory containing small +# .sed files which will contain edits to be applied to an arbitrary # tree of files upon the new domU. # # For example if we have the following sed file: diff --git a/roles/tmpfs b/roles/tmpfs index 70a09e8..fc51625 100755 --- a/roles/tmpfs +++ b/roles/tmpfs @@ -3,13 +3,9 @@ # This script is responsible for setting up /etc/fstab upon the # new instance. # -# This should be a simple job, but it is complicated by some of the -# differences between filesystems - some root filesystems will require -# the installation of new packages, and we have to handle that here. -# -# Steve +# Axel # -- -# http://www.steve.org.uk/ +# prefix=$1