Proxmox VE is a “bare metal” ISO Linux distribution that is a virtual machine platform. It is geared towards enterprise users and designed to be installed on enterprise grade hardware. The Proxmox VE distribution combines two virtual machine technologies; KVM and OpenVZ as well as a web interface to manage everything. Proxmox VE also integrates into its web interface a way to manage multiple computers as a cluster. For the rest of the article Proxmox VE shall be referred to as PVE. This article is written about PVE 1.1, the latest stable release.
Contents |
Installation
PVE comes as an ISO installer and is not designed to be added onto an existing OS. PVE is built on a stripped down version of Debian. PVE does not come with any graphical interface other than the web based one. It requires a “big iron” server. The server needs to support Intel or AMD virtualization processor extensions and hardware RAID is recommended. The reason for using such high end hardware is each server may have many other mission critical virtual computers running on it. If the server goes down you don't just lose that server you lose every virtual computer that is running on it. The install is really straight forward you only need to answer a few simple questions. Everything is installed and ready to use on a single machine in about 10-15 minutes. Installing a couple of additional hosts and setting them up to act as a PVE cluster can be done in less than an hour.
Interface
PVE's web interface is very elegant and exposes most underlying functionality of both KVM and OpenVZ. The interface is fairly intuitive but like all programs it has a learning curve. The interface gives basic information for your virtual machines, containers and for the host node. It provides stats on disk, memory and CPU usage. For the host node more advanced statistics are given that show a good overview of the host's health and potential bottlenecks.
The PVE web interface offers a Java-based VNC terminal so that you can log into your machines directly from the web interface. This method works well but has a few drawbacks. The most notable is there is no way to use the standalone VNC client to connect to a virtual machine. Also currently the open source Java releases do not support access to the host node's virtual consoles although that may change in the future.
The host node holds templates for the different types of OpenVZ containers that can be installed as well as ISO files for creation of KVM machines. PVE has a large selection of OpenVZ appliances that can be downloaded to do specific functions. Most of them are free however some like the Proxmox Mail Gateway are commercial offerings. Free offerings include Zenoss, Drupal, Joomla!, SugarCRM and Wordpress templates. You can also upload your own customized templates or ISO files. On PVE 1.1 (the latest version at time of writing) there is no way to set a description of the custom uploaded template so care must be used in creating the file name.
Creating new virtual machines is a snap and happens in three clicks. You choose the type of virtual machine you want; either OpenVZ or KVM. Then you choose the template or ISO file to use. Then you can select how large of a disk the virtual machine is to have and set the amount of RAM it is allowed to use. For OpenVZ you can also setup the root password and network parameters. While the web interface does a good job of exposing the basics for virtual machine creation it does not allow you to do some of the more advanced things that are possible especially with OpenVZ. Fortunately if you need those settings you can locally login (or ssh) to your PVE host node and run the underlying command line tools.
The web interface also provides a nice easy way to setup basic backups. There are three types of backups that can be made. They are snapshot which incurs no down time, suspend which incurs a short amount of down time and stop which incurs the most amount of down time. Snapshot is the default and there is not a compelling reason to use any of the other methods unless a machine fails to backup. The snapshot method works using LVM to copy currently running files that are in use by the OS at the time of backup. The suspend backup method works by backing up the files that it can on a running machine then it suspends the machine for a short time to get the files that were in use at the time of the backup. The stop method stops the machine completely and backs up all the files. The web interface allows you to schedule backups to happen at regular intervals. To make a non-scheduled backup you can use vzdump from the command line. The interface should make it easy to do a non-scheduled backup, hopefully this will be included in a future release. Backup restores are also command line only at this time, this needs to be changed as well. The backup leaves some things to be desired, I will discuss what I think needs to be added in the section entitled “Enhancements and Issues” below.
Proxmox VE Cluster
PVE can be run as a cluster. Clustering brings several advantages. The biggest is the ability to migrate your virtual machines between cluster nodes, live without shared storage. Another benefit is the ability to manage multiple servers from one unified web interface. You are able to create virtual machines from the cluster master node on any of the cluster slave nodes. If your master node fails the slave nodes are still able to function although the cluster is down so migrations will no longer be available, however the virtual machines will still work as normal.
Cluster setup is simple, it must be done from the command line though. Hopefully in future releases cluster setup will be able to be completed from the web interface. Overall clustering is a powerful tool that makes PVE greater than the sum of it parts.
Enhancements and Issues
Proxmox VE is a great program “as is” right now, however there are some enhancements that I would like to see be made, specifically:.
- Changes to the way backups are made tops my list for enhancements. Backups are made with the program vzdump. Vzdump does basic backups very well however some enhancements are needed to really push it to the next level. One of the things that I would like to see changed is the way the file is saved. Currently the file is saved with a fairly generic file name (vzdump$VEID.tgz) the change would be to just add the date to file so you can know when the backup took place. I currently do this with a helper script but it would be nice to have it all integrated. Also a way to clean up outdated backup files would be nice. For example vzdump --clean $numdays would delete all backups that are older than a certain number of days. Again I do this with a helper script but it would be nice to have the entire thing integrated into the web interface.
- Another issue with the backup is that there is no simple way to recover a single file easily. You can recover entire machines in seconds but single files are much much harder to recover. Currently the steps to recover a single file or set of files is to use vzdump to restore the machine. When the machine is brought on-line there is a IP address conflict causing a service outage until you can get the IP changed. You can then download the file(s) that you need and complete the recovery. In the future it would be nice to be able to mount the virtual hard drive in a read only state so you can then recover the files without having to bring up the entire machine and without risk of corrupting your backup since it mounted read only. Using LVM would be an easy way. Currently I use a separate system (backuppc) to be able to easily restore single files. This is not ideal because it doubles the storage requirements and adds to network complexity.
- The last issue with backups is in a PVE clustered environment when a machine is migrated to a different host its backup schedule does not follow it, this is a small issue and the only work around is to make sure that you check your backup schedule when a host is migrated. Hopefully a way will be devised that the backup schedule can follow the virtual machine and maintain the backup policy as machines are migrated.
- The next issue that affects only cluster users is the way networking is setup. Currently all cluster hosts must have the exact same network setup as the master host. This can be a major issue especially if you are using an advanced network setup. If all of the cluster nodes do not have the same exact same number of NIC's you are forced to use the lowest common denominator of the box with the least amount of network interface cards. The reason for this is when a new KVM machine or OpenVZ container using bridged Ethernet is created you select a NIC to attach to. If that NIC is not available on the host things will not work as expected. I have not figured out a reasonable work around. An enhancement would be to tailor the choice of NICs to what is available on physical hardware. There is still an issue as to what happens when the virtual machine is migrated to a host that does not contain the same number of NICs. How do other vendors like VMware or Xen handle this? Let me know in the comments.
The Future and Support
PVE is a great and usable virtualization environment as is now. Verison 2.0 will have some neat things built in like firewall, advanced user management, better resource monitoring and DRBD support. It will also support more storage options like iSCSI.
The PVE community is not huge but it is growing. There is a mailing list and the developers are active on it. Also the wiki contains quite a bit of documentation as well as video howtos. Also the KVM community and OpenVZ community can help with many issues. The company behind Proxmox VE also offers paid support on a per ticket basis.
Action Item: Practitioners should consider PVE as a viable alternative to other products like VMware or Xen. I have been using PVE since the initial 0.9 public release. I currently run about 30 virtual machines on a Proxmox VE cluster including several mission critical servers and have had no issues with the core functionality in the 8 months that I have been using it. It is missing some features that the commercial offerings provide but no other open source application offers the functionality provided by Proxmox VE.
Footnotes: ]
