Simple failover cluster on Ubuntu using CARP

Posted on April 4, 2012 by laurent.bel

I recently had the little challenge to build up a failover cluster on Ubuntu for SMTP services (postfix in my case).

Initially I had one single SMTP server running postfix. When the server is down, well… the service is down as well. So I decided to build up a second one, that would take over in case the first one crashes. What I want is a basic failover cluster (active/passive).

I wanted to keep it very simple and efficient, without going through the complex configuration of heartbeat for example.

I therefore decided to use Ucarp, a implementation of carp for Ubuntu.

Here is my architecture :

Server #1 : My first server where I configured postfix (IP : 172.17.0.75)
Server #2 : My second server where I configured postfix exactly like on Server #1 (IP : 172.17.0.76)
172.17.0.74 : The virtual IP address, created using Ucarp.

Ucarp is very simple : it works that way. If server #1 is up, then the virtual IP 172.17.0.74 is assigned to server #1. If server #1 is down, then the virtual IP 172.17.0.74 is assigned to server #2 (assuming server #2 is up). This way you have a simple failover cluster…

Here is how to set it up :

Step 1 : On Server #1 (172.17.0.75)

> sudo apt-get install ucarp

Edit the file /etc/network/interfaces

> sudo nano /etc/network/interfaces

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth1
iface eth1 inet static
        ################################
        # standard network configuration
        ################################
        address 172.17.0.75
        netmask 255.255.240.0
        gateway 172.17.1.254
        network 172.17.0.0
        broadcast 172.17.0.255

        ################################
        # ucarp configuration
        ################################
        # vid : The ID of the virtual server [1-255]
        ucarp-vid 1
        # vip : The virtual address
        ucarp-vip 172.17.0.74
        # password : A password used to encrypt Carp communications
        ucarp-password secret
        # advskew : Advertisement skew [1-255]
        ucarp-advskew 1
        # advbase : Interval in seconds that advertisements will occur
        ucarp-advbase 1
        # master : determine if this server is the master
        ucarp-master yes

# The carp network interface, on top of eth1
iface eth1:ucarp inet static
        address 172.17.0.74
        netmask 255.255.240.0

Restart the network interfaces, so that the ucarp config is taken into consideration

> sudo /etc/init.d/networking restart

Step 2 : On Server #2 (172.17.0.76)

> sudo apt-get install ucarp

Edit the file /etc/network/interfaces

> sudo nano /etc/network/interfaces

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth1
iface eth1 inet static
        ################################
        # standard network configuration
        ################################
        address 172.17.0.76
        netmask 255.255.240.0
        gateway 172.17.1.254
        network 172.17.0.0
        broadcast 172.17.0.255

        ################################
        # ucarp configuration
        ################################
        # vid : The ID of the virtual server [1-255]
        ucarp-vid 1
        # vip : The virtual address
        ucarp-vip 172.17.0.74
        # password : A password used to encrypt Carp communications
        ucarp-password secret
        # advskew : Advertisement skew [1-255]
        ucarp-advskew 100
        # advbase : Interval in seconds that advertisements will occur
        ucarp-advbase 1
        # master : determine if this server is the master
        ucarp-master no

# The carp network interface, on top of eth1
iface eth1:ucarp inet static
        address 172.17.0.74
        netmask 255.255.240.0

Restart the network interfaces, so that the ucarp config is taken into consideration

> sudo /etc/init.d/networking restart

Step 3 : Check that it works fine

While the two servers are running, check the interface on server #1 :

> sudo ifconfig

eth1      Link encap:Ethernet  HWaddr 00:0c:29:5b:d8:03
          inet addr:172.17.0.75  Bcast:172.17.0.255  Mask:255.255.240.0
          inet6 addr: fe80::20c:29ff:fe5b:d803/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:66814 errors:0 dropped:0 overruns:0 frame:0
          TX packets:21871 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:11618538 (11.6 MB)  TX bytes:10521832 (10.5 MB)

eth1:ucarp Link encap:Ethernet  HWaddr 00:0c:29:5b:d8:03
          inet addr:172.17.0.74  Bcast:172.17.15.255  Mask:255.255.240.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

You will see the section highlighted in red, that shows that the carp IP address 172.17.0.74 is active on the interface.

If you do the same, but this time on server #2, you’ll see that the carp IP is not active :

> sudo ifconfig

eth1      Link encap:Ethernet  HWaddr 00:0c:29:92:ba:ac
          inet addr:172.17.0.76  Bcast:172.17.0.255  Mask:255.255.240.0
          inet6 addr: fe80::20c:29ff:fe92:baac/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:67433 errors:0 dropped:0 overruns:0 frame:0
          TX packets:340 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:4644650 (4.6 MB)  TX bytes:73256 (73.2 KB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

If you then shutdown server #1, you’ll be able to see that the Carp IP address is transferred to server #2 :

> sudo ifconfig

eth1      Link encap:Ethernet  HWaddr 00:0c:29:92:ba:ac
          inet addr:172.17.0.76  Bcast:172.17.0.255  Mask:255.255.240.0
          inet6 addr: fe80::20c:29ff:fe92:baac/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:68775 errors:0 dropped:0 overruns:0 frame:0
          TX packets:402 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:4739385 (4.7 MB)  TX bytes:82180 (82.1 KB)

eth1:ucarp Link encap:Ethernet  HWaddr 00:0c:29:92:ba:ac
          inet addr:172.17.0.74  Bcast:172.17.15.255  Mask:255.255.240.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

If you turn back on server #1, you’ll see that server #1 will get the carp back. Server #1 is the master, and if it is up, it will get the Carp back.

Step 4 : Use it !!!

Now that it works fine, you can start using it.
Just make sure you use the CARP IP address 172.17.0.74 (instead of 75 or 76).

Conclusion :

This is a simple, a very fast way of setting up a failover cluster.The big advantages is that it is simple to setup and manage.

The disadvantages is that it only provide IP failover : the configuration of the services running on top of the server (postfix, apache, mysql, …) are not transferred, nor synchronized.

Side notes :

I’m running Ubuntu Server 10.04 LTS x64 in a virtualized environment over ESXi 5.
So if you wonder if it works as well on virtual machines, well the answer is yes !

Credits :

http://valeriytroshin.blogspot.fr/2011/08/carp-failover-redundancy-in-ubuntu-1104.html : Great and nearly only source of inspiration when I setup my servers !

VMWare ESXi (Free version) Hot Backup in powershell

Posted on March 30, 2012 by laurent.bel

I’m running a farm of around 10 VMWare ESXi servers (the free version of VMWare) and want to perform some backups of my virtual machines without shutting down the machines. In other words I want to backups my Virtual Machines while they are running. This might sound very easy to achieve if you are running the full (paid) version of VMWare vSphere, but it is slightly more tricky with the ESXi version since this feature is not included.

Let’s dive into the procedure to perform your backups using powershell. The overall procedure will take you around 1 hour :

Step 1 : Prepare your working environment

Grab a windows machine (Win 7 or Win Server 2008 R2 if possible)
Install Powershell 2.0 on top of it (if not already installed)
Open powershell and type “Set-ExecutionPolicy RemoteSigned”. Then close powershell.
Download and Install the latest VMware PowerShell CLIs. This will be used to manipulate VMWare ESXi servers through powershell. This will install a snap-in for powershell.
Download plink. This will be used to access the ESXi server through remote SSH.
Download and install WinRAR. We are going to use it to zip our virtual machines.

Step 2 : Enable SSH on your ESXi servers

Certain operations must be performed through an SSH connection to the ESXi servers. We need to enable SSH. Here is how to do so :

Open your vSphere Client and go into the configuration tab and select “Security Profile”

Click now on “properties”

A window will popup. Select “Remote Tech Support (SSH)” and click Options…

Click “Start” and then select the option “Start automatically”. Click OK to validate.

If you go back to your vSphere Top level summary page (home page), you’ll see a message warning you that SSH has been enabled.

Step 3 : Customize your Powershell script to execute backups

Copy this powershell script on your machine into a file called for example backupESX.ps1
Make sure plink.exe (downloaded before) is sitting next to your bakcupESX.ps1 file. By sitting next, I mean in the same folder ! (important)

#####################################################################################################################
#
# Title:         BackupESX.ps1
#
# Description:     Performs a hot (without machine interrupution) backup of a VMWare ESX virtual machine.
#        The machine is copied locally, then zipped and archived on a network share (typically
#        a NAS for storage). This script is intended to work for ESXi (reason why we use SSH)
#        and should also work for ESX. Tested and developped for ESX 4.1.0 u1.
#
# Imp. Note:    The following files might fail during backup, it is normal and they are not needed:
#        *.vswp, *.vmsn, *-delta.*, *.log
#        You might want to manually delete those file when restoring a Virtual machine
#
# Type:        PowerShell script
#
# Author:     Laurent Bel
#
# Version:     V1.0 - March 2011 - Initial version
#
#####################################################################################################################

# We add the snapin for VmWare
add-pssnapin VMware.VimAutomation.Core

# Pre Check
if (! $args.Count.Equals(4))
{
    Write-Output "Missing Parameters"
    Write-Output "Syntax: script-BackupESX.ps1 <ESXServerIP> <login> <pwd> <machineNameToBackup>"
    Write-Output "Sample: script-BackupESX.ps1 172.17.5.3 root MySecretPassword MyVirtualMachine"
    Write-Output ""
    Write-Output "Please press enter..."
    Read-Host
    Exit
}

# Variables global and parameters
$server = $args[0]
$login = $args[1]
$password = $args[2]
$machine = $args[3]
$date = Get-Date -Format yy-MM-dd--HH\hmm
$outputPath = "C:\Temp"
$rarExe = "C:\Program Files\WinRAR\Rar.exe"
$rarFile = "$outputPath\$machine-$date.rar"
$nasFolder = "\\myNASIP\subfolder"
$nasLogin = "yourNASLogin"
$nasPwd = "YourNASPassword"

# Ouput the summary of operation
echo "########## Operation summary ##########"
echo "Operation : Hot Backup of a ESXi server"
echo Server: $server
echo Login: $login
echo Password: $password
echo Machine: $machine
echo $date: date
echo "#######################################"

# We connect to ESX
$vh = Connect-VIServer -Server $server -Port 443 -User $login -Password $password

# We get the machine we are interested in
$vm = Get-VM -Server $vh | Where-Object {$_.Name -eq $machine}

# We get the ID of the VM and strip the beginning to get only the number
$vmid = $vm.Id -replace ("VirtualMachine-","")

# We get the datastore of the machine
$ds = Get-Datastore -Server $vh -VM $vm

# We snapshot the VM with Plink to have direct access to server
Start-Process .\plink.exe -ArgumentList "-ssh -P 22 -l $login -pw $password $server vim-cmd vmsvc/snapshot.create $vmid AutomaticBackupSnapshot$date 1 1" -Wait

# We add a PS Drive for the datastore to easily manipulate files
Remove-PSDrive -Name DS
$psd = New-PSDrive -Name DS -PSProvider ViMdatastore -Root \ -location $ds

# We copy the files from VH to local storage and close the PS drive once finished
mkdir $outputPath\$machine
Copy-DatastoreItem DS:\$machine\* -Destination $outputPath\$machine\
Remove-PSDrive -Name DS

# We remove the snapshot with Plink to have direct access to server
Start-Process .\plink.exe -ArgumentList "-ssh -P 22 -l $login -pw $password $server vim-cmd vmsvc/snapshot.remove $vmid" -Wait

# We disconnect from the server
Disconnect-VIServer -Server $vh -Confirm:$false

# We ZIP/RAR the extracted VM to reduce its size and delete the folder after completion
Start-Process $rarExe -ArgumentList "a $rarFile $outputPath\$machine\*.*" -Wait
Remove-Item $outputPath\$machine\ -Recurse -Confirm:$false

# We transfer the zipped/rared virtual machine to a NAS for archiving
net use A: /DELETE
net use A: $nasFolder /USER:$nasLogin $nasPwd
mkdir $nasFolder\$machine\VMBackup -Confirm:$false
Move-Item -Path $rarFile -Destination $nasFolder\$machine\VMBackup\$machine-$date.rar -Confirm:$false

# End of script
echo "########## Operation completed ##########"
echo "This is the end..."
echo "Machine: $machine has been archived"
echo "#########################################"

Adjust the section “variables global and parameter” so that it matches your needs.

Step 4 : Run the script backupESX.ps1

I strongly recommend you use a powershell debugging tool to start with in order to run the script, so that you can see what happens in details and fix errors you might face.

Conclusion :

I haven’t spent a lot of time explaining what I was doing in the script, and how it works, but the script is quite well documented and should be easy (at least not too difficult) to understand. Do not hesitate to leave comments on this article if you have any question. I’ll try to update the articles with feedback I’ll get.

Dynamics CRM 2011 – Error only secure content is displayed

Posted on December 16, 2011 by laurent.bel

Today I’m facing the following issue when I access my CRM platform :

Internet explorer complains about the fact that only secure content is displayed. Which means that some http is going through while my CRM platform is configured to use https. You’ll notive as well that the get started section is not displayed correctly.

You get exactly the same thing in the outlook plugin with a similar message that asks you if you want to display only the content that was delivered securely over https :

If have read a few articles that were talking about configuring IE to ask to mix secured and unsecured content. I did not like it, and wanted to understand why this content was not delivered through a secured channel.

I figured out that is comes from a configuration in the Dynamics CRM database that is not set correctly. After you have adjusted it, it will work smoothly. Here is the procedure to fix it :

Step 1: Open a SQL Server Management Studio on the CRM database server and open the MSCRM_CONFIG database. And perform the following query :

SELECT     HelpServerUrl
FROM         ConfigSettings

You’ll get something like that :

As you can see, the HelpServerUrl is indicating HTTP (and in my case even a wrong url because it points to a specific web front end instead of the load balancer url…).

Step 2 : Edit the value the you found in the HelpServerUrl to what you need. Especially HTTPS instead of HTTP.

Step 3 : Reboot your farm. CRM dynamics might cache those kind of values… so a reboot might be necessary (it was not the case for me though).

Done ! You’ll see a full page nicely displayed without any error or warning

Convertir un objet Image (ou Bitmap) en objet IPictureDisp

Posted on June 1, 2009 by laurent.bel

Ne vous méprenez pas, il est parfois très utile de pouvoir faire simplement ce type de conversion. J’ai rencontré ce problème lors de la réalisation d’un Add-in pour Outlook 2007 avec Visual Studio 2008 alors que je souhaitais ajouter une image à un CommandBarButton (bouton dans Outlook 2007).

L’opération semble simple en théorie. Dans la pratique vous serez confronté à une difficulté de taille : Être capable de créer un objet IPictureDisp contenant votre image.

Voici comment procéder pour ajouter une image (icône) à votre CommandBarButton.

ETAPE 1 : Ajout d’une image dans votre projet

Ajouter votre image à vos ressources dans Visual Studio 2008. Il suffit pour cela de cliquer sur le répertoire Properties dans votre projet Visual Studio, puis de cliquer sur Ressources et d’ajouter votre image.

ETAPE 2 : Ajouter la classe permettant de convertir vos images au format IPictureDisp

Ajouter une nouvelle classe à votre projet que vous appellerez AxHostConverter. Pour réaliser cela, il suffit faire clique droit dans VS 2008, puis Add New Item, Class…

Le code de cette classe sera le suivant :

class AxHostConverter : AxHost
{
    private AxHostConverter() : base(“”) { }

    static public stdole.IPictureDisp ImageToPictureDisp(Image image)
    {
        return (stdole.IPictureDisp)GetIPictureDispFromPicture(image);
    }

    static public Image PictureDispToImage(stdole.IPictureDisp pictureDisp)
    {
        return GetPictureFromIPicture(pictureDisp);
    }
}

Explications : Cette classe permettra de réaliser la conversion d’un objet Image (ou Bitmap puisque ce dernier hérite du premier !). L’explication technique (en version courte est simple) tient dans le fait que la classe AxHost permet de réaliser cette conversion via deux méthodes statiques. le problème est que ces méthodes ne sont pas accessible (protected) et qu’il est nécessaire d’encapsuler ces méthodes dans une classe de notre fabrication que nous exposerons publiquement afin de pouvoir les utiliser librement.

ETAPE 3 : Ajout d’une image à votre bouton

Via le code, juste après avoir créé votre bouton, vous pouvez faire un appel du type :

myCommandBarButton.Picture = AxHostConverter.ImageToPictureDisp((Image)Properties.Resources.monImage);
 

Note : “myCommandBarButton” devra être remplacé par le nom de votre bouton. “monImage” sera le nom de votre image dans les Ressources de votre projet Visual Studio. vous noterez également la conversion de la ressource Bitmap en objet Image afin de respecter la type d’entrée. Bitmap héritant de Image, cela ne pose aucun problème.

Conclusion :

Tout ça pour ça me direz vous. Et bien oui, mais parfois, une petite lutte acharnée permet de faire de belles choses.

Dans un prochain billet, je tâcherais d’expliquer comment gérer la transperence sur votre nouveau bouton ainsi créé.

Qu’allez-vous trouver sur ce blog

Posted on November 21, 2008 by laurent.bel

Un ami m’a fait récemment la réflexion que mon site comportait beaucoup de choses diverses sans qu’il y ait pour autant une cohérence d’ensemble entre les articles.

C’est vrai… et faux en même temps… La cohérence globale entre tous ces articles est simple : il s’agit de mon quotidien qui passe de l’un à l’autre en permanence.

Avant de commencer ce blog, je me suis naturellement posé la question de son contenu. J’ai hésité pendant un temps à dédier ce blog à SharePoint uniquement ! Je réalise après avoir écrit ces quelques articles qu’il est bien plus intéressant de brosser un panel plus large de technologies au gré du déroulement de mes journées !

Ce blog sera donc animé par la diversité !

1er article

Posted on November 17, 2008 by laurent.bel

Ok, il est vrai que ce premier article n’a pas un grand intérêt… mais la suite, espérons le, va devenir plus intéressante.

L’objectif initial est de réaliser un blog sur les technologies qui gravitent autour mon quotidien (boulot et perso) et de centraliser un certain nombre de tips, bonnes pratiques que je rencontre au fur et à mesure de la journée…

Laurent Bel's blogging

Mainly Microsoft related articles : SharePoint, SQL Server, Dynamics CRM, .NET, …

Category Archives: Uncategorized

Simple failover cluster on Ubuntu using CARP

Step 1 : On Server #1 (172.17.0.75)

Step 2 : On Server #2 (172.17.0.76)

Step 3 : Check that it works fine

Step 4 : Use it !!!

Conclusion :

Side notes :

Credits :

VMWare ESXi (Free version) Hot Backup in powershell

Step 1 : Prepare your working environment

Step 2 : Enable SSH on your ESXi servers

Step 3 : Customize your Powershell script to execute backups

Step 4 : Run the script backupESX.ps1

Conclusion :

Dynamics CRM 2011 – Error only secure content is displayed

Convertir un objet Image (ou Bitmap) en objet IPictureDisp

Qu’allez-vous trouver sur ce blog

1er article