Blog/ How to install and configure GlusterFS Server on Ubuntu

GlusterFS is a distributed File System. In this article, we'll show you how to create and configure GlusterFS Server on three nodes. You can create different types of GlusterFS volumes. We will only show you how to configure a replicated volume so that if you store a file on one machine, it will get replicated to all the nodes in the cluster. Suppose the hostnames of the three servers on which we want to install GlusterFS server are <hostname-1>, <hostname-2> and <hostname-3>. GlusterFS works better with hostnames instead of IP addresses so we'll use hostnames in the installation procedure. Here are the steps:

1) Install python-software-properties.

sudo apt-get install python-software-properties

2) Add the GlusterFS PPA.

sudo add-apt-repository ppa:gluster/glusterfs-3.5
sudo apt-get update

3) Install glusterfs-server.

sudo apt-get install glusterfs-server

4) Go to the first server and run the following command:

sudo gluster peer probe <hostname-2>
sudo gluster peer probe <hostname-3>

Replace <hostname-2> and <hostname-3> above by the host names of the remaining two servers. This command will form a cluster. To know the status of the cluster, execute the following:

sudo gluster peer status

The output should show that this server is connected to 2 peers.

5) Now execute the same command from the second or third server.

sudo gluster peer status

You will again see that there are 2 peers but instead of the hostname of the first server, it has registered its IP address. To fix this, run the following command from second or third server:

sudo gluster peer probe <hostname-1>

Now on executing sudo gluster peer status, you will see hostname of the first server and not its IP address.

6) Using fdisk, create a partition /dev/sdb on all the three servers. Format this partition using xfs filesystem.

sudo mkfs.xfs -i size=512 /dev/sdb1

You can also use ext4 filesystem if you prefer. To format the partition using ext4 filesystem, here is the command:

sudo mkfs.ext4 /dev/vdb1

7) Mount the above partition as a Gluster brick at /data on all the three servers. You can change the location of the mount if you want.

sudo mkdir /data
sudo mount /dev/vdb1 /data

8) The next step is to create one or more volumes in /data. The advantage of creating multiple volumes is that multiple applications can share the same three GlusterFS servers but use three different volumes. Each of these volumes can be mounted on the application server so that the applications can't access other applications' data. In this example, we will create two volumes: files1-volume and files2-volume at the location /data/files1 and /data/files2 respectively. You can use the volume names of your choice.

sudo mkdir /data/files1
sudo mkdir /data/files2

9) Create the two GlusterFS volumes.

sudo gluster volume create files1-volume replica 3 <hostname-1>:/data/files1 <hostname-2>:/data/files1 <hostname-3>:/data/files1
sudo gluster volume create files2-volume replica 3 <hostname-1>:/data/files2 <hostname-2>:/data/files2 <hostname-3>:/data/files2

Note the word replica. This indicates that we are creating a replicating volume. Also note the number 3 in the commands above. This denotes the number of GlusterFS servers in the cluster. In our case, we have 3. If you have more or less servers, you will need to change this number accordingly and provide information about the mounted GlusterFS brick on each of those servers after that.

You will see an output such as:

volume create: files1-volume: success: please start the volume to access data

10) Now is the time to start the volume. Make sure to do this step before you try to access the volume via a client library otherwise the client will hang. Execute the following commands:

sudo gluster volume start files1-volume
sudo gluster volume start files2-volume

11) To check the status of the volumes, run the following commands:

sudo gluster volume info files1-volume
sudo gluster volume info files2-volume

You should get output similar to:

Volume Name: files1-volume
Type: Replicate
Volume ID: 3c551edd-5944-4e0c-ba13-331ba627b5a6
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: <hostname-1>:/data/files1
Brick2: <hostname-2>:/data/files1
Brick3: <hostname-3>:/data/files1

12) To verify that replication is working, create files in /data/files1 and /data/files2 directories on one server and they should be accessible on the other servers.

In you want to connect to this GlusterFS server from an application server, see our article on how to connect from a client server to GlusterFS server.