{"id":6121,"date":"2021-02-15T13:10:17","date_gmt":"2021-02-15T07:40:17","guid":{"rendered":"https:\/\/www.skynats.com\/?p=6121"},"modified":"2021-10-28T10:48:37","modified_gmt":"2021-10-28T05:18:37","slug":"install-glusterfs-server-on-centos-7","status":"publish","type":"post","link":"https:\/\/www.skynats.com\/blog\/install-glusterfs-server-on-centos-7\/","title":{"rendered":"How To Install GlusterFS Server On CentOS 7"},"content":{"rendered":"\n

             <\/strong><\/p>\n\n\n\n

The glusterFS is an open-source, simple, elastic, and scalable file storage system that is suitable for high data such as media streaming, cloud storage, and the content delivery network (CDN).<\/p>\n\n\n\n

We often get a request from our client to install the GlusterFS server on centOS 7 as it is part of our server management plan<\/a>. You can also take assistance from our technical team support 24\/7 for further installation and configuration.<\/p>\n\n\n\n

Set up GlusterFS storage on RHEL 7.x and CentOS 7.x<\/strong><\/p>\n\n\n\n

Here we are considering 4 RHEL 7\/centOS 7 servers with minimal installation in order to assume the additional disk which is attached to the server to set up the glustesfs.<\/p>\n\n\n\n

Add and follow the below lines in the \/ etc\/ hosts file as you considered to have DNS server.<\/p>\n\n\n\n

192.xxx.xx.10 server1.com server1\n192.xxx.xx.20 server2.com server2\n192.xxx.xx.30 server3.com server3\n192.xxx.xx.40 server4.com server4\n<\/code><\/pre>\n\n\n\n
Run the following below commands to set up the gluster repo and EPEL repo.<\/p>\n\n\n\n
yum install wget\nyum install centos-release-gluster -y\nyum install epel-release -y\nyum install glusterfs-server -y\n<\/code><\/pre>\n\n\n\n
Once you have done with the setup, start and enable the GlusterFS services by using the following commands.<\/p>\n\n\n\n
systemctl start glusterd\nsystemctl enable glusterd\n<\/code><\/pre>\n\n\n\n
Later, allow the port on the firewall so that server can communicate and form the storage cluster. Run the commands.<\/p>\n\n\n\n
firewall-cmd --zone=public --add-port=24007-24008\/tcp --permanent\nfirewall-cmd --zone=public --add-port=24009\/tcp --permanent\nfirewall-cmd --zone=public --add-service=nfs --add-service=samba --add-service=samba-client --permanent\nfirewall-cmd --zone=public --add-port=111\/tcp --add-port=139\/tcp --add-port=445\/tcp --add-port=965\/tcp --add-port=2049\/tcp --add-port=38465-38469\/tcp --add-port=631\/tcp --add-port=111\/udp --add-port=963\/udp --add-port=49152-49251\/tcp --permanent\nfirewall-cmd --reload\n<\/code><\/pre>\n\n\n\n
Distribute the volume setup:<\/strong><\/p>\n\n\n\n
Let form the trusted storage pool that consists of server 1 and server 2. Then will create the brick on that and after that will create the distributed volume. Assume the raw space of 16 GB to allocate both servers.<\/p>\n\n\n\n
Run the commands from the server 12 consoles to form a trusted storage pool with server2.\u00a0<\/p>\n\n\n\n
gluster peer probe server2.com<\/code><\/pre>\n\n\n\n
Run commands and check the peer status in server 1. <\/p>\n\n\n\n
gluster peer status\n\n<\/code><\/pre>\n\n\n\n
Creating brick on server 1.<\/strong><\/p>\n\n\n\n
To set up the brick we need to create the logical volumes on the raw disk.<\/p>\n\n\n\n
Run the following command on server 1.<\/p>\n\n\n\n
pvcreate \/dev\/sdb \/dev\/vg_bricks\/dist_brick1 \/bricks\/dist_brick1 xfs rw,noatime,inode64,nouuid 1 2\nvgcreate vg_bricks \/dev\/sdb\nlvcreate -L 14G -T vg_bricks\/brickpool1\n<\/code><\/pre>\n\n\n\n
The above command is the brickpool1 in the name of a thin pool.<\/p>\n\n\n\n
Then create the logical volume of 3 GB.<\/p>\n\n\n\n
lvcreate -V 3G -T vg_bricks\/brickpool1 -n dist_brick1<\/code><\/pre>\n\n\n\n
Format the logical volume by using the xfs file system.<\/p>\n\n\n\n
mkfs.xfs -i size=512 \/dev\/vg_bricks\/dist_br<\/code><\/pre>\n\n\n\n
 Then mount the brick using the mount command.<\/p>\n\n\n\n
mount \/dev\/vg_bricks\/dist_brick1 \/bricks\/dist_brick1\/<\/code><\/pre>\n\n\n\n
If you want to mount it permanently and add the following line in \/etc\/ fsatb.<\/p>\n\n\n\n
\/dev\/vg_bricks\/dist_brick1 \/bricks\/dist_brick1 xfs rw,noatime,inode64,nouuid 1 2<\/code><\/pre>\n\n\n\n
Then create the directory with a brick under the mount point.<\/p>\n\n\n\n
mkdir \/bricks\/dist_brick1\/brick\n\n<\/code><\/pre>\n\n\n\n
Perform a similar set of commands on server 2.<\/p>\n\n\n\n
[root@server2 ~]# pvcreate \/dev\/sdb ; vgcreate vg_bricks \/dev\/sdb\n[root@server2 ~]# lvcreate -L 14G -T vg_bricks\/brickpool2\n[root@server2 ~]# lvcreate -V 3G -T vg_bricks\/brickpool2 -n dist_brick2\n[root@server2 ~]# mkfs.xfs -i size=512 \/dev\/vg_bricks\/dist_brick2\n[root@server2 ~]# mkdir -p \/bricks\/dist_brick2\n[root@server2 ~]# mount \/dev\/vg_bricks\/dist_brick2 \/bricks\/dist_brick2\/\n[root@server2 ~]# mkdir \/bricks\/dist_brick2\/brick\n<\/code><\/pre>\n\n\n\n
Follow the below gluster commands and create the distributed volume.<\/p>\n\n\n\n
[root@server1 ~]# gluster volume create distvol server1.example.com:\/bricks\/dist_brick1\/brick server2.example.com:\/bricks\/dist_brick2\/brick<\/p>\n\n\n\n
[root@server1 ~]# gluster volume start distvol<\/p>\n\n\n\n
volume start: distvol: success<\/p>\n\n\n\n
[root@server1 ~]#<\/p>\n\n\n\n
Verify the volume status. Run the commands.<\/p>\n\n\n\n
[root@server1 ~]# gluster volume info distvol<\/p>\n\n\n\n
MOUNT DISTRIBUTE VOLUMES ON CLIENT:<\/strong><\/p>\n\n\n\n
First, mount the volume by using the GlusterFS and then ensure the glusterfs-fuse package is installed on the client.<\/p>\n\n\n\n
Then log into the client console and run the following commands to install the glusterfs-fuse.<\/p>\n\n\n\n
.<\/p>\n\n\n\n
yum install glusterfs-fuse -y<\/code><\/pre>\n\n\n\n
Later, try to create the mount for distributed volume<\/p>\n\n\n\n
mkdir \/mnt\/distvol<\/code><\/pre>\n\n\n\n
Mount the ‘distvol’ by using the following commands.<\/p>\n\n\n\n
mount -t glusterfs -o acl server1.com:\/distvol \/mnt\/distvol\/<\/code><\/pre>\n\n\n\n
For the permanent mount, we need to add the below entry in the \/etc\/fstab file.<\/p>\n\n\n\n
server1.com:\/distvol \/mnt\/distvol glusterfs _netdev 0 0<\/code><\/pre>\n\n\n\n
Then run the DF command in order to verify the mount status volume.<\/p>\n\n\n\n
df -Th<\/code><\/pre>\n\n\n\n
Replicating the volume setup.<\/strong><\/p>\n\n\n\n
For the replication, use <\/strong>server 3 and server 4. and then assume the additional disk (\/dev\/sdb) for glusterfs is already assigned to the server.<\/strong><\/p>\n\n\n\n
Follow the steps to add server 3 and server 4 to the trusted storage pool.<\/p>\n\n\n\n
gluster peer probe server3.com<\/code><\/pre>\n\n\n\n
gluster peer probe server4.com<\/code><\/pre>\n\n\n\n
Run the commands to create and mount the brick on server 3.<\/p>\n\n\n\n
pvcreate \/dev\/sdb ; vgcreate vg_bricks \/dev\/sdb<\/code><\/pre>\n\n\n\n
lvcreate -L 14G -T vg_bricks\/brickpool4<\/code><\/pre>\n\n\n\n
lvcreate -V 3G -T vg_bricks\/brickpool4 -n shadow_brick2<\/code><\/pre>\n\n\n\n
mkfs.xfs -i size=512 \/dev\/vg_bricks\/shadow_brick2<\/code><\/pre>\n\n\n\n
mkdir -p \/bricks\/shadow_brick2<\/code><\/pre>\n\n\n\n
mount \/dev\/vg_bricks\/shadow_brick2 \/bricks\/shadow_brick2\/<\/code><\/pre>\n\n\n\n
mkdir \/bricks\/shadow_brick2\/brick<\/code><\/pre>\n\n\n\n
For the permanent mounting, run the below commands.<\/p>\n\n\n\n
\/dev\/vg_bricks\/shadow_brick1 \/bricks\/shadow_brick1\/ xfs rw,noatime,inode64,nouuid 1 2<\/code><\/pre>\n\n\n\n
Run the commands to create and mount the brick on server 3.<\/p>\n\n\n\n
pvcreate \/dev\/sdb ; vgcreate vg_bricks \/dev\/sdb<\/code><\/pre>\n\n\n\n
lvcreate -L 14G -T vg_bricks\/brickpool4<\/code><\/pre>\n\n\n\n
lvcreate -V 3G -T vg_bricks\/brickpool4 -n shadow_brick2<\/code><\/pre>\n\n\n\n
mkfs.xfs -i size=512 \/dev\/vg_bricks\/shadow_brick2<\/code><\/pre>\n\n\n\n
mkdir -p \/bricks\/shadow_brick2<\/code><\/pre>\n\n\n\n
mount \/dev\/vg_bricks\/shadow_brick2 \/bricks\/shadow_brick2\/<\/code><\/pre>\n\n\n\n
mkdir \/bricks\/shadow_brick2\/brick<\/code><\/pre>\n\n\n\n
For the permanent mounting, run the below commands.<\/p>\n\n\n\n
\/dev\/vg_bricks\/shadow_brick1 \/bricks\/shadow_brick1\/ xfs rw,noatime,inode64,nouuid 1 2<\/code><\/pre>\n\n\n\n
Run the commands to create and mount the brick on server 3.<\/p>\n\n\n\n
pvcreate \/dev\/sdb ; vgcreate vg_bricks \/dev\/sdb<\/code><\/pre>\n\n\n\n
lvcreate -L 14G -T vg_bricks\/brickpool4<\/code><\/pre>\n\n\n\n
lvcreate -V 3G -T vg_bricks\/brickpool4 -n shadow_brick2<\/code><\/pre>\n\n\n\n
mkfs.xfs -i size=512 \/dev\/vg_bricks\/shadow_brick2<\/code><\/pre>\n\n\n\n
mkdir -p \/bricks\/shadow_brick2<\/code><\/pre>\n\n\n\n
mount \/dev\/vg_bricks\/shadow_brick2 \/bricks\/shadow_brick2\/<\/code><\/pre>\n\n\n\n
mkdir \/bricks\/shadow_brick2\/brick<\/code><\/pre>\n\n\n\n
For the permanent mounting, run the below commands.<\/p>\n\n\n\n
\/dev\/vg_bricks\/shadow_brick1 \/bricks\/shadow_brick1\/ xfs rw,noatime,inode64,nouuid 1 2<\/code><\/pre>\n\n\n\n
Run the commands to create and mount the brick on server 3.<\/p>\n\n\n\n
pvcreate \/dev\/sdb ; vgcreate vg_bricks \/dev\/sdb<\/code><\/pre>\n\n\n\n
lvcreate -L 14G -T vg_bricks\/brickpool4<\/code><\/pre>\n\n\n\n
lvcreate -V 3G -T vg_bricks\/brickpool4 -n shadow_brick2<\/code><\/pre>\n\n\n\n
mkfs.xfs -i size=512 \/dev\/vg_bricks\/shadow_brick2<\/code><\/pre>\n\n\n\n
mkdir -p \/bricks\/shadow_brick2<\/code><\/pre>\n\n\n\n
mount \/dev\/vg_bricks\/shadow_brick2 \/bricks\/shadow_brick2\/<\/code><\/pre>\n\n\n\n
mkdir \/bricks\/shadow_brick2\/brick<\/code><\/pre>\n\n\n\n
For the permanent mounting, run the below commands.<\/p>\n\n\n\n
\/dev\/vg_bricks\/shadow_brick1 \/bricks\/shadow_brick1\/ xfs rw,noatime,inode64,nouuid 1 2<\/code><\/pre>\n\n\n\n
Run the commands to create and mount the brick on server 3.<\/p>\n\n\n\n
pvcreate \/dev\/sdb ; vgcreate vg_bricks \/dev\/sdb<\/code><\/pre>\n\n\n\n
lvcreate -L 14G -T vg_bricks\/brickpool4<\/code><\/pre>\n\n\n\n
lvcreate -V 3G -T vg_bricks\/brickpool4 -n shadow_brick2<\/code><\/pre>\n\n\n\n
mkfs.xfs -i size=512 \/dev\/vg_bricks\/shadow_brick2<\/code><\/pre>\n\n\n\n
mkdir -p \/bricks\/shadow_brick2<\/code><\/pre>\n\n\n\n
mount \/dev\/vg_bricks\/shadow_brick2 \/bricks\/shadow_brick2\/<\/code><\/pre>\n\n\n\n
mkdir \/bricks\/shadow_brick2\/brick<\/code><\/pre>\n\n\n\n
For the permanent mounting, run the below commands.<\/p>\n\n\n\n
\/dev\/vg_bricks\/shadow_brick1 \/bricks\/shadow_brick1\/ xfs rw,noatime,inode64,nouuid 1 2<\/code><\/pre>\n\n\n\n
Run the commands to create and mount the brick on server 3.<\/p>\n\n\n\n
pvcreate \/dev\/sdb ; vgcreate vg_bricks \/dev\/sdb<\/code><\/pre>\n\n\n\n
lvcreate -L 14G -T vg_bricks\/brickpool4<\/code><\/pre>\n\n\n\n
lvcreate -V 3G -T vg_bricks\/brickpool4 -n shadow_brick2<\/code><\/pre>\n\n\n\n
mkfs.xfs -i size=512 \/dev\/vg_bricks\/shadow_brick2<\/code><\/pre>\n\n\n\n
mkdir -p \/bricks\/shadow_brick2<\/code><\/pre>\n\n\n\n
mount \/dev\/vg_bricks\/shadow_brick2 \/bricks\/shadow_brick2\/<\/code><\/pre>\n\n\n\n
mkdir \/bricks\/shadow_brick2\/brick<\/code><\/pre>\n\n\n\n
For the permanent mounting, run the below commands.<\/p>\n\n\n\n
\n\/dev\/vg_bricks\/shadow_brick1 \/bricks\/shadow_brick1\/ xfs rw,noatime,inode64,nouuid 1 2\n<\/code><\/pre>\n\n\n\n
Perform a similar set of commands on server 4 for the creation and mounting of the brick by navigating the server 4 console.<\/p>\n\n\n\n
pvcreate \/dev\/sdb ; vgcreate vg_bricks \/dev\/sdb\nlvcreate -L 14G -T vg_bricks\/brickpool4\nlvcreate -V 3G -T vg_bricks\/brickpool4 -n shadow_brick2\nmkfs.xfs -i size=512 \/dev\/vg_bricks\/shadow_brick2\nmkdir -p \/bricks\/shadow_brick2\nmount \/dev\/vg_bricks\/shadow_brick2 \/bricks\/shadow_brick2\/\nmkdir \/bricks\/shadow_brick2\/brick\n\n<\/code><\/pre>\n\n\n\n
Use the below gluster commands to create the replicated volume.<\/p>\n\n\n\n
gluster volume create shadowvol replica 2 \nserver3.com:\/bricks\/shadow_brick1\/brick \nserver4.com:\/bricks\/shadow_brick2\/brick\n<\/code><\/pre>\n\n\n\n
volume create: shadowvol: success: please start the volume to access data.<\/p>\n\n\n\n
gluster volume start shadowvol<\/code><\/pre>\n\n\n\n
Run the commands to verify the volume info.<\/pre>\n\n\n\n
gluster volume info shadowvol<\/code><\/pre>\n\n\n\n
Add the below entry in the file “\/etc\/nfsmount.conf” on both storage server3 and the serevr4.<\/p>\n\n\n\n
Defaultvers=3<\/p>\n\n\n\n
Reboot both servers. Run following mount commands to volume “shadowvol”.<\/p>\n\n\n\n
mount -t nfs -o vers=3 server4.com:\/shadowvol \/mnt\/shadowvol\/<\/code><\/pre>\n\n\n\n
For permanent mount add the below entry in \/etc\/fstab file.<\/p>\n\n\n\n
server4.com:\/shadowvol \/mnt\/shadowvol\/ nfs vers=3 0 0<\/code><\/pre>\n\n\n\n
Later, verify the size and the mount status of the volume.<\/p>\n\n\n\n
df -Th<\/code><\/pre>\n\n\n\n
Distribute- replicate volume setup<\/strong>: <\/p>\n\n\n\n
In order to set up the distributed replicated volume use one brick from each server and then form the volume. let’s create the logical volume from the existing thin pool on the corresponding server.<\/p>\n\n\n\n
Run the command to create a brick on all 4 servers.<\/p>\n\n\n\n
Server 1<\/p>\n\n\n\n
lvcreate -V 3G -T vg_bricks\/brickpool1 -n prod_brick1<\/code><\/pre>\n\n\n\n
mkfs.xfs -i size=512 \/dev\/vg_bricks\/prod_brick1<\/code><\/pre>\n\n\n\n
mkdir -p \/bricks\/prod_brick1<\/code><\/pre>\n\n\n\n
mount \/dev\/vg_bricks\/prod_brick1 \/bricks\/prod_brick1\/<\/code><\/pre>\n\n\n\n
mkdir \/bricks\/prod_brick1\/brick<\/code><\/pre>\n\n\n\n
Server 2<\/p>\n\n\n\n
lvcreate -V 3G -T vg_bricks\/brickpool2 -n prod_brick2\nmkfs.xfs -i size=512 \/dev\/vg_bricks\/prod_brick2\nmkdir -p \/bricks\/prod_brick2\nmount \/dev\/vg_bricks\/prod_brick2 \/bricks\/prod_brick2\/\nmkdir \/bricks\/prod_brick2\/brick<\/code><\/pre>\n\n\n\n
Server 3<\/p>\n\n\n\n
lvcreate -V 3G -T vg_bricks\/brickpool3 -n prod_brick3<\/code><\/pre>\n\n\n\n
mkfs.xfs -i size=512 \/dev\/vg_bricks\/prod_brick3<\/code><\/pre>\n\n\n\n
mkdir -p \/bricks\/prod_brick3<\/code><\/pre>\n\n\n\n
mount \/dev\/vg_bricks\/prod_brick3 \/bricks\/prod_brick3\/<\/code><\/pre>\n\n\n\n
mkdir \/bricks\/prod_brick3\/brick<\/code><\/pre>\n\n\n\n
Server 4<\/p>\n\n\n\n
lvcreate -V 3G -T vg_bricks\/brickpool4 -n prod_brick4\nmkfs.xfs -i size=512 \/dev\/vg_bricks\/prod_brick4\nmkdir -p \/bricks\/prod_brick4\nmount \/dev\/vg_bricks\/prod_brick4 \/bricks\/prod_brick4\/\nmkdir \/bricks\/prod_brick4\/brick<\/code><\/pre>\n\n\n\n
Use the gluster commands and create the volume with the name ‘dist-rep-vol’.<\/strong><\/p>\n\n\n\n
gluster volume create dist-rep-vol replica 2 server1.example.com:\/bricks\/prod_brick1\/brick server2.example.com:\/bricks\/prod_brick2\/brick server3.example.com:\/bricks\/prod_brick3\/brick server4.example.com:\/bricks\/prod_brick4\/brick force<\/code><\/pre>\n\n\n\n
gluster volume start dist-rep-vol<\/p>\n\n\n\n
Run and verify the volume.<\/p>\n\n\n\n
gluster volume info dist-rep-vol<\/code><\/pre>\n\n\n\n
\"\"<\/figure>\n\n\n\n
Therefore, the volume of the first files will be distributed on any of the two bricks and then the files will be replicated into the remaining two bricks.<\/p>\n\n\n\n
Now, mount the volume on the client machine via gluster.<\/p>\n\n\n\n
Create the mount point to the volume.<\/p>\n\n\n\n
mkdir \/mnt\/dist-rep-vol<\/code><\/pre>\n\n\n\n
mount.glusterfs server1.com:\/dist-rep-vol \/mnt\/dist-rep-vol\/<\/p>\n\n\n\n
 Later, add the below entry in fstab for the permanent entry.<\/p>\n\n\n\n
server1.com:\/dist-rep-vol \/mnt\/dist-rep-vol\/ glusterfs _netdev 0 0<\/code><\/pre>\n\n\n\n
Run the following df commands and verify the size and volume.<\/p>\n\n\n\n
 df -Th<\/code><\/pre>\n\n\n\n
conclusion: <\/strong><\/p>\n\n\n\n
This blog will demonstrate how to install and configure the GlusterFS storage on RHEL7.x and CentOS 7.X.You can also take our technical team’s support for further installation and configuration.<\/p>\n","protected":false},"excerpt":{"rendered":"
              The glusterFS is an open-source, simple, elastic, and scalable file storage system that is suitable for high data such as media streaming, cloud storage, and the content delivery network (CDN). We often get a request from our client to install the GlusterFS server on centOS 7 as it is part of our server management […]<\/p>\n","protected":false},"author":7,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[68,208,253,230],"tags":[254,25,52,233,72],"class_list":["post-6121","post","type-post","status-publish","format-standard","hentry","category-server-management","category-centos-7","category-glusterfs","category-rhel-7","tag-centros-7","tag-cloud-management","tag-glusterfs","tag-rhel-8-7","tag-server-management"],"_links":{"self":[{"href":"https:\/\/www.skynats.com\/blog\/wp-json\/wp\/v2\/posts\/6121","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.skynats.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.skynats.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.skynats.com\/blog\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/www.skynats.com\/blog\/wp-json\/wp\/v2\/comments?post=6121"}],"version-history":[{"count":0,"href":"https:\/\/www.skynats.com\/blog\/wp-json\/wp\/v2\/posts\/6121\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.skynats.com\/blog\/wp-json\/wp\/v2\/media?parent=6121"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.skynats.com\/blog\/wp-json\/wp\/v2\/categories?post=6121"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.skynats.com\/blog\/wp-json\/wp\/v2\/tags?post=6121"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}