Oracle DBA Forum  

Вернуться   Oracle DBA Forum > Clustering | High Ability > DRBD, VVR, Lustre > DRBD, VVR, Lustre

Ответ
 
Опции темы Опции просмотра
  #11  
Старый 04.07.2011, 17:33
Marley Marley вне форума
Senior Member
 
Регистрация: 19.09.2009
Сообщений: 7,211
По умолчанию

Проверка работы:

Необходимо скопировать какой-либо файл в раздел /mnt/luster

Например, можно попробовать скопировать пакеты, оставшиеся от инсталляции lustre.

[[email protected] tmp]# cd /tmp
[[email protected] tmp]# cp * /mnt/lustre/


Далее, следует на втором клиенте проверить результат репликации.
Убедитесь, что скопированные пакеты доступны второму клиенту.

[[email protected] ~]# cd /mnt/lustre/
[[email protected] lustre]# ls
e2fsprogs-1.41.10.sun2-0redhat.rhel5.x86_64.rpm
kernel-2.6.18-194.17.1.el5_lustre.1.8.5-ext4.x86_64.rpm
kernel-2.6.18-194.17.1.el5_lustre.1.8.5.x86_64.rpm
lustre-1.8.5-2.6.18_194.17.1.el5_lustre.1.8.5-ext4.x86_64.rpm
lustre-1.8.5-2.6.18_194.17.1.el5_lustre.1.8.5.x86_64.rpm
lustre-client-1.8.5-2.6.18_194.17.1.el5_lustre.1.8.5.x86_64.rpm
lustre-client-modules-1.8.5-2.6.18_194.17.1.el5_lustre.1.8.5.x86_64.rpm
lustre-ldiskfs-3.1.4-2.6.18_194.17.1.el5_lustre.1.8.5-ext4.x86_64.rpm
lustre-ldiskfs-3.1.4-2.6.18_194.17.1.el5_lustre.1.8.5.x86_64.rpm
lustre-modules-1.8.5-2.6.18_194.17.1.el5_lustre.1.8.5-ext4.x86_64.rpm
lustre-modules-1.8.5-2.6.18_194.17.1.el5_lustre.1.8.5.x86_64.rpm

================================================== =====

[[email protected] /]# lctl dl

Код:
  0 UP mgs MGS MGS 13
  1 UP mgc [email protected] b4f12971-ad10-81d7-ae9f-cdf3b6db9e54 5
  2 UP mdt MDS MDS_uuid 3
  3 UP lov lustre-mdtlov lustre-mdtlov_UUID 4
  4 UP mds lustre-MDT0000 lustre-MDT0000_UUID 7
  5 UP osc lustre-OST0000-osc lustre-mdtlov_UUID 5
  6 UP osc lustre-OST0001-osc lustre-mdtlov_UUID 5
  7 UP osc lustre-OST0002-osc lustre-mdtlov_UUID 5
  8 UP osc lustre-OST0003-osc lustre-mdtlov_UUID 5
  9 UP osc lustre-OST0004-osc lustre-mdtlov_UUID 5
 10 UP osc lustre-OST0005-osc lustre-mdtlov_UUID 5
 11 UP osc lustre-OST0006-osc lustre-mdtlov_UUID 5
 12 UP osc lustre-OST0007-osc lustre-mdtlov_UUID 5
Командами
lctl --device 7 deactivate
lctl --device 7 activate

могу дективировать и активировать заново подмонтированный диск.
__________________
Чат форума (требуется аккаунт на github или twitter)

Последний раз редактировалось Marley; 05.07.2011 в 11:08.
Ответить с цитированием
  #12  
Старый 04.07.2011, 17:39
Marley Marley вне форума
Senior Member
 
Регистрация: 19.09.2009
Сообщений: 7,211
По умолчанию

======================================

Вариант, который был рассмотрен в примере от SUN

------------------------------------------------

// Servers

Metadata Server
Name: mds
Ip: 192.168.1.10


Object Store Server #1
Name: oss01
Ip: 192.168.1.11


Object Store Server #2
Name: oss02
Ip: 192.168.1.12


// Clients


Client #1
Name: client1
Ip: 192.168.1.21


Client #2
Name: client2
Ip: 192.168.1.22
__________________
Чат форума (требуется аккаунт на github или twitter)

Последний раз редактировалось Marley; 04.07.2011 в 17:58.
Ответить с цитированием
  #13  
Старый 04.07.2011, 17:40
Marley Marley вне форума
Senior Member
 
Регистрация: 19.09.2009
Сообщений: 7,211
По умолчанию

Creating the virtual volumes

Virtual volumes need to be created on both Object Store Servers and on the Metadata Server. The MDS requires only one virtual volume; six virtual volumes are created on each OSS in this example configuration.

Object Server #1:

На диске необходимо указать тип раздела LVM 0x8e


# fdisk -l

Код:
Disk /dev/hda: 42.9 GB, 42949672960 bytes
255 heads, 63 sectors/track, 5221 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/hda1   *           1          13      104391   83  Linux
/dev/hda2              14         274     2096482+  82  Linux swap / Solaris
/dev/hda3             275        2610    18763920   83  Linux
/dev/hda4            2611        5221    20972857+   5  Extended
/dev/hda5            2611        5221    20972826   83  Linux

Disk /dev/hdb: 42.9 GB, 42949672960 bytes
16 heads, 63 sectors/track, 83220 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes

   Device Boot      Start         End      Blocks   Id  System

[[email protected] ~]# fdisk /dev/hdb

Код:
The number of cylinders for this disk is set to 83220.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-83220, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-83220, default 83220):
Using default value 83220

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): L

 0  Empty           1e  Hidden W95 FAT1 80  Old Minix       bf  Solaris
 1  FAT12           24  NEC DOS         81  Minix / old Lin c1  DRDOS/sec (FAT-
 2  XENIX root      39  Plan 9          82  Linux swap / So c4  DRDOS/sec (FAT-
 3  XENIX usr       3c  PartitionMagic  83  Linux           c6  DRDOS/sec (FAT-
 4  FAT16 <32M      40  Venix 80286     84  OS/2 hidden C:  c7  Syrinx
 5  Extended        41  PPC PReP Boot   85  Linux extended  da  Non-FS data
 6  FAT16           42  SFS             86  NTFS volume set db  CP/M / CTOS / .
 7  HPFS/NTFS       4d  QNX4.x          87  NTFS volume set de  Dell Utility
 8  AIX             4e  QNX4.x 2nd part 88  Linux plaintext df  BootIt
 9  AIX bootable    4f  QNX4.x 3rd part 8e  Linux LVM       e1  DOS access
 a  OS/2 Boot Manag 50  OnTrack DM      93  Amoeba          e3  DOS R/O
 b  W95 FAT32       51  OnTrack DM6 Aux 94  Amoeba BBT      e4  SpeedStor
 c  W95 FAT32 (LBA) 52  CP/M            9f  BSD/OS          eb  BeOS fs
 e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a0  IBM Thinkpad hi ee  EFI GPT
 f  W95 Ext'd (LBA) 54  OnTrackDM6      a5  FreeBSD         ef  EFI (FAT-12/16/
10  OPUS            55  EZ-Drive        a6  OpenBSD         f0  Linux/PA-RISC b
11  Hidden FAT12    56  Golden Bow      a7  NeXTSTEP        f1  SpeedStor
12  Compaq diagnost 5c  Priam Edisk     a8  Darwin UFS      f4  SpeedStor
14  Hidden FAT16 <3 61  SpeedStor       a9  NetBSD          f2  DOS secondary
16  Hidden FAT16    63  GNU HURD or Sys ab  Darwin boot     fb  VMware VMFS
17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fc  VMware VMKCORE
18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fd  Linux raid auto
1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid fe  LANstep
1c  Hidden W95 FAT3 75  PC/IX           be  Solaris boot    ff  BBT
Hex code (type L to list codes): 8e
Changed system type of partition 1 to 8e (Linux LVM)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
# reboot



# fdisk -l /dev/hdb

Код:
Disk /dev/hdb: 42.9 GB, 42949672960 bytes
16 heads, 63 sectors/track, 83220 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/hdb1               1       83220    41942848+  8e  Linux LVM



// Use the Linux command pvcreate to create a physical volume

# pvcreate /dev/hdb1

Код:
  Physical volume "/dev/hdb1" successfully created
// Next, create a volume group:
# vgcreate vg00 /dev/hdb1

Код:
  Volume group "vg00" successfully created

// Once the volume group is created, create a number of Object Store Targets (OSTs) in this volume group.

# lvcreate --name vg00/ost1 --size 2G
# lvcreate --name vg00/ost2 --size 2G
# lvcreate --name vg00/ost3 --size 2G
# lvcreate --name vg00/ost4 --size 2G
# lvcreate --name vg00/ost5 --size 2G
# lvcreate --name vg00/ost6 --size 2G


// Use the ls command to list all logical volumes:

# ls -l /dev/vg00

Код:
total 0
lrwxrwxrwx 1 root root 21 Mar 30 09:19 ost1 -> /dev/mapper/vg00-ost1
lrwxrwxrwx 1 root root 21 Mar 30 09:19 ost2 -> /dev/mapper/vg00-ost2
lrwxrwxrwx 1 root root 21 Mar 30 09:19 ost3 -> /dev/mapper/vg00-ost3
lrwxrwxrwx 1 root root 21 Mar 30 09:19 ost4 -> /dev/mapper/vg00-ost4
lrwxrwxrwx 1 root root 21 Mar 30 09:19 ost5 -> /dev/mapper/vg00-ost5
lrwxrwxrwx 1 root root 21 Mar 30 09:19 ost6 -> /dev/mapper/vg00-ost6


======================================

Object Server #2:

Подключитесь пользователем root и повторите действия выполненные для Server #1, для создания 6 OSTs


======================================


Metadata Server:



Login as root to the MDS, and repeat Steps 1 to 5 to create a single logical volume (MDT) for the MDS.

As each file written to the Lustre file system requires 4 KB of storage space in the MDS, a 1 GB logical volume for the MDT is more than sufficient for most typical deployments.


lvcreate --name vg00/mdt --size 1G
__________________
Чат форума (требуется аккаунт на github или twitter)
Ответить с цитированием
  #14  
Старый 04.07.2011, 17:40
Marley Marley вне форума
Senior Member
 
Регистрация: 19.09.2009
Сообщений: 7,211
По умолчанию

Инсталляция Lustre

С сайта lustre.org копирую файлы из списка Required Files:

Backing filesystem creation and repair tools (MDS/MGS/OSS Only)
e2fsprogs-1.41.10.sun2-0redhat.oel5.x86_64.rpm 0.95 MB

Backing filesystem kernel module (MDS/MGS/OSS Only)
lustre-ldiskfs-3.1.4-2.6.18_194.17.1.0.1.el5_lustre.1.8.5-ext4.x86_64.rpm 1.26 MB

Backing filesystem kernel module (MDS/MGS/OSS Only)
lustre-ldiskfs-3.1.4-2.6.18_194.17.1.0.1.el5_lustre.1.8.5.x86_64.rpm 0.93 MB

Lustre-patched kernel (MDS/MGS/OSS Only)
kernel-2.6.18-194.17.1.0.1.el5_lustre.1.8.5-ext4.x86_64.rpm 18.84 MB

Lustre-patched kernel (MDS/MGS/OSS Only)
kernel-2.6.18-194.17.1.0.1.el5_lustre.1.8.5.x86_64.rpm 18.84 MB

Lustre modules (Client and Server for Lustre patched kernel)
lustre-modules-1.8.5-2.6.18_194.17.1.0.1.el5_lustre.1.8.5-ext4.x86_64.rpm 17.50 MB

Lustre modules (Client and Server for Lustre patched kernel)
lustre-modules-1.8.5-2.6.18_194.17.1.0.1.el5_lustre.1.8.5.x86_64.rpm 17.50 MB

Lustre userland tools (Client and server for patched Lustre kernel)
lustre-1.8.5-2.6.18_194.17.1.0.1.el5_lustre.1.8.5.x86_64.rpm 4.54 MB

Lustre userland tools (Client and server for patched Lustre kernel)
lustre-1.8.5-2.6.18_194.17.1.0.1.el5_lustre.1.8.5-ext4.x86_64.rpm 4.54 MB

Lustre client modules (Client for unpatched vendor kernel)
lustre-client-modules-1.8.5-2.6.18_194.17.1.0.1.el5_lustre.1.8.5.x86_64.rpm 14.41 MB

Lustre client userland tools (Client for unpatched vendor kernel)
lustre-client-1.8.5-2.6.18_194.17.1.0.1.el5_lustre.1.8.5.x86_64.rpm 4.54 MB

-------------------------------

При необходимости, можно на сервере поднять ftp
http://odba.ru/showpost.php?p=6690&postcount=6
__________________
Чат форума (требуется аккаунт на github или twitter)
Ответить с цитированием
  #15  
Старый 04.07.2011, 17:41
Marley Marley вне форума
Senior Member
 
Регистрация: 19.09.2009
Сообщений: 7,211
По умолчанию

# ls

Код:
e2fsprogs-1.41.10.sun2-0redhat.oel5.x86_64.rpm
kernel-2.6.18-194.17.1.0.1.el5_lustre.1.8.5-ext4.x86_64.rpm
kernel-2.6.18-194.17.1.0.1.el5_lustre.1.8.5.x86_64.rpm
lustre-1.8.5-2.6.18_194.17.1.0.1.el5_lustre.1.8.5-ext4.x86_64.rpm
lustre-1.8.5-2.6.18_194.17.1.0.1.el5_lustre.1.8.5.x86_64.rpm
lustre-client-1.8.5-2.6.18_194.17.1.0.1.el5_lustre.1.8.5.x86_64.rpm
lustre-client-modules-1.8.5-2.6.18_194.17.1.0.1.el5_lustre.1.8.5.x86_64.rpm
lustre-ldiskfs-3.1.4-2.6.18_194.17.1.0.1.el5_lustre.1.8.5-ext4.x86_64.rpm
lustre-ldiskfs-3.1.4-2.6.18_194.17.1.0.1.el5_lustre.1.8.5.x86_64.rpm
lustre-modules-1.8.5-2.6.18_194.17.1.0.1.el5_lustre.1.8.5-ext4.x86_64.rpm
lustre-modules-1.8.5-2.6.18_194.17.1.0.1.el5_lustre.1.8.5.x86_64.rpm


# rpm -Uvh --replacefiles e2fsprogs-1.41.10.sun2-0redhat.oel5.x86_64.rpm

Код:
Preparing...                ########################################### [100%]
   1:e2fsprogs              ########################################### [100%]

rpm -ihv --force kernel-2.6.18-194.17.1.0.1.el5_lustre.1.8.5-ext4.x86_64.rpm


rpm -ihv lustre-*ext4*.rpm

# vi /etc/grub.conf

Указываем, что стартовать должны с ядром:

Oracle Linux Server (2.6.18-194.17.1.0.1.el5_lustre.1.8.5)

Код:
default=0
timeout=1
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Oracle Linux Server (2.6.18-194.17.1.0.1.el5_lustre.1.8.5)
        root (hd0,0)
        kernel /vmlinuz-2.6.18-194.17.1.0.1.el5_lustre.1.8.5 ro root=LABEL=/
        initrd /initrd-2.6.18-194.17.1.0.1.el5_lustre.1.8.5.img
title Oracle Linux Server (2.6.32-100.26.2.el5uek)
        root (hd0,0)
        kernel /vmlinuz-2.6.32-100.26.2.el5 ro root=/dev/hda3
        initrd /initrd-2.6.32-100.26.2.el5.img

# reboot


# uname -a

Код:
Linux server1.localdomain 2.6.18-194.17.1.0.1.el5_lustre.1.8.5 #1 SMP Tue Nov 16 17:22:18 MST 2010 x86_64 x86_64 x86_64 GNU/Linux
__________________
Чат форума (требуется аккаунт на github или twitter)
Ответить с цитированием
  #16  
Старый 04.07.2011, 17:41
Marley Marley вне форума
Senior Member
 
Регистрация: 19.09.2009
Сообщений: 7,211
По умолчанию

Metadata Server


# mkfs.lustre --fsname lustre --mdt --mgs /dev/vg00/mdt


Код:
   Permanent disk data:
Target:     lustre-MDTffff
Index:      unassigned
Lustre FS:  lustre
Mount type: ldiskfs
Flags:      0x75
              (MDT MGS needs_index first_time update )
Persistent mount opts: iopen_nopriv,user_xattr,errors=remount-ro
Parameters: mdt.group_upcall=/usr/sbin/l_getgroups

checking for existing Lustre data: not found
device size = 1024MB
2 6 18
formatting backing filesystem ldiskfs on /dev/vg00/mdt
        target name  lustre-MDTffff
        4k blocks     262144
        options        -i 4096 -I 512 -q -O dir_index,uninit_groups -F
mkfs_cmd = mke2fs -j -b 4096 -L lustre-MDTffff  -i 4096 -I 512 -q -O dir_index,uninit_groups -F /dev/vg00/mdt 262144
Writing CONFIGS/mountdata


# mkdir /mdt


# mount -t lustre /dev/vg00/mdt /mdt
__________________
Чат форума (требуется аккаунт на github или twitter)
Ответить с цитированием
  #17  
Старый 04.07.2011, 17:41
Marley Marley вне форума
Senior Member
 
Регистрация: 19.09.2009
Сообщений: 7,211
По умолчанию

Выполнить на 2-х серверах Object Store Servers


# mkdir /mnt/ost1
# mkdir /mnt/ost2
# mkdir /mnt/ost3
# mkdir /mnt/ost4
# mkdir /mnt/ost5
# mkdir /mnt/ost6



# mkfs.lustre --fsname lustre --ost [email protected] /dev/vg00/ost1
# mkfs.lustre --fsname lustre --ost [email protected] /dev/vg00/ost2
# mkfs.lustre --fsname lustre --ost [email protected] /dev/vg00/ost3
# mkfs.lustre --fsname lustre --ost [email protected] /dev/vg00/ost4
# mkfs.lustre --fsname lustre --ost [email protected] /dev/vg00/ost5
# mkfs.lustre --fsname lustre --ost [email protected] /dev/vg00/ost6


Код:
   Permanent disk data:
Target:     lustre-OSTffff
Index:      unassigned
Lustre FS:  lustre
Mount type: ldiskfs
Flags:      0x72
              (OST needs_index first_time update )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: [email protected]

checking for existing Lustre data: not found
device size = 2048MB
2 6 18
formatting backing filesystem ldiskfs on /dev/vg00/ost1
        target name  lustre-OSTffff
        4k blocks     524288
        options        -J size=80 -i 16384 -I 256 -q -O dir_index,extents,uninit_groups -F
mkfs_cmd = mke2fs -j -b 4096 -L lustre-OSTffff  -J size=80 -i 16384 -I 256 -q -O dir_index,extents,uninit_groups -F /dev/vg00/ost1 524288
Writing CONFIGS/mountdata



# mount -t lustre /dev/vg00/ost1 /mnt/ost1
# mount -t lustre /dev/vg00/ost2 /mnt/ost2
# mount -t lustre /dev/vg00/ost3 /mnt/ost3
# mount -t lustre /dev/vg00/ost4 /mnt/ost4
# mount -t lustre /dev/vg00/ost5 /mnt/ost5
# mount -t lustre /dev/vg00/ost6 /mnt/ost6



# cat /proc/fs/lustre/devices

Код:
  0 UP mgc [email protected] b05005da-400b-0013-5644-f3c494720cae 5
  1 UP ost OSS OSS_uuid 3
  2 UP obdfilter lustre-OST0000 lustre-OST0000_UUID 5
  3 UP obdfilter lustre-OST0001 lustre-OST0001_UUID 5
  4 UP obdfilter lustre-OST0002 lustre-OST0002_UUID 5
  5 UP obdfilter lustre-OST0003 lustre-OST0003_UUID 5
  6 UP obdfilter lustre-OST0004 lustre-OST0004_UUID 5
  7 UP obdfilter lustre-OST0005 lustre-OST0005_UUID 5
__________________
Чат форума (требуется аккаунт на github или twitter)
Ответить с цитированием
  #18  
Старый 04.07.2011, 17:41
Marley Marley вне форума
Senior Member
 
Регистрация: 19.09.2009
Сообщений: 7,211
По умолчанию

Clients:


# rpm -ihv --force kernel-2.6.18-194.17.1.0.1.el5_lustre.1.8.5-ext4.x86_64.rpm

Код:
Preparing...                ########################################### [100%]
   1:kernel                 ########################################### [100%]
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_iget
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_free_blocks
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_get_group_desc
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ext_pblock
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_journal_start_sb
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol __ldiskfs_journal_stop
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_force_commit
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_ext_store_pblock
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_itable_unused_count
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_ext_search_right
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_mark_inode_dirty
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_xattr_get
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_ext_calc_credits_for_insert
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_xattr_set_handle
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_read_inode_bitmap
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_ext_insert_extent
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol __ldiskfs_journal_get_write_access
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_mb_new_blocks
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_ext_search_left
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_bread
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_ext_walk_space
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_map_inode_page
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol ldiskfs_mb_discard_inode_preallocations
WARNING: /lib/modules/2.6.18-194.17.1.0.1.el5_lustre.1.8.5/updates/kernel/fs/lustre/fsfilt_ldiskfs.ko needs unknown symbol __ldiskfs_handle_dirty_metadata



[[email protected] lustre]# rpm -Uvh --replacefiles e2fsprogs-1.41.10.sun2-0redhat.oel5.x86_64.rpm

Код:
Preparing...                ########################################### [100%]
   1:e2fsprogs              ########################################### [100%]

[[email protected] lustre]# rpm -ihv lustre-client-modules-1.8.5-2.6.18_194.17.1.0.1.el5_lustre.1.8.5.x86_64.rpm

Код:
Preparing...                ########################################### [100%]
   1:lustre-client-modules  ########################################### [100%]

Congratulations on finishing your Lustre installation!  To register
your copy of Lustre and find out more about Lustre Support, Service,
and Training offerings please visit

http://www.sun.com/software/products/lustre/lustre_reg.jsp

[[email protected] lustre]# rpm -ihv lustre-client-1.8.5-2.6.18_194.17.1.0.1.el5_lustre.1.8.5.x86_64.rpm

Код:
Preparing...                ########################################### [100%]
   1:lustre-client          ########################################### [100%]

# rpm -ihv lustre-modules-1.8.5-2.6.18_194.17.1.0.1.el5_lustre.1.8.5-ext4.x86_64.rpm

Код:
Preparing...                ########################################### [100%]
   1:lustre-modules         ########################################### [100%]
Congratulations on finishing your Lustre installation!  To register
your copy of Lustre and find out more about Lustre Support, Service,
and Training offerings please visit

http://www.sun.com/software/products/lustre/lustre_reg.jsp

отредактировать файл vi /etc/grub.conf


# reboot


# uname -a

Код:
Linux client2.localdomain 2.6.18-194.17.1.0.1.el5_lustre.1.8.5 #1 SMP Tue Nov 16 17:22:18 MST 2010 x86_64 x86_64 x86_64 GNU/Linux

# mkdir /mnt/lustre
# mount -t lustre [email protected]:/lustre /mnt/lustre




# lfs df -h

Код:
UUID                       bytes        Used   Available Use% Mounted on
lustre-MDT0000_UUID       895.9M       33.5M      811.2M   3% /mnt/lustre[MDT:0]
lustre-OST0000_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:0]
lustre-OST0001_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:1]
lustre-OST0002_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:2]
lustre-OST0003_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:3]
lustre-OST0004_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:4]
lustre-OST0005_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:5]
lustre-OST0006_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:6]
lustre-OST0007_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:7]
lustre-OST0008_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:8]
lustre-OST0009_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:9]
lustre-OST000a_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:10]
lustre-OST000b_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:11]

filesystem summary:        23.6G      998.3M       21.4G   4% /mnt/lustre

# lfs getstripe /mnt/lustre


Код:
/mnt/lustre
stripe_count:   1 stripe_size:    1048576 stripe_offset:  -1
/mnt/lustre/fstab
lmm_stripe_count:   1
lmm_stripe_size:    1048576
lmm_stripe_offset:  9
        obdidx           objid          objid            group
             9               2            0x2                0

# lfs setstripe /mnt/lustre 4M 0 1


# dd if=/dev/zero of=/mnt/lustre/test1 bs=10M count=100
# dd if=/dev/zero of=/mnt/lustre/test2 bs=10M count=100
# dd if=/dev/zero of=/mnt/lustre/test3 bs=10M count=100
# dd if=/dev/zero of=/mnt/lustre/test4 bs=10M count=100
# dd if=/dev/zero of=/mnt/lustre/test5 bs=10M count=100


Код:
100+0 records in
100+0 records out
1048576000 bytes (1.0 GB) copied, 28.4542 seconds, 36.9 MB/s



# lfs df -h


Код:
[[email protected] lustre]# lfs df -h
UUID                       bytes        Used   Available Use% Mounted on
lustre-MDT0000_UUID       895.9M       33.6M      811.1M   3% /mnt/lustre[MDT:0]
lustre-OST0000_UUID         2.0G        1.1G      830.2M  53% /mnt/lustre[OST:0]
lustre-OST0001_UUID         2.0G        1.1G      830.2M  53% /mnt/lustre[OST:1]
lustre-OST0002_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:2]
lustre-OST0003_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:3]
lustre-OST0004_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:4]
lustre-OST0005_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:5]
lustre-OST0006_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:6]
lustre-OST0007_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:7]
lustre-OST0008_UUID         2.0G        1.1G      830.2M  53% /mnt/lustre[OST:8]
lustre-OST0009_UUID         2.0G        1.1G      830.2M  53% /mnt/lustre[OST:9]
lustre-OST000a_UUID         2.0G       83.2M        1.8G   4% /mnt/lustre[OST:10]
lustre-OST000b_UUID         2.0G        1.1G      830.2M  53% /mnt/lustre[OST:11]

filesystem summary:        23.6G        5.9G       16.6G  24% /mnt/lustre



# lfs getstripe /mnt/lustre/test1

Код:
/mnt/lustre/test1
lmm_stripe_count:   1
lmm_stripe_size:    1048576
lmm_stripe_offset:  11
        obdidx           objid          objid            group
            11               2            0x2                0
__________________
Чат форума (требуется аккаунт на github или twitter)
Ответить с цитированием
  #19  
Старый 04.07.2011, 17:42
Marley Marley вне форума
Senior Member
 
Регистрация: 19.09.2009
Сообщений: 7,211
По умолчанию

MDS server:

Sometimes the file system gets unbalanced, often due to changed stripe settings. If an OST gets filled up and one tries to write more information to the file system involving said OST, an error occurs.


# lctl dl

Код:
  0 UP mgs MGS MGS 11
  1 UP mgc [email protected] a97899e8-dfe0-45f2-18fb-2734074f86bc 5
  2 UP mdt MDS MDS_uuid 3
  3 UP lov lustre-mdtlov lustre-mdtlov_UUID 4
  4 UP mds lustre-MDT0000 lustre-MDT0000_UUID 5
  5 UP osc lustre-OST0000-osc lustre-mdtlov_UUID 5
  6 UP osc lustre-OST0001-osc lustre-mdtlov_UUID 5
  7 UP osc lustre-OST0002-osc lustre-mdtlov_UUID 5
  8 UP osc lustre-OST0003-osc lustre-mdtlov_UUID 5
  9 UP osc lustre-OST0004-osc lustre-mdtlov_UUID 5
 10 UP osc lustre-OST0005-osc lustre-mdtlov_UUID 5
 11 UP osc lustre-OST0006-osc lustre-mdtlov_UUID 5
 12 UP osc lustre-OST0007-osc lustre-mdtlov_UUID 5
 13 UP osc lustre-OST0008-osc lustre-mdtlov_UUID 5
 14 UP osc lustre-OST0009-osc lustre-mdtlov_UUID 5
 15 UP osc lustre-OST000a-osc lustre-mdtlov_UUID 5
 16 UP osc lustre-OST000b-osc lustre-mdtlov_UUID 5

Use the lctl deactive command to take the full OST offline

# lctl --device 7 deactivate

# lctl dl

Код:
  0 UP mgs MGS MGS 11
  1 UP mgc [email protected] a97899e8-dfe0-45f2-18fb-2734074f86bc 5
  2 UP mdt MDS MDS_uuid 3
  3 UP lov lustre-mdtlov lustre-mdtlov_UUID 4
  4 UP mds lustre-MDT0000 lustre-MDT0000_UUID 5
  5 UP osc lustre-OST0000-osc lustre-mdtlov_UUID 5
  6 UP osc lustre-OST0001-osc lustre-mdtlov_UUID 5
  7 IN osc lustre-OST0002-osc lustre-mdtlov_UUID 5
  8 UP osc lustre-OST0003-osc lustre-mdtlov_UUID 5
  9 UP osc lustre-OST0004-osc lustre-mdtlov_UUID 5
 10 UP osc lustre-OST0005-osc lustre-mdtlov_UUID 5
 11 UP osc lustre-OST0006-osc lustre-mdtlov_UUID 5
 12 UP osc lustre-OST0007-osc lustre-mdtlov_UUID 5
 13 UP osc lustre-OST0008-osc lustre-mdtlov_UUID 5
 14 UP osc lustre-OST0009-osc lustre-mdtlov_UUID 5
 15 UP osc lustre-OST000a-osc lustre-mdtlov_UUID 5
 16 UP osc lustre-OST000b-osc lustre-mdtlov_UUID 5
# lctl --device 7 activate
__________________
Чат форума (требуется аккаунт на github или twitter)
Ответить с цитированием
Ответ

Метки
cluster, linux, lustre, oracle

Опции темы
Опции просмотра

Ваши права в разделе
Вы не можете создавать новые темы
Вы не можете отвечать в темах
Вы не можете прикреплять вложения
Вы не можете редактировать свои сообщения

BB коды Вкл.
Смайлы Вкл.
[IMG] код Вкл.
HTML код Выкл.

Быстрый переход


Текущее время: 21:00. Часовой пояс GMT +3.


Powered by vBulletin®