Experience: is what you get soon after you need it.

Experience: is what you get soon after you need it.

Rasul Allah (sal Allahu alaihi wa sallam) said: "Restore the trusts of those who trust you, and deal not falsely with him who deals falsely with you." [Abu Dawud, Tirmidhi]

Search This Blog

Saturday, September 12, 2015

Create and Manage a tablespace that uses NFS mounted file system file


Below we will discuss
1) How to create NFS filesystem and use it for Database files:
2) Configure DNFS client filesystem and use it for Database files bypassing Kernel NFS system.

1) How to create NFS filesystem client and use it for Database files:

Create a new NFS filesystem

Below I will creating a new NFS file system named /nfs

[oracle@collabn1 ~]$ df -Ph
Filesystem                       Size  Used Avail Use% Mounted on
/dev/mapper/vg_collabn1-lv_root   31G   20G   11G  66% /
tmpfs                            1.8G  758M  1.1G  43% /dev/shm
/dev/sda1                        477M   66M  382M  15% /boot
stage                            465G  280G  186G  61% /media/sf_stage
/dev/asm/shared-162              1.0G   79M  946M   8% /shared


[root@collabn1 ~]# cd /


[root@collabn1 /]# mkdir localfs nfs


Add the below lines in the fstab

[root@collabn1 ~]# vi /etc/fstab

# create an NFS mount on nfs which points back to localfs
localhost:/localfs /nfs nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,vers=3,timeo=600,noac

Add the below line to the exports file

[root@collabn1 ~]# cat /etc/exports
/nfs *(rw,no_root_squash,sync)

stop/start the nfs service:

[root@collabn1 /]# chkconfig nfs on

[root@collabn1 /]# service nfs stop
Shutting down NFS daemon:                                  [  OK  ]
Shutting down NFS mountd:                                  [  OK  ]
Shutting down NFS quotas:                                  [  OK  ]
Shutting down NFS services:                                [  OK  ]
Shutting down RPC idmapd:                                  [  OK  ]

[root@collabn1 /]# service nfs start
Starting NFS services:                                     [  OK  ]
Starting NFS quotas:                                       [  OK  ]
Starting NFS mountd:                                       [  OK  ]
Starting NFS daemon:                                       [  OK  ]
Starting RPC idmapd:                                       [  OK  ]


[root@collabn1 /]# mount /nfs

[root@collabn1 /]# df -Ph
Filesystem                       Size  Used Avail Use% Mounted on
/dev/mapper/vg_collabn1-lv_root   31G   20G   11G  66% /
tmpfs                            1.8G  758M  1.1G  43% /dev/shm
/dev/sda1                        477M   66M  382M  15% /boot
stage                            465G  280G  186G  61% /media/sf_stage
/dev/asm/shared-162              1.0G   79M  946M   8% /shared
localhost:/localfs                31G   20G   11G  66% /nfs

huray….. /nfs is mounted and available for use.



Create and Manage a tablespace that uses NFS mounted file system file

[root@collabn1 /]# cat /etc/fstab | grep nfs
localhost:/localfs /nfs nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,vers=3,timeo=600,noac

SHAIKDB>create tablespace nfstbs datafile '/nfs/nfstbs01.dbf' size 100m;       

Tablespace created.

SHAIKDB>


SHAIKDB>!ls -lrt /nfs

total 102412
-rw-r----- 1 oracle dba      104865792 Sep 12 19:56 nfstbs01.dbf


2) Configure DNFS client filesystem and use it for Database files bypassing Kernel NFS system.


With Oracle Database 11g release 2 (11.2), instead of using the operating system kernel NFS client, you can configure Oracle Database to access NFS V3 servers directly using an Oracle internal Direct NFS client.

To enable Oracle Database to use Direct NFS, the NFS file systems must be mounted and available over regular NFS mounts before you start installation. Direct NFS manages settings after installation. You should still set the kernel mount options as a backup, but for normal operation, Direct NFS will manage NFS mounts.

WARNING:
The DNFS Guide says to enable the init.ora param filesystemio_options to enable direct I/O support.
Doing this, all DB access to all files will be via DIO.
There is no way to only enable direct IO for certain files and exclude others.
Configure Direct NFS Client (DNFS)

SHAIKDB>select * from v$dnfs_servers;

no rows selected

SHAIKDB>show parameter filesystem

NAME                     TYPE     VALUE
------------------------------------ ----------- ------------------------------
filesystemio_options             string     none

i) Configure oranfstab file
Direct NFS Client can use a new configuration file or the mount tab file (/etc/mtab on Linux) to determine the mount point settings for NFS storage devices.

When the oranfstab file is placed in $ORACLE_HOME/dbs, the entries in the file are specific to a single database. When the oranfstab file is placed in /etc, then it is globally available to all Oracle databases

Direct NFS searches for mount entries in the following order:
1. $ORACLE_HOME/dbs/oranfstab
2. /etc/oranfstab
3. /etc/mtab

Here I will let Oracle use the /etc/mtab instead of using oranfstab file:

Enable the Direct NFS Client ODM Library


[root@collabn1 /]# cat /etc/mtab | grep nfs
sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw 0 0
nfsd /proc/fs/nfsd nfsd rw 0 0
localhost:/localfs /nfs nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,vers=3,timeo=600,noac,addr=127.0.0.1 0 0


cd $ORACLE_HOME/lib

mv libodm11.so libodm11.so_bak

ln -s libnfsodm11.so libodm11.so

[oracle@collabn1 lib]$ ls -lrt libodm11.*
-rw-r--r-- 1 oracle oinstall 7442 Aug 14  2009 libodm11.a
lrwxrwxrwx 1 oracle oinstall   12 Aug  1 17:53 libodm11.so_bak -> libodmd11.so
lrwxrwxrwx 1 oracle oinstall   14 Sep 12 21:12 libodm11.so -> libnfsodm11.so

On 11.2 you can enable it with the following make command

[oracle@collabn1 lib]$ cd $ORACLE_HOME/rdbms/lib

[oracle@collabn1 lib]$ make -f ins_rdbms.mk dnfs_on
rm -f /u01/app/oracle/product/11.2.0.2/SHAIKPROD/lib/libodm11.so; cp /u01/app/oracle/product/11.2.0.2/SHAIKPROD/lib/libnfsodm11.so /u01/app/oracle/product/11.2.0.2/SHAIKPROD/lib/libodm11.so


Re-start the Database instance.

SHAIKDB>shut abort
ORACLE instance shut down.
SHAIKDB>startup
ORACLE instance started.

Total System Global Area  521936896 bytes
Fixed Size           2214936 bytes
Variable Size         494928872 bytes
Database Buffers       20971520 bytes
Redo Buffers           3821568 bytes
Database mounted.
Database opened.


View the alert_log:
[oracle@collabn1 trace]$ tail -f alert_SHAIKDB.log

Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 2.0
Sat Sep 12 21:17:46 2015
-
-
--
ALTER DATABASE OPEN
Direct NFS: attempting to mount /localfs on filer localhost defined in mtab
Direct NFS: channel config is:
    channel id [0] local [] path [localhost]
Direct NFS: mount complete dir /localfs on localhost mntport 34962 nfsport 2049
Direct NFS: NFS3ERR 1 Not owner. path localhost mntport 34962 nfsport 2049
Direct NFS: NFS3ERR 1 Not owner. path localhost mntport 34962 nfsport 2049


SHAIKDB>select ts.name tsname,df.name dfname,bigfile,status from v$tablespace ts,v$datafile df where df.ts#=ts.ts#;

TSNAME       DFNAME                   BIG STATUS
---------- ---------------------------------------- --- -------
SYSTEM       /u01/app/oracle/SHAIKDB/system01.dbf     NO    SYSTEM
SYSAUX       /u01/app/oracle/SHAIKDB/sysaux01.dbf     NO    ONLINE
UNDOTBS1   /u01/app/oracle/SHAIKDB/undotbs01.dbf    NO    ONLINE
USERS       /u01/app/oracle/SHAIKDB/users01.dbf        NO    ONLINE
NFSTBS       /nfs/nfstbs01.dbf               NO    ONLINE




SHAIKDB>col svrname for a10
SHAIKDB>col dirname for a10

SHAIKDB>select * from v$dnfs_servers;

    ID SVRNAME    DIRNAME        MNTPORT    NFSPORT        WTMAX      RTMAX
---------- ---------- ---------- ---------- ---------- ---------- ----------
    1 localhost  /localfs          34962      2049       0       0


SHAIKDB>select * from v$dnfs_files;

no rows selected

SHAIKDB>create tablespace DNFS datafile '/localfs/dnfs01.dbf' size 100m;

Tablespace created.

SHAIKDB>select * from v$dnfs_files;

no rows selected

SHAIKDB>create table dnfs (col1 number) tablespace dnfs;

Table created.



SHAIKDB>begin
 2  for i in 1..10000 loop
 3   insert into dnfs values (i);
 4   commit;
 5  end loop;
 6  end;
 7  /

PL/SQL procedure successfully completed.


SHAIKDB>select count(*) from dnfs;

 COUNT(*)
----------
    10000

SHAIKDB>select ts.name tsname,df.name dfname,bigfile,status from v$tablespace ts,v$datafile df where df.ts#=ts.ts#;

TSNAME       DFNAME                   BIG STATUS
---------- ---------------------------------------- --- -------
SYSTEM       /u01/app/oracle/SHAIKDB/system01.dbf     NO    SYSTEM
SYSAUX       /u01/app/oracle/SHAIKDB/sysaux01.dbf     NO    ONLINE
UNDOTBS1   /u01/app/oracle/SHAIKDB/undotbs01.dbf    NO    ONLINE
USERS       /u01/app/oracle/SHAIKDB/users01.dbf        NO    ONLINE
NFSTBS       /nfs/nfstbs01.dbf                   NO    ONLINE
DNFS       /localfs/dnfs01.dbf                   NO    ONLINE

6 rows selected.

More info DNFS:
Step by Step - Configure Direct NFS Client (DNFS) on Linux (11g) (Doc ID 762374.1)
http://docs.oracle.com/cd/E11882_01/install.112/e22489/storage.htm#CWLIN276 

No comments: