• Veritas Cluster
• SysAdmin Pocket Survival Guide
• AIX Administration
• LPARS
• NIM
• Maximo Package Deployment

• AIX LVM
• PowerVM Virtualization on IBM System p
• IBM VIO
• Advanced Solaris 10
• Apache Cheat Sheet

• Linux Cheat Sheet
• VMWare Cheat Sheet
• SAN Storage
• VxVM EMC Cheat Sheet
• MDL Install WebLogic

• Hitachi SAN
• IBM DS Storage Tuning Guide
• Network Appliance Setup Guide
• NetApp On Solaris
• Htaccess Cheat Sheet

• HP-UX Administration
• wget
• Sun Packaging
• SQL Cheatsheet
• Veritas Cheat Sheet

===============================
Veritas Cluster
===============================

• Veritas Cluster Administration Guide

===============================
SysAdmin Pocket Survival Guide
===============================

• SysAdmin Pocket Survival Guide

===============================
AIX Administration /////////////////////////
===============================

• AIX Administration 101
• AIX CD Catalogue

===============================
LPARS /////////////////////////
===============================

===============================
NIM /////////////////////////
===============================

Commands Reference, Volume 4, n - r

nim Command

Purpose

Performs operations on Network Installation Management (NIM) objects.

Syntax

nim { -o Operation} [ -F ] [ -t Type ] [ -a Attribute=Value . . . ] {ObjectName}

Description

The nim command performs an operation on a NIM object. The type of operation performed is dependent on the type of object specified by the ObjectName parameter. Possible operations include initializing environments and managing resources. You can use the lsnim command to display the list of supported operations.

Flags

Security

Access Control: You must have root authority to run the nim command.

Examples

The following examples are grouped by operation.

activate

1. To start the managed wpar1 workload partition, type: nim -o activate wpar1
2. To start the managed wpar1 workload partition with additional startwpar command flags to start with verbose output, type:
   
   nim -o activate -a cmd_flags="-v" wpar1

allocate

1. To allocate resources to a diskless workstation with the name of syzygy and SPOT attribute value of spot1, type:

   nim -o allocate -a spot=spot1 syzygy

2. To perform a base system installation on the machine named krakatoa, resources must first be allocated by typing:
   
   nim -o allocate -a spot=myspot -a lpp_source=images krakatoa

   Then the NIM environment can be initialized to support the install by performing the bos_inst operation, type:

   nim -o bos_inst krakatoa

3. To install the software product, adt, into a standalone machine, stand1, given that the installable option, adt, resides in the lpp_source, images, type:

   nim -o allocate -a lpp_source=images stand1

   Then type:

   nim -o cust -a filesets="adt" stand1

4. To install software products into a standalone machine, stand1, such that the image for the installable option, adt,
   resides in the lpp_source, images, and the installp_bundle, bundle1, contains the name of the installable option, type:

   nim -o allocate -a lpp_source=images \
   -a installp_bundle=bundle1 stand1

   Then type:

   nim -o cust stand1

5. To automatically configure a machine with name resolution services after a BOS installation,
   create the file /exports/resolv.conf, with contents similar to the following:

   nameserver 129.35.143.253
   nameserver 9.3.199.2
   domain austin.ibm.com

   then type:

   nim -o define -t resolv_conf -a location=/exports/resolv.conf \
   -a server=master rconf1

   Prior to issuing the bos_inst operation, allocate this resource with other required and optional resources by typing:

   nim -o allocate -a spot=spot1 -a lpp_source=images1 \
   -a bosinst_data=bid1 -a resolv_conf=rconf1 client1

6. To allocate all resources applicable to standalone machines from the NIM resource group res_grp1, to the machine mac1, type:

   nim -o allocate -a group=res_grp1 mac1

alt_disk_install

1. To install a mksysb resource all_devices_mysysb to client roundrock, on hdisk4 and hdisk5, using the image_data resource image_data_shrink, with
   debug turned on, type:

   nim -o alt_disk_install -a source=mksysb\
   -a image_data=image_data_shrink\
   -a debug=yes\
   -a disk='hdisk4 hdisk5' roundrock

2. To clone a rootvg on client austin to hdisk2, but only run phase1 and phase2 (leaving the /alt_inst file systems mounted), type:

   nim -o alt_disk_install -a source=rootvg\
   -a disk='hdisk2'\
   -a phase=12 austin

bos_inst

1. To install the machine blowfish, using the resources spot1, images1, bosinst_data1, and rconf1, first allocate the resources by typing:
   

   nim -o allocate -a spot=spot1 -a lpp_source=images1 \
   -a bosinst_data=bosinst_data1 -a resolv_conf=rconf1 blowfish

   Then, perform the BOS installation by typing:

   nim -o bos_inst blowfish

2. To install the machine blowfish while allocating the resources spot1, images1, bosinst_data1, and rconf1 automatically when the bos_inst operation starts, type:

   nim -o bos_inst -a spot=spot1 -a lpp_source=images1 \
   -a bosinst_data=bosinst_data1 -a resolv_conf=rconf1 blowfish

3. To use the default resources when installing the machine mac1, type:

   > nim -o bos_inst mac1

4. To install a machine, deadfish, with spot1 and lpp_source1 and use an adapter_def resource, adapter_def1, to configure secondary adapters, type:

   nim -o bos_inst -a spot=spot1 -a lpp_source=lpp_source1 \
   -a adapter_def=adapter_def1 deadfish

5. To install the machine blowfish and accept software license agreements, type:

   nim -o bos_inst -a spot=spot1 -a lpp_source=images1 \
   -a accept_licenses=yes -a resolv_conf=rconf1 blowfish

change

1. Machines on the BLDG905 network use the gateway905 gateway to reach the OZ network. Machines on the OZ network use the gatewayOZ gateway to reach the BLDG905 network. To add a route between two networks named BLDG905 and OZ, type:

   nim -o change -a routing1="OZ gateway905 gatewayOZ" BLDG905

2. The adapter identified by the host name sailfish2.austin.ibm.com is attached to a token ring network. To define a secondary interface for this adapter on the NIM master and instructing NIM to locate the NIM network representing the attached ethernet network and, if not found, have NIM define a NIM network with subnetmask 255.255.255.128, type:

   nim -o change -a if2="find_net sailfish2.austin.ibm.com 0" \
   -a net_definition="tok 255.255.255.128" -a ring_speed2=16 master

   Note: A default name is generated for the network, and no routing information is specified for the new network.

3. To define default routes for the networks net1 and net2 that use default gateways gw1 and gw2 respectively, type the following two commands:

   nim -o change -a routing1="default gw1" net1 nim -o change -a routing1="default gw2" net2

4. To designate the resources defined by the resource group res_grp1 as the set of resources always allocated by default during any operation in which these resources are applicable, type:

   nim -o change -a default_res=res_grp1 master

check

1. To have NIM check on the usability of a SPOT named myspot, type:

   nim -o check myspot

2. To check the status of an lpp_source named images, type:

   nim -o check images

chwpar

nim -o chwpar -a cmd_flags="-R rset=rs/cpu23" wpar1

create

1. To create the wpar1 workload partition with host name and specification file resource basic_wpar, type:

   nim -o create -a wpar_spec=basic_wpar wpar1

2. To create the wpar1 workload partition with the wpar-specification file resource wpar1_spec, type:

   nim -o create -a wpar_spec=wpar1_spec wpar1

3. To create the wpar1 workload partition from the savewpar backup image resource wpar1_backup, type:

   nim -o create -a savewpar=wpar_backup wpar1

cust

1. To install a software product into a spot, spot1, such that the image for the installable option, adt, resides in the lpp_source, images, type:

   nim -o cust -a lpp_source=images -a filesets=adt spot1

2. To install a software product into a spot, spot1, such that the image for the installable option, adt, resides in the lpp_source, images, and the installp_bundle, bundle1, contains the name of the installable option, type:

   nim -o cust -a lpp_source=images -a installp_bundle=bundle1 spot1

3. To install a software product into a spot, spot1, such that the image for the installable option, adt, resides on a tape that is in the tape drive that is local to the machine where the spot resides, type:

   nim -o cust -a lpp_source=/dev/rmt0 -a filesets=adt spot1

4. To install a software product into a spot, spot1, such that the image for the installable option, adt, resides on a tape that is in the tape drive that is local to the machine where the spot resides, type:

   nim -o cust -a lpp_source=/dev/rmt0 -a filesets=adt spot1

5. To install all fileset updates associated with APAR IX12345, residing on the tape /dev/rmt0 into spot1 and any diskless or dataless clients to which spot1 is currently allocated, type:

   nim -F -o cust -afixes=IX12345 -a lpp_source=/dev/rmt0 spot1

6. To update all software installed on the client Standalone1, with the latest updates in the lpp_source named updt_images, type:

   nim -o allocate -a lpp_source=updt_images Standalone1 nim -o cust -afixes=update_all Standalone1

7. To install the machine catfish with the contents of the installp_bundle bundle1, first allocate the resources by typing:

   nim -o allocate -a installp_bundle=bundle1 \
   -a lpp_source=images1 catfish

   Then, perform the cust operation by typing:

   nim -o cust catfish

    

8. To update all software installed on the client Standalone1, with the latest updates in the lpp_source named updt_images, type:

   nim -o cust -a lpp_source=updt_images -a fixes=update_all \
   Standalone1

9. To install the machine catfish with the contents of the installp_bundle bundle1, while allocating this resource and the lpp_source images1 when the cust operation runs, type:

   nim -o cust -a installp_bundle=bundle1 -a lpp_source=images1 \
   catfish

10. To configure secondary adapters on a client machine, deadfish, using the secondary adapter configuration file in the adaper_def resource, adapter_def1, type:

    nim -o cust -a adapter_def=adapter_def1 deadfish

deactivate

1. To stop the managed wpar1 workload partition, type:

   nim -o deactivate wpar1

2. To force the stop of the managed wpar1 workload partition, type:

   nim -Fo deactivate wpar1

3. To stop the managed wpar1 workload partition with additional stopwpar command flags to halt after 85 seconds, type:

   nim -o deactivate -a cmd_flags="-t 85" wpar1

deallocate

To deallocate an lpp_source named images from the standalone machine client1, type:

nim -o deallocate -a lpp_source=images client1

define

1. To define an rspc uniprocessor dataless machine for AIX 5.1 and earlier on the token-ring network called net1 and call it altoid, type:

   nim -o define -t dataless -a if1="net1 fred 10005aa88500" \
   -a ring_speed=16 -a platform=rspc -a netboot_kernel=up \
   -a comments="Dataless client altoid"

   The comments attribute is optional and may contain any user-entered notes.

   Note: The if1 attribute is required.

2. To define a resource that is a directory containing installable images that is located on server altoid and has a path name of /usr/sys/inst.images, and name that resource images, type:

   nim -o define -t lpp_source -a server=altoid \
   -a location=/usr/sys/inst.images images

3. To create a new SPOT resource named myspot on the NIM master in the /export/exec directory, using an lpp_source named images, type:

   nim -o define -t spot -a server=master -a location=/export/exec \
   -a source=images myspot

4. To define a network object named BLDG905, with a subnetmask of 255.255.240.0 and an address of 129.35.129.0, type:

   nim -o define -t tok -a snm=255.255.240.0 \
   -a net_addr=129.35.129.0 BLDG905

5. To define an lpp_source, lppsrc1, that will be located on the master from a tape selecting a specific set of software products that are on the tape, bos.INed and bos.adt, type:

   nim -o define -t lpp_source -a location=/images2/lppsrc1 \
   -a source=/dev/rmt0 -a server=master -a packages="bos.INed \ bos.adt" lppsrc1

6. To define a standalone machine that is a Symmetrical Multi-Processor model for AIX 5.1 and earlier that has a BOOTP-enabled IPL ROM, its hostname is jupiter and it will also be known as jupiter in the NIM environment, using a token-ring network called net1 and a ring speed of 16, type:

   nim -o define -t standalone -a if1="net1 jupiter 0" \
   -a ring_speed=16 -a platform=rs6ksmp jupiter

7. To define a mksysb resource, mksysb1, from an existing mksysb image located in /resources/mksysb.image on the master, type:

   nim -o define -t mksysb -a server=master \
   -a location=/resources/mksysb.image mksysb1

8. To define a NIM network named ATMnet with a subnet mask of 255.255.240 and an address of 129.35.101.0 to represent an ATM network, use the generic network type as follows:

   nim -o define -t generic -a snm=255.255.240.0 \
   -a net.addr=129.35.101.0 ATMnet

9. To define a PowerPC PCI bus-based, symmetric multiprocessor computer for AIX 5.1 and earlier whose hostname is bluefish as a standalone machine on a token ring network called net1 and have the machine be known to the NIM environment as bluefish, type:

   nim -o define -t standalone -a platform=rspcsmp \
   -a if1="net1 bluefish 0" -a ring_speed=16 bluefish
   

10. To define a machine group named DisklsMacs1 with members that are NIM diskless machines named diskls1, diskls2, and diskls3, type:

    nim -o define -t mac_group -a add_member=diskls1 \
    -a add_member=diskls2 -a add_member=diskls3 DisklsMacs1

11. To define a resource group named DisklsRes1 with resources spot1, root1, dump1, paging1, home1, tmp1, type:

    nim -o define -t res_group -a spot=spot1 -a root=root1 \
    -a dump=dump1 -a paging=paging1 -a home=home1 -a tmp=tmp1 \
    DisklsRes1

12. To display the space required to define a mksysb resource, mksysb2, and create a mksysb image of the client, client1, during the resource definition where the image will be located in /resources/mksysb.image on the master, type:

    Note: This action only shows the space required for the operation, mksysb or resource creation does NOT take place.

    nim -o define -t mksysb -a server=master \
    -a location=/resources/mksysb.image -a source=client1 \
    -a mk_image=yes -a size_preview=yes mksysb2

13. To define a mksysb resource, mksysb2, and create the mksysb image of the client, client1, during the resource definition where the image will be located in /resources/mksysb.image on the master, type:

    nim -o define -t mksysb -a server=master \
    -a location=/resources/mksysb.image -a source=client1 \
    -a mk_image=yes mksysb2
    

14. To define a mksysb resource, mksysb2, and create a mksysb image of the client, client1, during the resource definition where the mksysb flags used to create the image are -em, and the image will be located in /resources/mksysb.image on the master, type:

    nim -o define -t mksysb -a server=master \
    -a location=/resources/mksysb.image -a source=client1 \
    -a mk_image=yes -a mksysb_flags=em mksysb2

15. To define an exclude_files resource, exclude_file1, located in /resources/exclude_file1 on the master, type:

    nim -o define -t exclude_files -a server=master \
    -a location=/resources/exclude_file1 exclude_file1

16. A machine called redfish, hostname redfish_t.lab.austin.ibm.com, has its primary interface attached to a token-ring network with ring speed of 16 Megabits. To define redfish as a standalone machine in the NIM environment and instructing NIM to locate the name of the network that the machine's primary interface is attached, type:

    nim -o define -t standalone -a if1="find_net \
    redfish_t.lab.austin.ibm.com 0" -a ring_speed1=16 redfish

17. A machine called bluefish, hostname is bluefish_e.lab.austin.ibm.com, has its primary interface attached to an ethernet network with cable_type of bnc. To define bluefish as a diskless machine in the NIM environment and instructing NIM to locate the name of the network that the machine's primary interface is attached, and if not found, have NIM define a NIM network with the name ent_net, subnetmask of 255.255.255.128 and default route using the gateway with hostname lab_gate, type:

    nim -o define -t diskless -a if1="find_net \
    bluefish_e.lab.austin.ibm.com 0" -a net_definition="ent \
    255.255.255.128 lab_gate 0 ent_net" -a cable_type=bnc bluefish

    Note: Specify 0 in place of the master gateway in the net_definition attribute if a default route for the master already exists, otherwise you must specify the master gateway.

18. To define the /export/nim/adapters directory as an adapter_def resource, adapter_def1, located on the master, type:

    nim -o define -t adapter_def -a server=master \
    -a location=/export/nim/adapters adapter_def1

    To populate the adapter_def resource with secondary adapter configuration files, run the nimadapters command.
19. To display the space required to define a savevg resource, savevg2, and create a savevg image of the client, client1, during the resource definition where the image will be located in /export/nim/savevg on the master and the volume_group to to backup is myvg, type:

    nim -o define -t savevg -a server=master \
    -a location=/export/nim/savevg/savevg2 -a source=client1 \
    -a mk_image=yes -a size_preview=yes -a volume_group=myvg savevg2

    Note:
    This action only shows the space required for the operation. savevg or resource creation does not take place.
20. To define a savevg resource, savevg2, and create the savevg image of the client, client1, during the resource definition where the image will be located in /export/nim/savevg on the master and the volume_group to backup is myvg, type:

    nim -o define -t savevg -a server=master \
    -a location=/export/nim/savevg -a source=client1 \
    -a mk_image=yes -a volume_group=myvg savevg2

21. To define a savevg resource, savevg2, and create a savevg image of the client, client1, during the resource definition where the savevg flags used to create the image are -em, and the image will be located in /export/nim/savevg on the master, type:

    nim -o define -t savevg -a server=master \
    -a location=/export/nim/savevg -a source=client1 \
    -a mk_image=yes -a volume_group=myvg -a savevg_flags=em savevg2

22. To define a vg_data resource, my_vg_data, located on the master at the location /export/nim, type:

    nim -o define -t vg_data -a server=master -a location=/export/nim/my_vg_data my_vg_data

23. To define the wpar1 workload partition managed by the yogi managing standalone machine with wpar1 as both the host name and the name of the workload partition on the managing system, type:

    nim -o define -a mgmt_profile1="yogi wpar1" -a if1="find_net wpar1 0" wpar1

24. To define a savewpar resource named wpar1backup and create the savewpar image of the yogi workload partition on the sterling server, type:

    nim -o define -t savewpar \
    -a server=sterling -a location=/resources/wpar1.image \
    -a source=wpar1 -a mk_image=yes wpar1backup

25. To define a savewpar resource named wpar1backup and create the savewpar image of the yogi workload partition on the sterling server, excluding file patterns in the exclude_files resource wparexclude, and passing the flag to the savewpar resource to exclude files and create a new image.data file, type:

    nim -o define -t savewpar \
    -a server=sterling -a location=/resources/wpar1.image -a source=wpar1 \
    -a exclude_files=wparexclude -a cmd_flags="-ei" mk_image=yes wpar1backup

destroy

1. To remove the managed wpar1 workload partition from its managing system, type: nim -o destroy wpar1
2. To force the removal of the managed wpar1 workload partition, type:

   nim -Fo destroy wpar1

dkls_init

1. To initialize the environment for a diskless workstation with the name of syzygy, using the resources spot1, root1, dump1, and paging1, first allocate the resources by typing:

   nim -o allocate -a spot=spot1 -a root=root1 -a dump=dump1 \
   -a paging=paging1 syzygy

   Then initialize the resources for the client machine by typing:

   nim -o dkls_init syzygy

2. To initialize the environment for a diskless workstation with the name of syzygy, type:

   nim -o dkls_init syzygy

3. To exclude the member named diskls2 from operations on the machine group DisklsMacs1, and then initialize the remaining members while allocating the diskless resources defined by the resource group named DisklsRes1, type the following two commands:

   nim -o select -a exclude=diskls2 DisklsMacs1 nim -o dkls_init -a group=DisklsRes1 DisklsMacs1

4. To initialize the group of diskless machines defined by the machine group dtgrp1, while allocating the required and optional resources defined by the resource group dk_resgrp1, when the dkls_init operation runs, type:

   nim -o dkls_init -a group=dtgrp1 dk_resgrp1

dtls_init

1. To initialize the environment for a dataless workstation with the name of syzygy, using the resources spot1, root1, and dump1, first allocate the resources by typing:

   nim -o allocate -a spot=spot1 -a root=root1 -a dump=dump1 syzygy

   Then initialize the resources for the client machine by typing:

   nim -o dtls_init syzygy

2. To initialize the environment for a dataless workstation with the name of syzygy, type:

   nim -o dtls_init syzygy

3. To exclude the member named dataless1 from operations on the machine group DatalsMacs1, and then initialize the remaining members while allocating the dataless resources defined by the resource group named DatalsRes1, type the following two commands:

   nim -o select -a exclude=datals2 DatalsMacs1 nim -o dtls_init -a group=DatalsMacs1 DatalsRes1

4. To initialize the group of dataless machines defined by the machine group DatalsMacs1, while allocating the required and optional resources defined by the resource group DatalsRes1, when the dtls_init operation runs, type:

   nim -o dtls_init -a group=DatalsMacs1 DatalsRes1

fix_query

To list information about fixes installed on client Standalone1 for 20 APAR numbers, create the file /tmp/apar.list with one APAR number per line, as shown:

IX123435 IX54321 IX99999 ...

then type:

nim -o define -t fix_bundle -alocation=/tmp/apar.list \
-aserver=master fix_bun nim -o allocate -a fix_bundle=fix_bun Standalone1 nim -o fix_query Standalone1

lppchk

1. To check fileset version and requisite consistency on the SPOT spot1, type:

   nim -o lppchk spot1
   

2. To verify the file checksums for all packages beginning with the name bos on NIM targets in the group of standalone machines macgrp1, and displaying detailed error information and updating the software database to match the actual file checksum when inconsistencies are found, type:

   nim -o lppchk -a lppchk_flags='-c -m3 -u' \
   -a filesets='bos*' macgrp1

   Because the lppchk operation runs in the background on group members by default, to view the output from the lppchk operation type:

   nim -o showlog -a log_type=lppchk macgrp1

lppmgr

1. To list the names of duplicate base level filesets which should be removed from lpp_source1 with space usage information, type:

   nim -o lppmgr -a lppmgr_flags="-lsb" lpp_source1

2. To remove duplicate base and update filesets and superseded updates from lpp_source1, type:

   nim -o lppmgr -a lppmgr_flags="-rbux" lpp_source1

3. To remove all non-SIMAGES (filesets not required for a bos install) from lpp_source1, type:

   nim -o lppmgr -a lppmgr_flags="-rX" lpp_source1

4. To remove all language support except 'C' from lpp_source1, type:

   nim -o lppmgr -a lppmgr_flags="-r -k C" lpp_source1

lswpar

1. To list the characteristics of the managed wpar1 workload partition, type:

   nim -o lswpar wpar1

2. To list the network characteristics of the managed wpar1 workload partition, type:

   nim -o lswpar -a cmd_flags="-N" wpar1

3. To list the general characteristics of the workload partitions managed by the global1 standalone system, type:

   nim -o lswpar -a cmd_flags="-G" global1

maint

1. To deinstall the software products bos.INed and adt from a spot, spot1, type:

   nim -o maint -a installp_flags="-u" \
   -a filesets="bos.INed adt" spot1

2. To deinstall the options bos.INed and adt from a spot, spot1, such that the installp_bundle, bundle2, contains the names of the installable options, type:

   nim -o maint -a installp_flags="-u" \
   -a installp_bundle=bundle2 spot1

3. To cleanup from an interrupted software installation on a spot, spot1, type:

   nim -o maint -a installp_flags="-C" spot1

4. From the master, to deinstall the software products bos.INed and adt from a standalone machine, stand1, type:

   nim -o maint -a installp_flags="-u" \
   -a filesets="bos.INed adt" stand1

5. From the master, to clean up from an interrupted software installation on a standalone machine, stand1, type:

   nim -o maint -a installp_flags="-C" stand1

6. From the master, to deinstall the software products bos.INed and adt from a standalone machine, stand1, such that installp_bundle, bundle2, contains the names of the installable options, type:

   nim -o maint -a installp_flags="-u" \
   -a installp_bundle=bundle2 stand1

maint_boot

To enable the NIM standalone client, stand1, to boot in maintenance mode, type:

nim -o maint_boot stand1

This sets up the maintenance boot operation, but you must initiate the network boot locally from stand1.

remove

To remove a resource named dump_files, type:

nim -o remove dump_files

showlog

To view the boot logs of the machines defined by the group DisklsMacs1, type:

nim -o showlog -a log_type=boot DisklsMacs1

showres

1. To show the contents of the config script script1, type:

   nim -o showres script1

2. To show the contents of the bosinst.data resource bosinst_data1, type:

   nim -o showres bosinst_data1

3. To list all the filesets in the lpp_source lpp_source1, type:

   nim -o showres lpp_source1

4. To list all the filesets in the lpp_source lpp_source1 relative to what is currently installed on the machine machine1, type:

   nim -o showres -a reference=machine1 lpp_source1

5. To list user instructions for the bos.INed and xlC.rte filesets on the lpp_source lpp_source1, type:

   nim -o showres -a filesets="bos.INed xlC.rte" \
   -a installp_flags="qi" lpp_source1

6. To list all problems fixed by software on the lpp_source lpp_source1, use:

   nim -o showres -a instfix_flags="T" lpp_source1

7. To show the contents of the secondary adapter configuration file in the adapter_def resource, adapter_def1, for client, deadfish, type:

   nim -o showres -a client=deadfish adapter_def1

8. To show the contents of every secondary adapter configuration file in the adapter_def resource, adapter_def1, type:

   nim -o showres adapter_def1

9. To show the contents of the savevg resource, savevg1, type:

   nim -o showres savevg1

syncwpar

1. To synchronize the software of the managed wpar1 workload partition with its managing system, type:

   nim -o syncwpar wpar1

2. To synchronize the software of all the workload partitions managed by the global1 standalone system, type:

   nim -o syncwpar -a cmd_flags="-A" global1

update

1. To add all the filesets on /dev/cd0 to lpp_source1, type:

   nim -o update -a packages=all -a source=/dev/cd0 lpp_source1

2. To add the bos.games 5.2.0.0 and bos.terminfo filesets to lpp_source1, type:

   nim -o update -a packages="bos.games 5.2.0.0 bos.terminfo" \
   -a source=/dev/cd0 lpp_source1

3. To remove bos.games from lpp_source1, type:

   nim -o update -a rm_images=yes -a packages="bos.games" lpp_source1

4. To recover the missing SIMAGES for lpp_source1 from the AIX Installation CD, type: nim -o update -a recover=yes -a source=/dev/cd0 lpp_source1

Files

/etc/niminfo

Contains variables used by NIM.

Related Information

The lsnim command, nimadapters command, nimclient command, nimconfig command, niminit command.

The .info file.

 

===============================
PowerVM Virtualization on IBM System p
===============================

 

IBM POWER5TM systems combine enhancements in the IBM PowerPCTM processor architecture with greatly enhanced firmware to significantly increase the virtualization capabilities of IBM POWERTM servers. The POWER hypervisor, the basis of the IBM Virtualization EngineTM technologies on POWER5 systems, delivers leading-edge mainframe virtualization technologies to the UNIX® marketplace. In addition to being able to create computing-intensive partitions with dedicated resources (processors, memory, and I/O adapters), customers can harness idle processor capacity to configure micropartitions with virtualized resources in order to consolidate many AIXTM, i5/OSTM, and Linux® servers onto a single platform. The POWER hypervisor provides support for virtualized processors, an IEEE virtual local area network (VLAN)- compatible virtual Ethernet switch, virtual small computer system interface (VSCSI) adapters, and virtual consoles. Many of these features are dependent upon, or take advantage of the new facilities provided in the POWER5 processor, including the hypervisor decrementer, a fast page mover, and simultaneous multithreading support. The technology behind the virtualization capabilities that are available on the POWER5 servers, enabling customers to better utilize the industry-leading computing capacity of the POWER5 processor, is discussed in this paper.

PowerVM Virtualization on System p.pdf
systems_power_hardware_blades_supported_environments.pdf
systems_power_hardware_blades_i_on_blade_readme.pdf
systems_i_Virtualization_Open_Storage.pdf

 

===============================
AIX LVM ///////////////////////////////////////////////
===============================

 

LVM Theory Physical Volume = PV is IBM speak for a disk (it could be worse some IBMers still refer to DASD! (Dyanical Access Storage Device which is a main frame term). They are: Named by AIX as hdisk0, hdisk1, hdisk2, ... Regardless of the underlying technology (SCSI, SSA sort of IBM's early SAN, SAN, RAID5 using the adapter) Disk and AIX automatic bad block reallocation Volume Group = VG is IBM speak for a group of disks Volume Group operations: Disk space always allocated within single VG All disks available in AIX or none - work as a group Can be exported to be attached to other AIX - allows High Availability HACMP First VG called rootvg Root Volume Group (rootvg) is created automatically while installing AIX is placed within this VG AIX files Initial paging Usually only the first disk Or two, to allow mirroring of rootvg Often internal disks Recommend: Keep to a small number of disks Other Volume Groups Other VG created by the System Admin. Name typically related to their use Optionally ending in "vg" Volume Groups One or more disks Can later add a disk to a VG Can remove disk from VG - if empty Disks within VG cut into chunks called Physical Partitions (PP) Minimum allocation unit Typically, 8, 16 or 32 MB (older 2GB disks = 4 MB) Nothing smaller can be allocated Each VG fixed PP size. Volume group has total, free and used PPs *Disks do not need to be the same size or even the same technology Logical Volumes It is a large piece of disk Can be used for lots of tasks like raw device or file system or paging space Created within single VG LV has many attributes like: can be one disk or more disks or all disks of VG mirrored or striped particular part of disk Sizes is in Physical Partitions - example: id PP = 16 MB and LV size of 64 PPs = 1 GB PPs allocated by policies min or max spread across disks min=fill disk then move on, max=evenly spread disk placement centre, middle, edge mirrors via extra copies stripes via stripe width MWC (Mirrored write consistancy) and others Logical Volumes Admin Can be grown in size Can be removed Can be moved to different disks within VG Can later have a filesystem created in it Attributes can be changed like: mirror added or removed or policy changed How AIX/LVM sees the underlying disks LVM in practice The disks can be apprached from three different directions and from there you can see the relationship to the others. As a reminder pv = physical volumes - real disk or LUN lv = logical volume vg volume group To the this use the ls<thing> commands: lspv lslv lsvg In more detail Lists volume group names: lsvg List VG details: lsvg <vgname> List LVs within VG: lsvg -l <vgname> List PV within VG: lsvg -p <vgname> Lists physical group names: lspv List PV details: lspv hdiskN List LVs within PV: lspv -l hdiskN List PP within PV: lspv -p hdiskN Lists LV names: lsvg -l <vgname> List LV details: lslv <lvname> List LVs within PV: lslv -l <lvname> List PP within PV: lslv -p <pvname> Show mirrored LV: lslv -m <lvname> When you create a Logical Volume with smitty you will see a panel like this: smitty create logical partition Add a Logical Volume   Logical volume NAME [scratch] VOLUME GROUP name testvg Number of LOGICAL PARTITIONS [64] # PHYSICAL VOLUME names [hdisk1 hdisk2 hdisk3] + Logical volume TYPE [jfs] POSITION on physical volume middle + RANGE of physical volumes minimum + MAXIMUM NUMBER of PHYSICAL VOLUMES [] # to use for allocation   Number of COPIES of each logical partition 2 + partition Mirror Write Consistency? yes + Allocate each logical partition copy on a SEPARATE physical volume? yes + RELOCATE the logical volume during reorganization? yes + Logical volume LABEL [] MAXIMUM NUMBER of LOGICAL PARTITIONS [512] # Enable BAD BLOCK relocation? yes + SCHEDULING POLICY for reading/writing logical partition copies parallel + Enable WRITE VERIFY? no + Some of the terms used are described below: Mirrors On the disk placement Mirror Scheduling Policy Mirror Write Consistency If you are terribly and extremely unlucky ... a crash could happen leaving the two mirrored copies of a block different. Reading the block could give you the old or new block. With a RDBMS, it does not mind disk is newer but must be consist when read. MWC reminds LVM to "resilver" these recently written blocks in recovery - to make them the same. Physical Partition (PP) Allocation Logical Volumes striping From AIX 4.3.3 (not backward compatible) LVM support striping Often called fine striping (i.e. not PP level striping) Recommend 32K, 64K or 128K stripe width = 4 to 16 times I/O size most databases do 4K or 8K I/O Excellent for balancing serial and random I/O across disks Excellent read ahead serial I/O boost Using LVM commands (with smit) Find unused disks: lspv Add them to a VG or create a VG: smitty lvm + VG + Add VG Check unused space in VG: lsvg <vgname> Create a LV: smitty lvm + LV + Add LV Create a Filesystem (JFS)smitty jfs + Add jfs + standard + which VG (see below) If the command changes the VG, PV or LV then the VG is locked i.e. one command at a time. This is needed to allow recovery of the change - if the system fails during the update. Creating Stripes Tips: Stripes LV must contain a whole number of items, that is the number of PPs must divide evenly into the number of disks Or put another way: each disk have the same number of PPs Hint: use multiples of No. of disks to make this easier LVM Create Mirrors There are three ways to create a mirror: One Step = create LV with mirror These can be slow as you can have to wait until mirror complete before creating next lv Create LV and later add mirror synchronise=yes again slow as you can have to wait for silvering #Create LV and later add mirror synchronise=no and then sychronise all mirrors together Use this when creating lots of lv because all mirrors can be created at the same time So (with smit) create lv's add mirrors sync=no syncvg -l <lvname> to remove "stale" partitions = resilver Warning: resilvering hurts performance often forgotten when recommending mirror breaking for backup purposes LVM Higher Level Management Commands Disown disks ready to connect them to another system varyoffvg <vgname> exportvg <vgname> or smitty lvm Activate newly attached disks cfgmgr – finds disks importvg hdiskN – any one of the disks varyonvg <vgname> – mounts file systems Moving LVs migratelv (not striped) Use smit lvm Can migrate LV on a disk to another disk Can empty whole disk to remove it later or replac a fault disk Disks can be added and removed from a VG extendvg and reducevg Use smit lvm recommended LVM rootvg good practice Small rootvg - 1 or 2 disks (for mksysb backup) Mirror rootvg for AIX disk protection Add disk hdisk1 chvg -Qn rootvg For hd1 to hd8 and hd9var copy the LV mklvcopy hd{1,2,3,4,5,6,7,8,9var} 2 hdisk1 syncvg -v rootvg bosboot -a bootlist -m normal hdisk0 hdisk1 Or mirrorvg -s rootvg this second option is recommended TIP: This command will show you formatted list of active Volume groups for i in $(lsvg -o);do lsvg $i;done \ | awk ' BEGIN { printf("%-10s\t%10s\t%10s\t%10s\t%10s\n","VG","Total(MB)","Free","USED","Disks") };/VOLUME GROUP:/ \ { printf("%-10s\t", $3) };/TOTAL PP/ { B=index($0,"(") + 1;E=index($0," megaby");D=E-B;printf("%10s\t", substr($0,B,D) );};/FREE PP/ \ { B=index($0,"(") + 1;E=index($0," megaby");D=E-B;printf("%10s\t", substr($0,B,D) );};/USED PP/ \ { B=index($0,"(") + 1;E=index($0," megaby");D=E-B;printf("%10s\t", substr($0,B,D) );};/ACTIVE PV/ { printf("%10s\t\n", $3) } ' Put other disks into other volume groups use min PP size ( 8MB or 16 MB) use 16 to 32 disks per volume group use different vg for data that might be moved elsewhere Avoid hot disks Spread data across disk with LVs or RAID5 Mirrors add protection and performance Use striping whenever possible Think spindles not disk size reorgvg this will move physical partitions (PP) around in an attempt to get PPs in the right place migratepv this can be used to spread PPs across move disks also to remove all PPs from a suspect disk before replacement (due to errors being reported) AVOIDING Disk Crashes Rule One: don't let this stop your system RAID5 or mirror everything Rule Two: monitor error logs Make sure you know when a disk failed Rule Three: call hardware support That is what they are for Rule Four: Don't meddle Only try, if you know what you are doing Rule Five: Read and practise Get the Redbooks and try it safely

 

===============================
IBM VIO ///////////////////////////////////////////////
===============================

 

Virtual I/O (VIO) and Virtualization

Briefly What is Virtualization?

The Virtual I/O Server is part of the IBM System p5 Advanced Power Virtualization hardware feature. Virtual I/O Server allows sharing of physical resources between LPARs including virtual SCSI and virtual networking. This allows more efficient utilization of physical resources through sharing between LPARs and facilitates server consolidation.

This allows a single machine to run multiple operating system (OS) images at the same time but each is isolated from the others. POWER4 based machines started this in 2001 by allowing many Logical Partitions (LPAR) to run on the same machine using but each using different CPUs, different memory sections and different PCI adapter slots. Next came with POWER4, the ability to dynamically change the CPU, memory and PCI adapters slots with the OS running. With the introduction of POWER5 in 2005, further Virtualization items have been added:

1. Less than a whole CPU can be allocated to a LPAR
2. The LPARs sharing CPUs can be setup to allow automatic load balancing (in milliseconds)
3. The sharing of disks, networks, optical devices (CD/DVD) using a VIO Server (see below)

If you want more, try these two links:

• For the business benefits of Advanced POWER Virtualization to save time and money - take this link Business Reasons for Virtualisation
• For the technical advantages of Advanced POWER Virtualization for performance and rapid deployment - take this link Technical Reasons for Virtualisation

Briefly, What is Virtual IO (VIO)?

This is a logical partition (LPAR) on a pSeries machine controlled by the HMC or IVM that owns hardware adapters like SCSI disks, Fibre-Channel disks, Ethernet or CD/DVD optical devices but allows other LPARs to access them or a part of them. This allows the device to be shared. The LPAR with the resources is called the VIO Server and the other LPARs using it are called VIO Clients. For example, instead of each LPAR having a SCSI adapter and SCSI disk to boot from they can shared one disk on the VIO Server. This reduces costs but eliminating adapters, adapter slots and disks. This Client:Server access is implemented over memory within the machine for speed.

For a more detailed explanation, please click here: VIOS Detail

Virtualization Best Practice

To aid the sharing of good practice, hints and tip the Best Practice guide is a constant work in progress and a community based self help document. If you need some guideance take a look or if you have a really good idea that can help others please add it here.

• Virtualization Best Practice

Virtualization Back to Power Basics Movies

You can watch and download ten small movies of hands-on use of PowerVM and Virtualization on the AIX6 and POWER6 Movie AIX Wiki page at:

• http://www.ibm.com/developerworks/wikis/display/WikiPtype/Movies

Look for the Back to Power Basics Series. Each is around 10 minutes long and is a good introduction to PowerVM and the Virtual I/O Server and many of the features it provides.

Virtualization Assessments

Use these questionaires to evaluate your company and your personal readiness for Virtualizition and APV technology to reap the beneifts of:

• Reduced costs,
• Higher peak performance,
• Rapid response to business needs and
• Flexibility.
  The five stages (see below) of Virtualization have a set of questions each. Then via a simple scoring system that will award a Bronze, Silver or Gold level. The questions that you answered "no" will suggest an action list that you use to plan improvements and develop skills..
• Click on this link to take you to the questionaries Virtualization Company Assessments for Technology Deployment and developing an Action Plan
• Click on this link to take you to the questionaries Virtualization Personal Assessments for Skills Development

What to do if you want to develop your p5 Virtualisation skills?

The below diagram shows the suggested roadmap for developing Virtualisation Skills:

Each level involes back ground reading, Redbooks and the education available as highlighted below (see the table further below for the meaning of the reference numbers):

Level	Skills	What to read?	Documentation	Redbooks	Education
1	Pre-requsites				AIX 5.3 System Administration If only using Linux and IVM then just Linux System Admin If using Linux and HMC then AIX System Admin is recommended See IBM Learning Services at Curriculum
2	Virtualisation Concepts & pSeries p5 Implementation	0, 4, 9, 10, 11
3	Product information	1, 2, 3, 9			235
4a	Basic Hands-on for IVM based Virtualisation	13, 18, 19, 20	16,17	103
4b	Basic Hands-on for HMC based Virtualisation	18, 6 (IBMers only), 21	16, 17	101	231 (POWER5), 232 (POWER4)
5	Advanced Hands-on for HMC based Virtualisation includes dual VIO Server, SAN and VLAN	5, 7, 8, 12, 15, 21		102	233 (Performance), 234(underdevelopment)

Key Places For More Information and Building Virtualization Skills

We can recommend the following places:

Reference Number	Type	Name	URL	Size	Comments
1	Website	VIO Server home	Link	1 page	the Home for VIO Server
2	Website	White papers, FAQ & Data sheet	Link	3 pages	from here
3	Website	Download of the updates	Link	1 page	the most important website
4	Whitepaper	Overview of Virtualization &setting up the VIO Server	Link	21 pages	Excellent start pack example with all the details and background
5	Whitepaper	Virtualization working with VLANS	Link	18 pages	Details VLAN paper from the developer
6	Powerpoint	Power5 Virtual I/O (VIO) Options (internal to IBM only)	Link	94 pages	diagrams of the correct setups and procs+cons for each.
7	Whitepaper	POWER5 Virtual SCSI Performance Study (internal to IBM only)	Link	21 pages	A case study with vSCSI and DS4500 SAN disks
8	Whitepaper	POWER5 Processors Virtual SCSI Throughput Analysis	Link	23 pages	A case study of vSCSI and how small the overhead is.
9	Website	Advanced POWER Virtualization on eServer p5	Link	1 pages	Lots of links to other articles and websiates
10	Whitepaper	IBM p5 570 Server Consolidation Using POWER5 Virtualization	Link	8 pages	Good whitepaper on appraoching consolidation with APV
11	Whitepaper	IBM p5 570 Workload Balancing Using POWER5 Virtualization	Link	18 pages	Good worked example of what to expect
12	Whitepaper	Virtual I/O Server-Performance/Sizing/QOS considerations	Link	3 pages	Tiny font be a good review
13	Whitepaper	Use the Integrated Virtualization Manager with Linux on POWER	Link	22 pages	Blow by blow HowTo
14	Whitepaper	How to work with VLANs using the IBM Virtual I/O Server	Link	9 pages	What you really need to know to get started with VLAN and APV
15	Certification	Certified p5 Virtualization Technical Support AIX 5L V5.3	Link	1	Book you exam here.
16	Documentation	Using the Virtual I/O Server	Link	154 pages	Impossibly to find as its URL changes so often !!!!
17	Documentation	VIO Server and PLM Commands Reference	Link	174 pages	trouble is the URL keeps changing - good luck finding it !!
18	Wiki	Virtualization Best Practice	Virtualization Best Practice	6 pages	User wiki additions to the Best Practice Redbook
19	Wiki	Integrated Virtualization Manager Wiki	IVM whitpaper	16 pages	Overview and excellent summary of the features
20	Wiki	Advanced IVM Topics	Link	21 pages	Good further reading after the IVM Redbook
21	Website	The definative list of what is supported	Link	4 pages	Check here for features supported
101	Redbook	Advanced POWER Virtualization on IBM p5 Servers Introduction and Basic Configuration, SG24-7940, public includes configurations and setup examples	Link	446 pages	Excellent reference work and must read if you serious on APV
102	Redbook	Advanced Power Virtualization on IBM p5 Servers Architecture and Performance Considerations, SG24-5768, follow on Redbook with advanced topics and details	Link	436 pages	Excellent reference work
103	Redbook	Integrated Virtualization Manager with VIO Server the IVM starter pack and Handbook	Link	90 pages	Excellent start for IVM
104	Redbook	Advanced Power Virtualization Best Practice	Link	150 pages	Excellent up to date extra ideas
230	Education	AIX pSeries System Administration Curriculum	Curriculum
231	Education	System p LPAR and Virtualization I: Planning and Configuration	Go to http://www.ibm.com/training and search for AU73	5 days
232	Education	System p Virtualization and LPAR for pSeries Power4 Administrators	Go to http://www.ibm.com/training and search for AU74	3 days
233	Education	System p LPAR & Virtualization III: Virtualization Performance Mgnt including VIO Server	Go to http://www.ibm.com/training and search for AU75	5 days
234	Education	System p LPAR and Virtualization II: Implementing Adv Configurations	Go to http://www.ibm.com/training and search for AU78	4 days
235	Education	eServer, pSeries and AIX Product Overview	Go to http://www.ibm.com/training and search for AM21	2 days

For more technical details a good place to start learning is the IBM Redbooks which are freely available for download in .PDF format:

Alternative solutions:

The Linux SUSE SLES 9 operating system can also be used as a VIO Server (but with different features and not supported in the same way as the IBM VIO Server)

• VIOS using SUSE SLES9

The postings on this site solely reflect the personal views of the authors and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.

Children Hide Children | View in hierarchy

VIOS Detail (PowerVM wiki)
VIOS using SUSE SLES9 (PowerVM wiki)
Virtualization Accessments (PowerVM wiki)
Virtualization Assessments (PowerVM wiki)
Virtualization Company Assessments (PowerVM wiki)
Virtualization Personal Assessments (PowerVM wiki)

 

What is a VIO Server?

pSeries servers from IBM have, since October 2001, allowed a machine to be divided into LPARs, with each LPAR running a different OS image -- effectively a server within a server. You can achieve this by logically splitting up a large machine into smaller units with CPU, memory, and PCI adapter slot allocations.

The new POWER5 machines (pSeries p5 and OpenPower servers) can also run an LPAR with less than one whole CPU -- up to ten LPARs per CPU. So, for example, on a four CPU machine, 20 LPARs can easily be running. With each LPAR needing a minimum of one SCSI adapter for disk I/O and one Ethernet adapter for networking, the example of 20 LPARs would require the server to have at least 40 PCI adapters. This is where the VIO Server helps.

The VIO Server owns real PCI adapters (Ethernet, SCSI, or SAN), but lets other LPARs share them remotely using the built-in Hypervisor services. These other LPARs are called Virtual I/O client partitions (VIO client). And because they don't need real physical disks or real physical Ethernet adapters to run, they can be created quickly and cheaply.

VIO Server implementations

There are different VIO Server implementations:

• Both APV and AOPV versions of the VIO Server are special-purpose, single-function appliances and are not intended to run general applications.
• The Linux VIO Server for pSeries p5 or OpenPower hardware first became available with the SUSE SLES 9 distribution. Unlike the VIO Server, this is just a copy of the Linux operating system. This means it can run other central services such as NFS, network installation, DNS, an Apache Web site, or Samba services. Some care should be taken that these functions do not interfere with the performance of the VIO Server service. This software is also available on the Debian Linux for POWER distribution.

There are different implementations for VIO clients. Actually, these are just the regular operating systems, but they include the device drivers for running as a VIO client.

• AIX 5.3 (only supported by the APV or AOPV VIO Server)
• Linux -- SUSE SLES 9
• Linux -- Red Hat EL 3 update 3 onwards and Red Hat EL 4
• Linux -- Debian for POWER

This article covers the VIO Server and the AIX and Linux VIO clients.

Virtual SCSI disks

The VIO Server provides a virtual SCSI disk service, as shown in Figure 1 below.

Figure 1. Virtual SCSI disk service

With the VIO Server, data is moved directly between the VIO client partition and the real disk device using Remote DMA (RDMA) protocols. In the SUSE VIO Server implementation, data moves from the VIO client to the VIO Server, and thence to the disk. This is "double buffering".

Figure 1 shows a single VIO Server providing virtual SCSI services to multiple VIO client partitions. Each VIO client operates as if it had a dedicated SCSI device but, in fact, each client device is a real disk partition (logical disk partition) on the VIO Server. Alternatively, on the VIO Server, it could use a complete disk (hdisk). The VIO Server and VIO client communicate using the internal pSeries Hypervisor firmware (PHYP) feature, which efficiently allows disk I/O requests to be transferred between the LPARs using a message-passing protocol.

In Figure 1 above, the VIO Server has a few disks that could be SCSI or fiber channel storage area network (SAN) disks. The disk subsystem hardware or a RAID5 SCSI adapter can provide data protection. The VIO clients use the VIO client device driver just as they would a regular local device disk to communicate with the matching server VIO device driver. Then the VIO Server actually does the disk transfers on behalf of the VIO client. There is a strict client/server relationship between the VIO client and the VIO Server.

Virtual Ethernet

The LPARs in the machine can use the virtual Ethernet switch service (in the Hypervisor) in a number of different ways.

Case one: Internal only networks

You can use the Virtual Ethernet to allow TCP/IP (Transmission Control Protocol/Internet Protocol) to communicate between the LPARs, as shown in Figure 2 below. This provides high-speed data transfer without any hardware adapters starting at roughly one Gbit per second (can be much higher), especially using larger block sizes. Figure 2 also shows that there is no client/server relationship between the LPARs -- all are equally using the Virtual Ethernet. There can be many Virtual Ethernets in one machine, where groups of LPARs can communicate only within the virtual Ethernet they're connected to, allowing fast communication and complete security without buying additional Ethernet adapters, cables, hubs, or routers.

Figure 2. Virtual Ethernet -- Private/internal only networks

Case two: Routing to a physical LAN

One LPAR on the Virtual Ethernet can also communicate externally to other machines using a real physical network on behalf of all the LPARs. In this case, this special LPAR is being used to route Ethernet packets between the internal Virtual Ethernet and the external physical Ethernet network. It will work well, but involves setting up TCP/IP routes between the two networks (internal and external) and can take time to set up. Figure 3 below shows one LPAR with a real physical Ethernet adapter providing standard network routing between the two Ethernets. Note that this is not using any VIO Server features.

Figure 3. Internal Virtual Ethernet with a bridge to the external LAN

Case three: Shared Ethernet Adapter (SEA) to a physical LAN

Here, the VIO Server is being used to bridge Ethernet packets between the internal Virtual Ethernet and the external physical Ethernet network so that all the LPARs appear as regular machines on the physical network. This is simple to set up and is the option used in the example in this article. In Figure 4, the VIO Server is being used to join the two networks using the SEA. Strictly speaking, the adapter is not shared. It's owned and controlled by the VIO Server; however, it also provides shared access to the real physical network.

Figure 4. Internal Virtual Ethernet with a SEA to the external LAN

Case four: Bridging with virtual LANs (VLANs)

This particular scenario is almost the same as Case three. The only difference is the number of VLANs within the machine using Virtual Ethernet. These are connected to VLANs on the external network with a bridging LPAR and a network router that supports VLAN. This complex scenario is beyond the scope of this article, but some hints are included and it's supported.

Back to top

Why use the VIO Server?

You can use a VIO Server in any number of scenarios. Below are five typical examples that would make good use of a VIO Server.

• Small machine with limited PCI slots
  You have one set of internal SCSI disks or you can split the SCSI disks in two 4-packs on the OpenPower 720 or p5-550. This gives you two LPARs (at most) using the internal disks. So, you might run a VIO Server to support the other LPARs. For example, try a VIO Server (0.5 of a CPU) with four to six clients (0.1 to 1 CPU). Typically, clients might be small -- four to 16 GB virtual SCSI disks and one Virtual Ethernet for the whole machine. Figure 5 shows multiple LPARs running on a single disk pack.
  Figure 5. Multiple LPARs
  
• Mid-range machines with extra small workloads
  This might be an eight or 16 CPU machine with large partitions for production use. But many system administrators also want a small number of extra LPARs. Rather than buy an extra machine, a VIO Server can easily host a half dozen smaller LPARs. For example, larger production LPARs might have one to four larger dedicated CPU(s), dedicated disk I/O(s), and network(s) each.

  The VIO Server is used for "bits and bobs" LPAR like test, development, training, practice, new application trials, and so on. Typically, VIO clients might have a couple of four GB to eight GB virtual SCSI disks and one or two virtual Ethernets.

  In Figure 6, three large production LPARs are running (they would have dedicated disks and Ethernet) with a few extra small VIO clients and one VIO Server on the machine using spare capacity. This "spare" capacity could be demanded by the production LPARs during peaks in their workload.

  
  Figure 6. Three large production LPARs
  
• Ranch or server farm style
  Lots of small server consolidation workloads from smaller or older machines or many small servers are required, but they are unlikely to peak at the same time. The machine is to run lots of LPARs, for example 10 to 20 clients on a four-way machine or many times that on larger machines. Each LPAR is for small applications, but not high demand (0.2, or 0.5 CPU up to 2 CPU). This could be server consolidation or, for example, the importance of data isolation from a collection of small Web servers.

  The VIO Server has one or two CPU(s), possible RAID 5 SCSI disks, or SAN disks. Typically, clients have one or more four GB virtual SCSI disks each and might have different groups of LPARs around a different Virtual Ethernet.

  Figure 7 shows dozens of VIO clients with a medium-sized VIO Server supporting them on what might be several disk packs.

  
  Figure 7. Different groups of LPARs
  
• Serious I/O setup only once (to reduce setup and management)
  The VIO Server has SAN disks connected by two to four Fibre Channel adapters and two Ethernet adapters to run Ether channel for redundancy and additional bandwidth. The VIO Server has load balancing and failover, but VIO clients have a much simpler disk and Ethernet setup. The VIO Server could have one to three CPU(s), but the VIO clients are larger, too.

  For example, one to eight CPU(s) run quite large applications. Typically, VIO clients could have hundreds of GB of virtual SCSI disks and many Virtual Ethernets. This complex setup is not covered in this article. Figure 8 shows two regular LPARs (it would have dedicated disks) and a fully configured VIO Server (large) with multiple paths to disks and Ethernet. This is supporting some large VIO client LPARs.

  
  Figure 8. Regular LPAR
  
• Serious with high availability backup
  Same as above, but with a second VIO Server for availability/throughput. There are arguments that for very high availability you should spread your access to virtual SCSI and Virtual Ethernet across two VIO Servers in order to continue running in case one VIO Server goes down.

  The counter argument is that VIO Server is only running a few device drivers. Devices drivers are extremely reliable. Also, anything that would crash one VIO server could also crash the second one. Figure 9 shows that instead of using local physical device drivers, the VIO client uses the virtual resource device drivers to communicate with the VIO Server, which does the real I/O. Except the virtual VIO Server device drivers and the physical resource device drivers, there is very little code running on the VIO Server. Little can go wrong on the VIO Server side.

  
  Figure 9. Figure 9. VIO Server
  

I'm not going to cover duplicated VIO Servers. Further details are in the Advanced POWER Virtualization on IBM p5 Servers redbook (Download Adobe Reader).

Back to top

Prerequisites

This section describes the software, hardware, skills, and type of network you'll need.

Software

Where do I get the VIO Server?

• For a pSeries p5 machine, the software is included in the APV feature. This runs AIX and Linux VIO clients.
• For an OpenPower machine, the software is included in the AOPV feature. This will only run the Linux VIO clients (AIX does not run on these machines).

Hardware

You'll need:

• An OpenPower or pSeries p5 machine with spare resources:
  • Some CPU resources -- can be less than one CPU
  • Memory -- 512MB per LPAR (if necessary, just 256MB)
  • Real Ethernet adapter
  • Time with CD drive -- unless network installation is preferred
  • SCSI adapter and a SCSI disk -- could equally use a SAN disk
• The hardware virtualization feature activated, which is needed for LPAR and VIO Server features, but optional on some POWER5 machines.
• The VIO Server software on CD-ROM. Network Installation Manager (NIM) is possible too, but not covered here.

Skills

This article doesn't show you a screen-by-screen level of detail and each input field. It's assumed you already understand:

Though the IBM VIO Server is based on AIX, it's quite different. You only use the VIO Server commands; some standard AIX features are not present (for example, smitty) and some typical AIX setups are not allowed, such as LVM- (Logical Volume Management) level mirroring of VIO client logical volumes. Don't think of the VIO Server as AIX, but AIX experience can be useful in understanding the terms used.

• Basic AIX systems administration such as installing from an mksysb image, configuring networks, AIX style volume groups, and logical volumes terms.
• If you intend to use a Linux VIO client, then you need:
  • Basic Linux systems administration such as installing an RPM (rpm -Uvh <package>.rpm), configuring an Ethernet network adapter (ifconfig eth0 <ipaddress> mask 255.255.255.0), and managing a filesystem (mount /dev/sda5 /mnt). These tasks are identical to working on the Intel platform, and there are many books, training courses, and Internet material covering the regular system administration commands and tasks.
  • How to install SuSE Linux in either text mode (on a dumb/ASCII screen) or with a VNC (Virtual Network Computing) session. Once you've installed Linux a couple of times, this becomes a routine task. For the VNC install, the boot prompt extra command is vnc=1 password=abc123. There are six characters for the password. The system also prompts you for the other details.
• Hardware Management Console (HMC):
  • How to install the HMC hardware and software
  • How to set it up (It's assumed this has already been done.)
  • How to use the HMC to create and start a simple LPAR and its profiles
• The pSeries p5 and OpenPower range of machines internals are like the names of the adapter positions. For example, the Tn names for internal adapters and Cn for real adapters in a PCI slot. You need to create the VIO Server LPAR with the right SCSI disks and Ethernet resources on the HMC, with and without the CD. Above n is the number of the slot; details are in the hardware manuals, redbooks, or on the large sticker on the outside of the machine covers.

Network

The VIO Server must be able to communicate directly with the HMC for advanced functions and error reporting. Since this is easily forgotten, I recommend a network like the one in Figure 10.

Figure 10. The network

Many sites also have other dedicated networks in addition to the above. For example, a network for remote backup or a network dedicated for systems administration in addition to the networks in Figure 11 below.

Back to top

Getting started with VIO Server

This section covers the steps and three extra common tasks to get started with the VIO Server.

1. Step 1. Logical diagram of the example setup
2. Step 2.Planning your setup
3. Step 3. Create the SUSE SLES 9 VIO Server LPAR
4. Step 4. Install SUSE SLES 9 VIO Server
5. Step 5. HMC defining the VIO Server -- Virtual Ethernet
6. Step 6. HMC defining the VIO Server -- virtual SCSI
7. Step 7. HMC create the VIO client LPARs
8. Step 8. Clean up the HMC
9. Step 9. VIO Server preparing for the clients:
   1. Step 9a. Virtual Ethernet
   2. Step 9b.Virtual SCSI using logical volume disk partition for Client X
   3. Step 9c.Virtual SCSI using a whole disk for Client Y
10. Step 10. VIO client LPAR installations
11. Step 11. *Backing up a VIO Server and VIO client
12. Step 12. *Cloning a client
13. Step 13. *Linux dynamic LPARs (DLPARs) and RAS

*These particular tasks are useful and recommended.

Please note that it takes longer to describe some steps than to actually implement!

Back to top

1. Logical diagram of the example

Figure 11 shows the SUSE VIO Server in an LPAR and two VIO client LPARs that are going to be set up for this article. It's a logical diagram of the example setup, which is explained in the rest of this article.

Figure 11. The SUSE SLES 9 Server LPAR

Ethernet

For simplicity, the VIO client LPARs are given Ethernet IP addresses within the address range of the regular physical Ethernet network in this computer room. The VIO Server bridges between physical and virtual networks, meaning that the client LPARs will appear like any other computer to users. This is the most likely option to be implemented and hides the Virtual Ethernet network completely from users in order to allow simple access to the client LPARs.

Disks

For the disks, let's use the internal SCSI adapter in the VIO Server and one disk. The first client's (Client X) virtual disk connects to a logical volume (disk partition) on the VIO Server. The second client's (Client Y) virtual disk is supported by a whole disk on the VIO Server. This shows all of the common types of setup -- the SEA network and disk partitions, plus allocating a whole disk. In practice, most people use a logical volume.

Back to top

2. Planning your setup

First, do some planning of the VIO Server and client logical partitions. Experience has shown that just creating LPARs without some planning causes problems and can waste a lot of time. Table 1 shows the planning I've done for this example, which is an OpenPower 720. Except for the references to PCI slots like C3, T6, and T14, which are machine dependant, the references could be for any pSeries p5 or OpenPower machine. In this example, the VIO client logical partitions are going to be Linux, but equally they could be running AIX. Notes are included where AIX VIO clients would be different.

Table 1. Planning

 

	VIO Server	Client A	Client B
Hostname	op34	op36	op37
Ethernet adapter	C3 bridging	Virtual	Virtual
IP address	9.137.62.34	9.137.62.36	9.137.62.37
Virtual LAN ID (port)	1	1	1
Mask	255.255.255.0	255.255.255.0	255.255.255.0
Gateway	9.137.62.1	9.137.62.1	9.137.62.1
DNS	9.137.62.2	9.137.62.2	9.137.62.2
CD adapter	T6 for install only	T6 for install only	T6 for install only
SCSI adapter	T14	Virtual	Virtual
Disk size	hdisk0 is 36 GB	4 GB
	hdisk1 is 36 GB for client Y		73 GB
Device on VIO Server		lv00	hdisk1
Virtual SCSI adapters	Slot 3 for Client X	Slot 3 to server slot 3
	Slot 4 for Client Y		Slot 3 to server slot 4
Profile names	Normal	Normal	Normal
	Normal with CD	Normal with CD	Normal with CD
CPU values:
Dedicated/shared CPU	Shared	Shared	Shared
CPU desired	0.4	0.3	0.3
CPU min	0.2	0.1	0.1
CPU max	1	2	2
Virtual processors	1	2	2
Memory values:
Memory	512 MB	2048 MB	256 MB

Back to top

3. Creating the VIO Server LPAR

Next, you need to create the VIO Server LPAR. You do this on the HMC and create a special VIO Server LPAR, but initially with no extra virtualization features. You'll add the virtual features later. The only feature that is different from a regular Linux LPAR is the LPAR Partition Environment feature on the first panel of the Create Logical Partition Wizard. Here you must not select the AIX or Linux option, but must select VIO Server. See Figure 12 below.

Figure 12. The SUSE SLES 9 Server LPAR

Create the LPAR and the first profile as above, using the details in Table 1. (This article assumes you are familiar with the HMC and creating LPARs. If you are not, see Resources for documents that describe how to create LPARs.) I call the LPAR profile that is normally used with a name of "Normal". Further hints:

• A VIO Server LPAR can use dedicated CPUs, which is a good idea if you have plenty of CPUs or are expecting to do lots of I/O for many VIO client LPARs; it avoids any delay in starting the I/O on the real adapters. Dedicated CPUs are running the VIO Server all the time.

   

• A VIO Server LPAR can use shared CPUs, which is a good idea if you don't have whole CPUs that can be assigned. This also means unused CPU cycles are given back to the shared pool for other LPARs to use. If the machine becomes heavily loaded, it can introduce tiny delays in starting the I/O on real adapters. Shared CPU partitions are time-sliced onto the CPU, along with other LPARs. Setting the VIO Server partition to Uncapped and with a high weight is generally a good idea.

   

• A simple CPU rule of thumb: Assign at least ten percent of those CPUs to the VIO Server for CPUs that are going to be used for the VIO Server and client partitions. For example, for five CPUs in the shared pool being used for both VIO Server and VIO clients, allocate 0.5 of a CPU to the VIO Server.

   

• A simple memory rule of thumb: Use 512 MB of memory.

   

• It's recommended to have an LPAR profile with the adapter connected to the CD drive included to make installing the IBM VIO Server from CD straightforward. Copy the Normal profile and rename it Normal with CD. Then change the new profile properties to include the CD SCSI adapter. This will be used to initially boot the LPAR with a DVD/CD drive for installing the VIO Server.

   

• If this is a new machine and you are the only user, installations go much faster if you assign the LPAR a whole CPU or more. If the LPAR is going to be assigned less than this in production, it can always be reduced later, but this simple trick might save you ten minutes per LPAR installation.

Back to top

4. Installing the VIO Server

Now install the VIO Server into this partition as a recover from mksysb image methods. AIX systems administration experts will be familiar with this. The basic steps are:

• On the HMC in the Activate LPAR dialog, boot the LPAR into the System Management Services (SMS) menu by selecting both Open a Terminal and the Advanced button and then Boot Mode = SMS. Once in the SMS menus, choose the Boot Options and select Install/Boot. Then choose List all Devices and carefully select the CD-ROM drive, Normal Boot, and Yes to leave the SMS menus.

   

• Read the instructions carefully and, if free, elect to install the VIO Server on hdisk0. (This is assumed in the rest of the article.)

   

• Warning: If you want to use a whole disk for your VIO client, then you need to make sure the recovery of the VIO Server mksysb is not spread across all the disks.

   

• Assuming you now have the VIO Server up and running, you need can add and set up the VIO Server virtualization features.

Important hints

If you have the DLPAR change software installed and working, it's possible to dynamically add Virtual Ethernet and virtual SCSI. In practice, I recommend you shutdown your VIO Server and VIO client LPARs during this initial setup to make sure it works the first time. If you set up DLPAR later on, you can then experiment, but remember DLPAR changes also have to be implemented identically in your LPAR profile -- if you want the same configuration next time, you need to reboot your LPAR.

In this article, I take a simple and ultra-safe approach to shut down the VIO Server, making changes to the VIO Server profile and restarting it to avoid any confusion and complications. So on the VIO Server, use shutdown.

If you make changes to an LPAR profile, the LPAR must be shutdown and then restarted from the HMC to pick up those changes. If you use shutdown restart in the VIO Server LPAR, then you'll have only the same resources that were available when the LPAR was previously started from the HMC. You need to completely stop the LPAR to get the new resources.

Back to top

5. Defining the VIO Server -- Virtual Ethernet

On the HMC, you can now define the Virtual Ethernet. First shutdown the VIO Server LPAR (as root run: shutdown). On the HMC, change your Normal profile properties by right-clicking the Profile and selecting Properties. You also need to select the VIO tab. Select Ethernet at the bottom and click Create. By default, this will be allocated to slot number 2 and a Port virtual LAN ID of 1. Any LPAR with the same Port virtual LAN ID will be able to communicate with each other. This is going to be set up as SEA. To do this, you want to log into the VIO Server over the network and set these two options:

• Select trunk adapter.
• Leave the IEEE 802.1Q compatible adapter unchecked -- this is only needed if you are using VLANs internally.

If you want different Virtual Ethernet LANs so that different groups of LPARS can communicate with each other, all you need to do is have different Port virtual LAN ID numbers. These complex configurations are not covered in this article.

In Figure 13, you should see the VIO Server in the lower half and VIO client in the top half. It shows that if the Port virtual LAN IDs are the same, then the LPARs can communicate. It also shows the additional settings for the VIO Server. (The trunk is selected and IEEE 802.1Q is not selected.) These additional settings are really for the bridging feature, as Virtual Ethernet does not really have a client/server relationship -- all LPARs are equal on the network.

Figure 13 also shows the Virtual Ethernet settings. At the bottom is the VIO Server (or any LPAR that will be doing the bridging to the real Ethernet adapter) and at the top is the VIO client, or any LPAR that only uses the Virtual Ethernet.

Figure 13. Virtual Ethernet settings

On the other Virtual Ethernet LPARs, you can use the ifconfig command to set up your network just as you would any network. If it's AIX, you can use smitty or websm. If it's SUSE, the YAST tool can be used. It finds the Virtual Ethernet adapter just like any other regardless of the tool selection. Figure 14 shows a SUSE example.

Figure 14. Non-bridging Virtual Ethernet LPARs

Back to top

6. Defining the VIO Server -- virtual SCSI

On the HMC, you can now define the two different virtual SCSI devices. These two types of virtual disks (a logical volume or a whole hdisk) appear identical on the HMC; only on the actual VIO Server LPAR are they set up any differently.

If not done already, shutdown the VIO Server LPAR (as root run: shutdown).

• On the HMC, select the VIO Server and change your Normal profile properties by right-clicking the Profile and selecting Properties. You also need to select the Virtual I/O tab.
• Select SCSI at the bottom and click Create.
• This is the VIO Server, so select Adapter Type: Server.
• Select Any Remote Partition and Slot can connect. Ideally, this should name the specific LPAR and slot to eliminate the risk of the wrong connection between server and client, but at this point you have not created the client partition, so you can't name it yet. This is fixed up later; see Clean up the HMC section for details.
• Select OK.
• Do this a second time for the second virtual SCSI.

The client LPARs are going to use the VIO Server slots 3 and 4. Any more SCSI adapters are optional in this example. In practice, the writer typically sets up a handful of extra virtual devices, so they can be used in the future without stopping the VIO Server or having to do dynamic changes. Unused virtual adapters cost very little, so it's not a waste.

In Figure 15, you have the eventual configuration showing how the VIO Server at the bottom and client at the top both explicitly refer to each other to eliminate errors. You'll reach this configuration in Step 11. I have not covered it yet, but the client LPAR is, of course, shown here too.

Figure 15. Configuration of VIO Server and client

Back to top

7. Creating the VIO client LPARs

Now you can create the two VIO client LPARs for the two different types of virtual SCSI used in the example. It's assumed you already know the procedure to create a regular LPAR; this section covers additional things you need to consider.

This might be obvious, but you don't need real adapters for your disks or Ethernet connection because you're going to use virtual resources for these.

I recommend you install the client LPAR using CD because it's simple, so you'll want to have the CD SCSI adapter within your LPAR. Once installed, this can be removed from the LPAR profile.

It's recommended that you create two identical LPAR profiles -- one with and one without the CD. Once installed, the writer uses NFS to remotely mount a filesystem containing the AIX and Linux CDs, so you don't need the CD drive from then on to install additional LPP or RPM packages.

Add the Virtual Ethernet adapter on the VIO screen with the same Port virtual LAN ID, which is 1 in this example, but do not select the trunk or IEEE 802.1Q compatible adapter options.

Add the virtual SCSI adapter as follows:

• Set the Adapter Type: Client.
• Explicitly name the Remote Partition (the LPAR in which you have the VIO Server).
• Explicitly name the Remote Partition Virtual Slot Number, which is slots 3 for the first client LPAR (Client X) and 4 for the second client LPAR (Client Y).

Don't forget, you have two client LPARs to create with the two different SCSI Remote Partition virtual slot numbers but the same Virtual Ethernet Port virtual LAN ID.

Back to top

8. Cleaning up the HMC

Now that you've created the client LPARs, you can go back to the VIO Server LPAR and connect up the virtual SCSI adapters explicitly to their virtual client LPARs and slots. This ensures that only the right client LPAR connects to the right virtual SCSI disk. It's a safety precaution and worth doing.

Reference Figures 14 and 15 to make sure everything is in order.

On the HMC, highlight the VIO Server LPAR profile and bring up its properties. In the VIO tab, select each Server SCSI resource and the Properties button. You also need to set:

• Only selected Remote Partition and Slot connect option
• The correct Remote Partition name
• The correct Remote Partition Virtual Slot number

In this example, you have two virtual SCSI adapters on the VIO Server to "clean up". This is very easy to get wrong, and this is why I planned it in advance (see Table 1).

Back to top

9. VIO Server -- preparing for the clients

You now have all the connections set up for the VIO Server and virtual clients, but still have to connect the virtual SCSI disk to a piece of real disk space and the virtual and real Ethernets using the SEA. This is done on the VIO Server only as follows. First, start the VIO Server LPAR again from the HMC.

9a. Virtual Ethernet

Once the VIO Server is running and assuming no network is set up, you need to find out the names of the virtual resources you have to work with using:

$ lsdev -virtual name status description nt2 Available Virtual I/O Ethernet Adapter (l-lan) vhost0 Available Virtual SCSI Server Adapter vhost1 Available Virtual SCSI Server Adapter vhost2 Available Virtual SCSI Server Adapter vhost3 Defined Virtual SCSI Server Adapter vsa0 Available LPAR Virtual Serial Adapter clientY Available Virtual Target Device - Logical Volume clientZ Available Virtual Target Device - Logical Volume

To see the real adapters, use:

$ lsdev -type adapter name status description ent0 Available 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902) ent1 Available 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902) ent2 Available Virtual I/O Ethernet Adapter (l-lan) ide0 Defined ATA/IDE Controller Device lai0 Defined GXT135P Graphics Adapter sisioa0 Defined PCI-X Dual Channel U320 SCSI RAID Adapter sisioa1 Available PCI-X Dual Channel U320 SCSI RAID Adapter . . .

If your real Ethernet adapter has more than one port, it can be confusing since your Virtual Ethernet will have a higher number. In the case of a two-port Ethernet card, the Virtual Ethernet name might be en2, as en1 can be the second port on the real adapter. Also, make sure you plug in the Ethernet cable into the right port.

On your machine, the resource names might be slightly different, so be careful in following this example.

Create the SEA between the real and Virtual Ethernets with (Note: Do not type the arrow or comments when you try this.) :

$ mkvdev -sea ent0 <- this is the real Ethernet -vadapter ent2 <- this is the Virtual Ethernet -default ent2 <- this simple setup it's to only one so it's the default -defaultid 1 <- this is the Port Virtual ID from the HMC

This returned the below results:

ent3 Available en3 et3 $

And created the SEA with a name of ent3. Take a look with:

$ lsdev -dev en3 name status description ent3 Available Shared Ethernet Adapter

This new SEA adapter is used in the mktcpip command below.

Now program the TCP/IP details on the SEA adapter. This command is used instead of smitty (which is not available on the VIO Server) or ifconfig (not allowed for the SEA). You'll, of course, have to decide your own hostname and address.

mktcpip -hostname op34 <- use you own hostname -inetaddr 9.137.62.34 <- use your IP address -interface en3 <- from the mkvdev command -netmask 255.255.255.0 <- normal TCPIP meaning -gateway 9.137.62.1 <- normal TCPIP meaning

You should now be able to ping your gateway: ping 9.137.62.1.

9b. Virtual SCSI using a logical volume for Client X

Once the VIO Server is running and before the virtual clients are started, you need to create the disk space and connect it to the virtual SCSI resource that the VIO client will try to attach to.

Check your disks:

$ lspv hdisk0 00c033eaf709961e rootvg active hdisk1 none None hdisk2 none None $

Here you see the VIO Server is using the first disk and the others are currently unused. Next, take a look at the free space on that first disk and the primary volume group called rootvg:

$ lsvg   rootvg   $ lsvg rootvg   VOLUME GROUP: rootvg VG IDENTIFIER : 00c033ea00004c000000010104ffa3fc VG STATE: active PP SIZE : 128 megabyte(s) VG PERMISSION : read/write TOTAL PPs: 271 (34688 megabytes) MAX LVs: 256 FREE PPs : 151(18328 megabytes) LVs: 14 USED PPs : 120(15636 megabytes) OPEN LVs: 12 QUORUM : 2 TOTAL PVs: 1 VG DESCRIPTORS : 2 STALE PVs: 0 STALE PPs : 0 ACTIVE PVs: 1 AUTO ON : yes MAX PPs per VG: 32512   MAX PPs per PV: 1016 MAX PVs : 32 LTG size (Dynamic): 256 kilobyte(s) AUTO SYNC: no HOT SPARE: no BB POLICY : relocatable $ lsvg -pv rootvg   rootvg:   PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION Hdisk0 active 271 55 22..09..00..00..24 $

Look at the disk view, too:

$ lspv hdisk0 PHYSICAL VOLUME: hdisk0 VOLUME GROUP: rootvg PV IDENTIFIER: 00c033eaf709961e VG IDENTIFIER 00c033ea00004c000000010104ffa3fc PV STATE: active STALE PARTITIONS: 0 ALLOCATABLE: yes PP SIZE: 128 megabyte(s) LOGICAL VOLUMES: 14 TOTAL PPs: 271 (34688 megabytes) VG DESCRIPTORS: 2 FREE PPs: 151(19328 megabytes) HOT SPARE: no USED PPs: 120 (15636 megabytes) MAX REQUEST: 256 kilobytes FREE DISTRIBUTION: 22..09..00..00..24 USED DISTRIBUTION: 33..45..54..54..30 $

Important things to note here are:

• The TOTAL PPs entry shows the disk is a 36 GB drive -- actually 34688 MB, but remember the disk sizes are quoted in millions and billions and not the binary numbers used here.
• The FREE PPs entry shows there is approximately 18 GB free space on the disk.
• The PP SIZE entry shows that the VIO Server is allocating disk space in a minimum of this amount.

To create a logical volume in the rootvg volume group, try:

$ mklv -lv lv00 rootvg 4G lv00

The lv00 confirms the name used to create the logical volume.

To connect this to the VIO client resource, the name of the resource (here it's clientx) is used to make it very clear which partition is using it, but you could use any suitable name:

$ mkvdev -vdev lv00 -vadapter vhost0 -dev clientx clientx Available

You can now start the VIO Client X and find its virtual SCSI resources. In this example, you've defined just one logical volume for this VIO client, but many logical volumes could be used. I recommend you keep them to a minimum to make the configuration simpler.

9c. Virtual SCSI using a whole disk for Client Y

In the above section, you found that hdisk1 was unused. This disk will be used to support the VIO Client Y. The disk must not be in a volume group. In the lspv command above, there is no logical volume group name next to the disk, so it's not in a volume group. To connect this disk to the virtual SCSI disk for VIO Client Y, try:

$ mkvdev -vdev hdisk1 -vadapter vhost1 -dev clienty clienty Available

9d. Checking the configuration

You can now see the configuration:

$ lsdev -virtual name status description ent2 Available Virtual I/O Ethernet Adapter (l-lan) vhost0 Available Virtual SCSI Server Adapter vhost1 Available Virtual SCSI Server Adapter ... vsa0 Available LPAR Virtual Serial Adapter clientx Available Virtual Target Device - Logical Volume clienty Available Virtual Target Device - Logical Volume ent3 Available Shared Ethernet Adapter $ $ lsdev -dev clientx -attr attribute value description user_settable LogicalUnitAddr 0x8100000000000000 Logical Unit Address False aix_tdev lv00 Target Device Name False $ lsdev -dev ent3 -attr attribute value description pvid 3 PVID to use for the SEA device pvid_adapter ent2 Default virtual adapter to use for non-VLAN-tagged real_adapter ent0 Physical adapter associated with the SEA thread 0 Thread mode enabled (1) or disabled (0) virt_adapters ent2 List of virtual adapters associated with the SEA

Back to top

10. VIO client LPAR installations

Now you can start up your VIO clients and install them. This can be AIX (but only if the hardware is pSeries p5 and not OpenPower), SUSES SLES 9, Red Hat 3 update 3 onwards, or Debian. They should find both the:

• Virtual Ethernet, which will be named a Virtual Ethernet and behave like a real physical adapter.
• Virtual SCSI disk, which is presented just like an SCSI disk, but it will only be the size of the underlying disk partition or disk.

These should install just like a regular real Ethernet and SCSI disk.

VIO client running AIX

For AIX, this installation should be just like a normal AIX partition.

VIO client running Linux

For Linux, here are some additional notes. Once running, the Virtual Ethernet looks and behaves like a very fast one GB real adapter:

clienta:~ # ifconfig eth0 Link encap:Ethernet HWaddr AE:38:00:00:D0:02 inet addr:9.137.62.178 Bcast:9.137.62.255 Mask:255.255.255.0 inet6 addr: fe80::ac38:ff:fe00:d002/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1075 errors:0 dropped:0 overruns:0 frame:0 TX packets:350 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:113566 (110.9 Kb) TX bytes:40940 (39.9 Kb) Interrupt:184

Once running, you can see the virtual SCSI disk is being treated just like a regular disk:

clienta:~ # fdisk -l Disk /dev/sda: 4194 MB, 4194304000 bytes 130 heads, 62 sectors/track, 1016 cylinders Units = cylinders of 8060 * 512 = 4126720 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 1 3999 41 PPC PReP Boot /dev/sda2 6 132 511810 82 Linux swap /dev/sda3 133 1016 3562520 83 Linux

And you'll find the IBM virtual SCSI client kernel module installed (see ibmveth and ibmvscsic below):

clienta:~ # lsmod Module Size Used by evdev 31416 0 joydev 31520 0 st 72688 0 ipv6 478560 93 sg 74176 0 usbcore 183644 1 ibmveth 44536 0 subfs 30168 2 dm_mod 108224 0 ibmvscsic 43072 2 sr_mod 44380 0 sd_mod 43792 3 scsi_mod 192024 5 st,sg,ibmvscsic,sr_mod,sd_mod

Network installation might be trickier, as you might have to activate the devices drivers for the installation tools to find the virtual adapters. Network installations are not covered in this article.

Depending on which release you use, the installer might give you, with a series of menus, options to install the IBM virtual SCSI client and Virtual Ethernet drivers before you start an installation properly. Later releases fully understand and install the device drivers for these virtual resources without manual intervention.

Back to top

11. Backing up a VIO Server and VIO client

Once you've created your client LPAR and set it up the way you like, you should consider backing up the operating system images. Backups are a large subject for which many books have been written. There are many backup solutions from both commercial applications and freely available tools in the AIX and Linux world. For AIX, IBM has the Tivoli® Storage Manager product for remote backups. For Linux, one of the popular freely available tools is Amanda (Advanced Maryland Automatic Network Disk Archiver), which provides remote backup with disk caching and tape library management. There is also a Linux "Backup and Recovery How To" on the Internet for more information.

This article just covers the special considerations for VIO Servers and VIO clients. Backups are important for at least these three reasons, and these apply to VIO systems, too:

• Recovery of files that are accidentally removed.
• Disk failure, assuming your disks are not already protected with a mirror or RAID 5, or you are very unlucky and lose more than one disk.
• Recreating the entire system for a disaster (total machine loss) from backups held off the site.

HMC

The HMC data includes definitions of the LPAR physical resources such as CPU, memory, PCI slots, and definitions of the LPAR virtual resources (such as the connections between VIO Server and VIO clients).

See Effective System Management Using the IBM Hardware Management Console for pSeries, SG24-7038-00.

If the HMC fails, the data is still held in the Service Processor (FSP) and can be read by a replacement or recovered HMC. It's vital that the configuration details are available in case of a disaster. HMC backups are documented in manuals, InfoCenter help files, and IBM Redbooks. It's also recommended that details of the LPARs are documented on paper. For example, something similar to the planning table used to create the LPARs in this article.

VIO Server

The VIO Server itself needs to be backed up. There is the backupios command for saving the rootvg volume group to either a tape, filesystem, CD, or DVD. The other volume group structure (not the contents) can be saved and restored with the savevgstruct and restorevgstrct commands. To save the contents of other volume groups (not rootvg), you'll have to make other arrangements such as using the oem_setup_env command, dd command, savevg command, tar command, or other backup solutions.

If the VIO Server is purely being used for virtual I/O, then you need to back up:

• The details of the client logical volumes or the details of hdisks. Details include number of logical volumes, their size, the disk layout, which clients, and their use.
• The contents of the client logical volumes or hdisks.

To recover the VIO Server, you can simply reinstall the original install image, which is a mksysb image in much the same way as you installed the VIO Server in the first place.

There are different approaches to backing up the VIO client images from the VIO Server. First, note that you have the option of doing hot or cold backups:

Hot backup

A hot backup is while the VIO client is running. This is dangerous and is not recommended.

Cold backup

A cold backup is the only sensible way to back up the client from the VIO Server. This is simply a matter of shutting down the VIO client. For Linux as root on the client, try: shutdown -fh now, and for AIX shutdown -Fh.

For the logical volume method, you have to use the dd command to copy the logical volume images to a file, tape, NFS, and so on.

For the whole hdisk method, the large size probably means a tape is the best option.

The cp command is not a good idea, since it copies a file using small blocks and is very inefficient and slow. A better command is dd with a large block size. For example, 64 KB blocks use the bs=64k option. To copy a logical volume, try:

dd bs=64k if=/dev/lv01 of=/backup/B

Alternatively, you can back up straight to a tape drive using a command like tar or backup. Some machines support a writeable DVD device that can also be used as a backup medium.

Because AIX and the VIO Server can perform DLPAR changes of PCI slots, a single tape drive and its associated SCSI adapter can be moved to the VIO Server for the backup period, and then removed so it can be used in other LPARs.

Recovery of a VIO client involves getting the disk image back in the right place and starting the VIO client LPAR again.

VIO client

The VIO client can be used to back up its own data just like a regular LPAR running AIX or Linux. With an AIX VIO client, the best backup method is the mksysb command.

It's unlikely that VIO client LPARs will have its own tape, since the purpose of a VIO client is to share physical resources to reduce hardware requirements. As with the VIO Server, the DLPAR changes of PCI slots can be used to temporarily introduce a tape driver to the client and then for it to back up its own data. Automating this process can be hard to coordinate between lots of client LPARs, but it can be done using scripts on a central machine. Some machines support a writeable DVD device that can also be used as a backup medium.

A second option is for the client to use another LPAR (possibly the VIO Server) or another machine to save the data using either a:

Remote tape drive

You can find lots of information on how to do this and make it secure on the Internet. You might need to check the speed of this mechanism and use a Linux tool called "buffer" to ensure the tape drive streams data onto the tape drive efficiently.

NFS server

To temporarily store the backup data before it's backed up to tape.

Remote Backup application

Discussed at the top of this section and uses a local client application to transfer data to the server machine to provide a backup service.

In all three cases, the high speed of the Virtual Ethernet can boost backup performance. Recovery using these methods can be harder work. With AIX, you can simply recover the mksysb image. With Linux, you might have to reinstall from the original CD-ROMs and then overwrite the running Linux with the backup.

Back to top

12. Cloning a client

Another option is to create a copy of the VIO client SCSI disk and use it as the virtual SCSI disk for a new VIO client LPAR.

For the logical volume, create another logical volume of identical size and use the dd command to create a copy of the original logical volume. Be careful with the header structure of the logical volume; AIX can keep some information in the first block.

For the whole disk, you would need a disk of identical size and then copy from the original to the new partition using the dd command.

You need to check that the cloned LPAR is not using the same Ethernet IP addresses as the original. Alternatively, you can clone the original LPAR before putting it on the network.

Back to top

13. Linux DLPARs and RAS

This is not really part of the VIO Server, but it's important for what IBM calls reliability, availability, and serviceability (RAS).

For AIX 5.3, VIO clients have the DLPAR and RAS features already installed.

For the Linux VIO clients, these need to be added as described below.

After installing Linux, it's strongly recommended that you install the IBM packages for both DLPARs (this works for physical and virtual resources) and the daemons and tools to increase RAS. This ensures that you get the expected reliability from your pSeries p5 or OpenPower machine. These RPMs put the LPAR in touch with the HMC for dynamic changes, reporting problems, and provides tools to use on the LPAR, too.

The tools can be downloaded from Service and productivity tools for Linux on POWER. At this Web site, select the right version of Linux tab then the HMC-Managed option to find the list. Below is the list of RPMs at the time of writing this article, but the version numbers might have since been updated since.

1. src-1.2.2.1-0.ppc.rpm 2. rsct.core.utils-2.3.4.2-0.ppc.rpm 3. rsct.core-2.3.4.2-0.ppc.rpm 4. csm.core-1.4.0.3-79.ppc.rpm 5. csm.client-1.4.0.3-79.ppc.rpm 6. devices.chrp.base.ServiceRM-2.2.0.0-1.ppc.rpm 7. DynamicRM-1.1-2.ppc.rpm 8. rpa-dlpar-1.0-11.ppc.rpm 9. rpa-pci-hotplug-1.0-8.ppc.rpm 10. librtas-1.1-12.ppc64.rpm

A prerequisite is the rdistcommand. For SUSE Linux, this is on the Linux SuSE SLES 9 CD3 and file name rdist-6.1.5-792.1.ppc.rpm.

Download these RPMs and install or update existing packages. You should find you can now do DLPAR changes. In addition, Linux can report problems to the HMC, which is used to forward problems to IBM (if set up) and used by hardware maintenance staff for diagnosis and correction.

A similar procedure is available for Red Hat and other Linux versions. See the above Web site for details about the packages you need to install.

Back to top

Conclusion

This article described the virtualization capabilities of IBM POWER5 servers and, through examples that apply equally to both pSeries p5 and eServer OpenPower systems, showed how to set up and use the VIO Server.

Virtualization is a hot topic in the computing industry. The POWER5-based machines provide opportunities for a significant reduction in operating costs for complex environments. Unlike software solutions available from other vendors, the POWER5 implementation uses advanced processor features (Hypervisor) and hardware features to create efficient and flexible virtualization capabilities.

Back to top

Acknowledgements

The author wishes to thank Dave Williams and Stephen Atkins, both of IBM UK, for reviewing and improving this article.

Resources

• Get more information on:
  • Setting up your IBM eServer OpenPower server
  • Setting up your Linux on pSeries server
  • Linux on Power Facts and Features
  • pSeries p5, OpenPower, and JS20 Facts and Features
• Read the Linux on Power Whitepaper.
• Get a full list of Linux on Power applications and more on the way.
• Find several Linux on Power developerWorks articles.
• Download IBM Redbooks:
  • Introduction to Advanced POWER Virtualization on IBM p5 Servers, Introduction and basic configuration , SG24-7940
  • Advanced Power Virtualization on IBM p5 Servers Architecture and Performance Considerations , SG24-5768
• Read the following whitepapers:
  • IBM p5 570 Server Consolidation Using POWER5 Virtualization
  • IBM p5 570 Workload Balancing Using POWER5 Virtualization
  • Virtual I/O Server-Performance/Sizing/QOS considerations
• Go to the InfoCenter for:
  • Using the Virtual I/O Server
  • VIO Server and PLM Commands Reference
• Visit the download site for the IBM RPMs.
• Visit the Virtual Innovation Center for Hardware for AIX development support. This is the primary source for all pSeries AIX development.
• Browse for books on these and other technical topics.
• Want more? The developerWorks AIX and UNIX zone hosts hundreds of informative articles and introductory, intermediate, and advanced tutorials on the eServer brand.
• Download Adobe Reader.
• Get involved in the developerWorks community by participating in developerWorks blogs.
• The IBM developerWorks team hosts hundreds of technical briefings around the world which you can attend at no charge.

About the author

		Nigel Griffiths is the EMEA Linux on POWER Hands-On Technical Advocate and EMEA p5 Virtualization Technical Focus Group Leader. He specializes in virtualization, performance, sizing, and tools. Nigel is also a certified consulting member of the IT-Specialist profession. You can contact Nigel at nag@uk.ibm.com.

 

===============================
Advanced Solaris 10
===============================

• Solaris 10 System Internals
• Solaris 10 SAN
• Solaris 10 ZFS
• Solaris 10 System Administrator Guide
• Solaris 10 System Administration Guide
• Solaris System Administration 101

 

===============================
Linux Cheat Sheet
===============================

 

• Linux Cheat Sheet
• Linux Cheat Sheet 2
• Linux Admin Quick Reference Guide

 

 

===============================
VMWare Cheat Sheet
===============================

 

• VMWare Quick Reference Guide

 

ESX Engineer Configuration Commands Cheat Sheet

I periodically set up ESX servers for customers. Over time I have developed a document that helps me when I am performing configurations on-site. This is by no means an inclusive list of commands, this is more a brain dump for me so I have access to this in case my laptop isn’t handy. If anybody has any additions, I would love to hear them!!

• Edit an ESX server to allow root SSH and SFTP access
  • vi /etc/ssh/sshd_config
  • Change PermitRootLogin no to yes
  • Restart ssh service using the command: service sshd restart

 

• Edit an ESX Server for NTP access (Internet in this case, could be a customer NTP server)
  • vi /etc/ntp.conf
  • Change OUR TIMESERVERS section, enter the server names – For the Internet use the following:
    • server 0.us.pool.ntp.org
    • server 1.us.pool.ntp.org
    • server 2.us.pool.ntp.org
  • Pipe the NTP Servers into step-tickers so NTP will update at NTP start using the following the commands:
    • Echo 0.us.pool.ntp.org >> /etc/ntp/step-tickers
    • Echo 1.us.pool.ntp.org >> /etc/ntp/step-tickers
    • Echo 2.us.pool.ntp.org >> /etc/ntp/step-tickers
  • Set NTP to start: chkconfig ntpd on
  • Set ESX Firewall to allow NTP Traffic: esxcfg-firewall –e ntpClient
  • Run an manual update: ntpdate 0.us.pool.ntp.org

 

• Patch the ESX Server with the latest patches (Update Manager is MUCH easier!)
• NOTE: The patches must be installed in “groups” based on their date starting with the oldest to the newest (i.e. apply the Septmber 9th patches, then the September 21st patches, etc)

 

• • SFTP the Patches into a directory on the server
  • Change into the folder and run the following as root
  • For ESX 3.0.X: tar –xvzf (name of patch).tgz
  • For ESX 3.5.X: unzip *.zip
  • change into the directory it creates
  • esxupdate –(2 dashes)noreboot update
  • change back to the root patch folder, rinse, repeat
  • When complete enter the following command to check the patch status: esxupdate query

 

 

• Commands to change the server ip, name, dns, gatewat, etc.
  • To check the config of the server from the command line: esxcfg-info
  • To check the vSwitch settings from the command line: esxcfg-nics -l
  • To change the hostname of the server or the default gateway
    • vi /etc/sysconfig/network
    • vi /etc/hosts with the new name
  • To change the DNS entries of the ESX server: vi /etc/resolv.conf
  • To change the IP Address and the Subnet Mask of the ESX Server:
    • esxcfg-vswif –i (IP Address) –n (Subnet Mask) (vswitch-name -> vswif0 for the Service Console by default)
  • To change the VLAN tag of the service console port:
    • esxcfg-vswitch vSwitch0 –p “Service Console” –v (VLAN NUMBER)
  • NOTE: If you modify the above settings, make sure you also modify the network section of the /root/anaconda-ks.cfg file as well.
  • To see what services the ESX firewall will allow: esxcfg-firewall –s
  • To change the root password from the command line: passwd
  • If you have changed the ip address or name and HA is acting up:
    • /opt/vmware/aam/bin/ft_gethostbyname
    • rename and/or delete FT_HOSTS file under /etc/opt/vmware/aam/

 

 

===============================
SAN Storage
===============================

 

===============================
VxVM EMC Cheat Sheet
===============================

 

VxVM-Cheat-Sheet-EMC

Written by Administrator

Friday, 03 October 2008 14:17

Veritas Volume Manager Cheat Sheet: The Setup: an Oracle database supporting an application called ABC, and given the following: Assuming that we need to create filesystems intended for Oracle and: we are using a SID of 'ABC' we have nine disks, c0t0d1 through c0t0d9 we will be creating six volumes: /u01/oradata/ABC ( 4gb ), /u02/oradata/ABC (4gb ), /u03/oradata/ABC (8gb ), /u04/oradata/ABC ( 4gb ), /u01/app/oracle/product/8.0.5_ABC ( 4gb ), and /u01/admin/ABC ( 4gb ). We added these disks to Volume Manager Control during installation with vxinstall, or later with vxdiskadd(1m). Both vxinstall and vxdiskadd can add either initialize a previously unused disk, or 'encapsulate' a disk already containing data, but not under Volume Manager control. we could take the following steps to set up our filesystems. Adding an EMC disk to Veritas Use the inq utility to see if you have SCSI visibility to the disk Use vxdisk -o alldgs list to see if Veritas has seen the disk if not, try the following, and then repeat the previous step: devfsadm vxdctl initdmp vxdctl enable   Creating a Disk Group in Volume Manager: Create a vxvm disk group for the Oracle SID using vxdg(1m). It's important that all of the disks/filesystems necessary for the ABC database to run are included in the disk group, otherwise the database could not be moved to another system. The easiest way to do this is to use vxdiskadd vxdiskadd c0t0d0 c0t0d1 c0t0d2 . . . Doing it the hard way: vxdg init ABCdg ABCdg01=c0t0d1 This would create a disk group called ABCdg containing one physical disk, c0t0d1, which will be referred to by it's name within vxvm, ABCdg01. You cannot initiate a disk group without specifying at least one disk as a member of the group. You should not include any disks destined to be part of an Oracle Database in the default vxvm group, rootdg. This allows you to use the vxdg 'deport' and 'import' commands to migrate an entire vxvm disk group to another host. Now that we've create the disk group for our database, we can add the rest of our disks to it: vxdg -g ABCdg adddisk ABCdg02=c0t0d2 vxdg -g ABCdg adddisk ABCdg09=c0t0d9   Creating Volumes With vxassist vxassist(1m) is a vxvm command that acts as a front-end to other vxvm commands, much as newfs(1m) acts as a simpler-to-use front-end to mkfs(1m). Using vxassist with it's -v flag will display the actual vxvm commands used during volume construction. vxassist also front-ends vxfs commands and normal UNIX utilities during volume construction. An example of this is using a single vxassist command to set up a volume. Using vxassist simplifies many vxvm tasks, but you should pay careful attention to the defaults you will inherit when setting up volumes using this tool. Understanding the contents of the /etc/default/vxassist file is important, especially when striping disks. To make our 8gb /u03/oradata/ABC volume, we could use the following command line: vxassist -g ABCdg make ABC01 8g Breaking it down into it's component parts: vxassist -g ABCdg # this volume will belong to the ABCdg disk group make # tells vxassist that we are creating a new volume ABC01 # name the volume 8g # specify the size of the volume Here we create the other five volumes: vxassist -g ABCdg make ABC02 8g vxassist -g ABCdg make ABC03 8g vxassist -g ABCdg make ABC04 8g vxassist -g ABCdg make ABC05 8g vxassist -g ABCdg make ABC06 8g   Creating a File System: We now lay filesystems down on our newly created volumes. We assume here that you've agreed with the Oracle DBA's to use a block size of 8192 for our filesystems. Not specifying a block size allow Veritas to dynamically assign one for you based upon the size of the volume. Our volumes would default to 1024 for a block size on our 4 GB volumes, and 2048 for our 8GB volume. We will specify the largefiles option, because the mkfs_vxfs(1m) command uses nolargefiles as a default, unlike mkfs_ufs(1m) on Solaris 2.6 or greater, which uses largefiles as a default. If we forgot to do so, we could use fsadm ( fsadm_vxfs(1m) ) to set the largefiles bit on. fsadm can also query a mounted filesystem for the current setting of this flag. Be certain that whatever flags you set during file system creation or modification are reflect in vfstab, or the resulting file system may be un-mount-able. mkfs -F vxfs -o bsize=8192,largefiles /dev/vx/rdsk/ABCdg/ABC01   Mounting a File System: Creating and mounting filesystems go hand in hand. You must be certain that any special flags set at creation time are reflected in /etc/vfstab, or on the command line if there is no entry in vfstab for your filesystem. mount -F vxfs -o largefiles /dev/vx/dsk/ABCdg/ABC01 /u03/oradata/ABC   Resizing a mounted volume and file system: /etc/vx/bin/vxresize -g rootdg -F vxfs archive 16g   Recovering disks under VM control: (if you want to live dangerously, try looking at dfwset01:/root/progs/hose_vx_dg.pl) Pre-reqs Do a vxprint -g mydg first to see what disks your volumes lie on! Do a vxdisk list and save it Do an inq -et and save it! You may wish to review moving a volume off of a particluar disk Procedure 1) comment the volume(s) you're destroying out of vfstab 2) umount the volume 3) stop it: vxvol stop volume01 4) remove it: vxedit -rf rm volume01 5) remove the disk(s) the volume was on from the disk group ( up to the last disk ) vxdg -g mydg rmdisk dmdiskname01 6) Remove the the disk(s) from VXVM control vxdisk rm c1t1d51s2 7) If you wish do get rid of the disk group ( in order to recover the last disk ), you must destroy the disk group: vxdg destroy mydg 8) remove the last disk vxdisk rm c2t7d9s2 9) If you're removing the disks from the EMC port, be sure to clean up: drvconfig disks vxdctl enable vxdctl initdmp   10) Freak out and realize you whacked the wrong thing, use /root/eotw to recover.   Renaming a VM Disk: Since names like "ABCdg10" are not as descriptive as they could be, you can rename it to something more useful. vxedit rename ABCdg10 ABCdg_hot_spare Using long disk names can be more descriptive, but will make using vxva trickier, as it will truncate your disk name somewhat if it is too long.   Moving a Disk Group to Another System: 1) umount the disks: umount /u01/app/oracle/product/8.0.5_ABC umount /u01/admin/ABC umount /u01/oradata/ABC umount /u02/oradata/ABC umount /u03/oradata/ABC umount /u04/oradata/ABC 2) Stop the volumes: vxvol -g ABCdg stopall 3) deport the disk group from one system: vxdg deport ABCdg 4) import them on another system: vxdg import ABCdg 5) Start all of the volumes on the new system and resync mirrors in the background: vxrecover -g ABCdg -sb 6) mount the filesystems mount /u01/app/oracle/product/8.0.5_ABC mount /u01/admin/ABC mount /u01/oradata/ABC mount /u02/oradata/ABC mount /u03/oradata/ABC mount /u04/oradata/ABC This procedure assumes that the second system can see the disks, and also that you have commented the file systems out of vfstab on the old system. Rename a Disk Group /root dfwns19 # df -k | grep reports /dev/vx/dsk/reportsdg/ncr01 /u01/app/oracle/admin/REPORTS /dev/vx/dsk/reportsdg/ncr02 /u01/app/oracle/product/8.0.5REPORTS /dev/vx/dsk/reportsdg/ncr03 /u01/oradata/REPORTS /dev/vx/dsk/reportsdg/ncr04 /u02/oradata/REPORTS /dev/vx/dsk/reportsdg/ncr05 /u03/oradata/REPORTS /dev/vx/dsk/reportsdg/ncr06 /u04/oradata/REPORTS /dev/vx/dsk/reportsdg/ncr07 /u11/oraarch/REPORTS dfwns19 # umount /u01/app/oracle/admin/REPORTS dfwns19 # umount /u01/app/oracle/product/8.0.5REPORTS dfwns19 # umount /u01/oradata/REPORTS dfwns19 # umount /u02/oradata/REPORTS dfwns19 # umount /u03/oradata/REPORTS dfwns19 # umount /u04/oradata/REPORTS dfwns19 # umount /u11/oraarch/REPORTS dfwns19 # vxvol -g reportsdg stopall dfwns19 # vxdg deport reportsdg dfwns19 # vxdg -n nclproddg import reportsdg dfwns19 # vxdg list Mirroring and then breaking mirrors # make an sd from your dm ( use the -g, or it won't work. The book is wrong ) # sdname dmname,start_offset,sd_length vxmake -g dwproddg sd dwprodd15-01 dwprodd15,0,8389440 # create a plex from your sd vxmake -g dwproddg plex dw01-02 sd=dwprodd15-01 # attach the plex to your volume vxplex -g dwproddg att dw01 dw01-02 # watch it silver ( check out the pause and slow options. badass ) /tmp dfwora03 # vxtask -l list Task: 162 RUNNING Type: ATCOPY Operation: PLXATT Vol dw01 0.0 Started: Tue Dec 05 03:47:29 2000 Throttle: 0 Progress: 18.33% 1537352 of 8388608 Blocks Work time: 1 minute, 26 seconds (06:23 remaining) # more than one sd per plex for plexen > one dm vxmake -g dwproddg sd dwprodd17-01 dwprodd17,0,8833920 vxmake -g dwproddg sd dwprodd18-01 dwprodd18,0,7944000 vxmake -g dwproddg plex dw03-02 sd=dwprodd17-01,dwprodd18-01 vxplex -g dwproddg att dw03 dw03-02 # now for the scary part. dis-associating the plexes and sd's vxplex -o rm dis dw01-01 # zap the dm from the dg vxdg -g dwproddg rmdisk dwprodd01 # nuke the da vxdisk rm c2t2d13s2 NAME STATE ID nclproddg enabled 949356971.2501.dfwns19 Renaming a disk group and volume bansecdbdg -> secproddg used as an example # Unmount all volumes in the bansecdbdg disk group. # Stop all volumes vxvol -g bansecdbdg stopall # Deport bansecdbdg vxdg deport bansecdbdg # Import bansecdbdg using the -n option vxdg -n secproddg import bansecdbdg # Recover the disk group by the new dg name vxrecover -g secproddg -sb # Rename each volume vxedit -g secproddg rename bansecdb01 sec01 vxedit -g secproddg rename bansecdb02 sec02 . . . vxedit -g secproddg rename bansecdb08 sec08 # Update vfstab, your mount points and remount the volumes Resizing a disk, while specifying the disk # Resize volume u11, and only use disk dev20dg23 # x requires the volume to grow (i.e. does not let you specify a # smaller than existing volume size), and b backgrounds the # task (according to the tech) ./vxresize -F vxfs -bx -g dev20dg u11 16g dev20dg23 Moving a volume off of a particluar disk # Move u11 off of dm dev20dg20 vxassist -g dev20dg move u11 !dev20dg20

 

===============================
Hitachi SAN
===============================

 

• Hitachi Dynamic Link Manager

 

AIX: Command Cheatsheet

dsmc q sched check TSM scheduler

prtconf system configuration

oslevel -r OS and ML level

netstat -v NIC statistics

lsps -a check usage of the swapfile

lsattr -El sys0 list attributes of sys0

lsvg -l list volume groups

df -k shows what lv a fs lives in

lslv -l list info on volume

lslv -l vgname shows filesystems within vg

lspv shows disks in volume group

chfs -a size=+131072 /filesystem extend 'filesystem' by 128MB

history -1000 | grep whatever command history search

du -sk ./* get usage stats

lpstat status of print queues

qconfig printer configuration

lprm -Pprinter_queue 676 Remove print job 676 from printer_queue

lscfg | grep fcs get fibercard info

lscfg -vl fcsx list stats for card fcsx

lscfg | grep U1.9-P1-I7/Q1 1st step to find port

lscfg -vl hdisk135 get port number

lscfg -vl fcs0 Get WWN for SAN request

fuser -cuxd who is using what

topas top equivalent

errclear 0 clears errpt

errpt -a shows errors on aix

extendvg Adds 1 or more hdisks to an existing VG

refresh -s inetd re-reads inetd.conf

dlnkmgr view path Reports status of SAN connection (Hitachi)

dlnkmgr view -sys Shows info on SAN subsystem

dlnkmgr set -lb off sets load balancing off

dlnkmgr set -afb off sets automatic failback off

lsvpcfg Reports status on SAN connection (Shark)

qcan -X -P Cancels all print jobs on

lp -dQUEUENAME filename send file to printer

snap -gc places a snap.pax.Z file in /tmp for IBM engineers to look at

datapath query device reports info on shark disk Reading in from a list:

while read line
do
set $line
chdev -l $1 (or whatever you want to do)
done < /tmp/inputfile > outputfil

 

coolcommand dlnkmgr - Hitachi SAN HDLM software

Description Available command parameters: # dlnkmgr help dlnkmgr { clear | help | offline | online | set | view } The view option allows you to see system and drive information # dlnkmgr view -help dlnkmgr view -sys [ -sfunc | -msrv | -adrv | -pdrv ] [-t] dlnkmgr view -path [ -c | -hdev HostDeviceName ] [-t] dlnkmgr view -path -item [pn] [dn] [lu] [cp] [type] [ic] [ie] [dnu] [hd] [ -hdev HostDeviceName ] [-t] dlnkmgr view -drv [-t] The clear option allows you to clear the errors counters # dlnkmgr clear -help dlnkmgr clear -pdst [-s] The offline option allows you to disactivate a dlmfdrv # dlnkmgr offline -help Format dlnkmgr offline [-path] -pathid AutoPATH_ID [-s] Valid value AutoPATH_ID {000000 - 999999}(Decimal) The online option allows you to activate a dlmfdrv # dlnkmgr online -help Format dlnkmgr online [-path] [-pathid AutoPATH_ID] [-s] Valid value AutoPATH_ID {000000 - 999999}(Decimal) The set option allows you to set various dlm values # dlnkmgr set -help Format dlnkmgr set { -lb { on | off } | -ellv LogLevel | -elfs LogSize | -systflv TraceLevel | -pchk { on [ -intvl Interval-Time ] | off } | -afb { on [ -intvl Interval-Time ] | off } | -rsv on ReserveLevel } [-s] Valid value LogLevel { 0 | 1 | 2 | 3 } (Default Value 3) LogSize { 100 - 9900 }(KB) (Default Value 1000) TraceLevel { 0 | 1 | 2 | 3 | 4 } (Default Value 0) Interval-Time { 1 - 1440 }(Minute) (Default Value 30) (pchk) Interval-Time { 1 - 1440 }(Minute) (Default Value 1) (afb) ReserveLevel on { 0 | 2 } (Default Value "on 0") Command to view path information # dlnkmgr view -path Paths:000002 OnlinePaths:000002 PathStatus IO-Count IO-Errors Online 49960 0 PathID PathName DskName iLU ChaPort Status Type IO-Count IO-Errors DNum HDevName 000000 08.14.0000000000075000.0000 HITACHI .OPEN-V .43194 0085 1G Online Own 25152 0 0 dlmfdrv2 000001 08.3D.0000000000085000.0000 HITACHI .OPEN-V .43194 0085 2G Online Own 24808 0 0 dlmfdrv3 KAPL01001-I The HDLM command completed normally. Operation name = view

Example dnlkmgr view -path

 

 

===============================
IBM DS Storage Tuning Guide
===============================

 

• IBM DS Storage Tuning Guide

 

===============================
Network Appliance Setup Guide
===============================

 

• NetApp Veritas Cluster Snap Mirror AIX, Solaris, HP-UX, Linux
• NetApp Pocket Survival Guide

 

===============================
NetApp On Solaris
===============================

 

How to Add a Network Appliance Network-Attached Storage (NAS) Quorum Device

When you use a Network Appliance (NetApp) network-attached storage (NAS) device as a quorum device, the following are required:

• You must install the iSCSI license from NetApp.

• You must configure an iSCSI LUN on the clustered filer for use as the quorum device.

• You must configure the NetApp NAS unit to use NTP for synchronizing time.

• At least one of the NTP servers selected for the clustered filer must be an NTP server for the Sun Cluster nodes.

• When booting the cluster, always boot the NAS device before you boot the cluster nodes.

  If you boot devices in the wrong order, your nodes cannot find the quorum device. If a node should fail in this situation, your cluster might be unable to remain in service. If a service disruption occurs, you must either reboot the entire cluster or remove the NetApp NAS quorum device and add it again.

• A cluster can use a NAS device for only a single quorum device.

  You can configure other shared storage if you need additional quorum devices. Additional clusters that use the same NAS device can use separate LUNs on that device as their quorum devices.

See the following Network Appliance NAS documentation for information about creating and setting up a Network Appliance NAS device and LUN. You can access the following documents at http://now.netapp.com.

Task	Network Appliance Documentation
Setting up a NAS device	System Administration File Access Management Guide
Setting up a LUN	Host Cluster Tool for Unix Installation Guide
Installing ONTAP software	Software Setup Guide, Upgrade Guide
Exporting volumes for the cluster	Data ONTAP Storage Management Guide
Installing NAS support software packages on cluster nodes	Log in to http://now.netapp.com. From the Software Download page, download the Host Cluster Tool for Unix Installation Guide.

 

 

See the following Sun Cluster documentation for information about installing a NetApp NAS storage device in a Sun Cluster environment: Sun Cluster 3.1 - 3.2 With Network-Attached Storage Devices Manual for Solaris OS.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands.

1. Make sure that all Sun Cluster nodes are online and can communicate with the NetApp clustered filer.

2. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

3. Start the clsetup utility.

   # clsetup

   The clsetup Main Menu is displayed.

4. Type the number that corresponds to the option for Quorum.

   The Quorum Menu is displayed.

5. Type the number that corresponds to the option for adding a quorum device. Then type yes to confirm that you are adding a quorum device.

   The clsetup utility asks what type of quorum device you want to add.

6. Type the number that corresponds to the option for a netapp_nas quorum device. Then type yes to confirm that you are adding a netapp_nas quorum device.

   The clsetup utility asks you provide the name of the new quorum device.

7. Type the name of the quorum device you are adding.

   The quorum device name can be any name you choose. The name is only used to process future administrative commands.

   The clsetup utility asks you to provide the name of the filer for the new quorum device.

8. Type the name of the filer of the new quorum device.

   This name is the network-accessible name or address of the filer.

   The clsetup utility asks you to provide the LUN ID for the filer.

9. Type the ID of the quorum device LUN on the filer.

   The clsetup utility asks if to the new quorum device should be added on the filer.

10. Type yes to continue adding the new quorum device.

    If the new quorum device is added successfully, the clsetup utility displays a message to that effect.

11. Verify that the quorum device has been added.

    # clquorum list -v

---

Example 6–2 Adding a NetApp NAS Quorum Device

The following example shows the clquorum command generated by clsetup when it adds a NetApp NAS quorum device. The example also shows a verification step.

Become superuser or assume a role that provides solaris.cluster. modify RBAC authorization on any cluster node. [Start the clsetup utility:] # clsetup [Select Quorum Add a quorum device] [Answer the questions when prompted.] [You will need the following information.] [Information: Example:] [Quorum Device Netapp_nas quorum device] [Name: qd1] [Filer: nas1.sun.com] [LUN ID: 0] [Verify that the clquorum command was completed successfully:] clquorum add -t netapp_nas -p filer=nas1.sun.com,-p lun_id=0 qd1 Command completed successfully. [Quit the clsetup Quorum Menu and Main Menu.] [Verify that the quorum device is added:] # clquorum list -v Quorums Type ------- ---- qd1 netapp_nas scphyshost-1 node scphyshost-2 node

---

 

===============================
HP-UX Administration
===============================

 

• HP-UX Administration 101
• HP-UX Boot

 

===============================
wget
===============================

 

wget is a nice tool for downloading resources from the internet. The basic usage is wget url:

wget http://linuxreviews.org/

Therefore, wget (manual page) + less (manual page) is all you need to surf the internet. The power of wget is that you may download sites recursive, meaning you also get all pages (and images and other data) linked on the front page:

wget -r http://linuxreviews.org/

But many sites do not want you to download their entire site. To prevent this, they check how browsers identify. Many sites refuses you to connect or sends a blank page if they detect you are not using a web-browser. You might get a message like:

Sorry, but the download manager you are using to view this site is not supported. We do not support use of such download managers as flashget, go!zilla, or getright

Wget has a very handy -U option for sites like this. Use -U My-browser to tell the site you are using some commonly accepted browser:

  wget  -r -p -U Mozilla http://www.stupidsite.com/restricedplace.html

The most important command line options are --limit-rate= and --wait=. You should add --wait=20 to pause 20 seconds between retrievals, this makes sure you are not manually added to a blacklist. --limit-rate defaults to bytes, add K to set KB/s. Example:

wget --wait=20 --limit-rate=20K -r -p -U Mozilla http://www.stupidsite.com/restricedplace.html

A web-site owner will probably get upset if you attempt to download his entire site using a simple wget http://foo.bar command. However, the web-site owner will not even notice you if you limit the download transfer rate and pause between fetching files.

Use --no-parent

--no-parent is a very handy option that guarantees wget will not download anything from the folders beneath the folder you want to acquire. Use this to make sure wget does not fetch more than it needs to if just just want to download the files in a folder.

===============================
Sun Packaging
===============================

 

Sun Packaging