Test 000-104: AIX 6.1 Administration

Test 000-104: AIX 6.1 Administration


What you see in ¡§Red¡¨ are the objectives of the exam defined by IBM:

http://www-03.ibm.com/certify/tests/obj104.shtml

I have tried to add notes to each item to some extent. This is not a replacement for IBM documents or
courses, but can be used as a wrap-up for the exam or as a reference for some admin tasks. The
document was not intended for public use in the first place, that is why you will typo mistakes,
formating or other problems in it. Hope these notes help you pass the exam with a better score :)

Note: References are mostly IBM redbooks, man pages and other freely-available IBM web resources.


Backup and Recovery (5%)


a. Recover from a lost root password
1. Boot the LPAR from AIX media, mksysb tape or NIM server. The boot resource should
have the same version and TL as the system you want to recover. For example, an AIX
6.1 with TL6 cannot be recovered by AIX 6.1 TL2 media or NIM resource.
2. Choose Start Maintenance Mode for System Recovery .
3. Select Access a Root Volume Group. A message displays explaining that you will not be
able to return to the Installation menus without rebooting if you change the root
volume group at this point.
4. Type 0 and press Enter.
5. Type the number of the appropriate volume group from the list and press Enter.
6. Select Access this Volume Group and start a shell by typing 1 and press Enter.
7. At the # (number sign) prompt, type the passwd command at the command line prompt
to reset the root password. For example:
# passwd
Changing password for "root"
root's New password:
Enter the new password again:
8. To write everything from the buffer to the hard disk and reboot the system, type the
following:
# sync;sync;sync;reboot

b. Backup AIX OS and data using AIX commands (mksysb, mkcd, tar, backup, etc)
mksysb:
Backup to tape (Note: Not all tape drives are bootable!):
# mksysb -iXV /dev/rmt0

Backup to filesystem (the filesystem path can be local or NFS-mounted):
# mksysb -iX /backups/mksysb31Mar2011.mksysb

Backup a client from NIM server (Note: /mksysbs in the following command should be NFS
exported to testlpar):
# nim -o define -t mksysb -a server=master -a location=/mksysbs/testlpar31Mar2011.mksysb -a
source=testlpar -a mk_image=yes -a mksysb_flags=XeA testlpar_31Mar2011_mksysb

Check the NIM resource in NIM server:
# lsnim -t mksysb

testlpar_31Mar2011_mksysb resources mksysb
Note: mksysb only backs up files and directories in rootvg that are mounted.


There are other methods to clone an AIX systems:
o Alternate Disk Install
o Tivoli Sysback
o Taking mirror disks of rootvg to another system!
o And probably more¡K
„h Mksysb image can be extracted from tape to be used in NIM server.:
o First you should find the block size of the tape when the mksysb has been performed:

# chdev -l rmt0 -a block_size=512
# tctl -f /dev/rmt0 rewind
# restore -s2 -xqvf /dev/rmt0.1 ./tapeblksz
# cat tapeblksz
1024 NONE
It means the mksysb backup has been made using block size of 1024.
# chdev -l rmt0 -a block_size=1024
# tctl -f /dev/rmt0 rewind
# dd if=/dev/rmt0.1 of=/mksysbs/mksysb1 bs=1024 fskip=3
¡P It is possible to show information about a mksysb image:
# lsmksysb -lf /tmp/mksysbfile <-- this will show infromation about filesystems and OS
level of the image.
(Actually lsmksysb is a soft link to listvgbackup. It means you could use "listvgbackup -lf
/tmp/mksysbfile" instead of above command as well)
savevg and restvg:
- The volume group should be vary-on and filesystems should be mounted.
- This will backup testvg into a file called vgbackup1:
o # savevg -if /backups/vgbackup1 testvg
- In order to exclude files, edit /etc/excluce.testvg.
- If you destroy the volume group, it can be restored by restvg:
o # restvg -f /backups/vgbackup1 hdisk1
mkszfile and mkvgdata:
When you use ¡§-i¡¨ switch with mksysb and savevg, they call mkszfile and mkvgdata respectively.
It will create /image.data for rootvg, /tmp/vgdata/testvg/testvg.data for a user-created
volume group like testvg and /tmp/wpardata/wpar1/image.data for a workload partition called
wpar1. If you need to change the characteristics of the restored volumes group, above files
should be edited and then mksysb, savevg are used without ¡§-i¡¨ switch.
Note: /usr/bin/mkszfile is a shell script that has two aliases: mkvgdata and mkwpardata. The
script runs differently based on the name of invoker file:
¡K
NAME=`/usr/bin/basename $0`
¡K
if [ $NAME = "mkszfile" ]
then
set -- `${getopt} XfmN $*` # mkszfile options
¡K
savewpar
savewpar cannot be used to create bootable tapes.
The command switches are very similar to savevg.
Example:
# savewpar ¡Vief /backups/wpar1backup wpar1
Note:
How to exclude files from and volume group or wpar backup:
- Create a file called /etc/exclude.rootvg, /etc/exclude.testvg or /etc/exclude.wpar1
- Put the ¡§pattern¡¨ you would like to exclude:
^./home „hexcludes /home filesystem
testfs „h excludes any file or directory that grep finds ¡§testfs¡¨ pattern it their path.
- # mksysb ¡VeX /mksysbs/newbackup
- # savevg ¡Vief /backups/vgbackup1 testvg
- # savewpar ¡Vief /backups/wpar1backup wpar1
Another way to exclude filesytems in a backup is to remove filesystem and its associated logical
volumes information from image.data (of rootvg or a workload partition) or testvg.data for a
user-created volume group named testvg.
mkcd /mkdvd
- Create multi-volume CDs from a mksysb, savevg, or savewpar backup image.
- Can generate a new backup or alternatively use existing mksysb, savevg or savewpar image.
- Generate CD or DVD images
o Images can be burnt now
o Images can be saved for later use
- # mkdvd ¡Vd /dev/cd0 „h bootable rootvg backup
- # mkdvd ¡Vd /dev/cd0 ¡VW wpar1
- # mkdvd ¡VS ¡VI /backups/ -C /backup -W wpar1 „h stop to burn and keeps the images in
/backups.
- # mkdvd ¡VSI /backups ¡VC /bakcups ¡Vv testvg
- There are so many command switches. You can use smit for more convenience.
Note:
mkdvd is an alias to mkcd
tar
# tar ¡Vcvf /dev/rmt0 /data „h backs up /data tree to rmt0 tape
# tar ¡Vtvf /dev/rmt0 „h lists the table of content
# tar ¡Vxvf /dev/rmt0 „h extracts (restores) /data
Note:
- When you use relative path, be careful when you restore the backup. You should go the
same directory to restore it.
- Tar can backup to file:
o # tar ¡Vcvf /backups/newbackup.tar /data
- You can use tar without the dash charater ¡§-¡§:
o # tar tvf /dev/rmt1
- You can backup many files and create a very big tar file, but each file cannot be bigger than
8GB. To dodge this problem you can use GNU tar. I have tested it with files of 80GB, and it
did not complain.
backup
Backup files by name:
- Use ¡§-i¡¨ flag.
- # find /home/Salehi | backup -ivqf /dev/rmt0
Backup filesysems by i-node:
- Need the filesysem to be un-mounted.
- ¡§backup -2¡¨ means level 2. If you use -u, it performs an incremental backup. ¡§u¡¨ means
update /etc/dumpdates
- # backup -1 -u -f /dev/rmt0 /data
c. Restore AIX OS and data using AIX commands, including listing backup media contents (restvg,
restore, tar, etc)
To restore a mksysb tape, just try boot from it. If the tape is not bootable, boot from AIX DVD
and then in SMS menus try to restore the mksysb by selecting the tape drive:
Normal Mode Boot „h Yes „h Start Maintenance Mode for System Recovery „h Install from a
System Backup
restvg
# restvg -f /backups/vgbackup1 hdisk1
restore
- To show the contents of a backup:
o # restore ¡VTvqf /backups/mydata.bak
- To extract all mine directory and its contents:
o # restore ¡Vxvqf /backups/mydata.bak /data/mine/
restwpar
# restwpar -f /backups/wpar1.bak -n wapr2 -d /newbasedir
System Initialization and Boot (7%)
a. Describe and modify the /etc/inittab and rc files
b. Describe the different run levels and boot modes
a,b,c and h are not true runlevels:
¡P they are processed only by telinit (not by init)
¡P A process started by these runlevels is not killed when init command changes runlevels.
c. Use commands to manage the boot list and create boot logical volumes (incl. changing the
boot list)
d. Describe the boot process (BIST, POST, mounts, cfgmgr)
AIX boot process:
1. POST and hardware checking
2. System ROS locates and loads the bootstrap code. It is operating system independent.
3. Software ROS (bootstrap) creates RAMFS, locates the BLV and turns control to it.
4. RAM filesystem includes a reduced version of ODM (such as PdDv), rc.boot ¡K
5. Base devices are configured and ¡§init¡¨ process will be started from RAMFS.
6. There is still no rootvg! But disks have been configured and are ready.
Now rc.boot will be called three times:
7. Phase1:
a. init process is already running. So it forks rc.boot 1
b. ODM is copied to RAMFS from BLV
c. ¡§cfgmgr ¡Vf¡¨ configures the necessary items to have rootvg disks.
8. Phase 2:
a. Rootvg is varied on.
b. fsck ¡Vf /dev/hd4 (root filesystem)
c. hd4 is mounted on /mnt in RAMFS
d. /usr and /var are checked and mounted
e. /var is checked and mounted
f. If system has been dumped before, ¡§copycore¡¨ command copies the dump from
/dev/hd6 (default) to /var/adm/ras.
g. /var is unmounted.
h. The primary paging space h6 is activated.
i. All /dev files are copied from RAMFS to disk
j. All customized ODM files from the RAM file system are copied to disk. Both ODM
versions from hd4 and hd5 are now synchronized.
k. Root filesystems are mounted.
9. Phase 3:
a. Rc.boot 3 (from disk)
b. /tmp is mounted
c. Syncvg rootvg
d. Cfgmgr ¡Vp2 for the rest of devices for normal boot. For service mode ¡Vp3 is invoked.
e. Cfgcon configures the console and boot messages are sent to the console
f. ODM of BLV and / are synched.
g. Syncd and errdemon are started.
h. Init turns the control to the next line of inittab
e. Interrupt the boot process and use SMS
f. Describe booting from different media (disk, network, tape, cd)
g. Perform system or partition startups, shutdowns and reboots
bootlist: Displays and alters the list of boot devices available to the system
bootlist has some modes:
normal: When the system is booted in normal mode
service: When the system is booted in service mode
prevboot: ¡§Some hardware platforms may attempt to boot from the previous boot
device before looking for a boot device in one of the other lists.¡¨
To show the normal bootlist:
# bootlist -m normal -o
To set the normal mode bootlist:
# bootlist -m normal cd0 hdisk0
To clear (invalidate) the service mode bootlist:
# bootlist -m service ¡Vi
When a partition is activated, you can choose the boot mode:
Normal: Uses ¡§normal mode¡¨ bootlist stored in NVRAM
SMS: Boot process stops at System Management Services menus.
DIAG_STORED: Uses ¡§service mode¡¨ bootlist and eventually shows diag menus.
DIAG_DEFAULT: Like DIAG_STORED, it is used for diag, but uses default boot list (not what you
have set using boot -m service)
OPEN_FIRMWARE: System boots to Open Firmware (used by service personnel)
Useful shutdown switches:
# shutdown -l (creates /etc/shutdown.log for diagnostics. ¡§-l¡¨ stands for ¡§log¡¨).
# shutdown -Fr (fast reboot)
System and Device Configuration (9%)
a. Add or remove devices (printers, tape, adapters, using cfgmgr, etc)
Add a device:
- Physically attach the device to the system. (The device may be hot-pluggable or not)
- If the system is powered-off, power it on. It will run cfgmgr by default. Otherwise, run
cfgmgr which will introduce the device into AIX ODM.
o If the device driver of the attached device does not exist in the system, install it
explicitly or have cfgmgr to install it:
# cfgmgr -i /dev/cd0
Remove a device:
# rmdev -l rmt0 (notice! This command only unconfigures the device, and do not removes it)
# rmdev -dl rmt0 (removes the device from ODM)
# rmdev -Rdl fcs0 (removes fcs0 and all its children recuresively)
# rmdev -p fcs0 (just removes the children, not fcs0 itself)
b. Determine / chance device attributes, including WWN, MAC addresses, etc. (lsdev, chdev,
lscfg, lsattr)
Chdev:
Changing the attributes of a device if it is busy:
# chdev -l ent0 -a ... -P (P stands for permanent)
Determine WWPN or FC adapter:
# fcstat fcs1 | grep -i "world wide port name"
World Wide Port Name: 0x10000000C97A34BF
Or:
# # lscfg -vl fcs1 | grep -i "network address"
Network Address.............10000000C97A34BF
Determining WWNN of FC adapter:
# fcstat fcs1 | grep -i "world wide node name"
World Wide Node Name: 0x20000000C97A34BF
Or:
# lscfg -vl fcs1 | grep -i z8
Device Specific.(Z8)........20000000C97A34BF
Determining Ethernet adapter MAC address:
# entstat -d ent0 | grep -i "hardware address"
Hardware Address: 00:14:5e:53:9d:40
Or:
# lscfg -vl ent0 | grep -i "network address"
Network Address.............00145E539D40
c. List, define and change paging space
List paging space:
# lsps -a
shows detailed output
# lsps -s
shows a summary
# mkps -s 1 -n -a testvg hdisk1
defines a paging space with one PP, starts now and at restart
# chps -s 1 paging00
adds one PP to the paging space
# chps -d 1 paging00
removes one PP from the paging space
# swapon /dev/paging00
activate the paging space now
# swapoff /dev/paging00
# rmps paging00
remove the paging space
d. Configure and manage print subsystem (print queues, default printer, print job management)
e. Configure system environment (timezone, /etc/environment, etc.)
f. Add / remove disks (including data migration tasks, using cfgmgr)
Network Administration (9%)
a. Configure the network (TCP/IP daemons, /etc/hosts, hostname, ifconfig, route,
/etc/resolv.conf, etc/netsvc.conf, /etc/ntpd.conf)
/etc/hosts:
You can add, change or delete entries from this file by hostent command. (Manual editing is still
available).
This adds a record to /etc/hosts with primary hostname of ¡§salehi¡¨ and an alias named ¡§mypc¡¨:
# hostent ¡Va 10.0.62.14 ¡§salehi mypc¡¨
To show the record associated with Salehi:
# hostent ¡Vs salehi
10.0.62.14 salehi mypc
Reserved host names:
timeserver
If you set timeserver in /etc/hosts, you get run setclock to get its time and set it to the current
system.
printserver
Identifies the default host to receive print requests.
hostname:
- ¡§hostname¡¨ command can show or ¡§temporarily¡¨ set the hostname of a system:
o # hostname newhostname (next reboot will roll it back. It is not permanent.)
- Another way to permanently set hostname:
o # chdev -l inet0 -a hostname=newhostname
o This will not change /etc/hosts
- Another way:
o # smit mkhostname
o This will not change /etc/hosts
- Another way:
o # mktcpip -h newhostname -a 10.0.84.79 -m 255.255.255.0 -i en0
o This will change /etc/hosts. (Actually adds the new host name as an alias of previous
value in /etc/hosts.)
Conclusion:
When you change hostname, always check /etc/hosts.
ifconfig:
To list all interfaces that are ¡§up¡¨ with details:
# ifconfig -au
To add IP to en0:
# ifconfig en0 10.1.2.3 netmask 255.255.255.0 up
To bring a network interface down:
# ifconfig en0 down
Note:
Changes made by ifconfig will be gone in next restart.
route:
To list the routing table:
# netstat ¡Vnr
To find the default gateway:
# netstat -nr | grep default | awk '{print $2}'
To establish a default gateway:
# route add 0 192.168.1.1
Add route to a destination (like 11.25.12.1) via a gateway (like 10.10.10.1):
# route add 11.25.12.1 10.10.10.1
To reach a network (like 50.1.3.0) via a gateway like 172.16.16.1 via en0:
# route add -net 192.168.10.0 10.0.62.14 ¡Vinterface 0
Or:
# chdev -l inet0 -a route=net,-hopcount,0,,-if,en0,,,,-static,50.1.3.0,172.16.16.1
To delete above route:
# route delete -net 50.1.3.0
# chdev -l inet0 -a delroute=net,-hopcount,0,,,50.1.3.0,172.16.16.1
Note:
The effect of route command is not permanent. Sometimes it is desirable to set routing via a
script when needed (like in HACMP environment). If you need to make it permanent, use ¡§chdev
-l inet0 ¡K¡¨ instead.
resolv.conf:
AIX uses some methods to map host names to their IP address:
- /etc/hosts
- DNS
- NIS
- LDAP
If /etc/resolv.conf does not exist:
it means the network is ¡§flat¡¨ and therefore /etc/hosts will be used for name resolution.
If /etc/resolv.conf exists:
We have ¡§domain network¡¨ and therefore resolver algorithm will be used.
File format:
A ¡§domain¡¨ entry tells the resolver routines which default domain name to append to names
that do not end with a . (period). There can be only one domain entry. This entry is of the form:
domain my.domain.com
¡§search¡¨ is another entry of this file that is mutually exclusive with ¡§domain¡¨. With ¡§search¡¨ you
can specify many domains to search within when you are resolving a name. The first domain in
the search list, is default domain.
¡§nameserver¡¨ entry specifies the remote domain name server.
- The address is dotted decimal
- You can specify more than one name server:
nameserver 192.9.21.1
nameserver 192.9.21.2
Note:
- If both ¡§domain¡¨ and ¡§search¡¨ entries exist, the one that appear last will be considered.
- If there is no default domain in /etc/resolv.conf, you should set it in the hostname.
- If you use LDAP, /etc/resolv.ldap should be configured.
- Name resolution order is specified in irs.conf and netsvc.conf and NSORDER environment
variable. NSORDER overrides the settings of netsvc.conf and netsvc.conf overrides irs.conf.
netsvc.conf:
It is used to specify the ordering of name resolution.
Syntax:
hosts = value [, value]
alias = value [, value]
Sample:
#checks /etc/hosts and then DNS for name resolution:
Hosts = local, bind
# checks /etc/aliases and then NIS to resolve aliases for sendmail:
alias = files, nis
/etc/aliases:
/etc/aliases is a link to /etc/mail/aliases
Contains the required aliases for the sendmail command.
moi: salehi
NSORDER:
If NSORDER environment variable is set, it overrides the settings of netsvc.conf and irs.conf
Example:
# export NSORDER=bind,nis,local
ntp.conf:
# startsrc -s xntpd
# lssrc -ls xntpd | grep peer
Sys peer: no peer, system is insane „h insane means ntp configuration is wrong!
In ntp.conf:
- Add this:
server 127.127.1.0
- and comment this:
#broadcastclient
# stopsrc -s xntpd
# startsrc -s xntpd -a ¡Vx (-x can be very important)
Wait for one or two miutes and then:
# lssrc -ls xntpd | grep peer
Sys peer: 127.127.1.0
flags: (configured)(refclock)(sys peer)
On ntp client side:
# ntpdate ¡Vd node1
If offset is more than 1000 seconds, change the time date manually and then try above
command again.
Note:
You can set the client to automatically sync the time with your server.
- Add a server entry in /etc/ntp.conf, but this time the address of your timeserver.
- Uncomment broadcastclient
- # stopsrc -s xntpd
- # startsrc -s xntpd -a ¡Vx (-x can be very important)
In order to start xntpd in system startup, change /etc/rc.tcpip. This can be done both in client
and server.
b. Configure network security (/etc/hosts.equiv, .rhosts, etc.)
First /etc/hosts.equiv and then $HOME/.rhosts will be checked to see whether the remote
r-command request is from a trusted host or not.
Sample:
toaster # all users from toaster are allowed
machine1 bob # only bob from machine1
+ lester # user lester from all machines
tron ¡Vjoel # user joel from tron host is not allowed.
tron # all userd from trom are allowd.
Note:
- For root user, only /.rhosts is checked.
- If /etc/hosts.equiv and $HOME/.rhosts have write permission for group or others, password
will be asked!
- The deny, or - (minus sign), statements must precede the accept, or + (plus sign),
- statements in the lists
- Generally it is not secure to use this kind of password-less communication. You can use SSH
key pairs, instead.
c. Verify network availability and debug network problems (ping, ifconfig, netstat, tcpdump,
iptrace)
tcpdump:
It prints the headers of packets on a network interface.
Example:
# tcpdump -i en0
To print all packets arriving at or departing from Salehi:
# tcpdump host salehi
Iptrace:
It provides interface-level packet tracing for IP protocol. It generates a log file that can be very
big.
iptrace can be started by issuing ¡§iptrace¡¨ command itself or by SRC. If not started by SRC, the
process should be stopped by ¡§kill -15¡¨. (-15 is SIGTERM or software termination signal).
Example:
# startsrc -s iptrace -a "/tmp/nettrace"
# stopsrc -s iptrace
# iptrace -i en0 -p telnet -s airmail /tmp/telnet.trace
# kill -15 234343
d. Understand and configure Etherchannel and teaming
e. Configure NFS (/etc/exports/, biod, nfsd, showmount, etc.)
/etc/exports:
If this file is present, at system startup /etc/rc.nfs brings up nfsd and mountd.
The entries of this file are like this:
Directory options
Example:
/soft # exports to the world
/usr2 -access=hermes:zip:tutorial # exports only to these systems
/usr/tps -root=hermes:zip # root access only to these systems
Important daemons and commands:
- nfsd:
o Services client requests for file system operations.
o Each daemon handles one request at a time. You can tune the max threads by chnfs
or chssys.
- mountd:
o It is an RPC that answers a client request to mount a filesystem.
- chnfs:
o # chnfs -n 10 -I (sets the number of nfsd daemons).
- exportfs:
o Exports and unexports directories to NFS clients.
o # exportfs -a (exports all in the /etc/exports)
o # exportfs /dir1 (exports only /dir1 which is in the /etc/exports)
o # exportfs -i /dir2 (exports only /dir1 which is not in the /etc/exports)
o # exportfs ¡Vu /dir2 (unexports /dir2)
Note:
You cannot export either a parent directory or a subdirectory of an exported directory within
the same file system.
biod:
It handles client requests for files. It is an old daemon and might be removed in future AIX
releases.
showmont:
# showmount -a (shows all clients that have mounted something on this server)
# showmount -e nfssrv1 (show which filesystems are exported from nfssrv1)
/etc/xtab:
Contains entries for currently mounted NFS directories. exportfs -u removes entries from this
file.
f. Configure and use CIFS (very basic)
Install bos.cifs_fs package in AIX and then ¡§smit cifs_fs¡¨. That¡¦s it! This will enable AIX to mount
Windows shared directories.
These ports should be opened: 137,138,139 and 445
Security and User Management (7%)
a. Add, delete, change user and group accounts
# mkuser -a mehdi <== mehdi will be admin
# mkuser -R LDAP Nava <== Nava will be authenticated by LDAP
# chuser shell=/usr/bin/bash mehdi <== changes the user's shell
How to reset the failed login count:
# chsec -f /etc/security/lastlog -a "unsuccessful_login_count=0" -s mehdi
b. Describe and modify user and group management related files, profiles, and set or change the
shell environment (/etc/security/user, /etc/security/limits, /etc/security/passwd,
/etc/profile/, .profile)
c. Demonstrate in-depth knowledge of the login process (is getty running, order of the
environment being set, etc.)
Login process:
1- When getty ¡V which is a long running process - detects a connection, it prompts for a
username and runs the login program to authenticate the user. So, getty is the first step
started from inittab:
cons:0123456789:respawn:/usr/sbin/getty /dev/console
2- getty prints a herald message from /etc/security/login.cfg to get the user name from
input.
3- getty calls login process to check whether password is needed to login or not. If
password is needed, another prompt will ask for it.
Note: If the second field of /etc/passwd is null, the user can login without password:
testuser::208:1::/home/testuser:/usr/bin/ksh
This method works only with telnet. ssh asks always for password.
4- Login process do the validation process
a. If login fails, a record is added to /etc/security/failedlogin
b. If login is successful:
a. /etc/environment
b. /etc/security/environ
c. /etc/security/limits
a. /etc/security/user
b. /etc/profile
c. $HOME/.profile (or .dtprofile for CDE)
b. Set permissions (in more depth than operator)
c. Configure RBAC (role-based access control)
The majority of the Enhanced RBAC commands are included in the bos.rte.security fileset.
Authorizations are assigned to roles, which may then be assigned to user.
KST stands for Kernel Security Tables
o lskst
Enhanced RBAC security database to be stored in LDAP
o System-defined authorizations cannot be stored in LDAP and will remain local to
each client system.
If enhanced_RBAC of sys0 is true, RBAC is enhanced. You can change it to false to go back to
Legacy RBAC.
Predefined roles:
o ISSO (Information System Security Officer)
„h The most powerful role
o SA: (System Administrator)
„h Cannot change passwords
o SO: (System Operator)
To list the roles:
- # lsrole ALL | awk '{print $1}'
AccountAdmin
BackupRestore
DomainAdmin
FSAdmin
SecPolicy
SysBoot
SysConfig
isso
sa
so
Add role to a user: (for example add shutdown and reboot privilege to user salehi)
- # lssecattr -c /usr/sbin/reboot | awk '{print $2}'
accessauths=aix.system.boot.reboot
- # lssecattr -c /usr/sbin/shutdown | awk '{print $2}'
accessauths=aix.system.boot.shutdown
- There might be an existing role that contains above authorizations:
# lsrole ALL | grep ¡§aix.system.boot.reboot¡¨ | awk '{print $1}'
SysBoot
- Assign the role:
# lsuser -a roles salehi
salehi roles=SysBoot
# chuser roles=SysBoot Salehi
# lsuser -a roles salehi
salehi roles=SysBoot
The user itself can list the roles:
# su - salehi -c "rolelist"
SysBoot System Boot Administration
Activate the role:
- If the user does not activate a role, it is still an ordinary user without any role.
- # swrole SysBoot (switches to SysBoot role)
- # swrole ALL (switches to all user roles)
- # rolelist ¡Ve (lists effective roles)
SysBoot System Boot Administration
Role authentication:
Be default user should provide password to activate a role. Because auth_mode=INVOKER.
# lsrole -a auth_mode SysBoot
SysBoot auth_mode=INVOKER
You can change it:
# chrole auth_mode=NONE SysBoot
# lsrole -a auth_mode SysBoot
SysBoot auth_mode=INVOKER
Create a user-defined role:
The goal is to assign a role to a user to enable him to change cron settings:
# lsauth ALL | grep cron | cut -f1 -d' '
aix.system.config.cron
Only ¡§sa¡¨ (system administrator) has this authorization:
# lsrole ALL | grep aix.system.config.cron | cut -f1 -d' '
sa
So we need to define a role:
# mkrole authorizations="aix.system.config.cron" cronRole
Assign the role to the user:
# chuser roles=cronRole salehi
Read the RBAC security database files and load the information from the database files into the
Kernel Security Tables (KST):
# setkst
Now Salehi can change root¡¦s crontab:
# su ¡V Salehi
# swrole ALL
# crontan ¡Ve root
Another example:
Grant write access to /etc/hosts to operator2 (you need to create a new authorization for it):
root:/> mkauth newauth
root:/> setsecattr -f writeauths=newauth /etc/hosts
root:/> mkrole authorizations=newauth etchostsRole
root:/> chuser roles=etchostsRole operator2
root:/> setkst
root:/> su - operator2
operator1:/home/operator2> swrole ALL
operator1:/home/operator2> vi /etc/hosts
Install and Maintain AIX (11%)
a. Determine correct installation source (CD/DVD, NIM, cloning, alternate disk install, etc)
Minimum memory supported by AIX 6.1 is 265 MB.
b. Determine correct installation type (preservation, migration, new/complete overwrite)
¡§New and complete overwrite¡¨ destroys everything on the specified disks.
¡§Migration¡¨ changes the AIX version and/or release (like from 5.3 to 6.1)
¡§Preservation¡¨ method keeps user data in rootvg intact. But removes /usr, /, /var and /tmp
c. Install, check and remove updates, TLs and fixes. Describe lpp statuses and tasks (commit,
apply, or reject using lslpp), and debug install errors using lppchk
# installp -r <package_name> <== rejects an applied software
# installp -c all <== commits all
# installp -C <== cleanup after a fialed or interrupted software install
# installp -acgYd /dev/cd0 cluster.* (install, commit, requisite install, accept license, path of
source media)
d. Describe various options to acquire updates and fixes (SUMA, FLRT)
List the SUMA global configuration settings:
# suma ¡Vc
Change SUMA global configuration settings:
# suma -c -a HTTP_PROXY=http://user:pass@proxysrv:8080
Download critical fixes now:
# suma -x -a Action='download' -a RqType=' Critical'
To see the difference between available fixes and what you in /soft/AIX/6.1/AIX61TL6:
# suma -x -a Action='Preview' -a DLTarget='/TL' -a FilterDir='/soft/AIX/6.1/AIX61TL6'
FLRT stands for Fix Level Recommendation Tool an IBM useful page.
e. Install additional IBM and Open Source licensed program products (rpm, rte, bff, etc.)
f. Install and configure a basic NIM environment (what it is and what must be configured)
nimconfig: (configures the nim master. requires bos.sysmgt.nim.master)
To define a NIM master only:
# nimconfig -a netname=NIMnet0 -a pif_name=en0
niminit: (configures the nim client)
# niminit -a name=testlpar -a master=nimsrv1 -a pif_name=en0 -a netboot_kernel=mp
nim: (performs operations on NIM resources)
# nim -o allocate -a spot=spot1 -a lpp_source=lppAIX61 nimclient1
# nim -Fo reset nimclient1
# nim -Fo deallocate -a subclass=all testlpar
Lots of operations are possible, like: define, change, create, restvg, ...
nimclient: (performs NIM operations in NIM client side)
# nimclient -l (shows the resrouces)
# nimclient -Fo reset (resets the NIM client)
g. Obtain and validate system and device firmware, including considerations for 'deferred' and
'concurrent' maintenance.
Concurrent update:
Firmware that can be applied and activated on running systems.
Deferred update:
Firmware can be concurrently applied but contains some fixes that can't be activated until the
next IPL because the fixes affect the IPL path.
Disruptive upgrade/update:
A platform IPL is required to activate. None of the content contained in the release/service pack
will be activated until the next IPL.
Activated Level of firmware:
The level running in memory. Normally when you apply the firmware, it is saved in NRAM, but
in next IPL it will be loaded to memory.
Accepted Leve of firmware:
The level saved on p-side of flash.
Logical Volume, File and Filesystem management (7%)
a. Enlarge and reduce file systems
b. Describe and differentiate between physical volumes and LVMs, logical volumes, physical and
logical partitions, and physical disk and physical partition size.
c. Manage Volume Groups including mirroring (mkvg, varyonvg, varyoffvg, extendvg, exportvg,
importvg, lsvg)
Volumg group quorum:
# chvg ¡VQn testvg <== truns off quorum
If quorum if set to "y", when the volume group loses quorum of VGDAs, it will be automatically
varied off.
If a volume group loses its quorm of disks, it can be varied on only force (varyonvg -f)
d. Describe and manage different types of Logical Volumes, including mirroring.
e. Describe and manage different types of filesystems and different logging methods (mkfs, chfs,
fsck, mount, snapshot, etc.)
# umount -f <== forces the umount, even if the path busy or for remote filesysems if the remote
server is not present.
# fcsk -p <== Does not display messages about minor problems but fixes them automatically.
mounting an ISO image:
Method1 (for older AIX versions):
Create a logical volume, dd the ISO image to the LV, then mount the LV:
# mklv -y dvd_lv testvg 5G
# dd if=isofile of=/dev/dvd_lv bs=1m
# mount -v cdrfs -o ro /dev/dvd_lv /mnt
How to unmount:
¡§umount¡¨ command is used to unmount the image.
Method2 (recommended):
Using loopback device in AIX 6.1 TL4+ and VIOS:
# mkdev -c loopback -s node -t loopback # this creates loop0 once forever.
# lsdev -Cc loopback
loop0 Available Loopback Device
# loopmount -i /soft/TSM/TSMserver.iso -l loop0 -o "-V cdrfs -o ro" -m /mnt
How to unmount:
If you unmount the image using ¡§umount¡¨ command, loop0 device will not be
unconfigured. You can use loopumopunt instead:
# loopumount -l loop -m /mnt
mounting an USB flash:
snapshot:
Split-mirror backup:
# chfs -a snapdir=/backup -a copy=3 /testfs
Now you can backup /backup. When you remove /backup, /testfs will be resynced automatically
which might take a very long time with unwanted I/O load.
Question: Is there any limitation for the number of snapshots of a filesystem? something like 15
or 16?
Yes: The maximum number of external snapshots per file system is 15, while the maximum
number of internal snapshots per file system is 64.
There is another method which uses "snapshot" command and used copy-on-wirte algorithm:
Changes will go to the snapshot storage. From AIX 6.1 onwards, you can use internal snapshots,
it means the space to store snapshot is inside the filesystem itself.
Create external snapshot:
# mklv -y newsnaplv -t jfs2 datavg 4
# snapshot -o snapfrom=/mksysbs newsnaplv <== newsnaplv is the snapshot device
or
# snapshot -o snapfrom=/mksysbs -o size=128MB <== create the snapshot LV automatically
Verify:
# snapshot -q /mksysbs
Snapshots for /mksysbs
Current Location 512-blocks Free Time
/dev/newsnaplv 2097152 2096384 Mon May 16 12:37:13 2011
* /dev/fslv06 524288 523520 Mon May 16 12:38:37 2011 <==
* means current snapshot
you can mount a snapshot:
# mount -o snapshot /dev/fslv06 /mnt
¡P /mnt will contain the contents of /mksysbs when you created the snapshot. (remember
the copy-on-write method).
¡P It is mounted as read-only by default.
How to rollback: <== this will remove the snapshot
You have changed something in /mksysbs filesystem and want to rollback:
# umount /mksysbs
# rollback -v /mksysbs /dev/fslv06
Delete the snapshot:
# snapshot -d /dev/fslv06
Note:
Internal snapshot should be enabled only at filesystem creation time:
# crfs -v jfs -m /testfs -g rootvg -A yes -a isnapshot=yes -a size=1G
copcy some file to /testfs.
# snapshot -o snapfrom=/testfs -n monsnap
# rollback -v -n monsnap /testfs
Shrinking filesystem and defragfs with a snapshot is not supported.
In order to backup the snapshot of a filesystem, use "backsnap" command.
f. Configure and manage symbolic and hard links
Hard link: Two file names that refer to the same i-node
- Source and target should be in the same filesystem
- ln: cannot hard link directory (only files)
- # ln source target
- If you remove source or target, the other one still refers to the i-node and works fine. I-node
will be removed if all references (links) are deleted.
Soft/symbolic link:
- points to the name of source file/directory, not the i-node
- can be used across filesystems
- # ln -s source target
- If source is removed, target will become a dangling reference (= a pointer that points to
something that does not exist).
g. Demonstrate understanding of multipath I/O
Multipath I/O or MPIO means establishing more than one path between the two ends of an I/O
stream like between AIX and a disk subsystem. The purpose of MPIO is to provide more
resilience and/or better I/O throughput.
- AIX native MPIO supports only failover (and no load balancing) for all MPIO-capable disk
subsystems.
- Each disk vendor should provide a special device driver to provide more advanced
algorithms like round-robin, extended round-robin. Examples are IBM SDDPCM (Subsystem
Device Driver Path Control Module), Hitachi HDLM (Dynamic Link Manager), EMC
PowerPath and so forth.
- AIX native MPIO commands:
# lspath
# mkpath
# chpath
# rmpath
Problem Determination and Resolution (15%)
a. Use logs to identify problems (errlog, alog, syslog, etc.)
b. Use the diag utility
c. Use traces, truss, snap and kdb
trace:
The trace daemon records selected system events.
Trace has different data collection modes:
- Alternate (default):
o All trace events are captured in the trace log file.
o If the log file reaches the max size, file is overwritten from beginning.
- Circular:
o Circular logging occurs within trace ¡§buffer¡¨. Log file is generated only when trace is
stopped.
o Useful when user knows when the problem occurs. So, if they stop the trace exactly
after they encounter the problem, buffer contains useful information that will be
save in log file.
o # trace -l
- Single buffer:
o Trace stops when the in-memory trace buffer fills up.
o The contents of the buffer are captured in the trace log file.
o # trace -f
- Buffer Allocation:
o By default, buffers are allocated from the kernel heap.
o If requested size is not fit into kernel heap, it will be allocated in separate segments
from pinned memory.
o # trace -b or -B
The default trace log file is /var/adm/ras/trcfile. This is a binary file that should be viewed by
trcrpt.
Running trace in interactive mode:
# trace
> ! anycommand
> q
Running trace in background:
# trace -a -o /tmp/my_trace_log; anycmd; trcstop
trcrpt:
Formats a report from the trace log with the format the is implied from /etc/trcfmt.
# trcrpt -o /tmp/newfile
truss:
truss command is useful for tracing system calls in one or more processes:
A simple example:
# truss -ea hostname
execve("/usr/bin/hostname", 0x2FF22C90, 0x20012ED8) argc: 1
argv: hostname
envp: AUTHSTATE=compat TERM=xterm SHELL=/usr/bin/bash
SSH_CLIENT=10.0.62.14 1781 22 SSH_TTY=/dev/pts/0
LOCPATH=/usr/lib/nls/loc USER=root ODMDIR=/etc/objrepos
MAIL=/usr/spool/mail/root
PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java5/jre/bin:/usr/java5/bin:
LOGIN=root PWD=/home/salehi LANG=C TZ=CST6CDT
PS1=\[\]\u\[\]@\[\]\h\[\]:$PWD\[\]>
SHLVL=1 HOME=/ LC__FASTMSG=true MAILMSG=[YOU HAVE NEW MAIL]
LOGNAME=root SSH_CONNECTION=10.0.62.14 1781 10.0.84.79 22
DISPLAY=salehi:0 _=/usr/bin/truss OLDPWD=/ AIXTHREAD_SCOPE=S
NLSPATH=/usr/lib/nls/msg/%L/%N:/usr/lib/nls/msg/%L/%N.cat
gethostname(0x2FF22AE4, 256) = 0
kioctl(1, 22528, 0x00000000, 0x00000000) = 0
testlpar
kwrite(1, " t e s t l p a r\n", 9) = 9
kfcntl(1, F_GETFL, 0x2FF22FFC) = 67110914
kfcntl(2, F_GETFL, 0x2FF22FFC) = 67110914
_exit(0)
As you see, ¡§-e¡¨ could be useful to find out what environment vairiables are passed to a
command or program.
snap:
snap command gathers extensive system configuration information.
To gather HACMP information:
# snap -e
To gather all system configuration except HACMP and create a compressed pax output:
# snap ¡Vca
The output pax file will be stored in /tmp/ibmsupt.
snap can be used to restore from dump device:
???
kdb:
kdb is an interactive utility that allows for the examining of a system or live dump or a running
kernel.
d. Describe and use ODM
e. Configure and use system dump devices
sysdumpdev -l and os forth...
f. Recover from a full file system
g. Troubleshoot common boot LED codes and access a system that will not boot
LEDs: 0c0...0c9 and 0cc are all related to dump
LED Description
201 Invalid boot image
223-229 Invalid boot list
551-555-557 Corrupted filesystem or JFS log
552-554-556 Corrupted superblock or ODM
553 Invalid /etc/inittab
C40 configuration files are being restored
C41 Could not determine the boot device
C42 Extracting data files from diskette
C43 Cannot access the install tape
C44 Initializing configuration database for target disks
C45 Cannot configure the console
C46 Normal installation processing
C47 Could not create PVID on disk
C48 Prompting for user input
C49 Could not create or form the JFS log
C50 Creating root volume group
C51 No paging devices were found
C52 Changing from RAM environment to disk environment
C53 /tmp is small for preservation installation
C54 Installation BOS or other packages
C55 Could not remove an LV in preservation installation
C56 Running user-defined customization
C57 Failure to restore BOS
C58 Displaying message to turn the key
C59 Could not copy info from RAM to disk
C61 Failure to create boot image
C62 Loading debug files
C63 Loading data files
C64 Failed to load data files
h. Troubleshoot installation hangs and failures
i. Debug shell script common interpreter problems (ksh, etc)
j. Recover a logical volume
k. Find and correct corrupted filesystems, superblocks, etc.
Process and Performance Management and Tuning (9%)
a. Use the system resource manager
b. Understand and use Workload Manager (WLM) at a basic level
# wlmassign --> Manually assigns processes to a Workload Management class
# mkclass -> Creates a Workload Management class
# lsclass
# chclass
# rmclass
# lswlmconf
# wlmstat
# wlmcntrl -->Starts or stops the Workload Manager.
# confsetcntrl
c. Use cron and at at a detailed level
The format of crontab file:
minute hour day_of_month month weekday command
d. Use tuning tools and parameters (ioo, vmo, no, /etc/tunables, etc)
e. Use performance monitoring tools (topas, netstat, vmstat, lvmstat, iostat, svmon, nmon)
f. Monitor and change process execution (ps, nice, kill)
Planning and Documentation (11%)
a. Understand Workload Partitions (WPARs) and when to use them
WPAR products consists of two parts:
The part that is included in AIX 6.1
WPAR products consists of two parts:
¡P The part that is included in AIX 6.1
¡P Workload Partition Manager.
¡P WPAR managre help "Live Application Mobilty" (even automatic mobility)
¡P Each WPAR uses /usr and /opt as read-only.
WPAR types:
¡P System partitoin
It is a miniture copy of AIX.
Create --> (defined state) --> run (active state) --> stop --> (defined state) --> remove
¡P application partition
The idea is that we put a WPAR around an application. When the applications start, WPAR is
created, and when it stoped, WPAR would be removed.
Basic commands:
# mkwpar -n wpar1
# lswpar
# startwpar wpar1
# stopwpar wpar1
Applicatioin mobity:
chkpwpar <-- checkpoints (or freezed the partitoin to a statefile)
restartwpar <-- resumes a WPAR probably on a different machine.
When you create a WPAR, in order to mark it as a mobile workload partition you need to specify
an NFS server. This NFS server will hold the state of WPAR during mobility.
You cannot move a WPAR to a different hardware version (like POWER5 to POWER6).
b. Plan HMC configuration (networking, redundancy, users, security, etc.)
c. Describe the use and function of VIO
d. Partition planning (micropartitioning, memory planning, HEA/IVE, processor allocation, etc)
e. Document a system (sysplan, etc)
f. Find appropriate resources (info center, key center, etc.)
g. Determine system redundancy requirements (avoiding single points of failure)
h. Describe applicability and use of Capacity on Demand
Permanent:
¡P It is a purchage agreement
¡P You cannot turn it off
¡P One processor or one GB or memroy
Trial CoD
¡P 30 contiguous days
On/Off CoD
¡P Temporary additonal processor or memory
¡P Activity is reported monthly to IBM
¡P Charged vased on number of days, even one minutes!
¡P Monthly charge
Utility CoD
¡P Similar to on/off, but charge is based on minutes rather than days.
¡P For Power6+
Capacity Backup:
¡P Reserve capacity for backup server
¡P Works up to 90 days
HMC and Partition Management (6%)
a. Apply HMC and Server fixes
b. Define, add, remove resources from an LPAR (DLPAR and partition profiles, etc.)
c. Backup and restore the HMC
d. Use the HMC and ASMI interface,
e. Understand and use IVM (options, functions, etc.)
f. Configure and use electronic service agent
ESA is a free software on AIX 5.3 TL6+ and if configured properly, sends error information to IBM to
aid in problem resolution.
ESA client is freely available on all IBM systems plus DS8000.
# smit esa_main
Starting electronic service agent:
# startsrc -s IBM.ESAGENT
Miscellaneous:
multibos:
¡P Manipulates multiple versions of BOS in rootvg. It means you have more than one operating
system in the rootvg disks. Except /, /usr, /var and /opt, all other filesystems and logical volumes
would be shared between BOS instances.
¡P It is like alternate disk install, but does not require additional disks.
¡P choosing between BOS instances is possible when you set boot list
¡P Setup:
# multibos -R <== Removes all standby BOS objects
# multibos -sXp <==To perform a standby BOS setup operation preview
# multibos -sX <==To perform a standby BOS setup operation
# multibos -sXp -M /soft/mksysb1 <==To perform a standby BOS setup operation preview from
an existing mksysb
# bootlist -m normal -o
hdisk0 blv=bos_hd5 pathid=1
hdisk0 blv=hd5 pathid=1
To make sure you are booting from the right instance, compare the boot device when AIX is
starting in SMS with what bootlist shows:
# bootlist -m normal -ov
'ibm,max-boot-devices' = 0x5
NVRAM variable: (boot-device=/vdevice/v-scsi@30000002/disk@8100000000000000:4
/vdevice/v-scsi@30000002/disk@8100000000000000:2)
Path name: (/vdevice/v-scsi@30000002/disk@8100000000000000:4)
match_specific_info: ut=disk/vscsi/vdisk
hdisk0 blv=bos_hd5 pathid=1
Path name: (/vdevice/v-scsi@30000002/disk@8100000000000000:2)
match_specific_info: ut=disk/vscsi/vdisk
hdisk0 blv=hd5 pathid=1
# alog -of /etc/multibos/logs/op.alog <== to view the log
# lsvg rootvg -l | grep bos_
bos_hd5 boot 1 1 1 closed/syncd N/A
bos_hd4 jfs2 10 10 1 closed/syncd /bos_inst
bos_hd2 jfs2 70 70 1 closed/syncd /bos_inst/usr
bos_hd9var jfs2 12 12 1 closed/syncd /bos_inst/var
bos_hd10opt jfs2 13 13 1 closed/syncd /bos_inst/opt
# multibos -S <== initiates an interactive session to the standby BOS
# multibos -Xac -l /TL <== applies a TL on standby BOS
How to change back the bootlist:
# bootlist -m normal -o hdisk0 blv=hd5
Encrypted filesystem:
EFS helps to protect data on filesystem by assigning each user a unique encryption key. When a user
requests access to a file, kernel checks the credentials. The cryptographic information is kept in the
extended attribute of the file. This is an additional granularity and flexibility to traditional access
permissions.
- How to enable EFS:
# efsenable -av
This will create /var/efs directory (that keeps keystores) and alters /etc/security/user and
group.
- Create two EFS-enabled filesystem:
# crfs -v jfs2 -g rootvg -m /sales -a size=100M -a efs=yes
# crfs -v jfs2 -g rootvg -m /finance -a size=100M -a efs=yes
- Make users to access each filesystem:
# mkuser saleman; passwd salesman
# mkuser financeman; passwd financeman
- passwd in previous step, causes to create a separate directory (here called keystore) for the
user in /etc/efs/users:
# ls /var/efs/users/
total 0
-rw------- 1 root system 0 Apr 26 05:52 .lock
drwx------ 2 root system 256 Apr 26 06:08 finance
drwx------ 2 root system 256 Apr 26 05:52 root
drwx------ 2 root system 256 Apr 26 06:08 sales
- demostration:
# mount /finance
# su -finance
# mkdir -p /finance/yearlyreport
# chmod -R 777 /finance/yearlyreport „h look at full permission
# efsmgr -E /finance/yearlyreport „h enables efs for the directory
# efsmgr -L /finance/yearlyreport „h list
EFS inheritance is set with algorithm: AES_128_CBC
Login back:
# su - finance
# touch /finance/yearlyreport/anewfile
touch: /finance/yearlyreport/anewfile cannot create
But you can load the keystore and run a command:
# efskeymgr ¡Vo <thecommand>
# efskeymgr ¡Vo bash „h this will open a bash session
Now you can touch the file.
# ls ¡VU „h for security information
drwxrwxrwxe 2 finance staff 256 Apr 26 08:29 yearlyreport
Some HMC tips:
¡P HMC web acces port is 443
¡P Each POWER system has three users by default in ASM: admin, general and HMC. The HMC user
is the one hardware management console uses to be authenticated against when it discovers
the machine.
Trusted Execution:
Trusted Execution is a security feature of AIX 6.1. To some extent it is similar to TCB, but:
¡P TCB should enabled at installation phase.
¡P TCB checks the integrity in time intervals using cron.
¡P TE check the integrity of command when they are invoked.
SEA on HEA:
Is SEA possible on HEA in promiscuous mode?
Answer: Yes
sugroup:
http://www.ibm.com/developerworks/aix/library/au-sugroup/index.html
/etc/objrepos/errnotify:
http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp?topic=/com.ibm.aix.baseadmn/doc/b
aseadmndita/HT_baseadmn_missingpv.htm
and
http://www.blacksheepnetworks.com/security/resources/aix-error-notification.html
Disabling JFS2 logging:
# mount -o log=NULL /testfs
Add more ¡K.
Hope this helps,
Mehdi