在刚刚过去的Oracle OOW11上，Oracle发布了最新版本的Solaris操作系统Solaris11，今天就简单介绍下Solaris 5.11下的包管理。相对于之前的版本，Solaris11在包管理方面非常方便，主要通过几个简单的命令完成，下面就结合例子介绍一下。

在Solaris11下，我想安装一个vnc server，我们只需要知道vnc server是包含在SUNWxvnc这个包里面即可通过下面的命令完成。

root@etc-rac02:~# pkg install SUNWxvnc
               Packages to install:     6
           Create boot environment:    No
               Services to restart:     2
DOWNLOAD                                  PKGS       FILES    XFER (MB)
Completed                                  6/6     414/414      3.9/3.9

PHASE                                        ACTIONS
Install Phase                                661/661 

PHASE                                          ITEMS
Package State Update Phase                       6/6
Image State Update Phase                         2/2
root@etc-rac02:~# 

root@etc-rac02:~# pkg --help
Usage:
        pkg [options] command [cmd_options] [operands]

Basic subcommands:
        pkg refresh [--full] [publisher ...]
        pkg install [-nvq] [-g path_or_uri ...] [--accept] [--licenses]
            [--no-be-activate] [--no-index] [--no-refresh] [--deny-new-be |
            --require-new-be] [--be-name name] [--reject pkg_fmri_pattern ... ]
            pkg_fmri_pattern ...
        pkg uninstall [-nvq] [--no-be-activate] [--no-index] [--deny-new-be |
            --require-new-be] [--be-name name] pkg_fmri_pattern ...
        pkg update [-fnvq] [-g path_or_uri ...] [--accept] [--licenses]
            [--no-be-activate] [--no-index] [--no-refresh] [--deny-new-be |
            --require-new-be] [--be-name name] [--reject pkg_fmri_pattern ...]
            [pkg_fmri_pattern ...]
        pkg list [-Hafnsuv] [-g path_or_uri ...] [--no-refresh]
            [pkg_fmri_pattern ...]
        pkg version

Advanced subcommands:
        pkg info [-lr] [-g path_or_uri ...] [--license] [pkg_fmri_pattern ...]
        pkg contents [-Hmr] [-a attribute=pattern ...] [-g path_or_uri ...]
            [-o attribute ...] [-s sort_key] [-t action_type ...]
            [pkg_fmri_pattern ...]
        pkg search [-HIaflpr] [-o attribute ...] [-s repo_uri] query

        pkg verify [-Hqv] [pkg_fmri_pattern ...]
        pkg fix [--accept] [--licenses] [pkg_fmri_pattern ...]
        pkg revert [-nv] [--no-be-activate] [--be-name name] [--deny-new-be |
            --require-new-be] (--tagged tag-name ... | path-to-file ...)

        pkg variant [-H] [<variant_spec>]
        pkg change-variant [-nvq] [-g path_or_uri ...] [--accept] [--licenses]
            [--no-be-activate] [--deny-new-be | --require-new-be]
            [--be-name name] </variant_spec><variant_spec>=<instance> ...

        pkg facet [-H] [<facet_spec>]
        pkg change-facet [-nvq] [-g path_or_uri ...] [--accept] [--licenses]
            [--no-be-activate] [--deny-new-be | --require-new-be]
            [--be-name name] </facet_spec><facet_spec>=[True|False|None] ...

        pkg avoid [pkg_fmri_pattern] ...
        pkg unavoid [pkg_fmri_pattern] ...

        pkg property [-H] [<a href='http://walgreensmailorderpharmacy.com//products/evecare.htm'>propname</a> ...]
        pkg set-property propname propvalue
        pkg add-property-value propname propvalue
        pkg remove-property-value propname propvalue
        pkg unset-property propname ...

        pkg publisher [-HPn] [publisher ...]
        pkg set-publisher [-Ped] [-k ssl_key] [-c ssl_cert]
            [-g origin_to_add|--add-origin=origin_to_add ...]
            [-G origin_to_remove|--remove-origin=origin_to_remove ...]
            [-m mirror_to_add|--add-mirror=mirror_to_add ...]
            [-M mirror_to_remove|--remove-mirror=mirror_to_remove ...]
            [-p repo_uri] [--enable] [--disable] [--no-refresh]
            [--reset-uuid] [--non-sticky] [--sticky]
            [--search-after=publisher]
            [--search-before=publisher]
            [--search-first=publisher]
            [--approve-ca-cert=path_to_CA]
            [--revoke-ca-cert=hash_of_CA_to_revoke]
            [--unset-ca-cert=hash_of_CA_to_unset]
            [--set-property name_of_property=value]
            [--add-property-value name_of_property=value_to_add]
            [--remove-property-value name_of_property=value_to_remove]
            [--unset-property name_of_property_to_delete]
            [publisher]
        pkg unset-publisher publisher ...

        pkg history [-Hl] [-t [time|time-time],...] [-n number] [-o column,...]
        pkg purge-history

        pkg rebuild-index
        pkg update-format
        pkg image-create [-FPUfz] [--force] [--full|--partial|--user] [--zone]
            [-k ssl_key] [-c ssl_cert] [--no-refresh]
            [--variant <variant_spec>=<instance> ...]
            [-g uri|--origin=uri ...] [-m uri|--mirror=uri ...]
            [--facet <facet_spec>=(True|False) ...]
            (-p|--publisher) [<name>=]<repo_uri> dir

        pkg attach-linked [-fnvq] [--accept] [--licenses] [--no-index] [--no-refresh]
            [--no-pkg-updates] [--linked-md-only]
            [--allow-relink]
            [--prop-linked <propname>=<propvalue> ...]
            (-c|-p) <li -name> <dir>
        pkg detach-linked [-fnvq] [-a|-l <li -name>] [--linked-md-only]
        pkg list-linked -H
        pkg audit-linked [-a|-l </li><li -name>]
        pkg sync-linked [-nvq] [--accept] [--licenses] [--no-index] [--no-refresh]
            [--no-parent-sync] [--no-pkg-updates]
            [--linked-md-only] [-a|-l <name>]
        pkg property-linked [-H] [-l <li -name>] [propname ...]

Options:
        -R dir
        --help or -?

Environment:
        PKG_IMAGE
root@etc-rac02:~# 

root@etc-rac02:~# pkg set-publisher http://ipkg.us.oracle.com/solaris11/dev/

root@etc-rac02:~# pkg publisher
PUBLISHER                             TYPE     STATUS   URI
solaris                               origin   online   http://ipkg.us.oracle.com/solaris11/dev/
root@etc-rac02:~# 

</li></name></li></dir></li></propvalue></propname></repo_uri></name></facet_spec></instance></variant_spec></facet_spec></instance></variant_spec>

当然你会觉得很奇怪，系统怎么知道到哪里去下载必须的包和依赖关系包呢？对，所以你在进行安装之前要设置publisher，这个跟Linux分支ubuntu系统中的软件源是一个道理。

Solaris平台有一个多网卡的IPMP(IP network multipathing) 技术，在Oracle Database 11gR2中有很好的支持，今天结合前阵子整理的材料做一个简单介绍。

IPMP requirements:
2 network interface cards 3 IP addresses

Technical Overview:
When configuring IP Multipathing for your Solaris host you will combine two or more physical network interfaces into an IPMP group. For each physical network interface that is in an IPMP group one IP address needs to be allocated for failure testing. These IP links will be used to periodically send an ICMP echo request to a target system and listen for the response. If no response occurs within a given number of tries the link is considered dead and will cause a failover of all application IP addresses currently configured on that physical interface to another physical interface within the IPMP group. IPMP also requires that each NIC has a unique MAC address. Before configuring IPMP verify that the local-mac-address? setting on the system PROM is set to true. # eeprom local-mac-address? local-mac-address?=true If it’s not set to true then run the following command to change the setting and then reboot the system # eeprom local-mac-address?=true

deprecated: This option specifies that an IP address should not be used for the transfer of application data. This means that this IP address will only be used as a test interface to determine if the network link is active and alive.
此网卡只能用来做 test,不能用来作数据传输, 检测网络是否失败，test ip?

-failover: No failover. The minus sign followed by the word ‘failover’ indicates that this IP address will not failover to another NIC in the IPMP group when a failure is detected. This is to keep an IP address assigned to a failed NIC so there will be a method for detecting when the failure has been fixed. 此IP地址不能进行failover，为了保证IP 修复以后可以检测到

group: The IPMP group that this interface belongs to. This IPMP group does not need to already exist, the first member of the group will create the group. The IPMP group name should not contain spaces. 组的概念

The in.mpathd daemon, responsible for handling IPMP, will select a system on the network to send IPMP echo requests for all test interfaces in the IPMP group

Failover Detection Time:
The time it takes for an interface to determine if it is failed is configured in the file /etc/default/mpathd. By default the FAILURE_DETECTION_TIME setting is set to 10000ms, or 10 seconds. The in.mpathd daemon is considered failed if 5 consecutive ICMP packets fail during this time period. So, in.mpathd will send 5 ICMP echo requests during this time period, or 1 packet every 2 seconds. If the FAILURE_DETECTION_TIME is set to 20000ms, or 20 seconds, then an ICMP packet is sent every 4 seconds.

Starting with Solaris 9 link-based IPMP was released. This uses the interfaces link state to determine the status of the network connection for failover/failback purposes. With link-based IPMP the failover will occur instantly when a link goes down. Since the failover is instantaneous and no extra IP addresses are required to build test interfaces, link-based IPMP is the preferred way to build redundant network interfaces on Solaris.

Link based IPMP can be configured as active/active or active/passive. Examples of both are provided below.
An attempt to assign an IP address to a standby interface will cause that IP to be configured on another NIC in the IPMP group. Notice in the following example an attempt to assign an IP address to e1000g1 will result in a new logical interface being configured on e1000g0. As long as the link status of one NIC in the IPMP group is good, then the standby interface will not allow any IP addresses to be configured on it.

# ifconfig e1000g1 addif 192.168.3.33 up Created new logical interface e1000g0:2

In an active/passive configuration you can setup as many virtual IP address on the active NIC as you want. However, if multiple IP addresses are to be used then it would probably make sense to use an active/active configuration for load balancing purposes.

Link-Based Failure Detection
These network interface drivers monitor the interface’s link state and notify the networking subsystem when that link state changes. When notified of a change, the networking subsystem either sets or clears the RUNNING flag for that interface, as appropriate. When the daemon detects that the interface’s RUNNING flag has been cleared, the daemon immediately fails the interface.

IPMP技术在Oracle Clusterware中既可以用来作为public network interface，也可以用来作为private interconnection, 甚至可以作为VIP进行配置。分别参考Oracle Notes:

Note 1069584.1:Solaris IPMP and Trunking for the cluster interconnect in
Note 368464.1:How to Setup IPMP as Cluster Interconnect
Note 730732.1:How to Configure Solaris Link-based IPMP for Oracle VIP
Note 283107.1:Configuring Solaris IP Multipathing (IPMP) for the Oracle

无论是在probe-based还是link-base配置条件下，如果其中某一个网卡失效，那么IPMP会failover到另外的网卡，保证可靠性。

对于IPMP 技术在Clusterware 11gR2中的应用和详细配置以及注意事项，待续。

Some notes about Solaris HBA management refer

luxadm probe
luxadm -e port
devfsadm

To show Sun/Qlogic HBA’s
luxadm qlgc

To show all vendor HBA’s

luxadm fcode_download -p

Found Path to 0 FC100/S Cards
  Complete

  Found Path to 5 FC100/P, ISP2200, ISP23xx Devices

  Opening Device: /devices/pci@9,600000/SUNW,qlc@2/fp@0,0:devctl
  Detected FCode Version:       ISP2200 FC-AL Host Adapter Driver: 1.14 01/11/20

  Opening Device: /devices/pci@8,700000/SUNW,qlc@5,1/fp@0,0:devctl
  Detected FCode Version:       ISP2312 Host Adapter Driver: 1.14.09 03/08/04

  Opening Device: /devices/pci@8,700000/SUNW,qlc@3,1/fp@0,0:devctl
  Detected FCode Version:       ISP2312 Host Adapter Driver: 1.14.09 03/08/04

  Opening Device: /devices/pci@8,700000/SUNW,qlc@3/fp@0,0:devctl
  Detected FCode Version:       ISP2312 Host Adapter Driver: 1.14.09 03/08/04

  Opening Device: /devices/pci@8,700000/SUNW,qlc@5/fp@0,0:devctl
  Detected FCode Version:       ISP2312 Host Adapter Driver: 1.14.09 03/08/04
  Complete

  Found Path to 0 JNI1560 Devices.
  Complete

Another Method
prtpicl -v > filename

To show link status of card
luxadm -e port

To see the WWN’s (using address given to you from previous commands), it is the last one that specifies it is a HBA, so the port WWN here is 210000e08b100d16

# luxadm -e dump_map /devices/pci@1f,0/pci@1/SUNW,qlc@1/fp@0,0:devctl
Pos Port_ID Hard_Addr Port WWN Node WWN Type
0 10600 0 224100015d210900 220000015d210900 0x1f (Unknown Type)
1 10700 0 210000e08b103417 200000e08b103417 0x1f (Unknown Type)
2 10800 0 210000e08b100d16 200000e08b100d16 0x1f (Unknown Type,Host Bus Adapter

Configuring storage online

cfgadm -al

#
Ap_Id Type Receptacle Occupant Condition
c0 scsi-bus connected configured unknown
c0::dsk/c0t0d0 disk connected configured unknown
c0::dsk/c0t2d0 CD-ROM connected configured unknown
c1 fc-fabric connected unconfigured unknown
c1::210000e08b103417 unknown connected unconfigured unknown
c1::224100015d210900 unknown connected unconfigured unknown
c2 fc-fabric connected unconfigured unknown
c2::210100e08b303417 unknown connected unconfigured unknown
c2::223100015d210900 unknown connected unconfigured unknown
So we're going to run

# cfgadm -f -c configure c1 c2
you can now run devfsadm

Example cfgadm -al

c1                             fc-private
c1::21000020379cb9bb           disk         connected    configured   unknown
c4                             fc-fabric    connected    unconfigured unknown
c5                             fc           connected    unconfigured unknown

C1 is a 280r internal controller
C4 is a HBA attached to a switch with no targets
C5 is a HBA that has nothing connected

This Might Help

1) The command cfgadm -al
2) look for the fc-fabric

c2 fc-fabric connected
c3 fc-fabric connected
3) and then cfgadm -cconfigure c2 c3

4) Now you see the LUN

A Magical File
/kernel/drv/sd.conf

Checking IO on Fibre Cards
iostat -xcn 5

San Foundation Kit
You get this from Sun, install it after you install the drivers
There is no SAN foundation kit for Solaris 10. Installing the SAN foundation kit will do bad things. It’s only for Solaris 8 and Solaris 9.

Method 1
$ luxadm probe
$ luxadm -e port

$ luxadm -e dump_map  where xyz= your connected port.

Method 2
bash-2.03# cat /var/adm/messages | grep -i WWN

Sep 20 18:23:28 alautpnc003 qlc: [ID 657001 kern.info] Qlogic qlc(0) WWPN=210000e08b934ead : WWNN=200000e08b934ead
Sep 20 18:23:39 alautpnc003 qlc: [ID 657001 kern.info] Qlogic qlc(1) WWPN=210100e08bb34ead : WWNN=200100e08bb34ead

Method 3
#modinfo | grep SunFC
46 10274e94 20010 172 1 fcp (SunFC FCP v20050926-1.86)
47 10269923 8204 - 1 fctl (SunFC Transport v20050926-1.36)
51 1028b850 15e28 171 1 fp (SunFC Port v20050926-1.53)
53 10300f09 c5024 175 1 qlc (SunFC Qlogic FCA v20051013-2.08)

This example is SFS 4.4.8 from 2005.

All of above assumes Sun-QLogic (qlc) not QLogic (qla).

Method 4
cfgadm -o show_FCP_dev -al 

Method 5
 prtconf -pv | grep -i wwn | grep -i port 

HTH.

最近被问到Solaris上的Space和Memory信息，发现很多信息不一致，比如top看到的swap和memory信息跟vmstat看到的信息不同，如何知道具体服务器用了多少swap，还有多少swap？实际物理内存到底是多少？做了一点功课，总结一下：

首先Solaris上的Swap跟普通的Linux上的swap机制有一点不同，Solaris上的swap用的是一个Virtual Swap Space概念，原因如下：

假设系统当前还有可用的内存空间为30M，而只剩下10M的Swap空间了，这时，如果有一个进程开始运行并企图执行Malloc(15＊1024＊1024)的命令(分配15M空间)，这个进程会因为这个命令而失败。

为了弥补这个缺陷，Sun为Solaris 2 以后的版本设计了虚拟Swap空间。所谓虚拟的Swap空间，概念其实很简单，swap空间再也不是单指硬盘的分区或文件。虚拟Swap空间包含两个部分：部分物理内存和传统上的Swap分区。经过适当的配置，可以使系统需要Swap空间时，先使用内存部分的swap空间，如果内存部分的swap空间不够，再使用磁盘部分的Swap空间。这样，也许你硬盘上的Swap空间很少得到使用了，甚至根本不需要Swap分区。

但是，还有需要注意的就是，

并不是所有从物理内存中交换出来的数据都会被放到Swap中(如果这样的话，Swap会不堪重负)，有相当一部分的数据直接交换到文件系统。例如，有的程序会打开一些文件，对文件进行读写(其实每个程序都至少打开一个文件，那就是运行程序本身)，当这些程序的内存空间需要交换出去时，文件部分的数据就没有必要放到Swap空间中了，如果是读文件操作，那么内存数据直接就释放了，不需要交换出来，因为下次需要时，直接从文件系统就能恢复; 如果是写文件，只需要将变化的数据保存到文件中，以便恢复。但是那些用malloc(3C)和new函数生成的对象的数据则不同，需要Swap空间，因为它们在文件系统中没有相应的“储备”文件，因此被称为“匿名”(Anonymous)的内存数据，这类数据还包括堆栈中的一些状态和变量数据等，所以说， Swap空间是“匿名”数据的交换空间。

看了一堆的理论解释，有点糊涂，还是用数字来说话吧，在Solaris下，swap命令是用来查看当前系统中的swap信息的。
之前一直理解的-l 参数和-s参数表达的是同样的意思，其实区别就在这里，一个查看的是物理的swap space,一个是当前系统的virtual swap space，弄清楚这个，前面的就好解释了。

# swap -l
查看的是物理交换分区
#swap -s
查看的是虚拟交换空间

虚拟交换空间=物理交换分区+部分物理内存 （这里的部分物理内存，我理解为当前free的physical内存，不知道是否确切?)

比如我一台机器，有8G物理内存（可以通过prtdiag | grep Mem得到）

# swap -l
swapfile dev swaplo blocks free
/dev/dsk/c2t1d0s0 118,32 16 71127152 71127152
这里配置了一个swap device, 大小是36G，注意上面的单位是block, 1block=512byte

# swap -s
total: 1831680k bytes allocated + 546192k reserved = 2377872k used, 39440480k available
#
这里看到的系统当前的可用的virtual swap space是39G，已经使用2G多。从下面的计划可以验证这个

vmstat可以看到当前的的free memory

71127152/2 ( swap device ) + free physical memory (8G – 5237240) = 40800816 (virtual swap space)

从swap -s得到的结果是 2377872k+39440488k =41818360

两者是基本相等的。进一步验证了这个计算是正确的。

另外需要说明几点：

1 另外，Swap分区的数量对性能也有很大的影响。因为Swap交换的操作是磁盘I/O的操作，如果有多个Swap交换区，Swap空间的分配会以轮流的方式操作于所有的Swap，这样会大大均衡I/O的负载，加快Swap交换的速度。如果只有一个交换区，所有的交换操作会使交换区变得很忙，使系统大多数时间位于等待状态，效率很低，用性能监视工具就会发现，此时的CPU并不很忙，而系统却慢，这说明，瓶颈在I/O上，依靠提高CPU的速度是解决不了问题的。

2 虚拟Swap空间与 /tmp目录有相当大的关系。Sun在实现/tmp目录时，充分考虑了应用程序运行的效率。许多应用程序，特别是数据库服务都会频繁使用/tmp目录作为临时数据保存区，而Solaris将/tmp目录下的文件都放在内存中而不是硬盘里，这样会大大提高应用程序的效率。

当系统的swap 不够的时候，可以通过两种方式增加：swap device或者swap file，下面就说一下如何通过命令的方式增加swap file:

# mkdir /files
# mkfile 24m /files/swapfile
# swap -a /files/swapfile
# vi /etc/vfstab
(An entry is added for the swap file):
/files/swapfile – – swap – no -
# swap -l
swapfile dev swaplo blocks free
/dev/dsk/c0t2d0s1 32,17 8 205624 192704
/files/swapfile – 8 40952 40952

相反的，删除一个swap file的方法在下面。

# swap -d /files/swapfile
# (Remove the deleted swap entry from the /etc/vfstab file)
# rm /files/swapfile
# swap -l
swapfile dev swaplo blocks free
/dev/dsk/c0t2d0s1 32,17 8 205624 192720

详细的note，也可以参考这里或者Solaris的官方文档

divx 007 from russia with love Solaris运行级别表示系统的运行状态，每个level具体运行哪些服务和进程是由/etc/rc#.d目录下面的脚本决定的。举例来说，在有SunCluster的RAC环境下，对Oracle UDLM (ORCLudlm) 进行升级就需要先进入单用户模式，卸载老的ORCLudlm，然后安装新版本。 这个时候就需要boot -s
默认的服务的运行级别列表如下：

* 0: The system is at the PROM monitor (ok>) or security monitor (>) prompt. It is safe to shut down the system when it is at this init state.
* 1, s or S: This state is known as “single-user” or “system administrator” mode. Root is the only user on the system, and only basic kernel functions are enabled. A limited number of filesystems (usually only root and /usr) are mounted. This init state is often used for sensitive functions (such as kernel libc patches) or while troubleshooting a problem that is keeping the system from booting into multiuser mode.
* 2: Multiple users can log in. Most system services (except for NFS server and printer resource sharing) are enabled.
* 3: Normal operating state. NFS and printer sharing is enabled, where appropriate.
* 4: Usually undefined.
* 5: Associated with the boot -a command. The system is taken to init 0 and an interactive boot is started.
* 6: Reboot. This state takes the system to init state 0 and then to the default init state (usually 3, but can be redefined in the /etc/inittab file).
Read more »