The Design and Implementation of the NetBSD rc.d system
netbsd中的rc.d系统的设计与实现
Luke Mewburn
Wasabi Systems, Inc.
lukem@wasabisystems.com
提要：
在这篇文章中，我将说明从netbsd1.5开始成为netbsd基本系统一部分的rc.d系统（启动机制）的设计与实现，它取代了继承自 4.4bsd的单/etc/rc启动文件。文章将讨论：不同unix（包含netbsd1.5以前的版本）的启动机制，（我们）耗费了6年的讨论，实现细节，在设计和实现上人性化的考量，以及从那以后的增强和发展。
介绍：
3年前，netbsd从传统4.4bsd的单一/etc/rc启动脚本转换到了现在的rc.d机制，这种机制用不同的脚本来管理服务和后台进程，这些脚本在系统启动后按照次序执行。
这篇文章将说明关于rc.d系统的设计动机，设计，和实现；netbsd怎样转换到这个系统的历史（2000.12月，netbsd1.5起用这个系统），以及将来的发展.
在netbsd中,这些变化带来了异议,并且大家踊跃的讨论其每一个变化.这些讨论将帮助我们有助我们了解它的设计与实现机理.
历史:
不同的unix对启动机制的实现是多样的.下面象详细介绍一些其原理.netbsd源自4.4bsd,将说明其有关的信息.solaris的启动机制也将被说明,因为它被用于大部分sysv系统.
1,4.4bsd的启动顺序(机制)极其简单.
当我们启动到多用户模式,kernel将运行init(/sbin/init),init使用sh(/bin/sh)启动/etc/rc, rc脚本将检查
文件系统的一致性,装载磁盘,运行系统进程...../etc/rc调用/etc/netstat配置网络和与其相干的服务, /etc/rc.local(如果它存在的话)启动本地添加的系统服务.当/etc/rc成功运行完毕,init将为每个/etc/ttys复制 (fork)一个自己(init),而ttys一般运行/usr/libexec/getty.
对系统服务的管理通常是编辑脚本/etc/rc,/etc/rc.local,/etc/netstat.一些时候,仅改变shell变量, 而另外的情况是添加,改变,移除其中的命令(即可).
4.4bsd并没有shutdown的有关脚本.当init收到sigterm信号,它将对每个terminal的进程发送 sighup信号,一般10秒后这些进程将收到这个信号,然后再过10秒后重复发送sigterm信号,又在过10秒,sigkil信号将被发送给它们. 所有进程退出后(即第3个10秒结束),init将会把系统切换到单用户模式,或者根据情况关机.
2,solaris 7
solaris系统源自sysv,它参考sysv4的启动,实现了sysv的启动机制.
系统运行时,处于8个不同的运行级别之一,每个运行级别运行不同的进程.通过init x(x为数字),系统可以随时切换到不同的级别.查看当前运行级别可以使用who -r命令.
系统启动时,kernel运行init,init根据/etc/inittab运行相应的进程.在/etc/inittab中有一个 rlevel运行级别的指示域,init将启动定义在每个action域的进程.下面是不同的运行级别:
0 关闭系统,..
s or S 单用户模式,所有文件系统都装载
1 单用户模式,所有文件系统都装载,并允许一般用户登陆
2 多用户模式,所有系统服务将运行,除了nfs服务进程
3 多用户模式,所有系统服务将运行,系统默认级别.
4 一般没有使用.
5 关闭系统,并试图关闭电源
6,关闭系统,进入0级别,并重起.
每个运行级别X,受/sbin/rcX脚本控制,而/etc/rcX.d目录包含这个级别执行的脚本./sbin/rcX将以 /etc/rcX.d/K*字母顺序关闭服务,而以/etc/rcX.d/S*的字母顺序启动服务.
加入一个新服务foo,需要添加/etc/rcX.d/S*foo到适当的运行级别,并且在所有不运行这个服务的运行级别中加入 /etc/rcX.d/K*foo.通常这些(S*,K*)文件是/etc/init.d目录中相应服务脚本的连接,/etc/init.d中的脚本实现了对相应服务的start up和shut down控制.
想删除这个服务foo,所有/etc/rc?.d/[KS]*foo都需要删除.
3,netbsd1.5以前的版本.
1.3以前的版本的启动机制是和4.4bsd类似的,只做了小小的修改.下面将详细说明这些变化.
netbsd1.3
1998发布的netbsd1.3,有2个变量被加入到了启动系统中./etc/rc.conf和/etc/rc.lkm.
/etc/rc.conf包含了变量,它们被/etc/rc和/etc/netstart用来控制服务是否启动.对每个服务 foo,2个变量是:
$foo=yes | no /yes即启动....
$foo_flags= //foo的参数
/etc/rc.conf的目的是将启动脚本同控制变量分开(数据和程序分开???),这样操作系统升级时,可以尽量避免丢失site- specific(用户定义)的配置.
/etv/rc.lkm被用来控制那些kernel模块在系统启动时装载.(而配置文件)/etc/lkm.conf控制分3个阶段:网络启动前的阶段,在装载非关键文件系统(关键文件系统是:/ /usr /var...)前的阶段,和所有文件系统被装载后的阶段.(为什么分3个阶段?)因为lkm(模块)可能在本地或者远程文件系统上. /etc/lkm.conf控制了/etc/rc.lkm的行为.
2,netbsd1.4版本
1999发布的1.4版本有加入2个变化:/etc/rc.shutdown和/etc/rc.wscons
/etc/rc.shutdown被shutdown使用在/sighup信号发送前.它很有用,因为有些服务必须按照顺序被关闭(比如,数据库程序),有些服务又需要更多信号才能彻底关闭.
/etc/rc.wscons被用来控制/wscons 驱动.它的配置文件是/etc/wscons.conf
3,1.5以前版本(变化)总结.
多用户模式启动时,init调用/etc/rc初始化系统./etc/rc调用/etc/netstat启动网络服务, /etc/rc.local启动本地服务,/etc/rc.lkm初始化kernel模块,/etc/rc.wscons则配置/wscons 控制台驱动.而所有这些启动脚本被/etc/rc.conf的变量控制.
系统关闭时,shutdown调用/etc/rc.shutdown来关闭那些在init发送sighup信号之前的服务.
3,设计思路
6年中,许多围绕着增强start-up系统的想法在netbsd邮递列表curren-users,tech-userlevel中讨论,它们是netbsd的开发者邮递列表.
There was no consensus on ` One True Design ';there was too much contention for that. What is described below is an amalgamation of what a few developers felt was a reasonable analysis of the problems and feedback as well as the most reasonable solution to support the widest variety of circumstances.
(大意是说:最后没有统一的解决方案,有很多争论,开发员的合理分析,以及反馈,最终将这些分析,想法,反馈融合起来,尽量适应多中环境的解决方案.下面说的就是这些)
1,旧的(启动机制)的问题.
旧的系统有下面一些问题:
除了手工编辑/etc/rc(还有其他一些脚本)或者移除某些东西,并没有其他办法控制相关性顺序(dependeny ordering)
This caused problems at various times, in situations such as workstations with remotely mounted /usr partitions, and these problems weren't completely resolved as was seen by observing various mailing discussions and a flurry of CVS commits to the source tree.
在系统启动后,很难手动控制个别的服务(如,restart dhcpd,关闭一个数据库...)
Whilst some people suggested that a system administrator who couldn't manually restart a service was incompetent, this doesn't resolve the issue that typing " /etc/rc.d/amd restart " is significantly easier that finding the process identifier of amd , killing it, examining /etc/rc for the syntax that amd is invoked with, searching /etc/rc.conf for any site-specific options, and manually typing in the resulting command.
Unfortunately, there was a slight tendency during some of the mailing list discussions to resort to attacks on people's competency in this manner. I consider this a form of computer based intellectual snobbery, and an unreasonable justification for why that person disliked a feature.
添加第三方的启动脚本不大容易,especially addition into arbitrary points in the boot sequence, including those installed by (semi-)automated procedures such as the NetBSD `pkg' tools.
2,新系统的要求
Given the problems in the old system, and observations of what other systems have done, including those described in See History., the following design considerations were defined.
Some of these considerations were not determined during discussion prior to implementation, but were identified once users were actively using the implementation.
1,依赖顺序(dependency ordering)
Dependency ordering is a strong requirement.
The following dependency ordering requirements were determined:
按照文件名的字母顺序
Some other systems (e.g., System V init.d) use an existing lexicographical ordering of filenames in a given directory, such as /etc/rc2.d/S69inet occurring before /etc/rc2.d/S70uucp , but experience has shown that this doesn't necessarily scale cleanly when adding local or third-party services into the order; often you end up with a lot of convoluted names around ` S99 '.
方便的添加第三方启动脚本
Some people proposed running /etc/rc.d/* out of /etc/rc.local , and retaining the existing /etc/rc semantics. This doesn't easily cater to a user who requires the ability to insert their own start-up items anywhere in the boot sequence (such as a cable modem authentication daemon required for networking).
在小的root分区系统上,不会导致/bin和/sbin的膨胀,使用/usr/bin的建议被否决,因为启动时,/usr很有可能还不能用 (没有被装载)
动态的依赖顺序
A lot of debate occurred regarding whether the dependency ordering is predetermined (e.g., by creating links to filenames or building a configuration file), or dynamically generated.
A predetermined order may be more obvious to determine the order (using ` ls ' or examining the configuration file instead of invoking a special command), but it can be difficult to add a service in at a given point on a system because generally ordering is not based on services provided.
A dynamic order may slow down boot slightly, but provides the flexibility of specifying start-up order in terms of dependencies.
For example, if service C depends on B which depends on A , and I have a new service D to install that must start after A and before C then I want to specify it in these terms, without having to worry about whether it starts before B , after B , or simultaneously with B .
关于如何实现动态依赖顺序的方法的讨论有很多:
# Using make and a Makefile .
# Using tsort , awk , and a few shell commands
# Providing a dedicated ordering tool which parsed the scripts for command directives in special comments to determine the order. If a script did not have a directive, it would be ordered last.
在经过讨论和实地测试后,我们决定使用一种动态依赖顺序的工具,rcorder.它是比较合适的.而使用make或者tsort以及awk需要将这些程序放在/bin中(肯定会导致资源紧张的root文件系统的膨胀),并且,使用在出错情况下,rcorder比较好反馈信息.
2,对单个服务的控制
Most people seem to agree that the ability to manipulate an individual service (via a script) is one of the benefits of the System V init.d start-up mechanism. Having a script that allows direct starting, stopping, and restarting of a service, as well as other per-service options like `reloading configuration files', significantly reduces system administrator overheads.
Having the same script be used by the start-up sequence is also highly desirable, as opposed to using a monolithic /etc/rc for booting and separate /etc/rc.d scripts for manual control (which had been suggested).
It is interesting to note that some System V init.d implementations often start multiple services in the one file, which defeats the purpose of providing per-service control files. An example is Solaris' /etc/init.d/inetsvc , which configures network interfaces, starts named and starts inetd .
3,支持第三方脚本
An important requirement is the ability to support third-party scripts, especially by allowing them to be inserted at any place in the boot sequence order.
The current system does support third-party scripts if they are installed into /etc/rc.d . There has been discussion about allowing for different directories to be used for local and third-party scripts, in order to provide a separate `name-space' to prevent possible conflicts with a local script and a future base system script, but so far none of the suggestions has been considered sufficiently complete to provide in the default system. This, however, does not prevent a site from implementing their own method.
4,维护/etc/rc.conf
etc/rc.conf was introduced in NetBSD 1.3, and most users seem fairly happy with the concept.
One of the concerns about a traditional System V init.d style mechanism is that the control of service start-up is managed by the existing of a link (or symbolic link) from /etc/rc2.d/S69inet to /etc/init.d/inetinit , which is difficult to manage in a traditional configuration change management environment (such as RCS). Similar concerns exist regarding the suggestion of using mode bits on files in /etc/rc.d to control start-up.
/etc/rc.conf was further enhanced as described in See Configuration improvements..
5,尽量代码重用
Traditional System V init.d implementations do not appear to re-use any code between scripts. From experience, maintaining local scripts in a traditional init.d environment is a maintenance nightmare. We achieved code re-use with common functions in /etc/rc.subr which results in the average /etc/rc.d script being a small (5-10 line) file. There were some concerns raised about using these common functions, but they weren't considered to be serious issues. (We have a C library and common Makefile fragments, so why not common shell functions?)
6,服务的关闭
he ability to shut down certain services at system shutdown time with /etc/rc.shutdown was a useful feature of the previous system and of other systems, and it makes sense to retain this feature.
In the initial implementation, we reverse the dependency order, and shut down any services which are tagged with a ` shutdown ' keyword (see See rcorder.) within the script. We may modify or enhance this behavior if observation of in-field use reveals a more complicated scheme is required.
7,避免使用运行级别
We avoided the use of System V run levels (also known as run states or init states) and /etc/inittab . This was the result of many discussions about the design, which can be summarized to:
# They're just too contentious; the /etc/inittab concept had the least number of advocates. Many people expressed the opinion (both during the design phase and post implementation) that they don't mind the /etc/rc.d idea but don't think an /etc/inittab , run-levels or /etc/rcN.d directories would improve things.
# There doesn't seem to be consistency between what each run-level means on various System V init.d implementations, or the exact semantics of what occurs at state change. Thus, using the argument of compatibility for system administrator ease of use isn't as relevant. Some systems (such as HP/UX 10.x) treat these as levels, where a transition from level 4 to level 2 executes the shut down scripts in level 3 and then level 2. Other systems (such as Solaris) treat these as separate run states, where a transition to a level runs all the stop scripts in that level and then all the start scripts. This can be confusing to an administrator, as well as not necessarily providing the optimal behavior.
# We currently support single-user mode (s), multi-user mode (2 or 3, depending on whether NFS serving is configured in /etc/rc.conf ), shutting down the system to single user mode (1), halting the system (0), rebooting the system (6), and powering off the system (5) (with the equivalent Solaris init state in parenthesis).
# If the ability to take the system from a given point in the order to another point in the order, then I feel that most people's requirements for what run-levels are touted to provide would be met. This is currently a work in progress.
# Whilst /etc/inittab provides for re-spawning of daemons, in practice very few daemons are actually started that way, and it's trivial to implement that feature in a few lines of shell script as a `wrapper' to the start of the daemon.
8,其他
After various discussions, we settled on the name /etc/rc.d instead of /etc/init.d , because the implementation was different enough from the System V /etc/init.d mechanism that we decided not to confuse people expecting the exact System V semantics. Many system administrators may be used to referring directly to /etc/init.d/foo or /sbin/init.d/bar when manipulating a service; a symbolic link from /etc/init.d or /sbin/init.d to /etc/rc.d on their systems could help retain their sanity.
The first implementation of /etc/rc.d that I released for evaluation supported all three start-up schemes; the original monolithic /etc/rc , a System V init.d (without run-levels), and the current /etc/rc.d . These were all built from the same sources, and a command was provided to generate the style that an administrator preferred. After feedback and discussion, this functionality was abandoned, because:
* It is very difficult to support multiple ways of starting the system when users have problems or questions, especially so in a volunteer project.
* Two of the methods ( /etc/rc , and System V init.d) do not have the ability to dynamically order the dependency list. In those situations, an administrator (or automatic application) would have to perform the extra step of `rebuild order' upon installation.
* The source scripts had various constraints to ensure that they could work as part of /etc/rc as well as acting as a stand-alone script in /etc/rc.d or /etc/init.d .
As architects of the NetBSD operating system, we have the responsibility to provide useful solutions to problems. In general, those solutions should be as flexible as possible, without introducing unnecessary flexibility, which will only cause confusion. Therefore, the alternative mechanisms were dropped.
That said, the current system is flexible enough that if a site decided to use a System V init.d approach, it is fairly trivial to populate /etc/rcN.d with a symbolic link farm to files in /etc/rc.d (using rcorder to build the dependency list), and modify /etc/rc to run the scripts in /etc/rcN.d/ in lexicographical order, or to even implement a System V /etc/inittab and run states.
Unfortunately, there is no easy solution for people who want to retain /etc/rc . However, as NetBSD is an Open Source project and allows for public access to the CVS source code repository (via anonymous CVS as well as via a WWW front-end [6]), nothing prevents users from reverting to the old style /etc/rc .
It is interesting that the people who argued the most to retain /etc/rc are probably those who are skilled enough to maintain this, and during the various discussions some even offered (some might say "threatened") to maintain their own copy of /etc/rc in their own public CVS server for those who wished to retain this functionality. Interestingly, over a year has passed since the implementation of this work and there is no evidence that any /etc/rc splinter work has actually occurred.
3,改进
The /etc/rc.conf mechanism was enhanced in two ways:
* The default configuration settings were moved from /etc/rc.conf to /etc/defaults/rc.conf , and /etc/rc.conf sources the former. Site specific configuration overrides are placed in /etc/rc.conf . This enables easier upgrades (both manual and automatic) of the default settings in /etc/defaults/rc.conf for new or changed services.
There was debate about this change, but a significant majority of users agreed with the change. Also, FreeBSD had made a similar change some time before, with a similar debate and outcome, and subsequent upgrade benefits observed which helped the case supporting the change.
* An optional per-service configuration file in /etc/rc.conf.d/SERVICE was added. This configuration file (if it exists) is read after /etc/rc.conf , to allow per-service overrides. This optional functionality was added to allow automated third-party installation mechanisms to easily add configuration data.
Migrating entirely away from /etc/rc.conf to a multitude of /etc/rc.conf.d/SERVICE files was considered, but no consensus was reached, and after a local trial, we decided that providing for the latter but retaining the former satisfies proponents of either side.
Recently (post NetBSD 1.5), a /sbin/chkconfig command has been added (similar to the equivalent command in IRIX) to manage /etc/rc.conf.d by displaying a setting or changing its value.
Thus, the order that configuration information for a given service foo is read in is as follows:
* foo sources /etc/rc.conf .
* /etc/rc.conf sources in /etc/defaults/rc.conf (if it exists), and machine specific overrides of the defaults are added at the end of /etc/rc.conf .
* A per-service configuration file in /etc/rc.conf.d/foo (if it exists) will be loaded. This allows for automated maintenance of /etc/rc.conf.d configuration files, whilst retaining the popular /etc/rc.conf semantics.
4,实现&结果
The system was implemented as described above in the design section, although the design was slightly fluid and did change as feedback was incorporated.
There are two elements to the post-implementation analysis; the human issues, and the technical details.
4.1. The human issues
There was a lot of feedback, debate, angst, flames, and hate-mail. The change has been one of the most contentious in the history of the project.
The first commits to the source code repository were made with the intention of providing a mostly complete implementation which was to be incrementally improved over a few months before the release of NetBSD 1.5.
Unfortunately, we made one of our largest implementation mistakes at this point; we didn't warn the user-base that this was our intention, and the commits were seen as a `stealth attack'. This was partly because we felt that there had been enough debate and announcing our intentions would have delayed the project another few months for a rehash of the same debate (which had been going on for five years at that point).
After the initial implementation, various technical and `religious' complaints were raised about the system. A summary of these is:
* " The use of `magic' functions [from /etc/rc.subr] is bad. "
It was felt that the code re-use that /etc/rc.subr promotes was sufficiently worthy to justify its continued use, as described in See Promote code re-use..
* " Switching from /etc/rc is not the BSD way, ... "
This particular objection was expected; it's a religious argument and the change was bound to annoy a certain section of the community.
Robert Elz, a long time user and contributor to BSD, had a good point to make about `the BSD way': " [the BSD way is to] find something that looks to be better (in the opinion of the small group deciding such things), implement it, and ship it. "[5]
In this case, the `small group' was the NetBSD core team, who voted in unanimous agreement for the work, with the proviso that it would be tweaked and improved as necessary, which is what occurred.
* " Why wasn't a System V init.d implemented? "
This was covered in See Avoid mandatory run levels..
Because some of the detractors were quite vocal in the complaints, there was a perception for a time that the work was against a majority decision. This was far from the truth; many users and developers had become jaded with the discussion over the years and did not bother to argue in support of the change, since they agreed with it in principle, if not in implementation particulars. This was borne out by the level of support for the change in the time since implementation.
