1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
|
<html>
<head>
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta http-equiv="Content-Language" content="en" />
<title>s6-rc: the s6-rc-compile program</title>
<meta name="Description" content="s6-rc: the s6-rc-compile program" />
<meta name="Keywords" content="s6-rc offline database compilation command rc init dependency state management services" />
<!-- <link rel="stylesheet" type="text/css" href="//skarnet.org/default.css" /> -->
</head>
<body>
<p>
<a href="index.html">s6-rc</a><br />
<a href="//skarnet.org/software/">Software</a><br />
<a href="//skarnet.org/">skarnet.org</a>
</p>
<h1> The s6-rc-compile program </h1>
<p>
s6-rc-compile is a service database compiler. It takes a series
of service definitions in the <em>source</em> format, and compiles
them into a directory, which is the <em>compiled</em> format.
</p>
<p>
The administrator can then examine that compiled database via
<a href="s6-rc-db.html">s6-rc-db</a>, put it into a place where
it will be registered at boot time as the current compiled database
by <a href="s6-rc-init.html">s6-rc-init</a>, or even live update
the current service database via
<a href="s6-rc-update.html">s6-rc-update</a>.
</p>
<h2> Interface </h2>
<pre>
s6-rc-compile [ -v <em>verbosity</em> ] [ -h <em>fdhuser</em> ] [ -b ] <em>compiled</em> <em>source...</em>
</pre>
<ul>
<li> s6-rc-compile analyzes every directory <em>source</em> in its
arguments. For every subdirectory <em>service</em> in <em>source</em>,
it expects to find a valid service definition in <em>service</em>. </li>
<li> s6-rc-compile outputs a compiled version of the service database
into <em>compiled</em>. This database contains information for all the
services declared in every <em>source</em> argument. </li>
</ul>
<h2> Exit codes </h2>
<ul>
<li> 0: success </li>
<li> 1: error in a source directory </li>
<li> 100: wrong usage </li>
<li> 111: system call failed </li>
</ul>
<h2> Options </h2>
<ul>
<li> <tt>-v <em>verbosity</em></tt> : be more or less
verbose. Default is 1: warning and error messages will be printed to
stderr. 0 silences warnings. 2 adds a bit more information about
what s6-rc-compile is doing. 3 or more is heavy debug output. </li>
<li> <tt>-h <em>fdhuser</em></tt> : arrange for the
<a href="//skarnet.org/software/s6/s6-fdholder-daemon.html">s6-fdholder-daemon</a>
program, which maintains the pipes for the longrun pipelines, to run
as user <em>fdhuser</em>. By default, it runs as the user owning
the supervision tree, i.e. most likely <tt>root</tt>. </li>
<li> <tt>-b</tt> : make
<a href="s6-rc-oneshot-run.html">s6-rc-oneshot-run</a>
invocations wait instead of fail on lock contention. This
should not change anything in practice, and you can ignore
that option. </li>
</ul>
<h2> Source format </h2>
<p>
s6-rc-compile scans every <em>source</em> directory to find
<em>service definition directories</em> in it. It ignores every
file that is not a directory, or that starts with a dot. For
every service definition directory that it finds, it creates a
service with the same name as the directory. Names cannot be
duplicated and cannot contain a slash or a newline; they can
contain spaces and tabs, but using anything else than alphanumerical
characters, underscores and dashes is discouraged - the s6-rc programs
will handle weird names just fine, but other tools, especially
shell scripts, may not. Names are also forbidden to use the reserved
<tt>s6rc-</tt> and <tt>s6-rc-</tt> prefixes.
</p>
<p>
Every service
definition directory <em>service</em> is expected to contain the following files:
</p>
<h3> For every service </h3>
<ul>
<li> A regular file named <tt>type</tt>, that contains only the text
<tt>oneshot</tt>, <tt>longrun</tt> or <tt>bundle</tt>, and a terminating
newline. This file declares the type of service defined by the
directory. </li>
</ul>
<h3> For bundles </h3>
<ul>
<li> A regular file named <tt>contents</tt>. This file must be a list
of service names, one per line. Whitespace at the beginning of a line
is ignored, but trailing whitespace is not. Lines starting with a <tt>#</tt>
character are ignored. The file defines the services that will be
represented by the bundle named <em>service</em>.
</ul>
<p>
It is possible to use bundle names in a <tt>contents</tt> file.
However, if s6-rc-compile detects a cycle in bundle definitions, it will
complain and exit 1.
</p>
<h3> For atomic services </h3>
<ul>
<li> An optional regular file named <tt>timeout-up</tt>. This file, if it exists,
must contain an integer, which is the maximum number of milliseconds
<a href="s6-rc.html">s6-rc</a> will wait for successful completion of the service
start; if starting the service takes longer than this value, s6-rc will declare
the transition a failure. If the file does not exist, or contains 0, no timeout
is defined and s6-rc will wait indefinitely for the service to start. </li>
<li> An optional regular file named <tt>timeout-down</tt>. This file, if it exists,
must contain an integer, which is the maximum number of milliseconds
<a href="s6-rc.html">s6-rc</a> will wait for successful completion of the service
stop; if stopping the service takes longer than this value, s6-rc will declare
the transition a failure. If the file does not exist, or contains 0, no timeout
is defined and s6-rc will wait indefinitely for the service to stop. </li>
<li> An optional regular file named <tt>dependencies</tt>. This file must be a list
of service names, one per line. Whitespace at the beginning of a line
is ignored, but trailing whitespace is not. Lines starting with a <tt>#</tt>
character are ignored. The file defines the <em>direct dependencies</em> of
<em>service</em>, i.e. the services that must be up in order for
<em>service</em> to work properly. </li>
</ul>
<p>
It is unnecessary to manually define complete sets of dependencies in the
<tt>dependencies</tt> file, because
<a href="s6-rc.html">s6-rc</a> will properly handle dependency chains.
If <em>A</em> depends on <em>B</em>, no matter the underlying
implementation of <em>B</em>, and the current implementation of <em>B</em>
depends on <em>C</em>, then you should just put <em>B</em> in
<tt><em>A</em>/dependencies</tt>; when starting the set,
<a href="s6-rc.html">s6-rc</a> will start <em>C</em> first, then
<em>B</em>, then <em>A</em>. If the underlying implementation of <em>B</em>
changes and does not depend on <em>C</em>, then you will just have to
modify the dependencies for <em>B</em>, and the definition of <em>A</em>
will still be correct.
</p>
<p>
Of course, if <em>A</em> depends on <em>C</em> anyway, you should add
both <em>B</em> and <em>C</em> to <tt><em>A</em>/dependencies</tt>.
</p>
<p>
If s6-rc-compile detects a cycle in dependencies across services, it will
complain and exit 1.
</p>
<h4> For oneshots </h4>
<ul>
<li> Two regular files named <tt>up</tt> and <tt>down</tt>, which
must each contain a single Unix command line. The files will be interpreted by the
<a href="//skarnet.org/software/execline/execlineb.html">execlineb</a>
lexer at compile time and the results will be stored into the
compiled database in an internal form. <tt>up</tt> will be run when
the service is started, and <tt>down</tt> will be executed when the service
is stopped. <tt>up</tt> is mandatory, but <tt>down</tt> is optional;
if no <tt>down</tt> file is provided in the source definition directory,
then it is treated as the empty script. If a script is empty,
then s6-rc will consider that the corresponding transition for this service
does nothing and always succeeds. </li>
</ul>
<p>
<tt>up</tt> and <tt>down</tt> are interpreted by
<a href="//skarnet.org/software/execline/execlineb.html">execlineb</a>, but
that does not mean they have to be entirely written in the
<a href="//skarnet.org/software/execline/">execline</a> language. The
<a href="//skarnet.org/software/execline/execlineb.html">execlineb</a>
lexer is only used because it can compile a Unix command line from a text file
and store the compiled result, whereas a shell would have to be invoked
everytime the script is run. There are many ways to write <tt>up</tt> and
<tt>down</tt> scripts:
</p>
<ul>
<li> They can be written as standard execline scripts, without the
initial shebang line or positional parameter management. </li>
<li> They can be written in any scripting language by invoking the interpreter
directly: for instance
<tt>/bin/sh -c "<em>script</em>"</tt>, where <em>script</em> is
a shell script. </li>
<li> They can also just call a script that will actually be stored somewhere else:
for instance, <tt>up</tt> can contain <tt>/etc/init.d/<em>service</em> start</tt>
while <tt>down</tt> contains <tt>/etc/init.d/<em>service</em> stop</tt>, and
<tt>/etc/init.d/<em>service</em></tt> is a System V-style init script. </li>
</ul>
<p>
Don't think you have to learn all the intricacies of the execline language
just because the <tt>up</tt> and <tt>down</tt> scripts get lexed by it.
You don't.
</p>
<h4> For longruns </h4>
<p>
The s6-rc service definition directory for a longrun service is similar to
a <a href="//skarnet.org/software/s6/servicedir.html">s6 service
directory</a>, but there are a few differences:
</p>
<ul>
<li> s6-rc-compile crafts the servicedir itself, based on what it
finds in the service definition directory. It does not copy everything
directly from the definition directory to the servicedir; only two
subdirectories will be copied verbatim, <tt>data</tt> and <tt>env</tt>.
So if you want to store service configuration data, to be used
by the run script, in the service directory, make sure it is in a
<tt>data/</tt> or <tt>env/</tt> subdirectory. </li>
<li> Definition directories cannot have a <tt>log</tt> subdirectory -
or if they do, it will be ignored. From s6-rc-compile's point of view,
logged s6 services must actually be defined as <em>two</em> separate
s6-rc services, one defined as a producer and one defined as a consumer,
making a pipeline of just two services; see below for more information
about pipelines. </li>
</ul>
<p>
The following files must or may appear in a longrun definition directory:
</p>
<ul>
<li> A mandatory regular file named <tt>run</tt>, as well as optional files
named <tt>finish</tt>, <tt>notification-fd</tt>, <tt>timeout-kill</tt>,
<tt>timeout-finish</tt>, <tt>nosetsid</tt>, <tt>max-death-tally</tt>,
and <tt>down-signal</tt>. These files will be copied, or recreated, in
the generated
<a href="//skarnet.org/software/s6/servicedir.html">service directory</a>:
they are meant to be used by the
<a href="//skarnet.org/software/s6/s6-supervise.html">s6-supervise</a>
process that will manage the longrun service. </li>
<li> Optional directories named <tt>data</tt> and <tt>env</tt>. These will
be copied verbatim into the generated service directory. </li>
<li> An optional file named <tt>producer-for</tt>. If this file exists, then
it must contain the name of another longrun service <em>servicelog</em>;
<em>service</em> is then declared as a producer for <em>servicelog</em>.
<em>servicelog</em> must also, in its own definition directory,
be declared as a consumer for at least <em>service</em>. </li>
<li> An optional file named <tt>consumer-for</tt>. If this file exists, then
it must contain a list of longrun services <em>serviceprod...</em>:
<em>service</em> is then declared as a consumer for all the services
in <em>serviceprod</em>. Each of those services
must also, in its own definition directory,
be declared as a producer for <em>service</em>. </li>
<li> An optional file named <tt>pipeline-name</tt>. If this file exists
along with a <tt>consumer-for</tt> file, and there is no
<tt>producer-for</tt> file, then a bundle will automatically be
created, named with the content of the <tt>pipeline-name</tt> file, and
containing all the services in the pipeline that ends at <em>service</em>.
See below for more about pipelining. The <tt>pipeline-name</tt> file
is ignored if <em>service</em> is not a last consumer. </li>
</ul>
<p>
Be aware that service directories will be relocated, and copied at boot time,
so if your run or finish scripts refer to files in the service directory
(probably in the <tt>data</tt> or <tt>env</tt> subdirectories!), they should
use relative paths, not absolute ones.
</p>
<p>
Note that you cannot create a <tt>./down</tt> file for
<a href="//skarnet.org/software/s6/s6-supervise.html">s6-supervise</a>
in a generated service
directory. Even if such a file exists in the definition directory, it will
be ignored - it will not be replicated in the service directory.
This is intentional:
<a href="s6-rc.html">s6-rc</a> internally uses <tt>./down</tt> files in
the service directories it manages, to mark longrun
services that are down.
</p>
<p>
The <tt>producer-for</tt>, <tt>consumer-for</tt> and <tt>pipeline-name</tt>
files are used to set up automatic longrun pipelining.
</p>
<h3> Longrun pipelining </h3>
<p>
Users of supervision suites know about logged services: a service acts
as a producer, and is coupled with another service, its logger; the
supervision system automatically maintains an open pipe between the
producer's stdout and the logger's stdin.
</p>
<p>
s6-rc comes with an extension of this mechanism. Rather than only
allowing two longrun services to be pipelined, it can set up an
indefinite number of longrun services this way.
</p>
<ul>
<li> A producer declares its direct consumer in a <tt>producer-for</tt> file. </li>
<li> Intermediate services declare both their direct producers in their
<tt>consumer-for</tt> file, and their direct consumer in their
<tt>producer-for</tt> file. </li>
<li> The last consumer only declares its direct producers in a <tt>consumer-for</tt> file. </li>
<li> The last consumer may also declare a name for the whole pipeline, in
its <tt>pipeline-name</tt> file. If it does so, then a bundle is automatically
created with
the given name, and it contains all the services in the pipeline. </li>
</ul>
<p>
s6-rc-compile will detect pipelines, and set up the service directories
so that every producer's stdout is connected to its consumer's stdin, and
that the pipes are not broken whenever one element in the chain dies.
</p>
<p>
A service can only be declared as a producer for <em>one</em> other
services, but it can be declared as a consumer for several other
services: the <em>consumer-for</em> can have several lines. This
means that one service can only send its output to one consumer, but
it can read its input from several producers. If there are several
producers to a service, they will all write to the same Unix pipe
that the service reads.
</p>
<p>
This means that what s6-rc calls <em>pipelines</em> are really
<em>funnels</em>: you can collapse multiple data streams into
a single data stream, at every step of your processing chain.
The <em>pipeline</em> terminology remains for history reasons:
previous versions of s6-rc could not handle multiple producers.
</p>
<p>
s6-rc-compile checks for pipeline consistency. It must see a
<tt>producer-for</tt> file in the producers' definition that is consistent
with the <tt>consumer-for</tt> file in the consumer's definition. It will
detect and reject cycles as well as collisions.
</p>
<p>
The pipe linking a set of producers with a consumer is created and stored at
run-time in a
<a href="//skarnet.org/software/s6/s6-fdholder-daemon.html">s6-fdholder-daemon</a>
instance managed by an automatically generated longrun service named
<tt>s6rc-fdholder</tt>.
</p>
<h2> Compiled database usage </h2>
<ul>
<li> Once it has been built with s6-rc-compile, a compiled database cannot be
modified. It (i.e. the directory containing the <em>n</em>, <em>db</em> and
<em>resolve.cdb</em> files as well as the <em>servicedirs</em> subdirectory)
can be moved around in the filesystem, provided the insides remain
untouched. </li>
<li> Once a compiled database is live (i.e. it has services running on it,
following an invocation of <a href="s6-rc-init.html">s6-rc-init</a> or
<a href="s6-rc-update.html">s6-rc-update</a>, it <em>must not move anymore</em>,
and it must not be deleted. The only way to "free" such a compiled database
for displacement or deletion is to replace it as the live one via another
call to <a href="s6-rc-update.html">s6-rc-update</a>. </li>
<li> No matter what user compiles the database, only root and the
user owning the supervision tree at run-time will be able
to operate the compiled database with the <a href="s6-rc.html">s6-rc</a>
command. </li>
</ul>
<h2> A complete example </h2>
<p>
The <tt>examples/source</tt> subdirectory of the s6-rc package contains a set
of service definition directories, which is actually a working, valid set for a
Linux system running
<a href="http://busybox.net/">busybox</a> and the
<a href="//skarnet.org/software/">skarnet.org</a> packages; of course, only
the service definition set has been kept, and private information has been
removed, so it won't work out-of-the-box without the proper specific files,
notably configuration in <tt>/etc</tt> - but nevertheless, you can browse the
source and understand what it does, and adapt it to your own needs. It will
compile as is with
<tt>s6-rc-compile</tt>, and you can examine the
resulting compiled database with
<tt><a href="s6-rc-db.html">s6-rc-db</a></tt>.
</p>
</body>
</html>
|
