Remake
Loading...
Searching...
No Matches
remake.cpp
Go to the documentation of this file.
1/* -*- mode: C++; indent-tabs-mode: t; c-basic-offset: 8; -*- */
2/**
3@mainpage Remake, a build system that bridges the gap between make and redo.
4
5As with <b>make</b>, <b>remake</b> uses a centralized rule file, which is
6named <b>Remakefile</b>. It contains rules with a <em>make</em>-like
7syntax:
8
9@verbatim
10target1 target2 ... : prerequisite1 prerequisite2 ...
11 shell script
12 that builds
13 the targets
14@endverbatim
15
16A target is known to be up-to-date if all its prerequisites are. If it
17has no known prerequisites yet the file already exits, it is assumed to
18be up-to-date. Obsolete targets are rebuilt thanks to the shell script
19provided by the rule.
20
21As with <b>redo</b>, <b>remake</b> supports dynamic dependencies in
22addition to these static dependencies. Whenever a script executes
23`remake prerequisite4 prerequisite5 ...`, these prerequisites are
24rebuilt if they are obsolete. (So <b>remake</b> acts like
25<b>redo-ifchange</b>.) Moreover, all the dependencies are stored in file
26<b>.remake</b> so that they are remembered in subsequent runs. Note that
27dynamic dependencies from previous runs are only used to decide whether a
28target is obsolete; they are not automatically rebuilt when they are
29obsolete yet a target depends on them. They will only be rebuilt once the
30dynamic call to <b>remake</b> is executed.
31
32In other words, the following two rules have almost the same behavior.
33
34@verbatim
35target1 target2 ... : prerequisite1 prerequisite2 ...
36 shell script
37
38target1 target2 ... :
39 remake prerequisite1 prerequisite2 ...
40 shell script
41@endverbatim
42
43(There is a difference if the targets already exist, have never been
44built before, and the prerequisites are either younger or obsolete, since
45the targets will not be rebuilt in the second case.)
46
47The above usage of dynamic dependencies is hardly useful. Their strength
48lies in the fact that they can be computed on the fly:
49
50@verbatim
51%.o : %.c
52 gcc -MMD -MF $@.d -o $@ -c $<
53 remake -r < $@.d
54 rm $@.d
55
56%.cmo : %.ml
57 ocamldep $< | remake -r $@
58 ocamlc -c $<
59
60after.xml: before.xml rules.xsl
61 xsltproc --load-trace -o after.xml rules.xsl before.xml 2> deps
62 remake `sed -n -e "\\,//,! s,^.*URL=\"\\‍([^\"]*\\‍).*\$,\\1,p" deps`
63 rm deps
64@endverbatim
65
66Note that the first rule fails if any of the header files included by
67a C source file has to be automatically generated. In that case, one
68should perform a first call to <b>remake</b> them before calling the
69compiler. (Dependencies from several calls to <b>remake</b> are
70cumulative, so they will all be remembered the next time.)
71
72\section sec-usage Usage
73
74Usage: <tt>remake <i>options</i> <i>targets</i></tt>
75
76Options:
77
78- `-B`, `--always-make`: Unconditionally make all targets.
79- `-d`: Echo script commands.
80- `-f FILE`: Read `FILE` as <b>Remakefile</b>.
81- `-j[N]`, `--jobs=[N]`: Allow `N` jobs at once;
82 infinite jobs with no argument.
83- `-k`, `--keep-going`: Keep going when some targets cannot be made.
84- `-r`: Look up targets from the dependencies on standard input.
85- `-s`, `--silent`, `--quiet`: Do not echo targets.
86
87\section sec-syntax Syntax
88
89Lines starting with a space character or a tabulation are assumed to be rule
90scripts. They are only allowed after a rule header.
91
92Lines starting with `#` are considered to be comments and are ignored.
93They do interrupt rule scripts though.
94
95Any other line is either a variable definition or a rule header. If such a
96line ends with a backslash, the following line break is ignored and the line
97extends to the next one.
98
99Variable definitions are a single name followed by equal followed by a list
100of names, possibly empty.
101
102Rule headers are a nonempty list of names, followed by a colon, followed by
103another list of names, possibly empty. Basically, the syntax of a rule is as
104follows:
105
106@verbatim
107targets : prerequisites
108 shell script
109@endverbatim
110
111List of names are space-separated sequences of names. If a name contains
112a space character, it should be put into double quotes. Names cannot be
113any of the following special characters `:$(),="`. Again, quotation
114should be used. Quotation marks can be escaped by a backslash inside
115quoted names.
116
117\subsection sec-variables Variables
118
119Variables can be used to factor lists of targets or prerequisites. They are
120expanded as they are encountered during <b>Remakefile</b> parsing.
121
122@verbatim
123VAR2 = a
124VAR1 = c d
125VAR2 += $(VAR1) b
126$(VAR2) e :
127@endverbatim
128
129Variable assignments can appear instead of prerequisites inside non-generic
130rules with no script. They are then expanded inside the corresponding
131generic rule.
132
133@verbatim
134foo.o: CFLAGS += -DBAR
135
136%.o : %.c
137 gcc $(CFLAGS) -MMD -MF $@.d -o $@ -c $<
138 remake -r < $@.d
139 rm $@.d
140@endverbatim
141
142Note: contrarily to <b>make</b>, variable names have to be enclosed in
143parentheses. For instance, `$y` is not a shorthand for <tt>\$(y)</tt> and
144is left unexpanded.
145
146\subsection sec-autovars Automatic variables
147
148The following special symbols can appear inside scripts:
149
150- `$<` expands to the first static prerequisite of the rule.
151- `$^` expands to all the static prerequisites of the rule, including
152 duplicates if any.
153- `$@` expands to the first target of the rule.
154- `$*` expands to the string that matched `%` in a generic rule.
155- `$$` expands to a single dollar symbol.
156
157Note: contrarily to <b>make</b>, there are no corresponding variables.
158For instance, `$^` is not a shorthand for `$(^)`. Another difference is
159that `$@` is always the first target, not the one that triggered the
160rule.
161
162\subsection sec-functions Built-in functions
163
164<b>remake</b> also supports a few built-in functions inspired from <b>make</b>.
165
166- <tt>$(addprefix <i>prefix</i>, <i>list</i>)</tt> returns the list obtained
167 by prepending its first argument to each element of its second argument.
168- <tt>$(addsuffix <i>suffix</i>, <i>list</i>)</tt> returns the list obtained
169 by appending its first argument to each element of its second argument.
170
171\subsection sec-order Order-only prerequisites
172
173If the static prerequisites of a rule contain a pipe symbol, prerequisites
174on its right do not cause the targets to become obsolete if they are newer
175(unless they are also dynamically registered as dependencies). They are
176meant to be used when the targets do not directly depend on them, but the
177computation of their dynamic dependencies does.
178
179@verbatim
180%.o : %.c | parser.h
181 gcc -MMD -MF $@.d -o $@ -c $<
182 remake -r < $@.d
183 rm $@.d
184
185parser.c parser.h: parser.y
186 yacc -d -o parser.c parser.y
187@endverbatim
188
189\subsection sec-static-pattern Static pattern rules
190
191A rule with the following structure is expanded into several rules, one
192per target.
193
194@verbatim
195targets: pattern1 pattern2 ...: prerequisites
196@endverbatim
197
198Every target is matched against one of the patterns containing the `%`
199character. A rule is then created using the patterns as targets, after
200having substituted `%` in the patterns and prerequisites. The automatic
201variable `$*` can be used in the script of the rule.
202
203\subsection sec-special-tgt Special targets
204
205Target `.PHONY` marks its prerequisites as being always obsolete.
206
207\subsection sec-special-var Special variables
208
209Variable `.OPTIONS` is handled specially. Its content enables some
210features of <b>remake</b> that are not enabled by default.
211
212- `variable-propagation`: When a variable is set in the prerequisite
213 part of a rule, it is propagated to the rules of all the targets this rule
214 depends on. This option also enables variables to be set on the command
215 line. Note that, as in <b>make</b>, this features introduces non-determinism:
216 the content of some variables will depend on the build order.
217
218\section sec-semantics Semantics
219
220\subsection src-obsolete When are targets obsolete?
221
222A target is obsolete:
223
224- if there is no file corresponding to the target, or to one of its siblings
225 in a multi-target rule,
226- if any of its dynamic prerequisites from a previous run or any of its static
227 prerequisites is obsolete,
228- if the latest file corresponding to its siblings or itself is older than any
229 of its dynamic prerequisites or static prerequisites.
230
231In all the other cases, it is assumed to be up-to-date (and so are all its
232siblings). Note that the last rule above says "latest" and not "earliest". While
233it might cause some obsolete targets to go unnoticed in corner cases, it allows
234for the following kind of rules:
235
236@verbatim
237config.h stamp-config_h: config.h.in config.status
238 ./config.status config.h
239 touch stamp-config_h
240@endverbatim
241
242A `config.status` file generally does not update header files (here
243`config.h`) if they would not change. As a consequence, if not for the
244`stamp-config_h` file above, a header would always be considered obsolete
245once one of its prerequisites is modified. Note that touching `config.h`
246rather than `stamp-config_h` would defeat the point of not updating it in
247the first place, since the program files would need to be rebuilt.
248
249Once all the static prerequisites of a target have been rebuilt, <b>remake</b>
250checks whether the target still needs to be built. If it was obsolete only
251because its prerequisites needed to be rebuilt and none of them changed, the
252target is assumed to be up-to-date.
253
254\subsection sec-rules How are targets (re)built?
255
256There are two kinds of rules. If any of the targets or prerequisites contains
257a `%` character, the rule is said to be <em>generic</em>. All the
258targets of the rule shall then contain a single `%` character. All the
259other rules are said to be <em>specific</em>.
260
261A rule is said to <em>match</em> a given target:
262
263- if it is specific and the target appears inside its target list,
264- if it is generic and there is a way to replace the `%` character
265 from one of its targets so that it matches the given target.
266
267When <b>remake</b> tries to build a given target, it looks for a specific rule
268that matches it. If there is one and its script is nonempty, it uses it to
269rebuild the target.
270
271Otherwise, it looks for a generic rule that matches the target. If there are
272several matching rules, it chooses the one with the shortest pattern (and if
273there are several ones, the earliest one). It then looks for specific rules
274that match each target of the generic rule. All the prerequisites of these
275specific rules are added to those of the generic rule. The script of the
276generic rule is used to build the target.
277
278Example:
279
280@verbatim
281t%1 t2%: p1 p%2
282 commands building t%1 and t2%
283
284t2z: p4
285 commands building t2z
286
287ty1: p3
288
289# t2x is built by the first rule (which also builds tx1) and its prerequisites are p1, px2
290# t2y is built by the first rule (which also builds ty1) and its prerequisites are p1, py2, p3
291# t2z is built by the second rule and its prerequisite is p4
292@endverbatim
293
294The set of rules from <b>Remakefile</b> is ill-formed:
295
296- if any specific rule matching a target of the generic rule has a nonempty script,
297- if any target of the generic rule is matched by a generic rule with a shorter pattern.
298
299\section sec-compilation Compilation
300
301- On Linux, MacOSX, and BSD: `g++ -o remake remake.cpp`
302- On Windows: `g++ -o remake.exe remake.cpp -lws2_32`
303
304Installing <b>remake</b> is needed only if <b>Remakefile</b> does not
305specify the path to the executable for its recursive calls. Thanks to its
306single source file, <b>remake</b> can be shipped inside other packages and
307built at configuration time.
308
309\section sec-differences Differences with other build systems
310
311Differences with <b>make</b>:
312
313- Dynamic dependencies are supported.
314- For rules with multiple targets, the shell script is executed only once
315 and is assumed to build all the targets. There is no need for
316 convoluted rules that are robust enough for parallel builds. For generic
317 rules, this is similar to the behavior of pattern rules from <b>gmake</b>.
318- As with <b>redo</b>, only one shell is run when executing a script,
319 rather than one per script line. Note that the shells are run with
320 option `-e`, thus causing them to exit as soon as an error is
321 encountered.
322- The prerequisites of generic rules (known as implicit rules in <b>make</b>
323 lingo) are not used to decide between several of them, which means that
324 <b>remake</b> does not select one for which it could satisfy the dependencies.
325- Variables and built-in functions are expanded as they are encountered
326 during <b>Remakefile</b> parsing.
327- Target-specific variables are not propagated, unless specifically enabled,
328 since this causes non-deterministic builds. This is the same for variables
329 set on the command line.
330
331Differences with <b>redo</b>:
332
333- As with <b>make</b>, it is possible to write the following kind of rules
334 in <b>remake</b>.
335@verbatim
336Remakefile: Remakefile.in ./config.status
337 ./config.status Remakefile
338@endverbatim
339- If a target is already built the first time <b>remake</b> runs, it still
340 uses the static prerequisites of rules mentioning it to check whether it
341 needs to be rebuilt. It does not assume it to be up-to-date. As with
342 <b>redo</b> though, if its obsolete status would be due to a dynamic
343 prerequisite, it will go unnoticed; it should be removed beforehand.
344- Multiple targets are supported.
345- <b>remake</b> has almost no features: no checksum-based dependencies, no
346 compatibility with job servers, etc.
347
348\section sec-limitations Limitations
349
350- If a rule script calls <b>remake</b>, the current working directory should
351 be the directory containing <b>Remakefile</b> (or the working directory
352 from the original <b>remake</b> if it was called with option `-f`).
353- As with <b>make</b>, variables passed on the command line should keep
354 the same values, to ensure deterministic builds.
355- Some cases of ill-formed rules are not caught by <b>remake</b> and can
356 thus lead to unpredictable behaviors.
357
358\section sec-links Links
359
360@see http://cr.yp.to/redo.html for the philosophy of <b>redo</b> and
361https://github.com/apenwarr/redo for an implementation and some comprehensive documentation.
362
363\section sec-licensing Licensing
364
365@author Guillaume Melquiond
366@version 0.14
367@date 2012-2020
368@copyright
369This program is free software: you can redistribute it and/or modify
370it under the terms of the GNU General Public License as published by
371the Free Software Foundation, either version 3 of the License, or
372(at your option) any later version.
373\n
374This program is distributed in the hope that it will be useful,
375but WITHOUT ANY WARRANTY; without even the implied warranty of
376MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
377GNU General Public License for more details.
378
379\section sec-internals Internals
380
381The parent <b>remake</b> process acts as a server. The other ones have a
382REMAKE_SOCKET environment variable that tells them how to contact the
383server. They send the content of the REMAKE_JOB_ID environment variable,
384so that the server can associate the child targets to the jobs that
385spawned them. They then wait for completion and exit with the status
386returned by the server. This is handled by #client_mode.
387
388The server calls #load_dependencies and #save_dependencies to serialize
389dynamic dependencies from <b>.remake</b>. It loads <b>Remakefile</b> with
390#load_rules. It then runs #server_mode, which calls #server_loop.
391
392When building a target, the following sequence of events happens:
393
394- #start calls #find_rule (and #find_generic_rule) to get the rule.
395- It then creates a pseudo-client if the rule has static dependencies, or
396 calls #run_script otherwise. In both cases, a new job is created; the
397 rule and the variables are stored into #jobs.
398- #run_script creates a shell process and stores it in #job_pids. It
399 increases #running_jobs.
400- The child process possibly calls <b>remake</b> with a list of targets.
401- #accept_client receives a build request from a child process and adds
402 it to #clients. It also records the new dependencies of the job into
403 #dependencies. It increases #waiting_jobs.
404- #handle_clients uses #get_status to look up the obsoleteness of the
405 targets.
406- Once the targets of a request have been built or one of them has failed,
407 #handle_clients calls #complete_request and removes the request from
408 #clients.
409- If the build targets come from a pseudo-client, #complete_request calls
410 #run_script. Otherwise it sends the reply to the corresponding child
411 process and decreases #waiting_jobs.
412- When a child process ends, #server_loop calls #finalize_job, which
413 removes the process from #job_pids, decreases #running_jobs, and calls
414 #complete_job.
415- #complete_job removes the job from #jobs and calls #update_status
416 to change the status of the targets. It also removes the target files in
417 case of failure.
418*/
419
420#ifdef _WIN32
421#define WIN32_LEAN_AND_MEAN
422#define WINDOWS
423#endif
424
425#include <fstream>
426#include <iostream>
427#include <list>
428#include <map>
429#include <set>
430#include <sstream>
431#include <string>
432#include <vector>
433#include <cassert>
434#include <cstdlib>
435#include <cstring>
436#include <ctime>
437#include <errno.h>
438#include <fcntl.h>
439#include <signal.h>
440#include <unistd.h>
441#include <sys/stat.h>
442#include <sys/types.h>
443
444#ifdef __APPLE__
445#define MACOSX
446#endif
447
448#ifdef __linux__
449#define LINUX
450#endif
451
452#ifdef WINDOWS
453#include <windows.h>
454#include <winbase.h>
455#include <winsock2.h>
456#define pid_t HANDLE
457typedef SOCKET socket_t;
458#else
459#include <sys/socket.h>
460#include <sys/un.h>
461#include <sys/wait.h>
462typedef int socket_t;
463enum { INVALID_SOCKET = -1 };
464extern char **environ;
465#endif
466
467#if defined(WINDOWS) || defined(MACOSX)
468enum { MSG_NOSIGNAL = 0 };
469#endif
470
471typedef std::list<std::string> string_list;
472
473typedef std::set<std::string> string_set;
474
475/**
476 * Reference-counted shared object.
477 * @note The default constructor delays the creation of the object until it
478 * is first dereferenced.
479 */
480template<class T>
482{
483 struct content
484 {
485 size_t cnt;
487 content(): cnt(1) {}
488 content(T const &t): cnt(1), val(t) {}
489 };
490 mutable content *ptr;
491 ref_ptr(): ptr(NULL) {}
492 ref_ptr(T const &t): ptr(new content(t)) {}
493 ref_ptr(ref_ptr const &p): ptr(p.ptr) { if (ptr) ++ptr->cnt; }
494 ~ref_ptr() { if (ptr && --ptr->cnt == 0) delete ptr; }
496 {
497 if (ptr == p.ptr) return *this;
498 if (ptr && --ptr->cnt == 0) delete ptr;
499 ptr = p.ptr;
500 if (ptr) ++ptr->cnt;
501 return *this;
502 }
503 T &operator*() const
504 {
505 if (!ptr) ptr = new content;
506 return ptr->val;
507 }
508 T *operator->() const { return &**this; }
509};
510
516
517typedef std::map<std::string, ref_ptr<dependency_t> > dependency_map;
518
519typedef std::map<std::string, string_list> variable_map;
520
521/**
522 * Build status of a target.
523 */
525{
526 Uptodate, ///< Target is up-to-date.
527 Todo, ///< Target is missing or obsolete.
528 Recheck, ///< Target has an obsolete dependency.
529 Running, ///< Target is being rebuilt.
530 RunningRecheck, ///< Static prerequisites are being rebuilt.
531 Remade, ///< Target was successfully rebuilt.
532 Failed ///< Build failed for target.
534
535/**
536 * Build status of a target.
537 */
539{
540 status_e status; ///< Actual status.
541 time_t last; ///< Last-modified date.
542};
543
544typedef std::map<std::string, status_t> status_map;
545
546/**
547 * Delayed assignment to a variable.
548 */
550{
551 bool append;
553};
554
555typedef std::map<std::string, assign_t> assign_map;
556
557/**
558 * A rule loaded from Remakefile.
559 */
560struct rule_t
561{
562 string_list targets; ///< Files produced by this rule.
563 string_list deps; ///< Dependencies used for an implicit call to remake at the start of the script.
564 string_list wdeps; ///< Like #deps, except that they are not registered as dependencies.
565 assign_map assigns; ///< Assignment of variables.
566 std::string stem; ///< Stem used to instantiate the rule, if any.
567 std::string script; ///< Shell script for building the targets.
568};
569
570typedef std::list<rule_t> rule_list;
571
572typedef std::map<std::string, ref_ptr<rule_t> > rule_map;
573
574/**
575 * A job created from a set of rules.
576 */
577
578struct job_t
579{
580 rule_t rule; ///< Original rule.
581 variable_map vars; ///< Values of local variables.
582};
583
584typedef std::map<int, job_t> job_map;
585
586typedef std::map<pid_t, int> pid_job_map;
587
588/**
589 * Client waiting for a request to complete.
590 *
591 * There are two kinds of clients:
592 * - real clients, which are instances of remake created by built scripts,
593 * - pseudo clients, which are created by the server to build specific targets.
594 *
595 * Among pseudo clients, there are two categories:
596 * - original clients, which are created for the targets passed on the
597 * command line by the user or for the initial regeneration of the rule file,
598 * - dependency clients, which are created to handle rules that have
599 * explicit dependencies and thus to emulate a call to remake.
600 */
602{
603 socket_t socket; ///< Socket used to reply to the client (invalid for pseudo clients).
604 int job_id; ///< Job for which the built script called remake and spawned the client (negative for original clients).
605 bool failed; ///< Whether some targets failed in mode -k.
606 string_list pending; ///< Targets not yet started.
607 string_set running; ///< Targets being built.
608 variable_map vars; ///< Variables set on request.
609 bool delayed; ///< Whether it is a dependency client and a script has to be started on request completion.
610 client_t(): socket(INVALID_SOCKET), job_id(-1), failed(false), delayed(false) {}
611};
612
613typedef std::list<client_t> client_list;
614
615/**
616 * Map from variable names to their content.
617 * Initialized with the values passed on the command line.
618 */
620
621/**
622 * Map from targets to their known dependencies.
623 */
625
626/**
627 * Map from targets to their build status.
628 */
630
631/**
632 * Set of generic rules loaded from Remakefile.
633 */
635
636/**
637 * Map from targets to specific rules loaded from Remakefile.
638 */
640
641/**
642 * Map of jobs being built.
643 */
645
646/**
647 * Map from jobs to shell pids.
648 */
650
651/**
652 * List of clients waiting for a request to complete.
653 * New clients are put to front, so that the build process is depth-first.
654 */
656
657/**
658 * Maximum number of parallel jobs (non-positive if unbounded).
659 * Can be modified by the -j option.
660 */
661static int max_active_jobs = 1;
662
663/**
664 * Whether to keep building targets in case of failure.
665 * Can be modified by the -k option.
666 */
667static bool keep_going = false;
668
669/**
670 * Number of jobs currently running:
671 * - it increases when a process is created in #run_script,
672 * - it decreases when a completion message is received in #finalize_job.
673 *
674 * @note There might be some jobs running while #clients is empty.
675 * Indeed, if a client requested two targets to be rebuilt, if they
676 * are running concurrently, if one of them fails, the client will
677 * get a failure notice and might terminate before the other target
678 * finishes.
679 */
680static int running_jobs = 0;
681
682/**
683 * Number of jobs currently waiting for a build request to finish:
684 * - it increases when a build request is received in #accept_client
685 * (since the client is presumably waiting for the reply),
686 * - it decreases when a reply is sent in #complete_request.
687 */
688static int waiting_jobs = 0;
689
690/**
691 * Global counter used to produce increasing job numbers.
692 * @see jobs
693 */
694static int job_counter = 0;
695
696/**
697 * Socket on which the server listens for client request.
698 */
700
701/**
702 * Whether the request of an original client failed.
703 */
704static bool build_failure;
705
706#ifndef WINDOWS
707/**
708 * Name of the server socket in the file system.
709 */
710static char *socket_name;
711#endif
712
713/**
714 * Name of the first target of the first specific rule, used for default run.
715 */
716static std::string first_target;
717
718/**
719 * Whether a short message should be displayed for each target.
720 */
721static bool show_targets = true;
722
723/**
724 * Whether script commands are echoed.
725 */
726static bool echo_scripts = false;
727
728/**
729 * Time at the start of the program.
730 */
731static time_t now = time(NULL);
732
733/**
734 * Directory with respect to which command-line names are relative.
735 */
736static std::string working_dir;
737
738/**
739 * Directory with respect to which targets are relative.
740 */
741static std::string prefix_dir;
742
743/**
744 * Whether the prefix directory is different from #working_dir.
745 */
747
748/**
749 * Whether target-specific variables are propagated to prerequisites.
750 */
751static bool propagate_vars = false;
752
753/**
754 * Whether targets are unconditionally obsolete.
755 */
756static bool obsolete_targets = false;
757
758#ifndef WINDOWS
759static sigset_t old_sigmask;
760
761static volatile sig_atomic_t got_SIGCHLD = 0;
762
763static void sigchld_handler(int)
764{
765 got_SIGCHLD = 1;
766}
767
768static void sigint_handler(int)
769{
770 // Child processes will receive the signal too, so just prevent
771 // new jobs from starting and wait for the running jobs to fail.
772 keep_going = false;
773}
774#endif
775
776struct log
777{
779 int depth;
780 log(): active(false), open(false), depth(0)
781 {
782 }
783 std::ostream &operator()()
784 {
785 if (open) std::cerr << std::endl;
786 assert(depth >= 0);
787 std::cerr << std::string(depth * 2, ' ');
788 open = false;
789 return std::cerr;
790 }
791 std::ostream &operator()(bool o)
792 {
793 if (o && open) std::cerr << std::endl;
794 if (!o) --depth;
795 assert(depth >= 0);
796 if (o || !open) std::cerr << std::string(depth * 2, ' ');
797 if (o) ++depth;
798 open = o;
799 return std::cerr;
800 }
801};
802
803static log debug;
804
806{
809 {
810 }
812 {
813 if (debug.active && still_open) debug(false) << "done\n";
814 }
815};
816
817#define DEBUG if (debug.active) debug()
818#define DEBUG_open log_auto_close auto_close; if (debug.active) debug(true)
819#define DEBUG_close if ((auto_close.still_open = false), debug.active) debug(false)
820
821/**
822 * Strong typedef for strings that need escaping.
823 * @note The string is stored as a reference, so the constructed object is
824 * meant to be immediately consumed.
825 */
827{
828 std::string const &input;
829 escape_string(std::string const &s): input(s) {}
830};
831
832/**
833 * Write the string in @a se to @a out if it does not contain any special
834 * characters, a quoted and escaped string otherwise.
835 */
836static std::ostream &operator<<(std::ostream &out, escape_string const &se)
837{
838 std::string const &s = se.input;
839 char const *quoted_char = ",: '";
840 char const *escaped_char = "\"\\$!";
841 bool need_quotes = false;
842 char *buf = NULL;
843 size_t len = s.length(), last = 0, j = 0;
844 for (size_t i = 0; i < len; ++i)
845 {
846 if (strchr(escaped_char, s[i]))
847 {
848 need_quotes = true;
849 if (!buf) buf = new char[len * 2];
850 memcpy(&buf[j], &s[last], i - last);
851 j += i - last;
852 buf[j++] = '\\';
853 buf[j++] = s[i];
854 last = i + 1;
855 }
856 if (!need_quotes && strchr(quoted_char, s[i]))
857 need_quotes = true;
858 }
859 if (!need_quotes) return out << s;
860 out << '"';
861 if (!buf) return out << s << '"';
862 out.write(buf, j);
863 out.write(&s[last], len - last);
864 delete[] buf;
865 return out << '"';
866}
867
868/**
869 * @defgroup paths Path helpers
870 *
871 * @{
872 */
873
874/**
875 * Initialize #working_dir.
876 */
877static void init_working_dir()
878{
879 char buf[1024];
880 char *res = getcwd(buf, sizeof(buf));
881 if (!res)
882 {
883 perror("Failed to get working directory");
884 exit(EXIT_FAILURE);
885 }
886 working_dir = buf;
887#ifdef WINDOWS
888 for (size_t i = 0, l = working_dir.size(); i != l; ++i)
889 {
890 if (working_dir[i] == '\\') working_dir[i] = '/';
891 }
892#endif
894}
895
896/**
897 * Initialize #prefix_dir and switch to it.
898 */
899static void init_prefix_dir()
900{
901 for (;;)
902 {
903 struct stat s;
904 if (stat((prefix_dir + "/Remakefile").c_str(), &s) == 0)
905 {
906 if (!changed_prefix_dir) return;
907 if (chdir(prefix_dir.c_str()))
908 {
909 perror("Failed to change working directory");
910 exit(EXIT_FAILURE);
911 }
912 if (show_targets)
913 {
914 std::cout << "remake: Entering directory `" << prefix_dir << '\'' << std::endl;
915 }
916 return;
917 }
918 size_t pos = prefix_dir.find_last_of('/');
919 if (pos == std::string::npos)
920 {
921 std::cerr << "Failed to locate Remakefile in the current directory or one of its parents" << std::endl;
922 exit(EXIT_FAILURE);
923 }
924 prefix_dir.erase(pos);
925 changed_prefix_dir = true;
926 }
927}
928
929/**
930 * Normalize an absolute path with respect to @a p.
931 * Paths outside the subtree are left unchanged.
932 */
933static std::string normalize_abs(std::string const &s, std::string const &p)
934{
935 size_t l = p.length();
936 if (s.compare(0, l, p)) return s;
937 size_t ll = s.length();
938 if (ll == l) return ".";
939 if (s[l] != '/')
940 {
941 size_t pos = s.rfind('/', l);
942 assert(pos != std::string::npos);
943 return s.substr(pos + 1);
944 }
945 if (ll == l + 1) return ".";
946 return s.substr(l + 1);
947}
948
949/**
950 * Normalize path @a s (possibly relative to @a w) with respect to @a p.
951 *
952 * - If both @a p and @a w are empty, the function just removes ".", "..", "//".
953 * - If only @a p is empty, the function returns an absolute path.
954 */
955static std::string normalize(std::string const &s, std::string const &w, std::string const &p)
956{
957#ifdef WINDOWS
958 char const *delim = "/\\";
959#else
960 char delim = '/';
961#endif
962 size_t pos = s.find_first_of(delim);
963 if (pos == std::string::npos && w == p) return s;
964 bool absolute = pos == 0;
965 if (!absolute && w != p && !w.empty())
966 return normalize(w + '/' + s, w, p);
967 size_t prev = 0, len = s.length();
968 string_list l;
969 for (;;)
970 {
971 if (pos != prev)
972 {
973 std::string n = s.substr(prev, pos - prev);
974 if (n == "..")
975 {
976 if (!l.empty()) l.pop_back();
977 else if (!absolute && !w.empty())
978 return normalize(w + '/' + s, w, p);
979 }
980 else if (n != ".")
981 l.push_back(n);
982 }
983 ++pos;
984 if (pos >= len) break;
985 prev = pos;
986 pos = s.find_first_of(delim, prev);
987 if (pos == std::string::npos) pos = len;
988 }
989 string_list::const_iterator i = l.begin(), i_end = l.end();
990 if (i == i_end) return absolute ? "/" : ".";
991 std::string n;
992 if (absolute) n.push_back('/');
993 n.append(*i);
994 for (++i; i != i_end; ++i)
995 {
996 n.push_back('/');
997 n.append(*i);
998 }
999 if (absolute && !p.empty()) return normalize_abs(n, p);
1000 return n;
1001}
1002
1003/**
1004 * Normalize the content of a list of targets.
1005 */
1006static void normalize_list(string_list &l, std::string const &w, std::string const &p)
1007{
1008 for (string_list::iterator i = l.begin(),
1009 i_end = l.end(); i != i_end; ++i)
1010 {
1011 *i = normalize(*i, w, p);
1012 }
1013}
1014
1015/** @} */
1016
1017/**
1018 * @defgroup lexer Lexer
1019 *
1020 * @{
1021 */
1022
1023/**
1024 * Skip spaces.
1025 */
1026static void skip_spaces(std::istream &in)
1027{
1028 char c;
1029 while (strchr(" \t", (c = in.get()))) {}
1030 if (in.good()) in.putback(c);
1031}
1032
1033/**
1034 * Skip empty lines.
1035 */
1036static void skip_empty(std::istream &in)
1037{
1038 char c;
1039 while (strchr("\r\n", (c = in.get()))) {}
1040 if (in.good()) in.putback(c);
1041}
1042
1043/**
1044 * Skip end of line. If @a multi is true, skip the following empty lines too.
1045 * @return true if there was a line to end.
1046 */
1047static bool skip_eol(std::istream &in, bool multi = false)
1048{
1049 char c = in.get();
1050 if (c == '\r') c = in.get();
1051 if (c != '\n' && in.good()) in.putback(c);
1052 if (c != '\n' && !in.eof()) return false;
1053 if (multi) skip_empty(in);
1054 return true;
1055}
1056
1057enum
1058{
1060 Word = 1 << 1,
1061 Colon = 1 << 2,
1062 Equal = 1 << 3,
1063 Dollarpar = 1 << 4,
1064 Rightpar = 1 << 5,
1065 Comma = 1 << 6,
1066 Plusequal = 1 << 7,
1067 Pipe = 1 << 8,
1068};
1069
1070/**
1071 * Skip spaces and peek at the next token.
1072 * If it is one of @a mask, skip it (if it is not Word) and return it.
1073 * @note For composite tokens allowed by @a mask, input characters might
1074 * have been eaten even for an Unexpected result.
1075 */
1076static int expect_token(std::istream &in, int mask)
1077{
1078 while (true)
1079 {
1080 skip_spaces(in);
1081 char c = in.peek();
1082 if (!in.good()) return Unexpected;
1083 int tok;
1084 switch (c)
1085 {
1086 case '\r':
1087 case '\n': return Unexpected;
1088 case ':': tok = Colon; break;
1089 case ',': tok = Comma; break;
1090 case '=': tok = Equal; break;
1091 case ')': tok = Rightpar; break;
1092 case '|': tok = Pipe; break;
1093 case '$':
1094 if (!(mask & Dollarpar)) return Unexpected;
1095 in.ignore(1);
1096 tok = Dollarpar;
1097 if (in.peek() != '(') return Unexpected;
1098 break;
1099 case '+':
1100 if (!(mask & Plusequal)) return Unexpected;
1101 in.ignore(1);
1102 tok = Plusequal;
1103 if (in.peek() != '=') return Unexpected;
1104 break;
1105 case '\\':
1106 in.ignore(1);
1107 if (skip_eol(in)) continue;
1108 in.putback('\\');
1109 return mask & Word ? Word : Unexpected;
1110 default:
1111 return mask & Word ? Word : Unexpected;
1112 }
1113 if (!(tok & mask)) return Unexpected;
1114 in.ignore(1);
1115 return tok;
1116 }
1117}
1118
1119/**
1120 * Read a (possibly quoted) word.
1121 */
1122static std::string read_word(std::istream &in, bool detect_equal = true)
1123{
1124 int c = in.peek();
1125 std::string res;
1126 if (!in.good()) return res;
1127 char const *separators = " \t\r\n$(),:";
1128 bool quoted = c == '"';
1129 if (quoted) in.ignore(1);
1130 bool plus = false;
1131 while (true)
1132 {
1133 c = in.peek();
1134 if (!in.good()) return res;
1135 if (quoted)
1136 {
1137 in.ignore(1);
1138 if (c == '\\')
1139 res += in.get();
1140 else if (c == '"')
1141 quoted = false;
1142 else
1143 res += c;
1144 continue;
1145 }
1146 if (detect_equal && c == '=')
1147 {
1148 if (plus) in.putback('+');
1149 return res;
1150 }
1151 if (plus)
1152 {
1153 res += '+';
1154 plus = false;
1155 }
1156 if (strchr(separators, c)) return res;
1157 in.ignore(1);
1158 if (detect_equal && c == '+') plus = true;
1159 else res += c;
1160 }
1161}
1162
1163/** @} */
1164
1165/**
1166 * @defgroup stream Token streams
1167 *
1168 * @{
1169 */
1170
1171/**
1172 * Possible results from word producers.
1173 */
1180
1181/**
1182 * Interface for word producers.
1183 */
1185{
1186 virtual ~generator() {}
1187 virtual input_status next(std::string &) = 0;
1188};
1189
1190/**
1191 * Generator for the words of a variable.
1192 */
1194{
1195 std::string name;
1196 string_list::const_iterator vcur, vend;
1197 variable_generator(std::string const &, variable_map const *);
1198 input_status next(std::string &);
1199};
1200
1202 variable_map const *local_variables): name(n)
1203{
1204 if (local_variables)
1205 {
1206 variable_map::const_iterator i = local_variables->find(name);
1207 if (i != local_variables->end())
1208 {
1209 vcur = i->second.begin();
1210 vend = i->second.end();
1211 return;
1212 }
1213 }
1214 variable_map::const_iterator i = variables.find(name);
1215 if (i == variables.end()) return;
1216 vcur = i->second.begin();
1217 vend = i->second.end();
1218}
1219
1221{
1222 if (vcur != vend)
1223 {
1224 res = *vcur;
1225 ++vcur;
1226 return Success;
1227 }
1228 return Eof;
1229}
1230
1231/**
1232 * Generator for the words of an input stream.
1233 */
1235{
1236 std::istream &in;
1240 input_generator(std::istream &i, variable_map const *lv, bool e = false)
1241 : in(i), nested(NULL), local_variables(lv), earliest_exit(e), done(false) {}
1242 input_status next(std::string &);
1243 ~input_generator() { assert(!nested); }
1244};
1245
1246static generator *get_function(input_generator const &, std::string const &);
1247
1249{
1250 if (nested)
1251 {
1252 restart:
1253 input_status s = nested->next(res);
1254 if (s == Success) return Success;
1255 delete nested;
1256 nested = NULL;
1257 if (s == SyntaxError) return SyntaxError;
1258 }
1259 if (done) return Eof;
1260 if (earliest_exit) done = true;
1261 switch (expect_token(in, Word | Dollarpar))
1262 {
1263 case Word:
1264 res = read_word(in, false);
1265 return Success;
1266 case Dollarpar:
1267 {
1268 std::string name = read_word(in, false);
1269 if (name.empty()) return SyntaxError;
1270 if (expect_token(in, Rightpar))
1272 else
1273 {
1274 nested = get_function(*this, name);
1275 if (!nested) return SyntaxError;
1276 }
1277 goto restart;
1278 }
1279 default:
1280 return Eof;
1281 }
1282}
1283
1284/**
1285 * Read a list of words from an input generator.
1286 * @return false if a syntax error was encountered.
1287 */
1289{
1290 while (true)
1291 {
1292 res.push_back(std::string());
1293 input_status s = in.next(res.back());
1294 if (s == Success) continue;
1295 res.pop_back();
1296 return s == Eof;
1297 }
1298}
1299
1300static bool read_words(std::istream &in, string_list &res)
1301{
1302 input_generator gen(in, NULL);
1303 return read_words(gen, res);
1304}
1305
1306/**
1307 * Generator for the result of function addprefix.
1308 */
1310{
1313 string_list::const_iterator prei;
1314 size_t prej, prel;
1315 std::string suf;
1316 addprefix_generator(input_generator const &, bool &);
1317 input_status next(std::string &);
1318};
1319
1321 : gen(top.in, top.local_variables)
1322{
1323 if (!read_words(gen, pre)) return;
1324 if (!expect_token(gen.in, Comma)) return;
1325 prej = 0;
1326 prel = pre.size();
1327 ok = true;
1328}
1329
1331{
1332 if (prej)
1333 {
1334 produce:
1335 if (prej == prel)
1336 {
1337 res = *prei + suf;
1338 prej = 0;
1339 }
1340 else
1341 {
1342 res = *prei++;
1343 ++prej;
1344 }
1345 return Success;
1346 }
1347 switch (gen.next(res))
1348 {
1349 case Success:
1350 if (!prel) return Success;
1351 prei = pre.begin();
1352 prej = 1;
1353 suf = res;
1354 goto produce;
1355 case Eof:
1357 default:
1358 return SyntaxError;
1359 }
1360}
1361
1362/**
1363 * Generator for the result of function addsuffix.
1364 */
1366{
1369 string_list::const_iterator sufi;
1370 size_t sufj, sufl;
1371 std::string pre;
1372 addsuffix_generator(input_generator const &, bool &);
1373 input_status next(std::string &);
1374};
1375
1377 : gen(top.in, top.local_variables)
1378{
1379 if (!read_words(gen, suf)) return;
1380 if (!expect_token(gen.in, Comma)) return;
1381 sufj = 0;
1382 sufl = suf.size();
1383 ok = true;
1384}
1385
1387{
1388 if (sufj)
1389 {
1390 if (sufj != sufl)
1391 {
1392 res = *sufi++;
1393 ++sufj;
1394 return Success;
1395 }
1396 sufj = 0;
1397 }
1398 switch (gen.next(res))
1399 {
1400 case Success:
1401 if (!sufl) return Success;
1402 sufi = suf.begin();
1403 sufj = 1;
1404 res += *sufi++;
1405 return Success;
1406 case Eof:
1408 default:
1409 return SyntaxError;
1410 }
1411}
1412
1413/**
1414 * Return a generator for function @a name.
1415 */
1416static generator *get_function(input_generator const &in, std::string const &name)
1417{
1418 skip_spaces(in.in);
1419 generator *g = NULL;
1420 bool ok = false;
1421 if (name == "addprefix") g = new addprefix_generator(in, ok);
1422 else if (name == "addsuffix") g = new addsuffix_generator(in, ok);
1423 if (!g || ok) return g;
1424 delete g;
1425 return NULL;
1426}
1427
1428/** @} */
1429
1430/**
1431 * @defgroup database Dependency database
1432 *
1433 * @{
1434 */
1435
1436/**
1437 * Load dependencies from @a in.
1438 */
1439static void load_dependencies(std::istream &in)
1440{
1441 if (false)
1442 {
1443 error:
1444 std::cerr << "Failed to load database" << std::endl;
1445 exit(EXIT_FAILURE);
1446 }
1447
1448 while (!in.eof())
1449 {
1450 string_list targets;
1451 if (!read_words(in, targets)) goto error;
1452 if (in.eof()) return;
1453 if (targets.empty()) goto error;
1454 DEBUG << "reading dependencies of target " << targets.front() << std::endl;
1455 if (in.get() != ':') goto error;
1457 dep->targets = targets;
1458 string_list deps;
1459 if (!read_words(in, deps)) goto error;
1460 dep->deps.insert(deps.begin(), deps.end());
1461 for (string_list::const_iterator i = targets.begin(),
1462 i_end = targets.end(); i != i_end; ++i)
1463 {
1464 dependencies[*i] = dep;
1465 }
1466 skip_empty(in);
1467 }
1468}
1469
1470/**
1471 * Load known dependencies from file `.remake`.
1472 */
1474{
1475 DEBUG_open << "Loading database... ";
1476 std::ifstream in(".remake");
1477 if (!in.good())
1478 {
1479 DEBUG_close << "not found\n";
1480 return;
1481 }
1483}
1484
1485
1486/**
1487 * Save all the dependencies in file `.remake`.
1488 */
1490{
1491 DEBUG_open << "Saving database... ";
1492 std::ofstream db(".remake");
1493 while (!dependencies.empty())
1494 {
1495 ref_ptr<dependency_t> dep = dependencies.begin()->second;
1496 for (string_list::const_iterator i = dep->targets.begin(),
1497 i_end = dep->targets.end(); i != i_end; ++i)
1498 {
1499 db << escape_string(*i) << ' ';
1500 dependencies.erase(*i);
1501 }
1502 db << ':';
1503 for (string_set::const_iterator i = dep->deps.begin(),
1504 i_end = dep->deps.end(); i != i_end; ++i)
1505 {
1506 db << ' ' << escape_string(*i);
1507 }
1508 db << std::endl;
1509 }
1510}
1511
1512/** @} */
1513
1514static void merge_rule(rule_t &dest, rule_t const &src);
1515static void instantiate_rule(std::string const &target, rule_t const &src, rule_t &dst);
1516
1517/**
1518 * @defgroup parser Rule parser
1519 *
1520 * @{
1521 */
1522
1523/**
1524 * Register a specific rule with an empty script:
1525 *
1526 * - Check that none of the targets already has an associated rule with a
1527 * nonempty script.
1528 * - Create a new rule with a single target for each target, if needed.
1529 * - Add the prerequisites of @a rule to all these associated rules.
1530 */
1531static void register_transparent_rule(rule_t const &rule, string_list const &targets)
1532{
1533 assert(rule.script.empty());
1534 for (string_list::const_iterator i = targets.begin(),
1535 i_end = targets.end(); i != i_end; ++i)
1536 {
1537 std::pair<rule_map::iterator, bool> j =
1538 specific_rules.insert(std::make_pair(*i, ref_ptr<rule_t>()));
1539 ref_ptr<rule_t> &r = j.first->second;
1540 if (j.second)
1541 {
1542 r = ref_ptr<rule_t>(rule);
1543 r->targets = string_list(1, *i);
1544 continue;
1545 }
1546 if (!r->script.empty())
1547 {
1548 std::cerr << "Failed to load rules: " << *i
1549 << " cannot be the target of several rules" << std::endl;
1551 }
1552 assert(r->targets.size() == 1 && r->targets.front() == *i);
1553 merge_rule(*r, rule);
1554 }
1555
1556 for (string_list::const_iterator i = targets.begin(),
1557 i_end = targets.end(); i != i_end; ++i)
1558 {
1560 if (dep->targets.empty()) dep->targets.push_back(*i);
1561 dep->deps.insert(rule.deps.begin(), rule.deps.end());
1562 }
1563}
1564
1565/**
1566 * Register a specific rule with a nonempty script:
1567 *
1568 * - Check that none of the targets already has an associated rule.
1569 * - Create a single shared rule and associate it to all the targets.
1570 * - Merge the prerequisites of all the targets into a single set and
1571 * add the prerequisites of the rule to it. (The preexisting
1572 * prerequisites, if any, come from a previous run.)
1573 */
1574static void register_scripted_rule(rule_t const &rule)
1575{
1576 ref_ptr<rule_t> r(rule);
1577 for (string_list::const_iterator i = rule.targets.begin(),
1578 i_end = rule.targets.end(); i != i_end; ++i)
1579 {
1580 std::pair<rule_map::iterator, bool> j =
1581 specific_rules.insert(std::make_pair(*i, r));
1582 if (j.second) continue;
1583 std::cerr << "Failed to load rules: " << *i
1584 << " cannot be the target of several rules" << std::endl;
1586 }
1587
1589 dep->targets = rule.targets;
1590 dep->deps.insert(rule.deps.begin(), rule.deps.end());
1591 for (string_list::const_iterator i = rule.targets.begin(),
1592 i_end = rule.targets.end(); i != i_end; ++i)
1593 {
1595 dep->deps.insert(d->deps.begin(), d->deps.end());
1596 d = dep;
1597 }
1598}
1599
1600/**
1601 * Register a specific rule.
1602 */
1603static void register_rule(rule_t const &rule)
1604{
1605 if (!rule.script.empty())
1606 {
1608 }
1609 else
1610 {
1611 // Swap away the targets to avoid costly copies when registering.
1612 rule_t &r = const_cast<rule_t &>(rule);
1613 string_list targets;
1614 targets.swap(r.targets);
1615 register_transparent_rule(r, targets);
1616 targets.swap(r.targets);
1617 }
1618
1619 // If there is no default target yet, mark it as such.
1620 if (first_target.empty())
1621 first_target = rule.targets.front();
1622}
1623
1624/**
1625 * Read a rule starting with target @a first, if nonempty.
1626 * Store into #generic_rules or #specific_rules depending on its genericity.
1627 */
1628static void load_rule(std::istream &in, std::string const &first)
1629{
1630 DEBUG_open << "Reading rule for target " << first << "... ";
1631 if (false)
1632 {
1633 error:
1634 DEBUG_close << "failed\n";
1635 std::cerr << "Failed to load rules: syntax error" << std::endl;
1637 }
1638
1639 // Read targets and check genericity.
1640 string_list targets;
1641 if (!read_words(in, targets)) goto error;
1642 if (!first.empty()) targets.push_front(first);
1643 else if (targets.empty()) goto error;
1644 else DEBUG << "actual target: " << targets.front() << std::endl;
1645 bool generic = false;
1646 normalize_list(targets, "", "");
1647 for (string_list::const_iterator i = targets.begin(),
1648 i_end = targets.end(); i != i_end; ++i)
1649 {
1650 if (i->empty()) goto error;
1651 if ((i->find('%') != std::string::npos) != generic)
1652 {
1653 if (i == targets.begin()) generic = true;
1654 else goto error;
1655 }
1656 }
1657 skip_spaces(in);
1658 if (in.get() != ':') goto error;
1659
1660 bool assignment = false, static_pattern = false;
1661
1662 rule_t rule;
1663 rule.targets.swap(targets);
1664
1665 // Read dependencies.
1666 {
1667 string_list v;
1668 if (expect_token(in, Word))
1669 {
1670 std::string d = read_word(in);
1671 if (int tok = expect_token(in, Equal | Plusequal))
1672 {
1673 if (!read_words(in, v)) goto error;
1674 assign_t &a = rule.assigns[d];
1675 a.append = tok == Plusequal;
1676 a.value.swap(v);
1677 assignment = true;
1678 goto end_line;
1679 }
1680 v.push_back(d);
1681 }
1682
1683 if (!read_words(in, v)) goto error;
1684 normalize_list(v, "", "");
1685 rule.deps.swap(v);
1686
1687 if (expect_token(in, Colon))
1688 {
1689 if (!read_words(in, v)) goto error;
1690 normalize_list(v, "", "");
1691 targets.swap(rule.targets);
1692 rule.targets.swap(rule.deps);
1693 rule.deps.swap(v);
1694 if (rule.targets.empty()) goto error;
1695 for (string_list::const_iterator i = rule.targets.begin(),
1696 i_end = rule.targets.end(); i != i_end; ++i)
1697 {
1698 if (i->find('%') == std::string::npos) goto error;
1699 }
1700 generic = false;
1701 static_pattern = true;
1702 }
1703
1704 if (expect_token(in, Pipe))
1705 {
1706 if (!read_words(in, v)) goto error;
1707 normalize_list(v, "", "");
1708 rule.wdeps.swap(v);
1709 }
1710 }
1711
1712 end_line:
1713 skip_spaces(in);
1714 if (!skip_eol(in, true)) goto error;
1715
1716 // Read script.
1717 std::ostringstream buf;
1718 while (true)
1719 {
1720 char c = in.get();
1721 if (!in.good()) break;
1722 if (c == '\t' || c == ' ')
1723 {
1724 in.get(*buf.rdbuf());
1725 if (in.fail() && !in.eof()) in.clear();
1726 }
1727 else if (c == '\r' || c == '\n')
1728 buf << c;
1729 else
1730 {
1731 in.putback(c);
1732 break;
1733 }
1734 }
1735 rule.script = buf.str();
1736
1737 // Register phony targets.
1738 if (rule.targets.front() == ".PHONY")
1739 {
1740 for (string_list::const_iterator i = rule.deps.begin(),
1741 i_end = rule.deps.end(); i != i_end; ++i)
1742 {
1743 status[*i].status = Todo;
1744 }
1745 return;
1746 }
1747
1748 // Add generic rules to the correct set.
1749 if (generic)
1750 {
1751 if (assignment) goto error;
1752 generic_rules.push_back(rule);
1753 return;
1754 }
1755
1756 if (!static_pattern)
1757 {
1758 if (!rule.script.empty() && assignment) goto error;
1759 register_rule(rule);
1760 return;
1761 }
1762
1763 for (string_list::const_iterator i = targets.begin(),
1764 i_end = targets.end(); i != i_end; ++i)
1765 {
1766 rule_t r;
1767 instantiate_rule(*i, rule, r);
1768 if (!r.stem.empty()) register_rule(r);
1769 }
1770}
1771
1772/**
1773 * Load rules from @a remakefile.
1774 * If some rules have dependencies and non-generic targets, add these
1775 * dependencies to the targets.
1776 */
1777static void load_rules(std::string const &remakefile)
1778{
1779 DEBUG_open << "Loading rules... ";
1780 if (false)
1781 {
1782 error:
1783 std::cerr << "Failed to load rules: syntax error" << std::endl;
1785 }
1786 std::ifstream in(remakefile.c_str());
1787 if (!in.good())
1788 {
1789 std::cerr << "Failed to load rules: no Remakefile found" << std::endl;
1791 }
1792 skip_empty(in);
1793
1795
1796 // Read rules
1797 while (in.good())
1798 {
1799 char c = in.peek();
1800 if (c == '#')
1801 {
1802 while (in.get() != '\n') {}
1803 skip_empty(in);
1804 continue;
1805 }
1806 if (c == ' ' || c == '\t') goto error;
1807 if (expect_token(in, Word))
1808 {
1809 std::string name = read_word(in);
1810 if (name.empty()) goto error;
1811 if (int tok = expect_token(in, Equal | Plusequal))
1812 {
1813 DEBUG << "Assignment to variable " << name << std::endl;
1814 string_list value;
1815 if (!read_words(in, value)) goto error;
1816 string_list &dest =
1817 *(name == ".OPTIONS" ? &options : &variables[name]);
1818 if (tok == Equal) dest.swap(value);
1819 else dest.splice(dest.end(), value);
1820 if (!skip_eol(in, true)) goto error;
1821 }
1822 else load_rule(in, name);
1823 }
1824 else load_rule(in, std::string());
1825 }
1826
1827 // Set actual options.
1828 for (string_list::const_iterator i = options.begin(),
1829 i_end = options.end(); i != i_end; ++i)
1830 {
1831 if (*i == "variable-propagation") propagate_vars = true;
1832 else
1833 {
1834 std::cerr << "Failed to load rules: unrecognized option" << std::endl;
1836 }
1837 }
1838}
1839
1840/** @} */
1841
1842/**
1843 * @defgroup rules Rule resolution
1844 *
1845 * @{
1846 */
1847
1848static void merge_rule(rule_t &dest, rule_t const &src)
1849{
1850 dest.deps.insert(dest.deps.end(), src.deps.begin(), src.deps.end());
1851 dest.wdeps.insert(dest.wdeps.end(), src.wdeps.begin(), src.wdeps.end());
1852 for (assign_map::const_iterator i = src.assigns.begin(),
1853 i_end = src.assigns.end(); i != i_end; ++i)
1854 {
1855 if (!i->second.append)
1856 {
1857 new_assign:
1858 dest.assigns[i->first] = i->second;
1859 continue;
1860 }
1861 assign_map::iterator j = dest.assigns.find(i->first);
1862 if (j == dest.assigns.end()) goto new_assign;
1863 j->second.value.insert(j->second.value.end(),
1864 i->second.value.begin(), i->second.value.end());
1865 }
1866}
1867
1868/**
1869 * Substitute a pattern into a list of strings.
1870 */
1871static void substitute_pattern(std::string const &pat, string_list const &src, string_list &dst)
1872{
1873 for (string_list::const_iterator i = src.begin(),
1874 i_end = src.end(); i != i_end; ++i)
1875 {
1876 size_t pos = i->find('%');
1877 if (pos == std::string::npos) dst.push_back(*i);
1878 else dst.push_back(i->substr(0, pos) + pat + i->substr(pos + 1));
1879 }
1880}
1881
1882/**
1883 * Instantiate a specific rule, given a target and a generic rule.
1884 * If the rule @a dst already contains a stem longer than the one found,
1885 * it is left unchanged.
1886 */
1887static void instantiate_rule(std::string const &target, rule_t const &src, rule_t &dst)
1888{
1889 size_t tlen = target.length(), plen = dst.stem.length();
1890 for (string_list::const_iterator j = src.targets.begin(),
1891 j_end = src.targets.end(); j != j_end; ++j)
1892 {
1893 size_t len = j->length();
1894 if (tlen < len) continue;
1895 if (plen && plen <= tlen - (len - 1)) continue;
1896 size_t pos = j->find('%');
1897 if (pos == std::string::npos) continue;
1898 size_t len2 = len - (pos + 1);
1899 if (j->compare(0, pos, target, 0, pos) ||
1900 j->compare(pos + 1, len2, target, tlen - len2, len2))
1901 continue;
1902 plen = tlen - (len - 1);
1903 dst = rule_t();
1904 dst.stem = target.substr(pos, plen);
1905 dst.script = src.script;
1906 substitute_pattern(dst.stem, src.targets, dst.targets);
1907 substitute_pattern(dst.stem, src.deps, dst.deps);
1908 substitute_pattern(dst.stem, src.wdeps, dst.wdeps);
1909 break;
1910 }
1911}
1912
1913/**
1914 * Find a generic rule matching @a target:
1915 * - the one leading to shorter matches has priority,
1916 * - among equivalent rules, the earliest one has priority.
1917 */
1918static void find_generic_rule(job_t &job, std::string const &target)
1919{
1920 for (rule_list::const_iterator i = generic_rules.begin(),
1921 i_end = generic_rules.end(); i != i_end; ++i)
1922 {
1923 instantiate_rule(target, *i, job.rule);
1924 }
1925}
1926
1927/**
1928 * Find a specific rule matching @a target. Return a generic one otherwise.
1929 * If there is both a specific rule with an empty script and a generic rule, the
1930 * generic one is returned after adding the dependencies of the specific one.
1931 */
1932static void find_rule(job_t &job, std::string const &target)
1933{
1934 rule_map::const_iterator i = specific_rules.find(target),
1935 i_end = specific_rules.end();
1936 // If there is a specific rule with a script, return it.
1937 if (i != i_end && !i->second->script.empty())
1938 {
1939 job.rule = *i->second;
1940 return;
1941 }
1943 // If there is no generic rule, return the specific rule (no script), if any.
1944 if (job.rule.targets.empty())
1945 {
1946 if (i != i_end)
1947 {
1948 job.rule = *i->second;
1949 return;
1950 }
1951 }
1952 // Optimize the lookup when there is only one target (already looked up).
1953 if (job.rule.targets.size() == 1)
1954 {
1955 if (i == i_end) return;
1956 merge_rule(job.rule, *i->second);
1957 return;
1958 }
1959 // Add the dependencies of the specific rules of every target to the
1960 // generic rule. If any of those rules has a nonempty script, error out.
1961 for (string_list::const_iterator j = job.rule.targets.begin(),
1962 j_end = job.rule.targets.end(); j != j_end; ++j)
1963 {
1964 i = specific_rules.find(*j);
1965 if (i == i_end) continue;
1966 if (!i->second->script.empty()) return;
1967 merge_rule(job.rule, *i->second);
1968 }
1969}
1970
1971/** @} */
1972
1973/**
1974 * @defgroup status Target status
1975 *
1976 * @{
1977 */
1978
1979/**
1980 * Compute and memoize the status of @a target:
1981 * - if the file does not exist, the target is obsolete,
1982 * - if any dependency is obsolete or younger than the file, it is obsolete,
1983 * - otherwise it is up-to-date.
1984 *
1985 * @note For rules with multiple targets, all the targets share the same
1986 * status. (If one is obsolete, they all are.) The second rule above
1987 * is modified in that case: the latest target is chosen, not the oldest!
1988 */
1989static status_t const &get_status(std::string const &target)
1990{
1991 std::pair<status_map::iterator,bool> i =
1992 status.insert(std::make_pair(target, status_t()));
1993 status_t &ts = i.first->second;
1994 if (!i.second) return ts;
1995 DEBUG_open << "Checking status of " << target << "... ";
1996 dependency_map::const_iterator j = dependencies.find(target);
1997 if (j == dependencies.end())
1998 {
1999 struct stat s;
2000 if (stat(target.c_str(), &s) != 0)
2001 {
2002 DEBUG_close << "missing\n";
2003 ts.status = Todo;
2004 ts.last = 0;
2005 return ts;
2006 }
2007 DEBUG_close << "up-to-date\n";
2008 ts.status = Uptodate;
2009 ts.last = s.st_mtime;
2010 return ts;
2011 }
2012 if (obsolete_targets)
2013 {
2014 DEBUG_close << "forcefully obsolete\n";
2015 ts.status = Todo;
2016 ts.last = 0;
2017 return ts;
2018 }
2019 dependency_t const &dep = *j->second;
2021 time_t latest = 0;
2022 for (string_list::const_iterator k = dep.targets.begin(),
2023 k_end = dep.targets.end(); k != k_end; ++k)
2024 {
2025 struct stat s;
2026 if (stat(k->c_str(), &s) != 0)
2027 {
2028 if (st == Uptodate) DEBUG_close << *k << " missing\n";
2029 s.st_mtime = 0;
2030 st = Todo;
2031 }
2032 status[*k].last = s.st_mtime;
2033 if (s.st_mtime > latest) latest = s.st_mtime;
2034 }
2035 if (st != Uptodate) goto update;
2036 for (string_set::const_iterator k = dep.deps.begin(),
2037 k_end = dep.deps.end(); k != k_end; ++k)
2038 {
2039 status_t const &ts_ = get_status(*k);
2040 if (latest < ts_.last)
2041 {
2042 DEBUG_close << "older than " << *k << std::endl;
2043 st = Todo;
2044 goto update;
2045 }
2046 if (ts_.status != Uptodate && st != Recheck)
2047 {
2048 DEBUG << "obsolete dependency " << *k << std::endl;
2049 st = Recheck;
2050 }
2051 }
2052 if (st == Uptodate) DEBUG_close << "all siblings up-to-date\n";
2053 update:
2054 for (string_list::const_iterator k = dep.targets.begin(),
2055 k_end = dep.targets.end(); k != k_end; ++k)
2056 {
2057 status[*k].status = st;
2058 }
2059 return ts;
2060}
2061
2062/**
2063 * Change the status of @a target to #Remade or #Uptodate depending on whether
2064 * its modification time changed.
2065 */
2066static void update_status(std::string const &target)
2067{
2068 DEBUG_open << "Rechecking status of " << target << "... ";
2069 status_map::iterator i = status.find(target);
2070 assert(i != status.end());
2071 status_t &ts = i->second;
2072 ts.status = Remade;
2073 if (ts.last >= now)
2074 {
2075 DEBUG_close << "possibly remade\n";
2076 return;
2077 }
2078 struct stat s;
2079 if (stat(target.c_str(), &s) != 0)
2080 {
2081 DEBUG_close << "missing\n";
2082 ts.last = 0;
2083 }
2084 else if (s.st_mtime != ts.last)
2085 {
2086 DEBUG_close << "remade\n";
2087 ts.last = s.st_mtime;
2088 }
2089 else
2090 {
2091 DEBUG_close << "unchanged\n";
2092 ts.status = Uptodate;
2093 }
2094}
2095
2096/**
2097 * Check whether all the prerequisites of @a target ended being up-to-date.
2098 */
2099static bool still_need_rebuild(std::string const &target)
2100{
2101 status_map::const_iterator i = status.find(target);
2102 assert(i != status.end());
2103 if (i->second.status != RunningRecheck) return true;
2104 DEBUG_open << "Rechecking obsoleteness of " << target << "... ";
2105 dependency_map::const_iterator j = dependencies.find(target);
2106 assert(j != dependencies.end());
2107 dependency_t const &dep = *j->second;
2108 for (string_set::const_iterator k = dep.deps.begin(),
2109 k_end = dep.deps.end(); k != k_end; ++k)
2110 {
2111 if (status[*k].status != Uptodate) return true;
2112 }
2113 for (string_list::const_iterator k = dep.targets.begin(),
2114 k_end = dep.targets.end(); k != k_end; ++k)
2115 {
2116 status[*k].status = Uptodate;
2117 }
2118 DEBUG_close << "no longer obsolete\n";
2119 return false;
2120}
2121
2122/** @} */
2123
2124/**
2125 * @defgroup server Server
2126 *
2127 * @{
2128 */
2129
2130/**
2131 * Handle job completion.
2132 */
2133static void complete_job(int job_id, bool success, bool started = true)
2134{
2135 DEBUG << "Completing job " << job_id << '\n';
2136 job_map::iterator i = jobs.find(job_id);
2137 assert(i != jobs.end());
2138 string_list const &targets = i->second.rule.targets;
2139 if (success)
2140 {
2141 bool show = show_targets && started;
2142 if (show) std::cout << "Finished";
2143 for (string_list::const_iterator j = targets.begin(),
2144 j_end = targets.end(); j != j_end; ++j)
2145 {
2146 update_status(*j);
2147 if (show) std::cout << ' ' << *j;
2148 }
2149 if (show) std::cout << std::endl;
2150 }
2151 else
2152 {
2153 std::cerr << "Failed to build";
2154 for (string_list::const_iterator j = targets.begin(),
2155 j_end = targets.end(); j != j_end; ++j)
2156 {
2157 std::cerr << ' ' << *j;
2158 update_status(*j);
2159 status_e &s = status[*j].status;
2160 if (s != Uptodate)
2161 {
2162 DEBUG << "Removing " << *j << '\n';
2163 remove(j->c_str());
2164 }
2165 s = Failed;
2166 }
2167 std::cerr << std::endl;
2168 }
2169 jobs.erase(i);
2170}
2171
2172/**
2173 * Return the script obtained by substituting variables.
2174 */
2175static std::string prepare_script(job_t const &job)
2176{
2177 std::string const &s = job.rule.script;
2178 std::istringstream in(s);
2179 std::ostringstream out;
2180 size_t len = s.size();
2181
2182 while (!in.eof())
2183 {
2184 size_t pos = in.tellg(), p = s.find('$', pos);
2185 if (p == std::string::npos || p == len - 1) p = len;
2186 out.write(&s[pos], p - pos);
2187 if (p == len) break;
2188 ++p;
2189 switch (s[p])
2190 {
2191 case '$':
2192 out << '$';
2193 in.seekg(p + 1);
2194 break;
2195 case '<':
2196 if (!job.rule.deps.empty())
2197 out << job.rule.deps.front();
2198 in.seekg(p + 1);
2199 break;
2200 case '^':
2201 {
2202 bool first = true;
2203 for (string_list::const_iterator i = job.rule.deps.begin(),
2204 i_end = job.rule.deps.end(); i != i_end; ++i)
2205 {
2206 if (first) first = false;
2207 else out << ' ';
2208 out << *i;
2209 }
2210 in.seekg(p + 1);
2211 break;
2212 }
2213 case '@':
2214 assert(!job.rule.targets.empty());
2215 out << job.rule.targets.front();
2216 in.seekg(p + 1);
2217 break;
2218 case '*':
2219 out << job.rule.stem;
2220 in.seekg(p + 1);
2221 break;
2222 case '(':
2223 {
2224 in.seekg(p - 1);
2225 bool first = true;
2226 input_generator gen(in, &job.vars, true);
2227 while (true)
2228 {
2229 std::string w;
2230 input_status s = gen.next(w);
2231 if (s == SyntaxError)
2232 {
2233 // TODO
2234 return "false";
2235 }
2236 if (s == Eof) break;
2237 if (first) first = false;
2238 else out << ' ';
2239 out << w;
2240 }
2241 break;
2242 }
2243 default:
2244 // Let dollars followed by an unrecognized character
2245 // go through. This differs from Make, which would
2246 // use a one-letter variable.
2247 out << '$';
2248 in.seekg(p);
2249 }
2250 }
2251
2252 return out.str();
2253}
2254
2255/**
2256 * Execute the script from @a rule.
2257 */
2258static status_e run_script(int job_id, job_t const &job)
2259{
2261 dep->targets = job.rule.targets;
2262 dep->deps.insert(job.rule.deps.begin(), job.rule.deps.end());
2263 if (show_targets) std::cout << "Building";
2264 for (string_list::const_iterator i = job.rule.targets.begin(),
2265 i_end = job.rule.targets.end(); i != i_end; ++i)
2266 {
2267 dependencies[*i] = dep;
2268 if (show_targets) std::cout << ' ' << *i;
2269 }
2270 if (show_targets) std::cout << std::endl;
2271
2272 std::string script = prepare_script(job);
2273
2274 std::ostringstream job_id_buf;
2275 job_id_buf << job_id;
2276 std::string job_id_ = job_id_buf.str();
2277
2278 DEBUG_open << "Starting script for job " << job_id << "... ";
2279 if (script.empty())
2280 {
2281 DEBUG_close << "no script\n";
2282 complete_job(job_id, true);
2283 return Remade;
2284 }
2285
2286 if (false)
2287 {
2288 error:
2289 DEBUG_close << "failed\n";
2290 complete_job(job_id, false);
2291 return Failed;
2292 }
2293
2294#ifdef WINDOWS
2295 HANDLE pfd[2];
2296 if (false)
2297 {
2298 error2:
2299 CloseHandle(pfd[0]);
2300 CloseHandle(pfd[1]);
2301 goto error;
2302 }
2303 if (!CreatePipe(&pfd[0], &pfd[1], NULL, 0))
2304 goto error;
2306 goto error2;
2308 ZeroMemory(&si, sizeof(STARTUPINFO));
2309 si.cb = sizeof(STARTUPINFO);
2310 si.hStdError = GetStdHandle(STD_ERROR_HANDLE);
2311 si.hStdOutput = GetStdHandle(STD_OUTPUT_HANDLE);
2312 si.hStdInput = pfd[0];
2313 si.dwFlags |= STARTF_USESTDHANDLES;
2316 if (!SetEnvironmentVariable("REMAKE_JOB_ID", job_id_.c_str()))
2317 goto error2;
2318 char const *argv = echo_scripts ? "SH.EXE -e -s -v" : "SH.EXE -e -s";
2319 if (!CreateProcess(NULL, (char *)argv, NULL, NULL,
2320 true, 0, NULL, NULL, &si, &pi))
2321 {
2322 goto error2;
2323 }
2324 CloseHandle(pi.hThread);
2325 DWORD len = script.length(), wlen;
2326 if (!WriteFile(pfd[1], script.c_str(), len, &wlen, NULL) || wlen < len)
2327 std::cerr << "Unexpected failure while sending script to shell" << std::endl;
2328 CloseHandle(pfd[0]);
2329 CloseHandle(pfd[1]);
2330 ++running_jobs;
2331 job_pids[pi.hProcess] = job_id;
2332 return Running;
2333#else
2334 int pfd[2];
2335 if (false)
2336 {
2337 error2:
2338 close(pfd[0]);
2339 close(pfd[1]);
2340 goto error;
2341 }
2342 if (pipe(pfd) == -1)
2343 goto error;
2344 if (setenv("REMAKE_JOB_ID", job_id_.c_str(), 1))
2345 goto error2;
2346 if (pid_t pid = vfork())
2347 {
2348 if (pid == -1) goto error2;
2349 ssize_t len = script.length();
2350 if (write(pfd[1], script.c_str(), len) < len)
2351 std::cerr << "Unexpected failure while sending script to shell" << std::endl;
2352 close(pfd[0]);
2353 close(pfd[1]);
2354 ++running_jobs;
2355 job_pids[pid] = job_id;
2356 return Running;
2357 }
2358 // Child process starts here. Notice the use of vfork above.
2359 char const *argv[5] = { "sh", "-e", "-s", NULL, NULL };
2360 if (echo_scripts) argv[3] = "-v";
2361 close(pfd[1]);
2362 if (pfd[0] != 0)
2363 {
2364 dup2(pfd[0], 0);
2365 close(pfd[0]);
2366 }
2368 execve("/bin/sh", (char **)argv, environ);
2370#endif
2371}
2372
2373/**
2374 * Create a job for @a target according to the loaded rules.
2375 * Mark all the targets from the rule as running and reset their dependencies.
2376 * Inherit variables from @a current, if enabled.
2377 * If the rule has dependencies, create a new client to build them just
2378 * before @a current, and change @a current so that it points to it.
2379 */
2380static status_e start(std::string const &target, client_list::iterator &current)
2381{
2382 int job_id = job_counter++;
2383 DEBUG_open << "Starting job " << job_id << " for " << target << "... ";
2384 job_t &job = jobs[job_id];
2386 if (job.rule.targets.empty())
2387 {
2388 status[target].status = Failed;
2389 DEBUG_close << "failed\n";
2390 std::cerr << "No rule for building " << target << std::endl;
2391 return Failed;
2392 }
2393 bool has_deps = !job.rule.deps.empty() || !job.rule.wdeps.empty();
2395 if (has_deps && status[target].status == Recheck)
2397 for (string_list::const_iterator i = job.rule.targets.begin(),
2398 i_end = job.rule.targets.end(); i != i_end; ++i)
2399 {
2400 status[*i].status = st;
2401 }
2402 if (propagate_vars) job.vars = current->vars;
2403 for (assign_map::const_iterator i = job.rule.assigns.begin(),
2404 i_end = job.rule.assigns.end(); i != i_end; ++i)
2405 {
2406 std::pair<variable_map::iterator, bool> k =
2407 job.vars.insert(std::make_pair(i->first, string_list()));
2408 string_list &v = k.first->second;
2409 if (i->second.append)
2410 {
2411 if (k.second)
2412 {
2413 variable_map::const_iterator j = variables.find(i->first);
2414 if (j != variables.end()) v = j->second;
2415 }
2416 }
2417 else if (!k.second) v.clear();
2418 v.insert(v.end(), i->second.value.begin(), i->second.value.end());
2419 }
2420 if (has_deps)
2421 {
2422 current = clients.insert(current, client_t());
2423 current->job_id = job_id;
2424 current->pending = job.rule.deps;
2425 current->pending.insert(current->pending.end(),
2426 job.rule.wdeps.begin(), job.rule.wdeps.end());
2427 if (propagate_vars) current->vars = job.vars;
2428 current->delayed = true;
2429 return RunningRecheck;
2430 }
2431 return run_script(job_id, job);
2432}
2433
2434/**
2435 * Send a reply to a client then remove it.
2436 * If the client was a dependency client, start the actual script.
2437 */
2438static void complete_request(client_t &client, bool success)
2439{
2440 DEBUG_open << "Completing request from client of job " << client.job_id << "... ";
2441 if (client.delayed)
2442 {
2443 assert(client.socket == INVALID_SOCKET);
2444 if (success)
2445 {
2446 job_map::const_iterator i = jobs.find(client.job_id);
2447 assert(i != jobs.end());
2448 if (still_need_rebuild(i->second.rule.targets.front()))
2449 run_script(client.job_id, i->second);
2450 else complete_job(client.job_id, true, false);
2451 }
2452 else complete_job(client.job_id, false);
2453 }
2454 else if (client.socket != INVALID_SOCKET)
2455 {
2456 char res = success ? 1 : 0;
2457 send(client.socket, &res, 1, MSG_NOSIGNAL);
2458 #ifdef WINDOWS
2459 closesocket(client.socket);
2460 #else
2461 close(client.socket);
2462 #endif
2463 --waiting_jobs;
2464 }
2465
2466 if (client.job_id < 0 && !success) build_failure = true;
2467}
2468
2469/**
2470 * Return whether there are slots for starting new jobs.
2471 */
2472static bool has_free_slots()
2473{
2474 if (max_active_jobs <= 0) return true;
2476}
2477
2478/**
2479 * Handle client requests:
2480 * - check for running targets that have finished,
2481 * - start as many pending targets as allowed,
2482 * - complete the request if there are neither running nor pending targets
2483 * left or if any of them failed.
2484 *
2485 * @return true if some child processes are still running.
2486 *
2487 * @post If there are pending requests, at least one child process is running.
2488 *
2489 * @invariant New free slots cannot appear during a run, since the only way to
2490 * decrease #running_jobs is #finalize_job and the only way to
2491 * increase #waiting_jobs is #accept_client. None of these functions
2492 * are called during a run. So breaking out as soon as there are no
2493 * free slots left is fine.
2494 */
2495static bool handle_clients()
2496{
2497 DEBUG_open << "Handling client requests... ";
2498 restart:
2499 bool need_restart = false;
2500
2501 for (client_list::iterator i = clients.begin(), i_next = i,
2502 i_end = clients.end(); i != i_end; i = i_next)
2503 {
2504 if (!has_free_slots()) break;
2505 ++i_next;
2506 DEBUG_open << "Handling client from job " << i->job_id << "... ";
2507
2508 // Remove running targets that have finished.
2509 for (string_set::iterator j = i->running.begin(), j_next = j,
2510 j_end = i->running.end(); j != j_end; j = j_next)
2511 {
2512 ++j_next;
2513 status_map::const_iterator k = status.find(*j);
2514 assert(k != status.end());
2515 switch (k->second.status)
2516 {
2517 case Running:
2518 case RunningRecheck:
2519 break;
2520 case Failed:
2521 i->failed = true;
2522 if (!keep_going) goto complete;
2523 // fallthrough
2524 case Uptodate:
2525 case Remade:
2526 i->running.erase(j);
2527 break;
2528 case Recheck:
2529 case Todo:
2530 assert(false);
2531 }
2532 }
2533
2534 // Start pending targets.
2535 while (!i->pending.empty())
2536 {
2537 std::string target = i->pending.front();
2538 i->pending.pop_front();
2539 switch (get_status(target).status)
2540 {
2541 case Running:
2542 case RunningRecheck:
2543 i->running.insert(target);
2544 break;
2545 case Failed:
2547 i->failed = true;
2548 if (!keep_going) goto complete;
2549 // fallthrough
2550 case Uptodate:
2551 case Remade:
2552 break;
2553 case Recheck:
2554 case Todo:
2555 client_list::iterator j = i;
2556 switch (start(target, i))
2557 {
2558 case Failed:
2559 goto pending_failed;
2560 case Running:
2561 // A shell was started, check for free slots.
2562 j->running.insert(target);
2563 if (!has_free_slots()) return true;
2564 break;
2565 case RunningRecheck:
2566 // Switch to the dependency client that was inserted.
2567 j->running.insert(target);
2568 i_next = j;
2569 break;
2570 case Remade:
2571 // Nothing to run.
2572 need_restart = true;
2573 break;
2574 default:
2575 assert(false);
2576 }
2577 }
2578 }
2579
2580 // Try to complete the request.
2581 // (This might start a new job if it was a dependency client.)
2582 if (i->running.empty() || i->failed)
2583 {
2584 complete:
2585 complete_request(*i, !i->failed);
2586 DEBUG_close << (i->failed ? "failed\n" : "finished\n");
2587 clients.erase(i);
2588 need_restart = true;
2589 }
2590 }
2591
2592 if (running_jobs != waiting_jobs) return true;
2593 if (running_jobs == 0 && clients.empty()) return false;
2594 if (need_restart) goto restart;
2595
2596 // There is a circular dependency.
2597 // Try to break it by completing one of the requests.
2598 assert(!clients.empty());
2599 std::cerr << "Circular dependency detected" << std::endl;
2600 client_list::iterator i = clients.begin();
2601 complete_request(*i, false);
2602 clients.erase(i);
2603 goto restart;
2604}
2605
2606/**
2607 * Create a named unix socket that listens for build requests. Also set
2608 * the REMAKE_SOCKET environment variable that will be inherited by all
2609 * the job scripts.
2610 */
2611static void create_server()
2612{
2613 if (false)
2614 {
2615 error:
2616 perror("Failed to create server");
2617#ifndef WINDOWS
2618 error2:
2619#endif
2621 }
2622 DEBUG_open << "Creating server... ";
2623
2624#ifdef WINDOWS
2625 // Prepare a windows socket.
2626 struct sockaddr_in socket_addr;
2627 socket_addr.sin_family = AF_INET;
2628 socket_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
2629 socket_addr.sin_port = 0;
2630
2631 // Create and listen to the socket.
2632 socket_fd = socket(AF_INET, SOCK_STREAM, 0);
2633 if (socket_fd == INVALID_SOCKET) goto error;
2635 goto error;
2636 if (bind(socket_fd, (struct sockaddr *)&socket_addr, sizeof(sockaddr_in)))
2637 goto error;
2638 int len = sizeof(sockaddr_in);
2639 if (getsockname(socket_fd, (struct sockaddr *)&socket_addr, &len))
2640 goto error;
2641 std::ostringstream buf;
2642 buf << socket_addr.sin_port;
2643 if (!SetEnvironmentVariable("REMAKE_SOCKET", buf.str().c_str()))
2644 goto error;
2645 if (listen(socket_fd, 1000)) goto error;
2646#else
2647 // Set signal handlers for SIGCHLD and SIGINT.
2648 // Block SIGCHLD (unblocked during select).
2652 if (sigprocmask(SIG_BLOCK, &sigmask, &old_sigmask) == -1) goto error;
2653 struct sigaction sa;
2654 sa.sa_flags = 0;
2655 sigemptyset(&sa.sa_mask);
2656 sa.sa_handler = &sigchld_handler;
2657 if (sigaction(SIGCHLD, &sa, NULL) == -1) goto error;
2658 sa.sa_handler = &sigint_handler;
2659 if (sigaction(SIGINT, &sa, NULL) == -1) goto error;
2660
2661 // Prepare a named unix socket in temporary directory.
2662 socket_name = tempnam(NULL, "rmk-");
2663 if (!socket_name) goto error2;
2664 struct sockaddr_un socket_addr;
2665 size_t len = strlen(socket_name);
2666 if (len >= sizeof(socket_addr.sun_path) - 1) goto error2;
2667 socket_addr.sun_family = AF_UNIX;
2668 strcpy(socket_addr.sun_path, socket_name);
2669 len += sizeof(socket_addr.sun_family);
2670 if (setenv("REMAKE_SOCKET", socket_name, 1)) goto error;
2671
2672 // Create and listen to the socket.
2673#ifdef LINUX
2674 socket_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0);
2675 if (socket_fd == INVALID_SOCKET) goto error;
2676#else
2677 socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
2678 if (socket_fd == INVALID_SOCKET) goto error;
2679 if (fcntl(socket_fd, F_SETFD, FD_CLOEXEC) < 0) goto error;
2680#endif
2681 if (bind(socket_fd, (struct sockaddr *)&socket_addr, len))
2682 goto error;
2683 if (listen(socket_fd, 1000)) goto error;
2684#endif
2685}
2686
2687/**
2688 * Accept a connection from a client, get the job it spawned from,
2689 * get the targets, and mark them as dependencies of the job targets.
2690 */
2691static void accept_client()
2692{
2693 DEBUG_open << "Handling client request... ";
2694
2695 // Accept connection.
2696#ifdef WINDOWS
2698 if (fd == INVALID_SOCKET) return;
2700 {
2701 error2:
2702 std::cerr << "Unexpected failure while setting connection with client" << std::endl;
2703 closesocket(fd);
2704 return;
2705 }
2706 // WSAEventSelect puts sockets into nonblocking mode, so disable it here.
2707 u_long nbio = 0;
2708 if (ioctlsocket(fd, FIONBIO, &nbio)) goto error2;
2709#elif defined(LINUX)
2711 if (fd < 0) return;
2712#else
2713 int fd = accept(socket_fd, NULL, NULL);
2714 if (fd < 0) return;
2715 if (fcntl(fd, F_SETFD, FD_CLOEXEC) < 0) return;
2716#endif
2717 clients.push_front(client_t());
2718 client_list::iterator proc = clients.begin();
2719
2720 if (false)
2721 {
2722 error:
2723 DEBUG_close << "failed\n";
2724 std::cerr << "Received an ill-formed client message" << std::endl;
2725 #ifdef WINDOWS
2726 closesocket(fd);
2727 #else
2728 close(fd);
2729 #endif
2730 clients.erase(proc);
2731 return;
2732 }
2733
2734 // Receive message. Stop when encountering two nuls in a row.
2735 std::vector<char> buf;
2736 size_t len = 0;
2737 while (len < sizeof(int) + 2 || buf[len - 1] || buf[len - 2])
2738 {
2739 buf.resize(len + 1024);
2740 ssize_t l = recv(fd, &buf[0] + len, 1024, 0);
2741 if (l <= 0) goto error;
2742 len += l;
2743 }
2744
2745 // Parse job that spawned the client.
2746 int job_id;
2747 memcpy(&job_id, &buf[0], sizeof(int));
2748 proc->socket = fd;
2749 proc->job_id = job_id;
2750 job_map::const_iterator i = jobs.find(job_id);
2751 if (i == jobs.end()) goto error;
2752 DEBUG << "receiving request from job " << job_id << std::endl;
2753 if (propagate_vars) proc->vars = i->second.vars;
2754
2755 // Parse the targets and the variable assignments.
2756 // Mark the targets as dependencies of the job targets.
2757 dependency_t &dep = *dependencies[i->second.rule.targets.front()];
2759 char const *p = &buf[0] + sizeof(int);
2760 while (true)
2761 {
2762 len = strlen(p);
2763 if (len == 0)
2764 {
2765 ++waiting_jobs;
2766 break;
2767 }
2768 switch (*p)
2769 {
2770 case 'T':
2771 {
2772 if (len == 1) goto error;
2773 std::string target(p + 1, p + len);
2774 DEBUG << "adding dependency " << target << " to job\n";
2775 proc->pending.push_back(target);
2776 dep.deps.insert(target);
2777 break;
2778 }
2779 case 'V':
2780 {
2781 if (len == 1) goto error;
2782 std::string var(p + 1, p + len);
2783 DEBUG << "adding variable " << var << " to job\n";
2784 last_var = &proc->vars[var];
2785 last_var->clear();
2786 break;
2787 }
2788 case 'W':
2789 {
2790 if (!last_var) goto error;
2791 last_var->push_back(std::string(p + 1, p + len));
2792 break;
2793 }
2794 default:
2795 goto error;
2796 }
2797 p += len + 1;
2798 }
2799
2800 if (!propagate_vars && !proc->vars.empty())
2801 {
2802 std::cerr << "Assignments are ignored unless 'variable-propagation' is enabled" << std::endl;
2803 proc->vars.clear();
2804 }
2805}
2806
2807/**
2808 * Handle child process exit status.
2809 */
2810static void finalize_job(pid_t pid, bool res)
2811{
2812 pid_job_map::iterator i = job_pids.find(pid);
2813 assert(i != job_pids.end());
2814 int job_id = i->second;
2815 job_pids.erase(i);
2816 --running_jobs;
2817 complete_job(job_id, res);
2818}
2819
2820/**
2821 * Loop until all the jobs have finished.
2822 *
2823 * @post There are no client requests left, not even virtual ones.
2824 */
2825static void server_loop()
2826{
2827 while (handle_clients())
2828 {
2829 DEBUG_open << "Handling events... ";
2830 #ifdef WINDOWS
2831 size_t len = job_pids.size() + 1;
2832 HANDLE h[len];
2833 int num = 0;
2834 for (pid_job_map::const_iterator i = job_pids.begin(),
2835 i_end = job_pids.end(); i != i_end; ++i, ++num)
2836 {
2837 h[num] = i->first;
2838 }
2840 h[num] = aev;
2845 if (len <= w)
2846 continue;
2847 if (w == len - 1)
2848 {
2849 accept_client();
2850 continue;
2851 }
2852 pid_t pid = h[w];
2853 DWORD s = 0;
2854 bool res = GetExitCodeProcess(pid, &s) && s == 0;
2857 #else
2860 fd_set fdset;
2861 FD_ZERO(&fdset);
2863 int ret = pselect(socket_fd + 1, &fdset, NULL, NULL, NULL, &emptymask);
2864 if (ret > 0 /* && FD_ISSET(socket_fd, &fdset)*/) accept_client();
2865 if (!got_SIGCHLD) continue;
2866 got_SIGCHLD = 0;
2867 pid_t pid;
2868 int status;
2869 while ((pid = waitpid(-1, &status, WNOHANG)) > 0)
2870 {
2871 bool res = WIFEXITED(status) && WEXITSTATUS(status) == 0;
2873 }
2874 #endif
2875 }
2876
2877 assert(clients.empty());
2878}
2879
2880/**
2881 * Load dependencies and rules, listen to client requests, and loop until
2882 * all the requests have completed.
2883 * If Remakefile is obsolete, perform a first run with it only, then reload
2884 * the rules, and perform a second with the original clients.
2885 */
2886static void server_mode(std::string const &remakefile, string_list const &targets)
2887{
2890 create_server();
2892 {
2893 clients.push_back(client_t());
2894 clients.back().pending.push_back(remakefile);
2895 server_loop();
2896 if (build_failure) goto early_exit;
2897 variables.clear();
2898 specific_rules.clear();
2899 generic_rules.clear();
2900 first_target.clear();
2902 }
2903 clients.push_back(client_t());
2904 if (!targets.empty()) clients.back().pending = targets;
2905 else if (!first_target.empty())
2906 clients.back().pending.push_back(first_target);
2907 server_loop();
2908 early_exit:
2910#ifndef WINDOWS
2913#endif
2916 {
2917 std::cout << "remake: Leaving directory `" << prefix_dir << '\'' << std::endl;
2918 }
2920}
2921
2922/** @} */
2923
2924/**
2925 * @defgroup client Client
2926 *
2927 * @{
2928 */
2929
2930/**
2931 * Connect to the server @a socket_name, send a request for building @a targets
2932 * with some @a variables, and exit with the status returned by the server.
2933 */
2934static void client_mode(char *socket_name, string_list const &targets)
2935{
2936 if (false)
2937 {
2938 error:
2939 perror("Failed to send targets to server");
2941 }
2942 if (targets.empty()) exit(EXIT_SUCCESS);
2943 DEBUG_open << "Connecting to server... ";
2944
2945 // Connect to server.
2946#ifdef WINDOWS
2947 struct sockaddr_in socket_addr;
2948 socket_fd = socket(AF_INET, SOCK_STREAM, 0);
2949 if (socket_fd == INVALID_SOCKET) goto error;
2950 socket_addr.sin_family = AF_INET;
2951 socket_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
2952 socket_addr.sin_port = atoi(socket_name);
2953 if (connect(socket_fd, (struct sockaddr *)&socket_addr, sizeof(sockaddr_in)))
2954 goto error;
2955#else
2956 struct sockaddr_un socket_addr;
2957 size_t len = strlen(socket_name);
2958 if (len >= sizeof(socket_addr.sun_path) - 1) exit(EXIT_FAILURE);
2959 socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
2960 if (socket_fd == INVALID_SOCKET) goto error;
2961 socket_addr.sun_family = AF_UNIX;
2962 strcpy(socket_addr.sun_path, socket_name);
2963 if (connect(socket_fd, (struct sockaddr *)&socket_addr, sizeof(socket_addr.sun_family) + len))
2964 goto error;
2965#ifdef MACOSX
2966 int set_option = 1;
2968 goto error;
2969#endif
2970#endif
2971
2972 // Send current job id.
2973 char *id = getenv("REMAKE_JOB_ID");
2974 int job_id = id ? atoi(id) : -1;
2975 if (send(socket_fd, (char *)&job_id, sizeof(job_id), MSG_NOSIGNAL) != sizeof(job_id))
2976 goto error;
2977
2978 // Send targets.
2979 for (string_list::const_iterator i = targets.begin(),
2980 i_end = targets.end(); i != i_end; ++i)
2981 {
2982 DEBUG_open << "Sending target " << *i << "... ";
2983 std::string s = 'T' + *i;
2984 ssize_t len = s.length() + 1;
2985 if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
2986 goto error;
2987 }
2988
2989 // Send variables.
2990 for (variable_map::const_iterator i = variables.begin(),
2991 i_end = variables.end(); i != i_end; ++i)
2992 {
2993 DEBUG_open << "Sending variable " << i->first << "... ";
2994 std::string s = 'V' + i->first;
2995 ssize_t len = s.length() + 1;
2996 if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
2997 goto error;
2998 for (string_list::const_iterator j = i->second.begin(),
2999 j_end = i->second.end(); j != j_end; ++j)
3000 {
3001 std::string s = 'W' + *j;
3002 len = s.length() + 1;
3003 if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
3004 goto error;
3005 }
3006 }
3007
3008 // Send terminating nul and wait for reply.
3009 char result = 0;
3010 if (send(socket_fd, &result, 1, MSG_NOSIGNAL) != 1) goto error;
3011 if (recv(socket_fd, &result, 1, 0) != 1) exit(EXIT_FAILURE);
3013}
3014
3015/** @} */
3016
3017/**
3018 * @defgroup ui User interface
3019 *
3020 * @{
3021 */
3022
3023/**
3024 * Display usage and exit with @a exit_status.
3025 */
3026static void usage(int exit_status)
3027{
3028 std::cerr << "Usage: remake [options] [target] ...\n"
3029 "Options\n"
3030 " -B, --always-make Unconditionally make all targets.\n"
3031 " -d Echo script commands.\n"
3032 " -d -d Print lots of debugging information.\n"
3033 " -f FILE Read FILE as Remakefile.\n"
3034 " -h, --help Print this message and exit.\n"
3035 " -j[N], --jobs=[N] Allow N jobs at once; infinite jobs with no arg.\n"
3036 " -k, --keep-going Keep going when some targets cannot be made.\n"
3037 " -r Look up targets from the dependencies on stdin.\n"
3038 " -s, --silent, --quiet Do not echo targets.\n";
3040}
3041
3042/**
3043 * This program behaves in two different ways.
3044 *
3045 * - If the environment contains the REMAKE_SOCKET variable, the client
3046 * connects to this socket and sends to the server its build targets.
3047 * It exits once it receives the server reply.
3048 *
3049 * - Otherwise, it creates a server that waits for build requests. It
3050 * also creates a pseudo-client that requests the targets passed on the
3051 * command line.
3052 */
3053int main(int argc, char *argv[])
3054{
3055 std::string remakefile;
3056 string_list targets;
3057 bool literal_targets = false;
3058 bool indirect_targets = false;
3059
3060 // Parse command-line arguments.
3061 for (int i = 1; i < argc; ++i)
3062 {
3063 std::string arg = argv[i];
3064 if (arg.empty()) usage(EXIT_FAILURE);
3065 if (literal_targets) goto new_target;
3066 if (arg == "-h" || arg == "--help") usage(EXIT_SUCCESS);
3067 if (arg == "-d")
3068 if (echo_scripts) debug.active = true;
3069 else echo_scripts = true;
3070 else if (arg == "-k" || arg =="--keep-going")
3071 keep_going = true;
3072 else if (arg == "-s" || arg == "--silent" || arg == "--quiet")
3073 show_targets = false;
3074 else if (arg == "-r")
3075 indirect_targets = true;
3076 else if (arg == "-B" || arg == "--always-make")
3077 obsolete_targets = true;
3078 else if (arg == "-f")
3079 {
3080 if (++i == argc) usage(EXIT_FAILURE);
3081 remakefile = argv[i];
3082 }
3083 else if (arg == "--")
3084 literal_targets = true;
3085 else if (arg.compare(0, 2, "-j") == 0)
3086 max_active_jobs = atoi(arg.c_str() + 2);
3087 else if (arg.compare(0, 7, "--jobs=") == 0)
3088 max_active_jobs = atoi(arg.c_str() + 7);
3089 else
3090 {
3091 if (arg[0] == '-') usage(EXIT_FAILURE);
3092 if (arg.find('=') != std::string::npos)
3093 {
3094 std::istringstream in(arg);
3095 std::string name = read_word(in);
3096 if (name.empty() || !expect_token(in, Equal)) usage(EXIT_FAILURE);
3097 read_words(in, variables[name]);
3098 continue;
3099 }
3100 new_target:
3101 targets.push_back(arg);
3102 DEBUG << "New target: " << arg << '\n';
3103 }
3104 }
3105
3108
3109 if (indirect_targets)
3110 {
3111 load_dependencies(std::cin);
3112 string_list l;
3113 targets.swap(l);
3114 if (l.empty() && !dependencies.empty())
3115 {
3116 l.push_back(dependencies.begin()->second->targets.front());
3117 }
3118 for (string_list::const_iterator i = l.begin(),
3119 i_end = l.end(); i != i_end; ++i)
3120 {
3121 dependency_map::const_iterator j = dependencies.find(*i);
3122 if (j == dependencies.end()) continue;
3123 dependency_t const &dep = *j->second;
3124 for (string_set::const_iterator k = dep.deps.begin(),
3125 k_end = dep.deps.end(); k != k_end; ++k)
3126 {
3127 targets.push_back(normalize(*k, working_dir, working_dir));
3128 }
3129 }
3130 dependencies.clear();
3131 }
3132
3133#ifdef WINDOWS
3135 if (WSAStartup(MAKEWORD(2,2), &wsaData))
3136 {
3137 std::cerr << "Unexpected failure while initializing Windows Socket" << std::endl;
3138 return 1;
3139 }
3140#endif
3141
3142 // Run as client if REMAKE_SOCKET is present in the environment.
3143 if (char *sn = getenv("REMAKE_SOCKET")) client_mode(sn, targets);
3144
3145 // Otherwise run as server.
3146 if (remakefile.empty())
3147 {
3148 remakefile = "Remakefile";
3150 }
3152 server_mode(remakefile, targets);
3153}
3154
3155/** @} */
static void client_mode(char *socket_name, string_list const &targets)
Definition remake.cpp:2934
static void save_dependencies()
Definition remake.cpp:1489
static void load_dependencies()
Definition remake.cpp:1473
static bool skip_eol(std::istream &in, bool multi=false)
Definition remake.cpp:1047
static int expect_token(std::istream &in, int mask)
Definition remake.cpp:1076
static void skip_empty(std::istream &in)
Definition remake.cpp:1036
static std::string read_word(std::istream &in, bool detect_equal=true)
Definition remake.cpp:1122
static void skip_spaces(std::istream &in)
Definition remake.cpp:1026
@ Plusequal
Definition remake.cpp:1066
@ Word
Definition remake.cpp:1060
@ Equal
Definition remake.cpp:1062
@ Unexpected
Definition remake.cpp:1059
@ Colon
Definition remake.cpp:1061
@ Dollarpar
Definition remake.cpp:1063
@ Pipe
Definition remake.cpp:1067
@ Comma
Definition remake.cpp:1065
@ Rightpar
Definition remake.cpp:1064
static void load_rules(std::string const &remakefile)
Definition remake.cpp:1777
static void register_transparent_rule(rule_t const &rule, string_list const &targets)
Definition remake.cpp:1531
static void register_scripted_rule(rule_t const &rule)
Definition remake.cpp:1574
static void register_rule(rule_t const &rule)
Definition remake.cpp:1603
static void load_rule(std::istream &in, std::string const &first)
Definition remake.cpp:1628
static std::string normalize_abs(std::string const &s, std::string const &p)
Definition remake.cpp:933
static std::string normalize(std::string const &s, std::string const &w, std::string const &p)
Definition remake.cpp:955
static void init_working_dir()
Definition remake.cpp:877
static void init_prefix_dir()
Definition remake.cpp:899
static void normalize_list(string_list &l, std::string const &w, std::string const &p)
Definition remake.cpp:1006
static void substitute_pattern(std::string const &pat, string_list const &src, string_list &dst)
Definition remake.cpp:1871
static void instantiate_rule(std::string const &target, rule_t const &src, rule_t &dst)
Definition remake.cpp:1887
static void find_generic_rule(job_t &job, std::string const &target)
Definition remake.cpp:1918
static void merge_rule(rule_t &dest, rule_t const &src)
Definition remake.cpp:1848
static void find_rule(job_t &job, std::string const &target)
Definition remake.cpp:1932
static void complete_request(client_t &client, bool success)
Definition remake.cpp:2438
static void accept_client()
Definition remake.cpp:2691
static bool handle_clients()
Definition remake.cpp:2495
static void create_server()
Definition remake.cpp:2611
static void finalize_job(pid_t pid, bool res)
Definition remake.cpp:2810
static std::string prepare_script(job_t const &job)
Definition remake.cpp:2175
static status_e start(std::string const &target, client_list::iterator &current)
Definition remake.cpp:2380
static status_e run_script(int job_id, job_t const &job)
Definition remake.cpp:2258
static bool has_free_slots()
Definition remake.cpp:2472
static void complete_job(int job_id, bool success, bool started=true)
Definition remake.cpp:2133
static void server_loop()
Definition remake.cpp:2825
static void server_mode(std::string const &remakefile, string_list const &targets)
Definition remake.cpp:2886
static bool still_need_rebuild(std::string const &target)
Definition remake.cpp:2099
static void update_status(std::string const &target)
Definition remake.cpp:2066
static status_t const & get_status(std::string const &target)
Definition remake.cpp:1989
input_status next(std::string &)
Definition remake.cpp:1220
addprefix_generator(input_generator const &, bool &)
Definition remake.cpp:1320
variable_generator(std::string const &, variable_map const *)
Definition remake.cpp:1201
static bool read_words(input_generator &in, string_list &res)
Definition remake.cpp:1288
input_status next(std::string &)
Definition remake.cpp:1386
input_status next(std::string &)
Definition remake.cpp:1248
addsuffix_generator(input_generator const &, bool &)
Definition remake.cpp:1376
static generator * get_function(input_generator const &, std::string const &)
Definition remake.cpp:1416
input_status
Definition remake.cpp:1175
input_status next(std::string &)
Definition remake.cpp:1330
@ SyntaxError
Definition remake.cpp:1177
@ Eof
Definition remake.cpp:1178
@ Success
Definition remake.cpp:1176
int main(int argc, char *argv[])
Definition remake.cpp:3053
static void usage(int exit_status)
Definition remake.cpp:3026
static bool keep_going
Definition remake.cpp:667
@ INVALID_SOCKET
Definition remake.cpp:463
static int max_active_jobs
Definition remake.cpp:661
static int job_counter
Definition remake.cpp:694
static bool build_failure
Definition remake.cpp:704
static void sigchld_handler(int)
Definition remake.cpp:763
std::map< int, job_t > job_map
Definition remake.cpp:584
std::map< std::string, status_t > status_map
Definition remake.cpp:544
int socket_t
Definition remake.cpp:462
std::map< std::string, ref_ptr< rule_t > > rule_map
Definition remake.cpp:572
static client_list clients
Definition remake.cpp:655
static int waiting_jobs
Definition remake.cpp:688
static time_t now
Definition remake.cpp:731
static std::string first_target
Definition remake.cpp:716
static rule_list generic_rules
Definition remake.cpp:634
std::list< std::string > string_list
Definition remake.cpp:471
static std::string working_dir
Definition remake.cpp:736
static status_map status
Definition remake.cpp:629
std::list< rule_t > rule_list
Definition remake.cpp:570
#define DEBUG_close
Definition remake.cpp:819
status_e
Definition remake.cpp:525
@ Failed
Build failed for target.
Definition remake.cpp:532
@ Todo
Target is missing or obsolete.
Definition remake.cpp:527
@ Running
Target is being rebuilt.
Definition remake.cpp:529
@ Recheck
Target has an obsolete dependency.
Definition remake.cpp:528
@ Remade
Target was successfully rebuilt.
Definition remake.cpp:531
@ Uptodate
Target is up-to-date.
Definition remake.cpp:526
@ RunningRecheck
Static prerequisites are being rebuilt.
Definition remake.cpp:530
static dependency_map dependencies
Definition remake.cpp:624
static std::ostream & operator<<(std::ostream &out, escape_string const &se)
Definition remake.cpp:836
static variable_map variables
Definition remake.cpp:619
std::set< std::string > string_set
Definition remake.cpp:473
static bool obsolete_targets
Definition remake.cpp:756
static bool changed_prefix_dir
Definition remake.cpp:746
static job_map jobs
Definition remake.cpp:644
std::map< pid_t, int > pid_job_map
Definition remake.cpp:586
static char * socket_name
Definition remake.cpp:710
static bool show_targets
Definition remake.cpp:721
static pid_job_map job_pids
Definition remake.cpp:649
char ** environ
std::map< std::string, ref_ptr< dependency_t > > dependency_map
Definition remake.cpp:517
std::map< std::string, assign_t > assign_map
Definition remake.cpp:555
static bool echo_scripts
Definition remake.cpp:726
static int running_jobs
Definition remake.cpp:680
static sigset_t old_sigmask
Definition remake.cpp:759
std::map< std::string, string_list > variable_map
Definition remake.cpp:519
#define DEBUG_open
Definition remake.cpp:818
#define DEBUG
Definition remake.cpp:817
static rule_map specific_rules
Definition remake.cpp:639
static socket_t socket_fd
Definition remake.cpp:699
static std::string prefix_dir
Definition remake.cpp:741
static bool propagate_vars
Definition remake.cpp:751
static void sigint_handler(int)
Definition remake.cpp:768
static log debug
Definition remake.cpp:803
static volatile sig_atomic_t got_SIGCHLD
Definition remake.cpp:761
std::list< client_t > client_list
Definition remake.cpp:613
input_generator gen
Definition remake.cpp:1311
string_list::const_iterator prei
Definition remake.cpp:1313
std::string suf
Definition remake.cpp:1315
string_list pre
Definition remake.cpp:1312
string_list suf
Definition remake.cpp:1368
string_list::const_iterator sufi
Definition remake.cpp:1369
input_generator gen
Definition remake.cpp:1367
std::string pre
Definition remake.cpp:1371
bool append
Definition remake.cpp:551
string_list value
Definition remake.cpp:552
string_list pending
Targets not yet started.
Definition remake.cpp:606
socket_t socket
Socket used to reply to the client (invalid for pseudo clients).
Definition remake.cpp:603
bool delayed
Whether it is a dependency client and a script has to be started on request completion.
Definition remake.cpp:609
bool failed
Whether some targets failed in mode -k.
Definition remake.cpp:605
int job_id
Job for which the built script called remake and spawned the client (negative for original clients).
Definition remake.cpp:604
string_set running
Targets being built.
Definition remake.cpp:607
variable_map vars
Variables set on request.
Definition remake.cpp:608
string_list targets
Definition remake.cpp:513
string_set deps
Definition remake.cpp:514
escape_string(std::string const &s)
Definition remake.cpp:829
std::string const & input
Definition remake.cpp:828
virtual ~generator()
Definition remake.cpp:1186
virtual input_status next(std::string &)=0
std::istream & in
Definition remake.cpp:1236
generator * nested
Definition remake.cpp:1237
variable_map const * local_variables
Definition remake.cpp:1238
input_generator(std::istream &i, variable_map const *lv, bool e=false)
Definition remake.cpp:1240
variable_map vars
Values of local variables.
Definition remake.cpp:581
rule_t rule
Original rule.
Definition remake.cpp:580
bool open
Definition remake.cpp:778
std::ostream & operator()(bool o)
Definition remake.cpp:791
bool active
Definition remake.cpp:778
int depth
Definition remake.cpp:779
log()
Definition remake.cpp:780
std::ostream & operator()()
Definition remake.cpp:783
content(T const &t)
Definition remake.cpp:488
~ref_ptr()
Definition remake.cpp:494
ref_ptr(T const &t)
Definition remake.cpp:492
T & operator*() const
Definition remake.cpp:503
ref_ptr()
Definition remake.cpp:491
ref_ptr(ref_ptr const &p)
Definition remake.cpp:493
content * ptr
Definition remake.cpp:490
T * operator->() const
Definition remake.cpp:508
ref_ptr & operator=(ref_ptr const &p)
Definition remake.cpp:495
assign_map assigns
Assignment of variables.
Definition remake.cpp:565
string_list wdeps
Like deps, except that they are not registered as dependencies.
Definition remake.cpp:564
std::string script
Shell script for building the targets.
Definition remake.cpp:567
string_list targets
Files produced by this rule.
Definition remake.cpp:562
std::string stem
Stem used to instantiate the rule, if any.
Definition remake.cpp:566
string_list deps
Dependencies used for an implicit call to remake at the start of the script.
Definition remake.cpp:563
status_e status
Actual status.
Definition remake.cpp:540
time_t last
Last-modified date.
Definition remake.cpp:541
string_list::const_iterator vend
Definition remake.cpp:1196
string_list::const_iterator vcur
Definition remake.cpp:1196
std::string name
Definition remake.cpp:1195