2024-04-09 08:08:21 -07:00
|
|
|
/* $OpenBSD: engine.c,v 1.74 2024/04/09 15:08:21 cheloha Exp $ */
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
/*
|
|
|
|
* Copyright (c) 2012 Marc Espie.
|
|
|
|
*
|
|
|
|
* Extensive code modifications for the OpenBSD project.
|
|
|
|
*
|
|
|
|
* Redistribution and use in source and binary forms, with or without
|
|
|
|
* modification, are permitted provided that the following conditions
|
|
|
|
* are met:
|
|
|
|
* 1. Redistributions of source code must retain the above copyright
|
|
|
|
* notice, this list of conditions and the following disclaimer.
|
|
|
|
* 2. Redistributions in binary form must reproduce the above copyright
|
|
|
|
* notice, this list of conditions and the following disclaimer in the
|
|
|
|
* documentation and/or other materials provided with the distribution.
|
|
|
|
*
|
|
|
|
* THIS SOFTWARE IS PROVIDED BY THE OPENBSD PROJECT AND CONTRIBUTORS
|
|
|
|
* ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
|
|
|
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
|
|
|
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE OPENBSD
|
|
|
|
* PROJECT OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
|
|
|
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
|
|
|
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
|
|
|
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
|
|
|
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
|
|
|
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
|
|
|
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
|
|
|
*/
|
2007-09-16 03:39:07 -07:00
|
|
|
/*
|
|
|
|
* Copyright (c) 1988, 1989, 1990 The Regents of the University of California.
|
|
|
|
* Copyright (c) 1988, 1989 by Adam de Boor
|
|
|
|
* Copyright (c) 1989 by Berkeley Softworks
|
|
|
|
* All rights reserved.
|
|
|
|
*
|
|
|
|
* This code is derived from software contributed to Berkeley by
|
|
|
|
* Adam de Boor.
|
|
|
|
*
|
|
|
|
* Redistribution and use in source and binary forms, with or without
|
|
|
|
* modification, are permitted provided that the following conditions
|
|
|
|
* are met:
|
|
|
|
* 1. Redistributions of source code must retain the above copyright
|
|
|
|
* notice, this list of conditions and the following disclaimer.
|
|
|
|
* 2. Redistributions in binary form must reproduce the above copyright
|
|
|
|
* notice, this list of conditions and the following disclaimer in the
|
|
|
|
* documentation and/or other materials provided with the distribution.
|
|
|
|
* 3. Neither the name of the University nor the names of its contributors
|
|
|
|
* may be used to endorse or promote products derived from this software
|
|
|
|
* without specific prior written permission.
|
|
|
|
*
|
|
|
|
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
|
|
|
|
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
|
|
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
|
|
|
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
|
|
|
|
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
|
|
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
|
|
|
|
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
|
|
|
|
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
|
|
|
|
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
|
|
|
|
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
|
|
|
* SUCH DAMAGE.
|
|
|
|
*/
|
|
|
|
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
#include <sys/types.h>
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
#include <sys/time.h>
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
#include <sys/wait.h>
|
2008-11-03 23:22:35 -08:00
|
|
|
#include <assert.h>
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
#include <ctype.h>
|
2007-09-16 03:39:07 -07:00
|
|
|
#include <errno.h>
|
|
|
|
#include <fcntl.h>
|
2013-04-23 07:32:53 -07:00
|
|
|
#include <limits.h>
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
#include <signal.h>
|
2013-04-23 07:32:53 -07:00
|
|
|
#include <stdint.h>
|
|
|
|
#include <stdio.h>
|
|
|
|
#include <stdlib.h>
|
|
|
|
#include <string.h>
|
|
|
|
#include <unistd.h>
|
2007-09-16 03:39:07 -07:00
|
|
|
#include "defines.h"
|
2023-08-30 23:53:28 -07:00
|
|
|
#include "cmd_exec.h"
|
2007-09-16 03:39:07 -07:00
|
|
|
#include "dir.h"
|
|
|
|
#include "engine.h"
|
|
|
|
#include "arch.h"
|
|
|
|
#include "gnode.h"
|
|
|
|
#include "targ.h"
|
|
|
|
#include "var.h"
|
|
|
|
#include "extern.h"
|
|
|
|
#include "lst.h"
|
|
|
|
#include "timestamp.h"
|
2023-05-29 21:42:21 -07:00
|
|
|
#include "main.h"
|
2007-09-16 03:39:07 -07:00
|
|
|
#include "make.h"
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
#include "pathnames.h"
|
|
|
|
#include "error.h"
|
|
|
|
#include "memory.h"
|
2007-11-17 08:39:45 -08:00
|
|
|
#include "buf.h"
|
2009-05-10 04:07:37 -07:00
|
|
|
#include "job.h"
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
#include "lowparse.h"
|
2007-09-16 03:39:07 -07:00
|
|
|
|
|
|
|
static void MakeTimeStamp(void *, void *);
|
2007-09-17 05:07:22 -07:00
|
|
|
static int rewrite_time(const char *);
|
2012-10-02 03:29:30 -07:00
|
|
|
static void list_parents(GNode *, FILE *);
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
|
2012-10-06 02:32:40 -07:00
|
|
|
/* XXX due to a bug in make's logic, targets looking like *.a or -l*
|
|
|
|
* have been silently dropped when make couldn't figure them out.
|
|
|
|
* Now, we warn about them until all Makefile bugs have been fixed.
|
|
|
|
*/
|
|
|
|
static bool
|
|
|
|
drop_silently(const char *s)
|
|
|
|
{
|
|
|
|
size_t len;
|
|
|
|
|
|
|
|
if (s[0] == '-' && s[1] == 'l')
|
|
|
|
return true;
|
|
|
|
|
|
|
|
len = strlen(s);
|
|
|
|
if (len >=2 && s[len-2] == '.' && s[len-1] == 'a')
|
|
|
|
return true;
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2007-09-16 03:39:07 -07:00
|
|
|
bool
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
node_find_valid_commands(GNode *gn)
|
2007-09-16 03:39:07 -07:00
|
|
|
{
|
2012-10-09 12:50:44 -07:00
|
|
|
if (DEBUG(DOUBLE) && (gn->type & OP_DOUBLE))
|
|
|
|
fprintf(stderr, "Warning: target %s had >1 lists of "
|
|
|
|
"shell commands (ignoring later ones)\n", gn->name);
|
2012-10-02 03:29:30 -07:00
|
|
|
if (OP_NOP(gn->type) && Lst_IsEmpty(&gn->commands)) {
|
2012-10-06 02:32:40 -07:00
|
|
|
if (drop_silently(gn->name)) {
|
2012-10-02 03:29:30 -07:00
|
|
|
printf("Warning: target %s", gn->name);
|
|
|
|
list_parents(gn, stdout);
|
2012-10-06 02:32:40 -07:00
|
|
|
printf(" does not have any command (BUG)\n");
|
2012-10-02 03:29:30 -07:00
|
|
|
return true;
|
|
|
|
}
|
2007-09-16 07:36:57 -07:00
|
|
|
/*
|
|
|
|
* No commands. Look for .DEFAULT rule from which we might infer
|
|
|
|
* commands
|
|
|
|
*/
|
2010-04-25 06:59:53 -07:00
|
|
|
if ((gn->type & OP_NODEFAULT) == 0 &&
|
2008-01-02 07:37:22 -08:00
|
|
|
(DEFAULT->type & OP_DUMMY) == 0 &&
|
2007-09-17 05:42:09 -07:00
|
|
|
!Lst_IsEmpty(&DEFAULT->commands)) {
|
2007-09-16 07:36:57 -07:00
|
|
|
/*
|
|
|
|
* Make only looks for a .DEFAULT if the node was never
|
|
|
|
* the target of an operator, so that's what we do too.
|
|
|
|
* If a .DEFAULT was given, we substitute its commands
|
|
|
|
* for gn's commands and set the IMPSRC variable to be
|
|
|
|
* the target's name The DEFAULT node acts like a
|
|
|
|
* transformation rule, in that gn also inherits any
|
|
|
|
* attributes or sources attached to .DEFAULT itself.
|
|
|
|
*/
|
|
|
|
Make_HandleUse(DEFAULT, gn);
|
2007-11-17 08:39:45 -08:00
|
|
|
Var(IMPSRC_INDEX, gn) = Var(TARGET_INDEX, gn);
|
2007-09-16 07:36:57 -07:00
|
|
|
} else if (is_out_of_date(Dir_MTime(gn))) {
|
|
|
|
/*
|
|
|
|
* The node wasn't the target of an operator we have no
|
|
|
|
* .DEFAULT rule to go on and the target doesn't
|
|
|
|
* already exist. There's nothing more we can do for
|
2010-04-25 06:59:53 -07:00
|
|
|
* this branch.
|
2007-09-16 07:36:57 -07:00
|
|
|
*/
|
2008-11-03 23:22:35 -08:00
|
|
|
return false;
|
|
|
|
}
|
2007-09-16 03:39:07 -07:00
|
|
|
}
|
2007-09-16 07:36:57 -07:00
|
|
|
return true;
|
2007-09-16 03:39:07 -07:00
|
|
|
}
|
|
|
|
|
2012-10-02 03:29:30 -07:00
|
|
|
static void
|
|
|
|
list_parents(GNode *gn, FILE *out)
|
|
|
|
{
|
|
|
|
LstNode ln;
|
|
|
|
bool first = true;
|
|
|
|
|
|
|
|
for (ln = Lst_First(&gn->parents); ln != NULL; ln = Lst_Adv(ln)) {
|
|
|
|
GNode *p = Lst_Datum(ln);
|
|
|
|
if (!p->must_make)
|
|
|
|
continue;
|
|
|
|
if (first) {
|
|
|
|
fprintf(out, " (prerequisite of:");
|
|
|
|
first = false;
|
|
|
|
}
|
|
|
|
fprintf(out, " %s", p->name);
|
|
|
|
}
|
|
|
|
if (!first)
|
|
|
|
fprintf(out, ")");
|
|
|
|
}
|
|
|
|
|
2008-11-03 23:22:35 -08:00
|
|
|
void
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
node_failure(GNode *gn)
|
2008-11-03 23:22:35 -08:00
|
|
|
{
|
|
|
|
/*
|
|
|
|
If the -k flag wasn't given, we stop in
|
|
|
|
* our tracks, otherwise we just don't update this
|
|
|
|
* node's parents so they never get examined.
|
|
|
|
*/
|
2012-10-02 03:29:30 -07:00
|
|
|
const char *diag;
|
|
|
|
FILE *out;
|
2008-11-03 23:22:35 -08:00
|
|
|
|
|
|
|
if (gn->type & OP_OPTIONAL) {
|
2012-10-02 03:29:30 -07:00
|
|
|
out = stdout;
|
|
|
|
diag = "(ignored)";
|
2008-11-03 23:22:35 -08:00
|
|
|
} else if (keepgoing) {
|
2012-10-02 03:29:30 -07:00
|
|
|
out = stdout;
|
|
|
|
diag = "(continuing)";
|
2008-11-03 23:22:35 -08:00
|
|
|
} else {
|
2012-10-02 03:29:30 -07:00
|
|
|
out = stderr;
|
|
|
|
diag = "";
|
|
|
|
}
|
|
|
|
fprintf(out, "make: don't know how to make %s", gn->name);
|
|
|
|
list_parents(gn, out);
|
|
|
|
fprintf(out, "%s\n", diag);
|
|
|
|
if (out == stdout)
|
|
|
|
fflush(stdout);
|
|
|
|
else {
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
print_errors();
|
2023-05-29 21:42:21 -07:00
|
|
|
dump_unreadable();
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
Punt(NULL);
|
2008-11-03 23:22:35 -08:00
|
|
|
}
|
|
|
|
}
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
|
2007-09-17 05:07:22 -07:00
|
|
|
/* touch files the hard way, by writing stuff to them */
|
|
|
|
static int
|
|
|
|
rewrite_time(const char *name)
|
|
|
|
{
|
|
|
|
int fd;
|
|
|
|
char c;
|
|
|
|
|
|
|
|
fd = open(name, O_RDWR | O_CREAT, 0666);
|
|
|
|
if (fd < 0)
|
|
|
|
return -1;
|
|
|
|
/*
|
|
|
|
* Read and write a byte to the file to change
|
|
|
|
* the modification time.
|
|
|
|
*/
|
|
|
|
if (read(fd, &c, 1) == 1) {
|
|
|
|
(void)lseek(fd, 0, SEEK_SET);
|
|
|
|
(void)write(fd, &c, 1);
|
|
|
|
}
|
|
|
|
|
|
|
|
(void)close(fd);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2007-09-16 03:39:07 -07:00
|
|
|
void
|
2007-11-03 07:05:39 -07:00
|
|
|
Job_Touch(GNode *gn)
|
2007-09-16 03:39:07 -07:00
|
|
|
{
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
handle_all_signals();
|
2020-01-26 04:41:21 -08:00
|
|
|
if (gn->type & (OP_USE|OP_OPTIONAL|OP_PHONY)) {
|
2007-09-16 03:39:07 -07:00
|
|
|
/*
|
2007-09-17 05:42:09 -07:00
|
|
|
* .JOIN, .USE, and .OPTIONAL targets are "virtual" targets
|
|
|
|
* and, as such, shouldn't really be created.
|
2009-08-16 02:47:06 -07:00
|
|
|
* Likewise, .PHONY targets are not really files
|
2007-09-16 03:39:07 -07:00
|
|
|
*/
|
2007-09-16 07:36:57 -07:00
|
|
|
return;
|
|
|
|
}
|
2007-09-16 03:39:07 -07:00
|
|
|
|
2018-11-27 01:33:48 -08:00
|
|
|
if (!Targ_Silent(gn)) {
|
2007-09-16 07:36:57 -07:00
|
|
|
(void)fprintf(stdout, "touch %s\n", gn->name);
|
2007-09-16 03:39:07 -07:00
|
|
|
(void)fflush(stdout);
|
|
|
|
}
|
2007-09-16 07:36:57 -07:00
|
|
|
|
|
|
|
if (noExecute) {
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (gn->type & OP_ARCHV) {
|
|
|
|
Arch_Touch(gn);
|
|
|
|
} else {
|
|
|
|
const char *file = gn->path != NULL ? gn->path : gn->name;
|
|
|
|
|
2024-04-09 08:08:21 -07:00
|
|
|
if (utimes(file, NULL) == -1){
|
2007-09-17 05:07:22 -07:00
|
|
|
if (rewrite_time(file) == -1) {
|
2014-10-31 06:29:42 -07:00
|
|
|
(void)fprintf(stderr,
|
2007-09-17 02:28:36 -07:00
|
|
|
"*** couldn't touch %s: %s", file,
|
2007-09-16 07:36:57 -07:00
|
|
|
strerror(errno));
|
2007-09-17 05:42:09 -07:00
|
|
|
}
|
2007-09-16 07:36:57 -07:00
|
|
|
}
|
|
|
|
}
|
2007-09-16 03:39:07 -07:00
|
|
|
}
|
|
|
|
|
|
|
|
void
|
2007-09-17 05:10:35 -07:00
|
|
|
Make_TimeStamp(GNode *parent, GNode *child)
|
2007-09-16 03:39:07 -07:00
|
|
|
{
|
2013-05-30 01:58:38 -07:00
|
|
|
if (is_strictly_before(parent->youngest->mtime, child->mtime)) {
|
2013-05-25 04:54:14 -07:00
|
|
|
parent->youngest = child;
|
2013-05-14 11:47:40 -07:00
|
|
|
}
|
2007-09-16 03:39:07 -07:00
|
|
|
}
|
|
|
|
|
|
|
|
void
|
2007-09-17 05:42:09 -07:00
|
|
|
Make_HandleUse(GNode *cgn, /* The .USE node */
|
2007-09-16 03:39:07 -07:00
|
|
|
GNode *pgn) /* The target of the .USE node */
|
|
|
|
{
|
2007-09-16 07:36:57 -07:00
|
|
|
GNode *gn; /* A child of the .USE node */
|
|
|
|
LstNode ln; /* An element in the children list */
|
|
|
|
|
2008-11-03 23:22:35 -08:00
|
|
|
assert(cgn->type & (OP_USE|OP_TRANSFORM));
|
2007-09-16 03:39:07 -07:00
|
|
|
|
2017-01-29 02:04:13 -08:00
|
|
|
if (pgn == NULL)
|
|
|
|
Fatal("Trying to apply .USE to '%s' without a parent",
|
|
|
|
cgn->name);
|
|
|
|
|
2008-11-03 23:22:35 -08:00
|
|
|
if ((cgn->type & OP_USE) || Lst_IsEmpty(&pgn->commands)) {
|
|
|
|
/* .USE or transformation and target has no commands
|
|
|
|
* -- append the child's commands to the parent. */
|
|
|
|
Lst_Concat(&pgn->commands, &cgn->commands);
|
|
|
|
}
|
2007-09-16 03:39:07 -07:00
|
|
|
|
2008-11-03 23:22:35 -08:00
|
|
|
for (ln = Lst_First(&cgn->children); ln != NULL;
|
|
|
|
ln = Lst_Adv(ln)) {
|
2016-10-21 09:12:38 -07:00
|
|
|
gn = Lst_Datum(ln);
|
2007-09-16 03:39:07 -07:00
|
|
|
|
2008-11-03 23:22:35 -08:00
|
|
|
if (Lst_AddNew(&pgn->children, gn)) {
|
|
|
|
Lst_AtEnd(&gn->parents, pgn);
|
2019-12-21 07:29:25 -08:00
|
|
|
pgn->children_left++;
|
2008-11-03 23:22:35 -08:00
|
|
|
}
|
2007-09-16 03:39:07 -07:00
|
|
|
}
|
2008-11-03 23:22:35 -08:00
|
|
|
|
2012-10-09 12:50:44 -07:00
|
|
|
if (DEBUG(DOUBLE) && (cgn->type & OP_DOUBLE))
|
|
|
|
fprintf(stderr,
|
|
|
|
"Warning: .USE %s expanded in %s had >1 lists of "
|
|
|
|
"shell commands (ignoring later ones)\n",
|
|
|
|
cgn->name, pgn->name);
|
|
|
|
pgn->type |= cgn->type & ~(OP_OPMASK|OP_USE|OP_TRANSFORM|OP_DOUBLE);
|
2008-11-03 23:22:35 -08:00
|
|
|
|
|
|
|
/*
|
2019-12-21 07:29:25 -08:00
|
|
|
* This child node is now built, so we decrement the count of
|
|
|
|
* not yet built children in the parent... We also remove the child
|
2008-11-03 23:22:35 -08:00
|
|
|
* from the parent's list to accurately reflect the number of
|
2019-12-21 07:29:25 -08:00
|
|
|
* remaining children the parent has. This is used by Make_Run to
|
2008-11-03 23:22:35 -08:00
|
|
|
* decide whether to queue the parent or examine its children...
|
|
|
|
*/
|
|
|
|
if (cgn->type & OP_USE)
|
2019-12-21 07:29:25 -08:00
|
|
|
pgn->children_left--;
|
2007-09-16 03:39:07 -07:00
|
|
|
}
|
|
|
|
|
2007-11-17 08:39:45 -08:00
|
|
|
void
|
|
|
|
Make_DoAllVar(GNode *gn)
|
2007-09-16 03:39:07 -07:00
|
|
|
{
|
2007-11-17 08:39:45 -08:00
|
|
|
GNode *child;
|
|
|
|
LstNode ln;
|
|
|
|
BUFFER allsrc, oodate;
|
|
|
|
char *target;
|
|
|
|
bool do_oodate;
|
|
|
|
int oodate_count, allsrc_count = 0;
|
|
|
|
|
|
|
|
oodate_count = 0;
|
|
|
|
allsrc_count = 0;
|
|
|
|
|
2012-11-21 15:21:54 -08:00
|
|
|
Var(OODATE_INDEX, gn) = "";
|
|
|
|
Var(ALLSRC_INDEX, gn) = "";
|
|
|
|
|
2007-11-17 08:39:45 -08:00
|
|
|
for (ln = Lst_First(&gn->children); ln != NULL; ln = Lst_Adv(ln)) {
|
2016-10-21 09:12:38 -07:00
|
|
|
child = Lst_Datum(ln);
|
2020-01-26 04:41:21 -08:00
|
|
|
if ((child->type & (OP_USE|OP_INVISIBLE)) != 0)
|
2007-11-17 08:39:45 -08:00
|
|
|
continue;
|
2007-09-17 05:42:09 -07:00
|
|
|
if (OP_NOP(child->type) ||
|
2007-11-17 08:39:45 -08:00
|
|
|
(target = Var(TARGET_INDEX, child)) == NULL) {
|
2007-09-16 07:36:57 -07:00
|
|
|
/*
|
|
|
|
* this node is only source; use the specific pathname
|
|
|
|
* for it
|
|
|
|
*/
|
2007-09-17 05:42:09 -07:00
|
|
|
target = child->path != NULL ? child->path :
|
|
|
|
child->name;
|
2007-09-16 07:36:57 -07:00
|
|
|
}
|
2007-09-16 03:39:07 -07:00
|
|
|
|
2007-11-17 08:39:45 -08:00
|
|
|
/*
|
|
|
|
* It goes in the OODATE variable if the parent is younger than
|
|
|
|
* the child or if the child has been modified more recently
|
|
|
|
* than the start of the make. This is to keep make from
|
|
|
|
* getting confused if something else updates the parent after
|
|
|
|
* the make starts (shouldn't happen, I know, but sometimes it
|
|
|
|
* does). In such a case, if we've updated the kid, the parent
|
|
|
|
* is likely to have a modification time later than that of the
|
|
|
|
* kid and anything that relies on the OODATE variable will be
|
|
|
|
* hosed.
|
|
|
|
*/
|
|
|
|
do_oodate = false;
|
2020-01-26 04:41:21 -08:00
|
|
|
if (is_strictly_before(gn->mtime, child->mtime) ||
|
2013-05-22 05:14:08 -07:00
|
|
|
(!is_strictly_before(child->mtime, starttime) &&
|
2019-12-21 07:28:16 -08:00
|
|
|
child->built_status == REBUILT))
|
2007-11-17 08:39:45 -08:00
|
|
|
do_oodate = true;
|
|
|
|
if (do_oodate) {
|
|
|
|
oodate_count++;
|
|
|
|
if (oodate_count == 1)
|
|
|
|
Var(OODATE_INDEX, gn) = target;
|
|
|
|
else {
|
|
|
|
if (oodate_count == 2) {
|
|
|
|
Buf_Init(&oodate, 0);
|
2010-04-25 06:59:53 -07:00
|
|
|
Buf_AddString(&oodate,
|
2007-11-17 08:39:45 -08:00
|
|
|
Var(OODATE_INDEX, gn));
|
|
|
|
}
|
|
|
|
Buf_AddSpace(&oodate);
|
|
|
|
Buf_AddString(&oodate, target);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
allsrc_count++;
|
|
|
|
if (allsrc_count == 1)
|
|
|
|
Var(ALLSRC_INDEX, gn) = target;
|
|
|
|
else {
|
|
|
|
if (allsrc_count == 2) {
|
|
|
|
Buf_Init(&allsrc, 0);
|
2010-04-25 06:59:53 -07:00
|
|
|
Buf_AddString(&allsrc,
|
2007-11-17 08:39:45 -08:00
|
|
|
Var(ALLSRC_INDEX, gn));
|
|
|
|
}
|
|
|
|
Buf_AddSpace(&allsrc);
|
|
|
|
Buf_AddString(&allsrc, target);
|
2007-09-16 07:36:57 -07:00
|
|
|
}
|
2007-09-16 03:39:07 -07:00
|
|
|
}
|
|
|
|
|
2007-11-17 08:39:45 -08:00
|
|
|
if (allsrc_count > 1)
|
|
|
|
Var(ALLSRC_INDEX, gn) = Buf_Retrieve(&allsrc);
|
|
|
|
if (oodate_count > 1)
|
|
|
|
Var(OODATE_INDEX, gn) = Buf_Retrieve(&oodate);
|
2007-09-16 03:39:07 -07:00
|
|
|
|
2007-11-06 13:12:23 -08:00
|
|
|
if (gn->impliedsrc)
|
2007-11-17 08:39:45 -08:00
|
|
|
Var(IMPSRC_INDEX, gn) = Var(TARGET_INDEX, gn->impliedsrc);
|
2007-09-16 03:39:07 -07:00
|
|
|
}
|
|
|
|
|
|
|
|
/* Wrapper to call Make_TimeStamp from a forEach loop. */
|
|
|
|
static void
|
2007-09-17 05:42:09 -07:00
|
|
|
MakeTimeStamp(void *parent, void *child)
|
2007-09-16 03:39:07 -07:00
|
|
|
{
|
2015-01-23 05:18:40 -08:00
|
|
|
Make_TimeStamp(parent, child);
|
2007-09-16 03:39:07 -07:00
|
|
|
}
|
|
|
|
|
|
|
|
bool
|
2007-09-17 05:42:09 -07:00
|
|
|
Make_OODate(GNode *gn)
|
2007-09-16 03:39:07 -07:00
|
|
|
{
|
2007-09-17 05:42:09 -07:00
|
|
|
bool oodate;
|
2007-09-16 07:36:57 -07:00
|
|
|
|
2007-09-16 03:39:07 -07:00
|
|
|
/*
|
2007-09-16 07:36:57 -07:00
|
|
|
* Certain types of targets needn't even be sought as their datedness
|
|
|
|
* doesn't depend on their modification time...
|
2007-09-16 03:39:07 -07:00
|
|
|
*/
|
2020-01-26 04:41:21 -08:00
|
|
|
if ((gn->type & (OP_USE|OP_PHONY)) == 0) {
|
2007-09-16 07:36:57 -07:00
|
|
|
(void)Dir_MTime(gn);
|
|
|
|
if (DEBUG(MAKE)) {
|
2007-09-17 05:42:09 -07:00
|
|
|
if (!is_out_of_date(gn->mtime))
|
2007-09-17 02:28:36 -07:00
|
|
|
printf("modified %s...",
|
2013-05-22 05:14:08 -07:00
|
|
|
time_to_string(&gn->mtime));
|
2007-09-17 05:42:09 -07:00
|
|
|
else
|
2007-09-16 07:36:57 -07:00
|
|
|
printf("non-existent...");
|
|
|
|
}
|
2007-09-16 03:39:07 -07:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2019-12-21 07:29:25 -08:00
|
|
|
* A target is rebuilt in one of the following circumstances:
|
2007-09-17 05:42:09 -07:00
|
|
|
* - its modification time is smaller than that of its youngest child
|
|
|
|
* and it would actually be run (has commands or type OP_NOP)
|
|
|
|
* - it's the object of a force operator
|
|
|
|
* - it has no children, was on the lhs of an operator and doesn't
|
|
|
|
* exist already.
|
2007-09-16 07:36:57 -07:00
|
|
|
*
|
2007-09-16 03:39:07 -07:00
|
|
|
*/
|
2007-09-16 07:36:57 -07:00
|
|
|
if (gn->type & OP_USE) {
|
|
|
|
/*
|
|
|
|
* If the node is a USE node it is *never* out of date
|
|
|
|
* no matter *what*.
|
|
|
|
*/
|
2007-09-17 05:42:09 -07:00
|
|
|
if (DEBUG(MAKE))
|
2007-09-16 07:36:57 -07:00
|
|
|
printf(".USE node...");
|
|
|
|
oodate = false;
|
2020-01-26 04:41:21 -08:00
|
|
|
} else if (gn->type & (OP_FORCE|OP_PHONY)) {
|
2007-09-16 07:36:57 -07:00
|
|
|
/*
|
2007-09-17 05:42:09 -07:00
|
|
|
* A node which is the object of the force (!) operator or which
|
|
|
|
* has the .EXEC attribute is always considered out-of-date.
|
2007-09-16 07:36:57 -07:00
|
|
|
*/
|
|
|
|
if (DEBUG(MAKE)) {
|
2007-09-17 05:42:09 -07:00
|
|
|
if (gn->type & OP_FORCE)
|
2007-09-16 07:36:57 -07:00
|
|
|
printf("! operator...");
|
2007-09-17 05:42:09 -07:00
|
|
|
else if (gn->type & OP_PHONY)
|
2007-09-16 07:36:57 -07:00
|
|
|
printf(".PHONY node...");
|
2007-09-17 05:42:09 -07:00
|
|
|
else
|
2007-09-16 07:36:57 -07:00
|
|
|
printf(".EXEC node...");
|
|
|
|
}
|
|
|
|
oodate = true;
|
2013-05-30 01:58:38 -07:00
|
|
|
} else if (is_strictly_before(gn->mtime, gn->youngest->mtime) ||
|
|
|
|
(gn == gn->youngest &&
|
2007-09-16 07:36:57 -07:00
|
|
|
(is_out_of_date(gn->mtime) || (gn->type & OP_DOUBLEDEP)))) {
|
|
|
|
/*
|
|
|
|
* A node whose modification time is less than that of its
|
2013-05-30 01:58:38 -07:00
|
|
|
* youngest child or that has no children (gn->youngest == gn)
|
|
|
|
* and either doesn't exist (mtime == OUT_OF_DATE)
|
2007-09-17 05:42:09 -07:00
|
|
|
* or was the object of a :: operator is out-of-date.
|
2007-09-16 07:36:57 -07:00
|
|
|
*/
|
|
|
|
if (DEBUG(MAKE)) {
|
2013-05-30 01:58:38 -07:00
|
|
|
if (is_strictly_before(gn->mtime, gn->youngest->mtime))
|
2013-05-14 11:47:40 -07:00
|
|
|
printf("modified before source(%s)...",
|
|
|
|
gn->youngest->name);
|
2007-09-17 05:42:09 -07:00
|
|
|
else if (is_out_of_date(gn->mtime))
|
2007-09-16 07:36:57 -07:00
|
|
|
printf("non-existent and no sources...");
|
2007-09-17 05:42:09 -07:00
|
|
|
else
|
2007-09-16 07:36:57 -07:00
|
|
|
printf(":: operator and no sources...");
|
|
|
|
}
|
|
|
|
oodate = true;
|
|
|
|
} else {
|
|
|
|
oodate = false;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If the target isn't out-of-date, the parents need to know its
|
|
|
|
* modification time. Note that targets that appear to be out-of-date
|
|
|
|
* but aren't, because they have no commands and aren't of type OP_NOP,
|
|
|
|
* have their mtime stay below their children's mtime to keep parents
|
|
|
|
* from thinking they're out-of-date.
|
|
|
|
*/
|
|
|
|
if (!oodate)
|
|
|
|
Lst_ForEach(&gn->parents, MakeTimeStamp, gn);
|
|
|
|
|
|
|
|
return oodate;
|
2007-09-16 03:39:07 -07:00
|
|
|
}
|
|
|
|
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
void
|
|
|
|
job_attach_node(Job *job, GNode *node)
|
|
|
|
{
|
|
|
|
job->node = node;
|
2012-10-02 03:29:30 -07:00
|
|
|
job->node->built_status = BUILDING;
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
job->next_cmd = Lst_First(&node->commands);
|
|
|
|
job->exit_type = JOB_EXIT_OKAY;
|
|
|
|
job->location = NULL;
|
|
|
|
job->flags = 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
void
|
2020-01-13 06:56:59 -08:00
|
|
|
handle_job_status(Job *job, int status)
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
{
|
2012-10-06 02:32:40 -07:00
|
|
|
bool silent;
|
2012-10-18 10:54:43 -07:00
|
|
|
int dying;
|
2012-10-06 02:32:40 -07:00
|
|
|
|
|
|
|
/* if there's one job running and we don't keep going, no need
|
|
|
|
* to report right now.
|
|
|
|
*/
|
|
|
|
if ((job->flags & JOB_ERRCHECK) && !keepgoing && runningJobs == NULL)
|
|
|
|
silent = !DEBUG(JOB);
|
|
|
|
else
|
|
|
|
silent = false;
|
|
|
|
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
debug_job_printf("Process %ld (%s) exited with status %d.\n",
|
|
|
|
(long)job->pid, job->node->name, status);
|
|
|
|
|
|
|
|
/* classify status */
|
|
|
|
if (WIFEXITED(status)) {
|
|
|
|
job->code = WEXITSTATUS(status);/* exited */
|
2012-10-18 10:54:43 -07:00
|
|
|
if (job->code != 0) {
|
|
|
|
/* if we're already dying from that signal, be silent */
|
|
|
|
if (!silent && job->code > 128
|
|
|
|
&& job->code <= 128 + _NSIG) {
|
|
|
|
dying = check_dying_signal();
|
|
|
|
silent = dying && job->code == dying + 128;
|
|
|
|
}
|
2012-10-06 02:32:40 -07:00
|
|
|
if (!silent)
|
|
|
|
printf("*** Error %d", job->code);
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
job->exit_type = JOB_EXIT_BAD;
|
|
|
|
} else
|
|
|
|
job->exit_type = JOB_EXIT_OKAY;
|
|
|
|
} else {
|
|
|
|
job->exit_type = JOB_SIGNALED;
|
|
|
|
job->code = WTERMSIG(status); /* signaled */
|
2012-10-18 10:54:43 -07:00
|
|
|
/* if we're already dying from that signal, be silent */
|
|
|
|
if (!silent) {
|
|
|
|
dying = check_dying_signal();
|
|
|
|
silent = dying && job->code == dying;
|
|
|
|
}
|
2012-10-06 02:32:40 -07:00
|
|
|
if (!silent)
|
|
|
|
printf("*** Signal %d", job->code);
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
}
|
|
|
|
|
|
|
|
/* if there is a problem, what's going on ? */
|
|
|
|
if (job->exit_type != JOB_EXIT_OKAY) {
|
2012-10-06 02:32:40 -07:00
|
|
|
if (!silent)
|
|
|
|
printf(" in target '%s'", job->node->name);
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
if (job->flags & JOB_ERRCHECK) {
|
|
|
|
job->node->built_status = ERROR;
|
|
|
|
if (!keepgoing) {
|
2012-10-06 02:32:40 -07:00
|
|
|
if (!silent)
|
|
|
|
printf("\n");
|
2020-01-13 07:12:58 -08:00
|
|
|
job->flags |= JOB_KEEPERROR;
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
/* XXX don't free the command */
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
printf(", line %lu of %s", job->location->lineno,
|
|
|
|
job->location->fname);
|
2020-01-13 06:15:21 -08:00
|
|
|
/* Parallel make already determined whether
|
|
|
|
* JOB_IS_EXPENSIVE, perform the computation for
|
|
|
|
* sequential make to figure out whether to display the
|
|
|
|
* command or not. */
|
2020-01-13 06:51:50 -08:00
|
|
|
if ((job->flags & JOB_SILENT) && sequential)
|
2020-01-13 06:14:24 -08:00
|
|
|
determine_expensive_job(job);
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
if ((job->flags & (JOB_SILENT | JOB_IS_EXPENSIVE))
|
|
|
|
== JOB_SILENT)
|
|
|
|
printf(": %s", job->cmd);
|
|
|
|
/* Abort the current target,
|
|
|
|
* but let others continue. */
|
|
|
|
printf(" (continuing)\n");
|
|
|
|
} else {
|
|
|
|
/* Continue executing commands for
|
|
|
|
* this target. If we return 0,
|
|
|
|
* this will happen... */
|
|
|
|
printf(" (ignored)\n");
|
|
|
|
job->exit_type = JOB_EXIT_OKAY;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
free(job->cmd);
|
|
|
|
}
|
|
|
|
|
|
|
|
int
|
|
|
|
run_gnode(GNode *gn)
|
|
|
|
{
|
|
|
|
if (!gn || (gn->type & OP_DUMMY))
|
|
|
|
return NOSUCHNODE;
|
|
|
|
|
2020-01-13 07:24:31 -08:00
|
|
|
Job_Make(gn);
|
|
|
|
loop_handle_running_jobs();
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
return gn->built_status;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
static bool
|
2017-07-24 05:08:15 -07:00
|
|
|
do_run_command(Job *job, const char *pre)
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
{
|
|
|
|
bool silent; /* Don't print command */
|
|
|
|
bool doExecute; /* Execute the command */
|
|
|
|
bool errCheck; /* Check errors */
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
pid_t cpid; /* Child pid */
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
const char *cmd = job->cmd;
|
2018-11-27 01:33:48 -08:00
|
|
|
silent = Targ_Silent(job->node);
|
|
|
|
errCheck = !Targ_Ignore(job->node);
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
if (job->node->type & OP_MAKE)
|
|
|
|
doExecute = true;
|
|
|
|
else
|
|
|
|
doExecute = !noExecute;
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
|
|
|
|
/* How can we execute a null command ? we warn the user that the
|
|
|
|
* command expanded to nothing (is this the right thing to do?). */
|
2008-01-29 14:23:10 -08:00
|
|
|
if (*cmd == '\0') {
|
2017-07-24 05:08:15 -07:00
|
|
|
Parse_Error(PARSE_WARNING,
|
|
|
|
"'%s' expands to '' while building %s",
|
|
|
|
pre, job->node->name);
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
return false;
|
2010-04-25 06:59:53 -07:00
|
|
|
}
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
|
|
|
|
for (;; cmd++) {
|
|
|
|
if (*cmd == '@')
|
|
|
|
silent = DEBUG(LOUD) ? false : true;
|
|
|
|
else if (*cmd == '-')
|
|
|
|
errCheck = false;
|
|
|
|
else if (*cmd == '+')
|
|
|
|
doExecute = true;
|
|
|
|
else
|
|
|
|
break;
|
|
|
|
}
|
2013-11-22 07:47:35 -08:00
|
|
|
while (ISSPACE(*cmd))
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
cmd++;
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
/* Print the command before fork if make -n or !silent*/
|
|
|
|
if ( noExecute || !silent)
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
printf("%s\n", cmd);
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
|
|
|
|
if (silent)
|
|
|
|
job->flags |= JOB_SILENT;
|
|
|
|
else
|
|
|
|
job->flags &= ~JOB_SILENT;
|
|
|
|
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
/* If we're not supposed to execute any commands, this is as far as
|
|
|
|
* we go... */
|
|
|
|
if (!doExecute)
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
return false;
|
|
|
|
/* always flush for other stuff */
|
|
|
|
fflush(stdout);
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
|
2017-07-09 08:28:00 -07:00
|
|
|
/* Optimization: bypass comments entirely */
|
|
|
|
if (*cmd == '#')
|
|
|
|
return false;
|
|
|
|
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
/* Fork and execute the single command. If the fork fails, we abort. */
|
|
|
|
switch (cpid = fork()) {
|
|
|
|
case -1:
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
Punt("Could not fork");
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
/*NOTREACHED*/
|
|
|
|
case 0:
|
turns out there WAS something fishy in signal handling in the "generic"
reaper. Specifically, the sigprocmask/wait/sigsuspend dance is correct for
the main process, BUT you have to remember to reset the signal mask to
something sane for the child (this was a duh! moment, that bug is very
stupid)
Finally, bluhm@ saw the actual issue. Kudoes to him.
The change to "unify" sequential and parallel make made the bug reproducible
under some circumstances
(because in the parallel make case, many things may happen in different
successions, so you don't get the wrong signal mask that often, but the
sequential case is reproducible, and using the "streamlined" reaper meant the
fork would occur WITHIN the signal loop instead of OUTSIDE)
So:
- discover signal state early on through Sigset_init;
- introduce reset_signal_mask to get back to the initial state;
- call reset_signal_mask systematically after fork
This organisation thanks to cmd_exec. SOME cmd_exec happens before Job_Init
happens, some afterwards (variables are still lazy and both !!= and :sh will
occur AFTER parsing), so both fork() need to be protected.
okay bluhm@
thx to sthen@ and naddy@ and mpi@ for helping out.
2020-01-16 08:07:18 -08:00
|
|
|
reset_signal_mask();
|
2012-10-02 03:29:30 -07:00
|
|
|
/* put a random delay unless we're the only job running
|
|
|
|
* and there's nothing left to do.
|
|
|
|
*/
|
|
|
|
if (random_delay)
|
2020-01-13 07:15:17 -08:00
|
|
|
if (!(runningJobs == NULL && nothing_left_to_build()))
|
2013-08-26 07:15:07 -07:00
|
|
|
usleep(arc4random_uniform(random_delay));
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
run_command(cmd, errCheck);
|
|
|
|
/*NOTREACHED*/
|
|
|
|
default:
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
job->pid = cpid;
|
|
|
|
job->next = runningJobs;
|
|
|
|
runningJobs = job;
|
|
|
|
if (errCheck)
|
|
|
|
job->flags |= JOB_ERRCHECK;
|
|
|
|
else
|
|
|
|
job->flags &= ~JOB_ERRCHECK;
|
|
|
|
debug_job_printf("Running %ld (%s) %s\n", (long)job->pid,
|
|
|
|
job->node->name, (noExecute || !silent) ? "" : cmd);
|
|
|
|
return true;
|
2008-01-29 14:23:10 -08:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
bool
|
|
|
|
job_run_next(Job *job)
|
2008-01-29 14:23:10 -08:00
|
|
|
{
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
bool started;
|
|
|
|
GNode *gn = job->node;
|
|
|
|
|
|
|
|
while (job->next_cmd != NULL) {
|
|
|
|
struct command *command = Lst_Datum(job->next_cmd);
|
|
|
|
|
|
|
|
handle_all_signals();
|
|
|
|
job->location = &command->location;
|
|
|
|
Parse_SetLocation(job->location);
|
2019-12-21 07:31:54 -08:00
|
|
|
job->cmd = Var_Subst(command->string, &gn->localvars, false);
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
job->next_cmd = Lst_Adv(job->next_cmd);
|
2012-08-25 01:12:56 -07:00
|
|
|
if (fatal_errors)
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
Punt(NULL);
|
2017-07-24 05:08:15 -07:00
|
|
|
started = do_run_command(job, command->string);
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
if (started)
|
|
|
|
return false;
|
2009-05-10 04:07:37 -07:00
|
|
|
else
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
free(job->cmd);
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
}
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
job->exit_type = JOB_EXIT_OKAY;
|
|
|
|
return true;
|
Work done at p2k7.
This is a really big step towards getting parallel make to work.
Note that this is not yet complete. There are still a few `details' to
fix before this works 100%. Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).
Tested by lots of people, thanks go to miod@, and robert@ among other people.
Quick summary of what this does:
- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.
2007-11-02 10:27:24 -07:00
|
|
|
}
|
major overhaul of the way make handle jobs, inspired by dpb:
instead of forking a "job" per target, and having that job further fork
separate commands, have make maintain a list of jobs, indexed by pid
of currently running commands, and handle process termination
continuation-style. This has lots of benefits:
- make is responsible for most printing, so we no longer need pipes nor
job control: make -j jobs see the tty.
- no more special-casing for jobs that don't really execute anything.
- unify code for make -jn and make -B, including signal handlers and
job waiting. So make -n, make -q, +cmd now run commands in the same
way in all cases.
- unified more accurate error-reporting, as make knows precisely which
command failed. Commands are tagged with their lines, and we display failing
commands in silent mode.
- fine-grained "expensive" command handling (recursion limiter). Do it
per-command instead of per-target.
Moreover, signal response is now simpler, as we just block the signals
in a small critical sections, test for events, and sigpause (thanks a lot
to guenther@ and millert@), so running make is now almost always paused
without any busy-waiting.
Thanks to everyone who tested and gave input.
2012-09-21 00:55:20 -07:00
|
|
|
|