In the RSYNC protocol a file's last modification time and its size are
used to determine whether sending a (partial) copy over the wire is needed.
Previously, when RRDP data structures are serialized to disk, the mtime of
files in DIR_VALID ended up being UTIME_NOW.
Thus, the mtimes of files obtained through RRDP will never match the mtimes
of the same files available through RSYNC - causing each and every file to
be added to the file transfer list.
Instead, use the internal timestamps of RPKI files as the last modified
timestamp. Specifically, for Signed Objects (ROAs, MFTs, GBRs, TAKs, ASPAs)
the CMS signing-time, for .cer files the X.509 notBefore, and for .crl files
the CRL lastUpdate. This results in a surprising optimization for the number
files which have to be transfered.
OK claudio@
Make rpki-client choose the verification time of the time it is invoked
rather than always getting the current system time for every certificate
verification. This will result in output that is not variable on run-time.
Using the time of invocation does not work well with fast publishing CAs. It
can take a few minutes to reach a repo and that CA may have issued certificates
that are not yet valid if that startup time of rpki-client is used to validate.
This still keeps the -P option to specify a fixed validation time.
OK beck@ job@ tb@
The generally rather poor quality RFC 3779 code in libcrypto also performs
abysmally. Flame graphs show that nearly 20% of the parser process is spent
in addr_contains() alone. There is room for improvement in addr_contains()
itself - the containment check for prefixes could be optimized quite a bit.
We can avoid a lot of the most expensive work for certificates with tons of
resources close to the TA by using the verifier's partial chains flag.
More precisely, in the tree of already validated certs look for the first
one that has no inherited RFC 3779 resources and use that as 'trust anchor'
for our chains via the X509_V_FLAG_PARTIAL_CHAIN flag. This way we can be
sure that a leaf's delegated resources are properly covered and at the same
time significantly shorten most paths validated.
Job's and my testing indicates that this avoids 30-50% of overhead and works
equally well with LibreSSL and OpenSSL >= 1.1. The main bottlenecks in the
parser process now appear to be SHA-2 and RSA/BIGNUM, two well-known pain
points in libcrypto.
This is based on a hint by beck and was discussed extensively with beck,
claudio and job during and after m2k23.
ok claudio job
rather than always getting the current system time for every certificate
verification. This will result in output that is not variable on run-time.
ok tb@ claudio@
This fixes some wrong accounting for repositories that are referenced
from more than one TAL. It changes the ometric lable output a little bit
since there are repository metrics that no longer include the 'name' label.
OK tb@
Unify common code paths which find the exact expiry moment into a new
helper function. Additionally, the new helper offers more accuracy by
checking more applicable CRLs whether their 'nextupdate' is 'sooner'.
tb@ noted: The helper adds a multiplier of log(#crls), but that's
certainly acceptable as it is still very cheap.
OK tb@
Change the way the validated ASPA tree is built since OpenBGPD config
follows more the ASPA profile and puts the optional AFI to each provider
ASnum instead of duplicated everything into an IPv4 and IPv6 tree.
The JSON output of ASPA is still the same.
The inclusion of the aspa-set can currently be disabled by the -A flag.
OK tb@
RFC 8182 requires the session_id to be a version 4 random UUID (using
variant 1). Now checking the version and variant is currently disabled
because there is at least one CA with a session_id that is all random
and therefor the version check triggers there.
Joint work with job@. OK job@, tb@
Many statistic values are now accounted by repository via repo_stat_inc()
At end of the run sum_stats() accumulates these stats per TAL and globally.
The new output file metrics is written when the -m output flag is specified.
The metrics file is written in OpenMetrics format (with a few tweaks to
allow node_exporter to parse the file as well). The ometric code is a copy
from bgpctl(8) and should be kept in sync.
OK tb@
RFC 9092 describes a scheme in which an authenticator is appended to a
geofeed (RFC 8805) file. It is a digest of the main body of the file
signed by the private key of the relevant RPKI certificate for a covering
address range. The authenticator is a detached CMS signature.
with and OK tb@
Unfortunately we can't yet error out when accessMethods other than
signedObject are encountered in the SubjectInformationAccess extension
because there is pollution in the ecosystem.
OK tb@
Add support validation of Signed Objects containing Trust Anchor Keys
(TAKs - aka 'Signed TALs'). Signed TALs provide a mechanism for RIRs
to distribute and sign the next Trust Anchor with the current Trust
Anchor. This might be an improvement over visiting RIR websites and
copy+pasting TAL data by hand.
OK tb@
Unify conformance checking of Trust Anchors, ROAs, ASPAs, RSCs - none of which
may have any 'inherit' elements in the RFC 3779 IP/AS Resources extension of
the X509 certificate.
OK tb@
The abort is done in a way that waits for any inflight files or http
requests to finish before removing the rrdp state and before sending
the rrdp done message indicating failure.
OK tb@ and benno@
ASPA objects are published in the RPKI and can be used to detect and
mitigate BGP route leaks. Validated ASPA Payloads are visible through
filemode (-f) and the JSON output format (-j).
With feedback from tb@
OK claudio@ tb@
The resources delegated in the RFC 3779 extensions of the EE cert for
ROAs or RSCs can be a subset of the resources in the auth chain. So far
we compared that the resources of ROAs and RSCs are covered by the auth
chain, which is not entirely correct. Extract the necessary data from
the EE cert into rpki-client's own data structures, then verify that
the EE cert's resources cover the ones claimed in the ROA or RSC.
Do this as part or ROA and RSC parsing, that the EE cert's resources are
covered by the auth chain is checked in valid_x509() later on.
All this is a bit more annoying and intrusive than it should be...
ok claudio job
Blocking outbound connections towards RPKI publication servers based
on IP or IPv6 address in external instrumentation like HTTP proxies
or pf(4) rules is somewhat unwieldy. It might be easier for operators
if we offer a mechanism that cuts at the CA cert SIA parsing step.
OK claudio@ tb@
Since the ASN.1 template conversions, we have three copies of mostly dead
code that validates that the econtent version is at its default value 0.
Until a new standard bumps this version and we decide to support that,
we're better off with only one copy of this code.
ok claudio
Change signatures of various functions to avoid using struct parse and
expose sbgp_as_{id,range}() and sbgp_addr{,_range}() so they can be used
from rsc.c. This is a mostly mechanical diff.
ok claudio job
Factor out POSIX portable filename check into a new valid_filename() and
rename the previous valid_filename() to valid_mft_filename(). Fixes and
supersedes imcomplete checks in the RSC code. Avoids truncation via
strndup() in case of embedded NULs.
input/ok claudio
This implements decoding support for draft-ietf-sidrops-rpki-rsc-06
There are three major outstanding issues:
* The wire image might still change to conform to the more widely deployed
3779 API in libressl/openssl. IETF discussion ongoing.
* Whether the resources listed in the ResourceBlock are contained within
the EE's RFC 3779 extension is not hooked up yet.
* There is a fair bit of duplicity between rsc.c and cert.c, look for XXX
OK tb@
proc_parser_cert_validate() and proc_parser_root_cert() adjust
parse_load_certchain() and parse_load_ta() respectivly.
Also cleanup the functions in parser.c and make it possible to call
ta_parse and cert_parse with a NULL cert.
OK tb@
makes sense to totally split it out. Duplicate proc_parser_cert_validate()
and proc_parser_root_cert() for now.
The valid_x509() plus the required static functions are moved to validate.c.
The crl_tree code moved into crl.c similar to the auth_tree handling in
cert.c. All the proc functions are now tagged with __attribute(noreturn)
which allows to remove the errx() after them.
OK tb@
Verify the CRL referenced from the mft against the mft's fileAndHash info.
If the CRL matches then load it and use it to validate this mft. If the
mft validated OK add the now also valid CRL to the auth store for later use.
Before the newest CRL was always selected but that has negative consequences
because it is common practice to revoke the previous MFT's EE cert and with
that the cache is turned useless as soon as a new CRL is used. Also there
was a possibility that the CRL used for validation of the MFT was not the
one later used.
Both RFC6486 and draft-ietf-sidrops-6486bis are unclear about this part
of the validation process. We opted in favor of the chached MFT.
With and OK tb@
AIA and CRL URIs can reuse the code in sbgp_sia_location and thus get the
same checks as the SIA locations. This eliminates some more duplication
and makes the checks more stringent in that embedded NULs and "./" are no
longer allowed.
ok claudio
method to sync the cache. It just depends on all deltas to work but has
no method to check if the result is consistent with the source.
Because of this do not unlink files which were fetched via RRDP. Instead
move them back to their .rrdp directory and hope the will be cleaned up.
This should help to keep the cache coherent in some edge cases.
OK tb@
the parse and some checks and cert_parse() or ta_parse() to do the
additional checks for regular certs or TAs.
With this adjust the cert parser in -f mode to identify TAs (by checking
if it is self signed) and adjust the validation in that case. Now -f
should be able to parse and show all object correctly.
With and OK tb@
to work more like x509_get_expire(). They will return an error if the
extension extraction failed but not if it was not present. The callers
must now do that check but most did already.
With this cert_parse_inner() no longer cares about TA vs non-TA certs.
Feedback and OK tb@
Create a common x509_convert_seqnum() function to convert the various
ASN1_INTEGERs into hexnumbers and use this for mft and crl handling.
With and OK tb@, also OK job@
For this introduce x509_get_time() that converts a ASN1_TIME to time_t.
Also move time2str() to print.c where it makes more sense.
This needs more work but that will happen in tree.
OK tb@
There is a chicken-egg here since manifests reference the CRL themselves.
We may also have two CRLs available, in which case we check against the
one with the newer thisUpdate time.
The RFC situation is a bit of a mess with abundant complexity, unclear
recommendations and requirements and draft specs that also need to be
considered. This is a first version that works with future improvements
to be landed later.
Joint work with claudio, prompted by a question by job
ok claudio job
parsing the CRL, so it makes more sense to have it there. It will also
make an upcoming change easier. While there, rename free_crl() into
crl_free() for consistency with all other *_free() functions.
input/ok claudio
selecting the newest file this opens both the new (from rrdp or rsync)
and old (valid) MFT. It then compares the manifest number and based on
that the 'newer' MFT is selected.
The MFT file and hash check is also changed to always try both locations
and selecting whatever matches up with the hash. The selction is passed
back to the the main process and used later on to open exactly the same
file as was checked against the hash.
The MFT parsing code has been split up into multiple steps so that the
files can be parsed, compared and then fully validated.
In most cases this makes no difference but it prevents replay attacks
using old but still valid files.
With and OK tb@
The file still needs to match its hash to make the MFT valid but then
there will only be a warning printed. Parsing of other files from that
MFT are not influenced.
OK tb@
poll loop. In the main process move the timeout handling for repositories
into a single function that does the timeouts and the calculation of the
timeout in one go.
OK tb@
of looking it up again. For this valid_roa() needs to be moved up in
proc_parser_roa() also move out the assignment of the TAL id. Not the
right thing to alter an object in a validation function.
OK tb@
rpki-client uses the same idiom to determine the file type in too many
places. Use one function that determines the appropriate RTYPE from the
file name. Add that type to struct mftfile and use this new member to
simplify queue_add_from_mft*().
input/ok claudio
printed in human readable form and is also verified against the valid
cache of rpki-client.
To validate the file the chain is explored backwards by looking at the
Authority Information Access URI and X509v3 CRL Distribution Point of
the cert. Once the trust anchor is found this chain can be verified.
Feedback and OK job@ tb@
successfully been verified by rpki-client.
With this the rsync and rrdp directories are more of a temporary storage
location. New files are downloaded there and then moved to the valid
directory at the end. In -n mode only the valid directory is looked at with
the exception of the ta directory holding the trust anchors.
A file can now be in two different locations so adjust all the code paths
that open files to check both locations.
One nice side-effect of this is that the RRDP handling in the main process
got simplified. There is no longer the need for temporary RRDP directories.
OK tb@
the rrdp directory. This is used before a snapshot download to ensure
that the snapshot is applied to a clean repo.
Similar cleanup happens if the transfer fails. In that case remove the
temp directory contents only.
This uses a new function remove_contents() to remove everything below
a base directory (a bit like rm -r X/*).
OK tb@
based on information from the repository, a local path and the filename.
This simplifies some code both in the main process and the parser.
For this to work repositories are passed to the parser before any other
entity of this repository is passed. Struct entity is extended to include
the repoid and the path along the file(name).
Input and OK tb@ & job@
Introduce an additional timeout for each publication point.
The limits are large enough to accomodate normal operating levels.
With and OK benno@ job@ tb@ beck@ deraadt@
Since they are stored in an array just blast the full array in and out
of the io buffers at once instead of iterating element by element.
It also allows to remove a lot of extra code.
OK benno@ job@
