Update the crypto submodule to the top of the Mbed Crypto development
branch. This brings in a version of Mbed Crypto that enables building
Mbed Crypto tests that depend on private headers, like
'psa_crypto_invasive.h'.
This also requires updating our config.h to include new configuration
options added to Mbed Crypto. MBEDTLS_PSA_ITS_FILE_C replaces
MBEDTLS_PSA_CRYPTO_STORAGE_FILE_C and MBEDTLS_PSA_CRYPTO_STORAGE_ITS_C.
MBEDTLS_PSA_INJECT_ENTROPY replaces MBEDTLS_PSA_HAS_ITS_IO.
* public/pr/2421: (68 commits)
Fix unused variable warning in ssl_parse_certificate_coordinate()
Add missing compile time guard in ssl_client2
Update programs/ssl/query_config.c
ssl_client2: Reset peer CRT info string on reconnect
Add further debug statements on assertion failures
Fix typo in documentation of ssl_parse_certificate_chain()
Add debug output in case of assertion failure
Fix typo in SSL ticket documentation
Add config sanity check for !MBEDTLS_SSL_KEEP_PEER_CERTIFICATE
ssl_client2: Zeroize peer CRT info buffer when reconnecting
Reintroduce numerous ssl-opt.sh tests if !MBEDTLS_SSL_KEEP_PEER_CERT
ssl_client2: Extract peer CRT info from verification callback
Improve documentation of mbedtls_ssl_get_peer_cert()
Improve documentation of MBEDTLS_SSL_KEEP_PEER_CERTIFICATE
Fix indentation of Doxygen comment in ssl_internal.h
Set peer CRT length only after successful allocation
Remove question in comment about verify flags on cli vs. server
Remove misleading and redundant guard around restartable ECC field
Add test for !MBEDTLS_SSL_KEEP_PEER_CERTIFICATE to all.sh
Free peer CRT chain immediately after verifying it
...
* restricted/pr/528:
Update query_config.c
Fix failure in SSLv3 per-version suites test
Adjust DES exclude lists in test scripts
Clarify 3DES changes in ChangeLog
Fix documentation for 3DES removal
Exclude 3DES tests in test scripts
Fix wording of ChangeLog and 3DES_REMOVE docs
Reduce priority of 3DES ciphersuites
* public/pr/2028:
Update the crypto submodule to a78c958
Fix ChangeLog entry to correct release version
Fix typo in x509write test data
Add ChangeLog entry for unused bits in bitstrings
Improve docs for named bitstrings and their usage
Add tests for (named) bitstring to suite_asn1write
Add new function mbedtls_asn1_write_named_bitstring()
Add a new function mbedtls_asn1_write_named_bitstring() that removes
trailing 0s at the end of DER encoded bitstrings. The function is
implemented according to Hanno Becker's suggestions.
This commit also changes the functions x509write_crt_set_ns_cert_type
and crt_set_key_usage to call the new function as the use named
bitstrings instead of the regular bitstrings.
The ecp_get_type function comes handy in higher level modules and tests
as well. It is not inline anymore, to enable alternative implementations
to implement it for themselves.
mbedtls_ecp_read_key() module returned without an error even when
importing keys corresponding to the requested group was not
implemented.
We change this and return an error when the requested group is not
supported and make the remaining import/export functions more robust.
`MBEDTLS_SSL__ECP_RESTARTABLE` is only defined if
`MBEDTLS_KEY_EXCHANGE_ECDHE_ECDSA_ENABLED` is set, which
requires `MBEDTLS_X509_PARSE_C` to be set (this is checked
in `check_config.`). The additional `MBEDTLS_X509_PARSE_C`
guard around the `ecrs_peer_cert` field is therefore not
necessary; moreover, it's misleading, because it hasn't
been used consistently throughout the code.
When removing the (session-local) copy of the peer's CRT chain, we must
keep a handshake-local copy of the peer's public key, as (naturally) every
key exchange will make use of that public key at some point to verify that
the peer actually owns the corresponding private key (e.g., verify signatures
from ServerKeyExchange or CertificateVerify, or encrypt a PMS in a RSA-based
exchange, or extract static (EC)DH parameters).
This commit adds a PK context field `peer_pubkey` to the handshake parameter
structure `mbedtls_handshake_params_init()` and adapts the init and free
functions accordingly. It does not yet make actual use of the new field.
This commit adds an ASN.1 buffer field `pk_raw` to `mbedtls_x509_crt`
which stores the bounds of the raw public key data within an X.509 CRT.
This will be useful in subsequent commits to extract the peer's public
key from its certificate chain.
`mbedtls_ssl_parse_certificate()` parses the peer's certificate chain
directly into the `peer_cert` field of the `mbedtls_ssl_session`
structure being established. To allow to optionally remove this field
from the session structure, this commit changes this to parse the peer's
chain into a local variable instead first, which can then either be freed
after CRT verification - in case the chain should not be stored - or
mapped to the `peer_cert` if it should be kept. For now, only the latter
is implemented.
A subsequent commit will need this function in the session ticket
and session cache implementations. As the latter are server-side,
this commit also removes the MBEDTLS_SSL_CLI_C guard.
For now, the function is declared in ssl_internal.h and hence not
part of the public API.
This commit introduces a static helper function
`mbedtls_ssl_ciphersuite_uses_srv_cert()`
which determines whether a ciphersuite may make use of server-side CRTs.
This function is in turn uses in `mbedtls_ssl_parse_certificate()` to
skip certificate parsing for ciphersuites which don't involve CRTs.
Note: Ciphersuites not using server-side CRTs don't allow client-side CRTs
either, so it is safe to guard `mbedtls_ssl_{parse/write}_certificate()`
this way.
Note: Previously, the code uses a positive check over the suites
- MBEDTLS_KEY_EXCHANGE_PSK
- MBEDTLS_KEY_EXCHANGE_DHE_PSK
- MBEDTLS_KEY_EXCHANGE_ECDHE_PSK
- MBEDTLS_KEY_EXCHANGE_ECJPAKE,
while now, it uses a negative check over `mbedtls_ssl_ciphersuite_uses_srv_cert()`,
which checks for the suites
- MBEDTLS_KEY_EXCHANGE_RSA
- MBEDTLS_KEY_EXCHANGE_RSA_PSK
- MBEDTLS_KEY_EXCHANGE_DHE_RSA
- MBEDTLS_KEY_EXCHANGE_ECDH_RSA
- MBEDTLS_KEY_EXCHANGE_ECDHE_RSA
- MBEDTLS_KEY_EXCHANGE_ECDH_ECDSA
- MBEDTLS_KEY_EXCHANGE_ECDHE_ECDSA
This is equivalent since, together, those are all ciphersuites.
Quoting ssl_ciphersuites.h:
```
typedef enum {
MBEDTLS_KEY_EXCHANGE_NONE = 0,
MBEDTLS_KEY_EXCHANGE_RSA,
MBEDTLS_KEY_EXCHANGE_DHE_RSA,
MBEDTLS_KEY_EXCHANGE_ECDHE_RSA,
MBEDTLS_KEY_EXCHANGE_ECDHE_ECDSA,
MBEDTLS_KEY_EXCHANGE_PSK,
MBEDTLS_KEY_EXCHANGE_DHE_PSK,
MBEDTLS_KEY_EXCHANGE_RSA_PSK,
MBEDTLS_KEY_EXCHANGE_ECDHE_PSK,
MBEDTLS_KEY_EXCHANGE_ECDH_RSA,
MBEDTLS_KEY_EXCHANGE_ECDH_ECDSA,
MBEDTLS_KEY_EXCHANGE_ECJPAKE,
} mbedtls_key_exchange_type_t;
```
The function `mbedtls_mpi_write_binary()` writes big endian byte order,
but we need to be able to write little endian in some caseses. (For
example when handling keys corresponding to Montgomery curves.)
Used `echo xx | tac -rs ..` to transform the test data to little endian.
The private keys used in ECDH differ in the case of Weierstrass and
Montgomery curves. They have different constraints, the former is based
on big endian, the latter little endian byte order. The fundamental
approach is different too:
- Weierstrass keys have to be in the right interval, otherwise they are
rejected.
- Any byte array of the right size is a valid Montgomery key and it
needs to be masked before interpreting it as a number.
Historically it was sufficient to use mbedtls_mpi_read_binary() to read
private keys, but as a preparation to improve support for Montgomery
curves we add mbedtls_ecp_read_key() to enable uniform treatment of EC
keys.
For the masking the `mbedtls_mpi_set_bit()` function is used. This is
suboptimal but seems to provide the best trade-off at this time.
Alternatives considered:
- Making a copy of the input buffer (less efficient)
- removing the `const` constraint from the input buffer (breaks the api
and makes it less user friendly)
- applying the mask directly to the limbs (violates the api between the
modules and creates and unwanted dependency)
The function `mbedtls_mpi_read_binary()` expects big endian byte order,
but we need to be able to read from little endian in some caseses. (For
example when handling keys corresponding to Montgomery curves.)
Used `echo xx | tac -rs .. | tr [a-z] [A-Z]` to transform the test data
to little endian and `echo "ibase=16;xx" | bc` to convert to decimal.
Define MBEDTLS_ECDH_LEGACY_CONTEXT in config.h instead of hard-coding
this in ecdh.h so that its absence can be tested. Document it as
experimental so that we reserve the right to change it in the future.
This commit improves hygiene and formatting of macro definitions
throughout the library. Specifically:
- It adds brackets around parameters to avoid unintended
interpretation of arguments, e.g. due to operator precedence.
- It adds uses of the `do { ... } while( 0 )` idiom for macros that
can be used as commands.
This is the first in a series of commits adding client-side
support for PSA-based ECDHE.
Previously, the state of an ECDHE key agreement was maintained
in the field mbedtls_ssl_handshake_params::ecdh_ctx, of type
::mbedtls_ecdh_context and manipulated through the ECDH API.
The ECDH API will be superseeded by the PSA Crypto API for key
agreement, which needs the following data:
(a) A raw buffer holding the public part of the key agreement
received from our peer.
(b) A key slot holding the private part of the key agreement.
(c) The algorithm to use.
The commit adds fields to ::mbedtls_ssl_handshake_params
representing these three inputs to PSA-based key agreement.
Specifically, it adds a field for the key slot holding the
ECDH private key, a field for the EC curve identifier, and
a buffer holding the peer's public key.
Note: Storing the peer's public key buffer is slightly
inefficient, as one could perform the ECDH computation
as soon as the peer sends its public key, either working
with in-place or using a stack-buffer to reformat the
public key before passing it to PSA. This optimization
is left for a later commit.
Silence a compiler warning about implicit fallthrough by using a comment
format the compiler understand to mean that the fallthrough is
intentional.
In file included from library/cipher.c:63:0:
include/mbedtls/psa_util.h: In function ‘mbedtls_psa_translate_cipher_mode’:
include/mbedtls/psa_util.h:91:15: error: this statement may fall through [-Werror=implicit-fallthrough=]
if( taglen == 0 )
^
include/mbedtls/psa_util.h:94:9: note: here
default:
^~~~~~~
cc1: all warnings being treated as errors
$ gcc --version
gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Context:
The existing API `mbedtls_x509_parse_crt_der()` for parsing DER
encoded X.509 CRTs unconditionally makes creates a copy of the
input buffer in RAM. While this comes at the benefit of easy use,
-- specifically: allowing the user to free or re-use the input
buffer right after the call -- it creates a significant memory
overhead, as the CRT is duplicated in memory (at least temporarily).
This might not be tolerable a resource constrained device.
As a remedy, this commit adds a new X.509 API call
`mbedtls_x509_parse_crt_der_nocopy()`
which has the same signature as `mbedtls_x509_parse_crt_der()`
and almost the same semantics, with one difference: The input
buffer must persist and be unmodified for the lifetime of the
established instance of `mbedtls_x509_crt`, that is, until
`mbedtls_x509_crt_free()` is called.
Resolve incompatibilties in the RSA module where changes made for
parameter validation prevent Mbed Crypto from working. Mbed Crypto
depends on being able to pass zero-length buffers that are NULL to RSA
encryption functions.
This reverts commit 2f660d047d.
Context: There are two public key writing functions in Mbed TLS. First,
mbedtls_pk_write_pubkey(), which exports a public key in the form of a
SubjectPublicKey structure containing the raw keying material
(for example, EC point coordinates for an EC public key, without
reference to the underlying curve). Secondly, mbedtls_pk_write_pubkey_der(),
which exports a public key in the form of a SubjectPublicKeyInfo structure,
wrapping the SubjectPublicKey structure by additional information
identifying the type of public key (and for ECC, e.g., it'd also contain
the ECC group identifier). The implementation of mbedtls_pk_write_pubkey_der()
calls mbedtls_pk_write_pubkey() first and then adds the corresponding
algorithm identifier wrapper.
Both of these functions need to be provided for PSA-based opaque PK contexts,
based on PSA's public key export function.
Previously, PSA used the SubjectPublicKeyInfo structure as its export format,
so mbedtls_pk_write_pubkey_der() could be easily implemented, while
mbedtls_pk_write_pubkey() would need to trim the output of the PSA export.
The previous implementation of mbedtls_pk_write_pubkey() is not quite right
because it calls PSA export doesn't do any trimming, hence exporting the large
SubjectPublicKeyInfo structure instead of the small SubjectPublicKey.
mbedtls_pk_write_pubkey_der(), in turn, immediately returns after calling
mbedtls_pk_write_pubkey(), hence also returning the SubjectPublicKeyInfo
structure, which is correct.
By now, the PSA public key export format has changed to the smaller
SubjectPublicKey structure. This means that, now, mbedtls_pk_write_pubkey()
can be implemented by just calling the PSA export, and that
mbedtls_pk_write_pubkey_der() needs to add the algorithm information around
it, just as in the other types of PK contexts. While not correct for the
old format, the existing code for mbedtls_pk_write_pubkey() is therefore
correct for the new PSA public key format, and needs no change apart from
the missing pointer shift in the last commit.
The implementation of mbedtls_pk_write_pubkey_der() needs a special code
path for PSA-based opaque PK contexts, as the PK context only contains
the PSA key handle, and the PSA API needs to be used to extract the
underlying EC curve to be able to write the AlgorithmParameter structure
that's part of the SubjectPublicKeyInfo structure.
That's what this commit does, (hopefully) making both
mbedtls_pk_write_pubkey() and mbedtls_pk_write_pubkey_der() export
the correctly formatted public key based on the new PSA public key format.
Additional changes to temporarily enable running tests:
ssl_srv.c and test_suite_ecdh use mbedtls_ecp_group_load instead of
mbedtls_ecdh_setup
test_suite_ctr_drbg uses mbedtls_ctr_drbg_update instead of
mbedtls_ctr_drbg_update_ret
The file oid.c had conditional inclusion of functions based on a config.h
define that belongs to X.509, which is backwards. For now, just include those
functions unconditionally and rely on the linker to garbage-collect them if
not used.
In the longer term X.509-specific functions are likely to be removed from
libmbedcrypto, but at this step the goal is to preserve the API (and even ABI)
of libmbedcrypto for as long as possible while separating the source trees of
Mbed Crypto and Mbed TLS.
As agreed during the workshop, temporarily move definitions to oid.h even if
they might not semantically belong here, as a short-term measure allowing to
build libmbecrypto on its own (without X.509 files present in the source tree)
but still provide all the things Mbed TLS currently expects, and more
specifically preserve the API and ABI exposed by libmbedtls.
As there are some definitions that are defined regardless of
whether MBEDTLS_ECP_RESTARTABLE is defined or not, these definitions
need to be moved outside the MBEDTLS_ECP_ALT guards. This is a simple
move as MBEDTLS_ECP_ALT and MBEDTLS_ECP_RESTARTABLE are mutually
exclusive options.
Document when a context must be initialized or not, when it must be
set up or not, and whether it needs a private key or a public key will
do.
The implementation is sometimes more liberal than the documentation,
accepting a non-set-up context as a context that can't perform the
requested information. This preserves backward compatibility.
For mbedtls_pk_parse_key and mbedtls_pk_parse_keyfile, the password is
optional. Clarify what this means: NULL is ok and means no password.
Validate parameters and test accordingly.
A 0-length buffer for the key is a legitimate edge case. Ensure that
it works, even with buf=NULL. Document the key and keylen parameters.
There are already test cases for parsing an empty buffer. A subsequent
commit will add tests for writing to an empty buffer.
The previous introduction of constant deprecation macros
in platform_util.h lead to failure of tests/scrips/check-names.sh
because the regular expressions in the latter choked on the brackets
in the part `__attribute__((deprecated))` of the definition of the
helper type `mbedtls_deprecated_{numeric|string}_constant_t`.
Postponing any further study and potential robustness improvements
in check-names.sh to another time, this commit circumvents this
problem by temporarily abbreviating `__attribute__((deprecated))`
as `MBEDTLS_DEPRECATED`, which doesn't lead to problems with
check-names.sh.
This commit introduces macros
* MBEDTLS_DEPRECATED_STRING_CONSTANT
* MBEDTLS_DEPRECATED_NUMERIC_CONSTANT
to platform_util.h which can be used to deprecate public macro constants.
Their definition is essentially taken from dhm.h where the
MBEDTLS_DEPRECATED_STRING_CONSTANT was used to deprecate
insecure hardcoded DHM primes.
Add inclusion to configration file in header files,
instead of relying on other header files to include
the configuration file. This issue resolves#1371
- Be specific about the constraints: be a readable/writable buffer of length
X, be an initialized context, be a context initialized and bound to a key...
- Always use full sentences with all the required pronouns.
It's better if the macro receives the condition as an expression rather than a
string - that way it can choose to use it as is or stringify it. Also, the
documentation states that the parameter is an expression, not a string.
The test framework for validation of parameters depends on the macro
MBEDTLS_PARAM_FAILED() being set to its default value when building the
library. So far the test framework attempted to define this macro but this was
the wrong place - this definition wouldn't be picked by the library.
Instead, a different approach is taken: skip those tests when the macro is
defined in config.h, as in that case we have no way to know if it will indeed
end up calling mbedtls_param_failed() as we need it to.
This commit was tested by manually ensuring that aes_invalid_params:
- passes (and is not skipped) in the default configuration
- is skipped when MBEDTLS_PARAM_FAILED() is defined in config.h
The previous prototype gave warnings are the strings produced by #cond and
__FILE__ are const, so we shouldn't implicitly cast them to non-const.
While at it modifying most example programs:
- include the header that has the function declaration, so that the definition
can be checked to match by the compiler
- fix whitespace
- make it work even if PLATFORM_C is not defined:
- CHECK_PARAMS is not documented as depending on PLATFORM_C and there is
no reason why it should
- so, remove the corresponding #if defined in each program...
- and add missing #defines for mbedtls_exit when needed
The result has been tested (make all test with -Werror) with the following
configurations:
- full with CHECK_PARAMS with PLATFORM_C
- full with CHECK_PARAMS without PLATFORM_C
- full without CHECK_PARAMS without PLATFORM_C
- full without CHECK_PARAMS with PLATFORM_C
Additionally, it has been manually tested that adding
mbedtls_aes_init( NULL );
near the normal call to mbedtls_aes_init() in programs/aes/aescrypt2.c has the
expected effect when running the program.
It was inconsistent between files: sometimes 3 arguments, sometimes one.
Align to 1 argument for the macro and 3 for the function, because:
- we don't need 3 arguments for the macro, it can add __FILE__ and __LINE__
in its expansion, while the function needs them as parameters to be correct;
- people who re-defined the macro should have flexibility, and 3 arguments
can give the impression they they don't have as much as they actually do;
- the design document has the macro with 1 argument, so let's stick to that.
The function called through the macro MBEDTLS_PARAM_FAILED() must be supplied by
users and makes no sense as a library function, apart from debug and test.
The SSL module accesses ECDH context members directly to print debug
information. This can't work with the new context, where we can't make
assumptions about the implementation of the context. This commit adds
new debug functions to complete the encapsulation of the ECDH context
and work around the problem.
We want to support alternative software implementations and we extend
the ECDH context to enable this. The actual functional change that makes
use of the new context is out of scope for this commit.
Changing the context breaks the API and therefore it has to be
excluded from the default configuration by a compile time flag.
We add the compile time flag to the module header instead of
`config.h`, because this is not a standalone feature, it only
enables adding new implementations in the future.
The new context features a union of the individual implementations
and a selector that chooses the implementation in use. An alternative
is to use an opaque context and function pointers, like for example the
PK module does it, but it is more dangerous, error prone and tedious to
implement.
We leave the group ID and the point format at the top level of the
structure, because they are very simple and adding an abstraction
layer around them away does not come with any obvious benefit.
Other alternatives considered:
- Using the module level replacement mechanism in the ECP module. This
would have made the use of the replacement feature more difficult and
the benefit limited.
- Replacing our Montgomery implementations with a new one directly. This
would have prevented using Montgomery curves across implementations.
(For example use implementation A for Curve448 and implementation B for
Curve22519.) Also it would have been inflexible and limited to
Montgomery curves.
- Encoding the implementation selector and the alternative context in
`mbedtls_ecp_point` somehow and rewriting `mbedtls_ecp_mul()` to
dispatch between implementations. This would have been a dangerous and
ugly hack, and very likely to break legacy applications.
- Same as above just with hardcoding the selector and using a compile
time option to make the selection. Rejected for the same reasons as
above.
- Using the PK module to provide to provide an entry point for
alternative implementations. Like most of the above options this
wouldn't have come with a new compile time option, but conceptually
would have been very out of place and would have meant much more work to
complete the abstraction around the context.
In retrospect:
- We could have used the group ID as the selector, but this would have
made the code less flexible and only marginally simpler. On the other
hand it would have allowed to get rid of the compile time option if a
tight integration of the alternative is possible. (It does not seem
possible at this point.)
- We could have used the same approach we do in this commit to the
`mbedtls_ecp_point` structure. Completing the abstraction around this
structure would have been a much bigger and much riskier code change
with increase in memory footprint, potential decrease in performance
and no immediate benefit.
In the future we want to support alternative ECDH implementations. We
can't make assumptions about the structure of the context they might
use, and therefore shouldn't access the members of
`mbedtls_ecdh_context`.
Currently the lifecycle of the context can't be done without direct
manipulation. This commit adds `mbedtls_ecdh_setup()` to complete
covering the context lifecycle with functions.
`mbedtls_ecp_tls_read_group()` both parses the group ID and loads the
group into the structure provided. We want to support alternative
implementations of ECDH in the future and for that we need to parse the
group ID without populating an `mbedtls_ecp_group` structure (because
alternative implementations might not use that).
This commit moves the part that parses the group ID to a new function.
There is no need to test the new function directly, because the tests
for `mbedtls_ecp_tls_read_group()` are already implicitly testing it.
There is no intended change in behaviour in this commit.
