Bring Mbed TLS 2.18.0 and 2.18.1 release changes back into the
development branch. We had branched to release 2.18.0 and 2.18.1 in
order to allow those releases to go out without having to block work on
the `development` branch.
Manually resolve conflicts in the Changelog by moving all freshly addded
changes to a new, unreleased version entry.
Reject changes to include/mbedtls/platform.h made in the mbedtls-2.18
branch, as that file is now sourced from Mbed Crypto.
* mbedtls-2.18:
platform: Include stdarg.h where needed
Update Mbed Crypto to contain mbed-crypto#152
CMake: Add a subdirectory build regression test
README: Enable builds as a CMake subproject
ChangeLog: Enable builds as a CMake subproject
Remove use of CMAKE_SOURCE_DIR
Update library version to 2.18.0
This commit introduces a new SSL error code
`MBEDTLS_ERR_SSL_VERSION_MISMATCH`
which can be used to indicate operation failure due to a
mismatch of version or configuration.
It is put to use in the implementation of `mbedtls_ssl_session_load()`
to signal the attempt to de-serialize a session which has been serialized
in a build of Mbed TLS using a different version or configuration.
This commit improves the test exercising the behaviour of
session deserialization when facing an unexpected version
or config, by testing ver/cfg corruption at any bit in the
ver/cfg header of the serialized data; previously, it had
only tested the first bit of each byte.
The chosen fix matches what's currently done in the baremetal branch - except
the `#ifdef` have been adapted because now in baremetal the digest is not kept
if renegotiation is disabled.
We have explicit recommendations to use US spelling for technical writing, so
let's apply this to code as well for uniformity. (My fingers tend to prefer UK
spelling, so this needs to be fixed in many places.)
sed -i 's/\([Ss]eriali\)s/\1z/g' **/*.[ch] **/*.function **/*.data ChangeLog
This test works regardless of the serialisation format and embedded pointers
in it, contrary to the load-save test, though it requires more maintenance of
the test code (sync the member list with the struct definition).
This uncovered a bug that led to a double-free (in practice, in general could
be free() on any invalid value): initially the session structure is loaded
with `memcpy()` which copies the previous values of pointers peer_cert and
ticket to heap-allocated buffers (or any other value if the input is
attacker-controlled). Now if we exit before we got a chance to replace those
invalid values with valid ones (for example because the input buffer is too
small, or because the second malloc() failed), then the next call to
session_free() is going to call free() on invalid pointers.
This bug is fixed in this commit by always setting the pointers to NULL right
after they've been read from the serialised state, so that the invalid values
can never be used.
(An alternative would be to NULL-ify them when writing, which was rejected
mostly because we need to do it when reading anyway (as the consequences of
free(invalid) are too severe to take any risk), so doing it when writing as
well is redundant and a waste of code size.)
Also, while thinking about what happens in case of errors, it became apparent
to me that it was bad practice to leave the session structure in an
half-initialised state and rely on the caller to call session_free(), so this
commit also ensures we always clear the structure when loading failed.
This test appeared to be passing for the wrong reason, it's actually not
appropriate for the current implementation. The serialised data contains
values of pointers to heap-allocated buffers. There is no reason these should
be identical after a load-save pair. They just happened to be identical when I
first ran the test due to the place of session_free() in the test code and the
fact that the libc's malloc() reused the same buffers. The test no longer
passes if other malloc() implementations are used (for example, when compiling
with asan which avoids re-using the buffer, probably for better error
detection).
So, disable this test for now (we can re-enable it when we changed how
sessions are serialised, which will be done in a future PR, hence the name of
the dummy macro in depends_on). In the next commit we're going to add a test
that save-load is the identity instead - which will be more work in testing as
it will require checking each field manually, but at least is reliable.
This initial test ensures that a load-save function is the identity. It is so
far incomplete in that it only tests sessions without tickets or certificate.
This will be improved in the next commits.
* crypto/pr/212: (337 commits)
Make TODO comments consistent
Fix PSA tests
Fix psa_generate_random for >1024 bytes
Add tests to generate more random than MBEDTLS_CTR_DRBG_MAX_REQUEST
Fix double free in psa_generate_key when psa_generate_random fails
Fix copypasta in test data
Avoid a lowercase letter in a macro name
Correct some comments
Fix PSA init/deinit in mbedtls_xxx tests when using PSA
Make psa_calculate_key_bits return psa_key_bits_t
Adjust secure element code to the new ITS interface
More refactoring: consolidate attribute validation
Fix policy validity check on key creation.
Add test function for import with a bad policy
Test key creation with an invalid type (0 and nonzero)
Remove "allocated" flag from key slots
Take advantage of psa_core_key_attributes_t internally #2
Store the key size in the slot in memory
Take advantage of psa_core_key_attributes_t internally: key loading
Switch storage functions over to psa_core_key_attributes_t
...
Drivers that allow destroying a key must have a destroy method. This
test bug was previously not caught because of an implementation bug
that lost the error triggered by the missing destroy method.
Add a flow where the key is imported or fake-generated in the secure
element, then call psa_export_public_key and do the software
verification with the public key.
Factor common code of ram_import and ram_fake_generate into a common
auxiliary function.
Reject key types that aren't supported by this test code.
Report the bit size correctly for EC key pairs.
The methods to import and generate a key in a secure element drivers
were written for an earlier version of the application-side interface.
Now that there is a psa_key_attributes_t structure that combines all
key metadata including its lifetime (location), type, size, policy and
extra type-specific data (domain parameters), pass that to drivers
instead of separate arguments for each piece of metadata. This makes
the interface less cluttered.
Update parameter names and descriptions to follow general conventions.
Document the public-key output on key generation more precisely.
Explain that it is optional in a driver, and when a driver would
implement it. Declare that it is optional in the core, too (which
means that a crypto core might not support drivers for secure elements
that do need this feature).
Update the implementation and the tests accordingly.
Register an existing key in a secure element.
Minimal implementation that doesn't call any driver method and just
lets the application declare whatever it wants.
Pass the key creation method (import/generate/derive/copy) to the
driver methods to allocate or validate a slot number. This allows
drivers to enforce policies such as "this key slot can only be used
for keys generated inside the secure element".
Allow the application to choose the slot number in a secure element,
rather than always letting the driver choose.
With this commit, any application may request any slot. In an
implementation with isolation, it's up to the service to filter key
creation requests and apply policies to limit which applications can
request which slot.
Test the behavior of the getter/setter functions.
Test that psa_get_key_slot_number() reports a slot number for a key in
a secure element, and doesn't report a slot number for a key that is
not in a secure element.
Test that psa_get_key_slot_number() reports the correct slot number
for a key in a secure element.
Add tests that call psa_generate_random() (possibly via
psa_generate_key()) with a size that's larger than
MBEDTLS_CTR_DRBG_MAX_REQUEST. This causes psa_generate_random() to
fail because it calls mbedtls_ctr_drbg_random() without taking the
maximum request size of CTR_DRBG into account.
Non-regression test for #206
Conflicts and changes:
* Files that do not exist in Mbed Crypto and have changed in Mbed TLS:
These files should not exist in Mbed Crypto. Keep them deleted.
* tests/data_files/test-ca.server1.db: new file in Mbed TLS, don't create
it in Mbed Crypto.
* tests/data_files/rsa_pkcs1_1024_clear.pem: do create this file in
Mbed Crypto. I don't see why it would be kept out.
* tests/data_files/Makefile: don't take any of the changes in sections
that have been removed in Crypto. Do take in the certificate
expiration date updates and the extra .crt.der rules (even if Crypto
doesn't actually use those certificates: removing them would be out
of scope of the present merge).
* tests/suites/helpers.function: consecutive additions, take
both (order indifferent).
In tests of mbedtls_cipher_xxx and mbedtls_pk_xxx with
MBEDTLS_USE_PSA_CRYPTO enabled, initialize and deinitialize the PSA
subsystem in every function. Before, the tests were only passing
because the first function to be called happened to call
psa_crypto_init() but not mbedtls_psa_crypto_free(). In some
configurations (not tested on CI), psa_crypto_init() was not called so
the tests using PSA failed.
Call PSA_DONE() at the end of each test function. This ensures that no
resources are leaked in the form of PSA crypto slot contents.
Incidentally, this also fixes a build error due to
test_helper_psa_done() being unused in test_suite_pk: the fact that it
wasn't used betrayed the missing calls to PSA_DONE().
Conflict resolution:
* `scripts/config.pl`:
Take the exclusion of `MBEDTLS_PSA_CRYPTO_SE_C` from the API branch.
Take the removal of `MBEDTLS_PSA_CRYPTO_STORAGE_ITS_C` (obsolete) from
the development branch.
* `tests/scripts/all.sh`:
Multiple instances of factoring a sequence of `config.pl` calls into
a mere `config.pl baremetal` in the development branch, and a change in
the composition of `baremetal` in the API branch. In each case, take the
version from development.
* `tests/suites/test_suite_psa_crypto_slot_management.function`:
A function became non-static in development and disappeared in the API
branch. Keep the version from the API branch. Functions need to be
non-static if they're defined but unused in some configurations,
which is not the case for any function in this file at the moment.
* `tests/suites/test_suite_psa_crypto.function`:
Consecutive changes in the two branches, reconciled.
Resolve conflicts by performing the following operations:
- Reject changes related to building a crypto submodule, since Mbed
Crypto is the crypto submodule.
- Reject X.509, NET, and SSL changes.
- Reject changes to README, as Mbed Crypto is a different project from
Mbed TLS, with a different README.
- Avoid adding mention of ssl-opt.sh in a comment near some modified
code in include/CMakeLists.txt (around where ENABLE_TESTING as added).
- Align config.pl in Mbed TLS with config.pl in Mbed Crypto where PSA
options are concerned, to make future merging easier. There is no
reason for the two to be different in this regard, now that Mbed TLS
always depends on Mbed Crypto. Remaining differences are only the
PSA_CRYPTO_KEY_FILE_ID_ENCODES_OWNER option and the absence of X.509,
NET, and SSL related options in Mbed Crypto's config.pl.
- Align config.h in Mbed Crypto with Mbed TLS's copy, with a few notable
exceptions:
- Leave CMAC on by default.
- Leave storage on by default (including ITS emulation).
- Avoid documenting the PSA Crypto API as is in beta stage in
documentation for MBEDTLS_PSA_CRYPTO_C.
The only remaining differences are a lack of X.509, NET, and SSL
options in Mbed Crypto's config.h, as well as an additional
Mbed-Crypto-specific PSA_CRYPTO_KEY_FILE_ID_ENCODES_OWNER option.
Documentation for the check params feature and related macros is also
updated to match Mbed TLS's description.
- Reject tests/data_files/Makefile changes to generate DER versions of
CRTs and keys, as none of those are used by Mbed Crypto tests.
- Add the "no PEM and no filesystem" test to all.sh, without ssl-opt.sh
run, as Mbed Crypto doesn't have ssl-opt.sh. Also remove use of PSA
Crypto storage and ITS emulation, since those depend on filesystem
support.
- Reject addition of test when no ciphersuites have MAC to all.sh, as
the option being tested, MBEDTLS_SSL_SOME_MODES_USE_MAC, is not
present in Mbed Crypto.
- Use baremetal config in all.sh, as Mbed Crypto's baremetal
configuration does exclude the net module (as it doesn't exist in Mbed
Crypto)
- Reject cmake_subproject_build changes, continuing to link only
libmbedcrypto.
- Reject changes to visualc and associated templates. Mbed Crypto
doesn't need additional logic to handle submodule-sourced headers.
- Avoid adding fuzzers from Mbed TLS. The only relevant fuzzers are the
privkey and pubkey fuzzers, but non-trivial work would be required to
integrate those into Mbed Crypto (more than is comfortable in a merge
commit).
- Reject addition of Docker wrappers for compat.sh and ssl-opt.sh, as
those are not present in Mbed Crypto.
- Remove calls to SSL-related scripts from basic-in-docker.sh
Fix test errors by performing the following:
- Avoid using a link that Doxygen can't seem to resolve in Mbed Crypto,
but can resolve in Mbed TLS. In documentation for
MBEDTLS_CHECK_PARAMS, don't attempt to link to MBEDTLS_PARAM_FAILED.
* origin/development: (339 commits)
Do not build fuzz on windows
No booleans and import config
Removing space before opening parenthesis
Style corrections
Syntax fix
Fixes warnings from MSVC
Add a linker flag to enable gcov in basic-build-test.sh
Update crypto submodule to a revision with the HAVEGE header changes
Test with MBEDTLS_ECP_RESTARTABLE
Allow TODO in code
Use the docstring in the command line help
Split _abi_compliance_command into smaller functions
Record the commits that were compared
Document how to build the typical argument for -s
Allow running /somewhere/else/path/to/abi_check.py
tests: Limit each log to 10 GiB
Warn if VLAs are used
Remove redundant compiler flag
Consistently spell -Wextra
Fix parsing issue when int parameter is in base 16
...
65528 bits is more than any reasonable key until we start supporting
post-quantum cryptography.
This limit is chosen to allow bit-sizes to be stored in 16 bits, with
65535 left to indicate an invalid value. It's a whole number of bytes,
which facilitates some calculations, in particular allowing a key of
exactly PSA_CRYPTO_MAX_STORAGE_SIZE to be created but not one bit
more.
As a resource usage limit, this is arguably too large, but that's out
of scope of the current commit.
Test that key import, generation and derivation reject overly large
sizes.
Move the "core attributes" to a substructure of psa_key_attribute_t.
The motivation is to be able to use the new structure
psa_core_key_attributes_t internally.
Add a few test cases to ensure that alg=0 in policy does not allow
using the key for an operation.
Add a test case to ensure that ANY_HASH does not have a wildcard
meaning for HMAC.
Add a parameter to the key import method of a secure element driver to
make it report the key size in bits. This is necessary (otherwise the
core has no idea what the bit-size is), and making import report it is
easier than adding a separate method (for other key creation methods,
this information is an input, not an output).
Run all functions that take a key handle as input with a key that is
in a secure element. All calls are expected to error out one way or
another (not permitted by policy, invalid key type, method not
implemented in the secure element, ...). The goal of this test is to
ensure that nothing bad happens (e.g. invalid pointer dereference).
Run with various key types and algorithms to get good coverage.
The following provides more information on this PR:
- PSA stands for Platform Security Architecture.
- Add support for use of psa_trusted_storage_api internal_trusted_storage.h v1.0.0
as the interface to the psa_trusted_storage_linux backend (i.e. for persistent
storage when MBEDTLS_PSA_ITS_FILE_C is not defined). This requires changes
to psa_crypto_its.h and psa_crypto_storage.c to migrate to the new API.
Stored keys must contain lifetime information. The lifetime used to be
implied by the location of the key, back when applications supplied
the lifetime value when opening the key. Now that all keys' metadata
are stored in a central location, this location needs to store the
lifetime explicitly.
Pass information via a key attribute structure rather than as separate
parameters to psa_crypto_storage functions. This makes it easier to
maintain the code when the metadata of a key evolves.
This has negligible impact on code size (+4B with "gcc -Os" on x86_64).
This test case was only executed if the SHA-512 module was enabled and
MBEDTLS_ENTROPY_FORCE_SHA256 was not enabled, so "config.pl full"
didn't have a chance to reach it even if that enabled
MBEDTLS_PLATFORM_NV_SEED_ALT.
Now all it takes to enable this test is MBEDTLS_PLATFORM_NV_SEED_ALT
and its requirements, and the near-ubiquitous MD module.
Call mbedtls_entropy_free on test failure.
Restore the previous NV seed functions which the call to
mbedtls_platform_set_nv_seed() changed. This didn't break anything,
but only because the NV seed functions used for these tests happened
to work for the tests that got executed later in the .data file.
memset has undefined behavior when either pointer can be NULL, which
is the case when it's the result of malloc/calloc with a size of 0.
The memset calls here are useless anyway since they come immediately
after calloc.
Make some functions non-static, to avoid Wunused function warnings. Make
a function scoped variable block scoped instead, to avoid Wunused
variable warnings in some configurations.
* origin/pr/2660:
Fix parsing issue when int parameter is in base 16
Refactor receive_uint32()
Refactor get_byte function
Make the script portable to both pythons
Update the test encoding to support python3
update the test script
Secure element support has its own source file, and in addition
requires many hooks in other files. This is a nontrivial amount of
code, so make it optional (but default on).
This change affects the psa_key_derivation_s structure. With the buffer
removed from the union, it is empty if MBEDTLS_MD_C is not defined.
We can avoid undefined behaviour by adding a new dummy field that is
always present or make the whole union conditional on MBEDTLS_MD_C.
In this latter case the initialiser macro has to depend on MBEDTLS_MD_C
as well. Furthermore the first structure would be either
psa_hkdf_key_derivation_t or psa_tls12_prf_key_derivation_t both of
which are very deep and would make the initialisation macro difficult
to maintain, therefore we go with the first option.
Part of the tests are adapted in this commit, another part is already
covered by the derive_input tests and some of them are not applicable to
the new API (the new API does not request capacity at the setup stage).
The test coverage temporarily drops with this commit, the two test cases
conserning capacity will be re-added in a later commit.
In the 1.0 API some functionality has been split from the
psa_key_derivation_setup() function and is now done with the
psa_key_derivation_input_*() functions. The new tests maintain the
existing test coverage of this functionality.
Add the compile time option PSA_PRE_1_0_KEY_DERIVATION. If this is not
turned on, then the function `psa_key_derivation()` is removed.
Most of the tests regarding key derivation haven't been adapted to the
new API yet and some of them have only been adapted partially. When this
new option is turned off, the tests using the old API and test cases
using the old API of partially adapted tests are skipped.
The sole purpose of this option is to make the transition to the new API
smoother. Once the transition is complete it can and should be removed
along with the old API and its implementation.
Call `greentea_getc()` 8 times, and then `unhexify` once, instead of
calling `receive_byte()`, which inside calls `greentea_getc()` twice,
for every hex digit.
Since Python3 handles encoding differently than Python2,
a change in the way the data is encoded and sent to the target is needed.
1. Change the test data to be sent as hex string
2. Convert the characters to binary bytes.
This is done because the mbed tools translate the encoding differently
(mbed-greentea, and mbed-htrunner)
* origin/pr/2697:
Update crypto submodule
Add all.sh component that exercises invalid_param checks
Remove mbedtls_param_failed from programs
Make it easier to define MBEDTLS_PARAM_FAILED as assert
Make test suites compatible with #include <assert.h>
Pass -m32 to the linker as well
Don't systematically rebuild programs
* origin/pr/2053:
Clarify ChangeLog entry for fix to #1628
Add Changelog entry for clang test-ref-configs.pl fix
Enable more compiler warnings in tests/Makefile
Change file scoping of test helpers.function
Create a new header file psa_helpers.h and put the one helper macro
that isn't specific to PSA crypto there. Use this header file in the
ITS test suite.
This file isn't like the other .function files: it isn't concatenated
by a separate preprocessing script, but included via C preprocessing.
Rename this file to .h. This isn't a normal C header, because it
defines auxiliary functions. But the functions aren't big and we only
have one compilation unit per executable, so this is good enough for
what we're doing.
* origin/pr/2443: (25 commits)
Fix documentation of X.509 parsing test
Add X.509 CRT parsing test for mixed time-encodings
Improve X.509 CRT parsing test names
Add negative X.509 parsing tests for v3Ext in v1/v2 CRT
Add negative X.509 parsing tests for IssuerID/SubjectID in v1 CRT
Improve name of X.509 CRT parsing test
Always use the same X.509 alg structure inside and outside of TBS
Fix test dependencies in X.509 CRT parsing suite
Fix non-DER length encoding in two X.509 CRT parsing tests
Fix test case name formatting in X.509 parsing suite
Use ASN.1 NULL TLVs when testing invalid tags
Shorten X.509 CRT parsing test names
Extend negative testing for X.509 Signature parsing
Extend negative testing for X.509 SignatureAlgorithm parsing
Extend negative testing for X.509 v3 Extension parsing
Extend negative testing for X.509 SubjectID parsing
Extend negative testing for X.509 IssuerID parsing
Extend negative testing for X.509 SubjectPublicKeyInfo parsing
Extend negative testing for X.509 Subject parsing
Extend negative testing for X.509 Validity parsing
...
* origin/pr/2430:
Document support for MD2 and MD4 in programs/x509/cert_write
Correct name of X.509 parsing test for well-formed, ill-signed CRT
Add test cases exercising successful verification of MD2/MD4/MD5 CRT
Add test case exercising verification of valid MD2 CRT
Add MD[245] test CRTs to tree
Add instructions for MD[245] test CRTs to tests/data_files/Makefile
Add suppport for MD2 to CSR and CRT writing example programs
Remove use of MD2 in further x509parse tests
Convert further x509parse tests to use lower-case hex data
Correct placement of ChangeLog entry
Adapt ChangeLog
Use SHA-256 instead of MD2 in X.509 CRT parsing tests
Consistently use lower case hex data in X.509 parsing tests
* origin/pr/2260:
Update crypto submodule
Remove heading spaces in tests/data_files/Makefile
Re-generate library/certs.c from script
Add new line at the end of test-ca2.key.enc
Use strict syntax to annotate origin of test data in certs.c
Add run to all.sh exercising !MBEDTLS_PEM_PARSE_C + !MBEDTLS_FS_IO
Allow DHM self test to run without MBEDTLS_PEM_PARSE_C
ssl-opt.sh: Auto-skip tests that use files if MBEDTLS_FS_IO unset
Document origin of hardcoded certificates in library/certs.c
Adapt ChangeLog
Rename server1.der to server1.crt.der
Add DER encoded files to git tree
Add build instructions to generate DER versions of CRTs and keys
Document "none" value for ca_path/ca_file in ssl_client2/ssl_server2
ssl_server2: Skip CA setup if `ca_path` or `ca_file` argument "none"
ssl_client2: Skip CA setup if `ca_path` or `ca_file` argument "none"
Correct white spaces in ssl_server2 and ssl_client2
Adapt ssl_client2 to parse DER encoded test CRTs if PEM is disabled
Adapt ssl_server2 to parse DER encoded test CRTs if PEM is disabled
Don't use the macro name assert. It's technically permitted as long as
<assert.h> is not included, but it's fragile, because it means the
code and any header that it includes must not include <assert.h>.
Don't use the macro name assert. It's technically permitted as long as
<assert.h> is not included, but it's fragile, because it means the
code and any header that it includes must not include <assert.h>.
Increase the SO versions of libmbedx509 and libmbedtls due to the
addition of fields in publicly visible (non-opaque) structs:
- mbedtls_ssl_config
- mbedtls_ssl_context
- mbedtls_x509_crt
Remove the "Decrypt empty buffer" test, as ChaCha20 is a stream cipher
and 0 bytes encrypted is identical to a 0 length buffer. The "ChaCha20
Encrypt and decrypt 0 bytes" test will test decryption of a 0 length
buffer.
Previously, even in the Chacha20 and Chacha20-Poly1305 tests, we would
test that decryption of an empty buffer would work with
MBEDTLS_CIPHER_AES_128_CBC.
Make the cipher used with the dec_empty_buf() test configurable, so that
Chacha20 and Chacha20-Poly1305 empty buffer tests can use ciphers other
than AES CBC. Then, make the Chacha20 and Chacha20-Poly1305 empty buffer
tests use the MBEDTLS_CIPHER_CHACHA20 and
MBEDTLS_CIPHER_CHACHA20_POLY1305 cipher suites.
Remove the "Decrypt empty buffer" test, as ChaCha20 is a stream cipher
and 0 bytes encrypted is identical to a 0 length buffer. The "ChaCha20
Encrypt and decrypt 0 bytes" test will test decryption of a 0 length
buffer.
Previously, even in the Chacha20 and Chacha20-Poly1305 tests, we would
test that decryption of an empty buffer would work with
MBEDTLS_CIPHER_AES_128_CBC.
Make the cipher used with the dec_empty_buf() test configurable, so that
Chacha20 and Chacha20-Poly1305 empty buffer tests can use ciphers other
than AES CBC. Then, make the Chacha20 and Chacha20-Poly1305 empty buffer
tests use the MBEDTLS_CIPHER_CHACHA20 and
MBEDTLS_CIPHER_CHACHA20_POLY1305 cipher suites.
