The ssl-opt.sh test cases using session resumption tend to fail occasionally
on the CI due to a race condition in how ssl_server2 and ssl_client2 handle
the reconnection cycle.
The server does the following in order:
- S1 send application data
- S2 send a close_notify alert
- S3 close the client socket
- S4 wait for a "new connection" (actually a new datagram)
- S5 start a handshake
The client does the following in order:
- C1 wait for and read application data from the server
- C2 send a close_notify alert
- C3 close the server socket
- C4 reset session data and re-open a server socket
- C5 start a handshake
If the client has been able to send the close_notify (C2) and if has been
delivered to the server before if closes the client socket (S3), when the
server reaches S4, the datagram that we start the new connection will be the
ClientHello and everything will be fine.
However if S3 wins the race and happens before the close_notify is delivered,
in S4 the close_notify is what will be seen as the first datagram in a new
connection, and then in S5 this will rightfully be rejected as not being a
valid ClientHello and the server will close the connection (and go wait for
another one). The client will then fail to read from the socket and exit
non-zero and the ssl-opt.sh harness will correctly report this as a failure.
In order to avoid this race condition in test using ssl_client2 and
ssl_server2, this commits introduces a new command-line option
skip_close_notify to ssl_client2 and uses it in all ssl-opt.sh tests that use
session resumption with DTLS and ssl_server2.
This works because ssl_server2 knows how many messages it expects in each
direction and in what order, and closes the connection after that rather than
relying on close_notify (which is also why there was a race in the first
place).
Tests that use another server (in practice there are two of them, using
OpenSSL as a server) wouldn't work with skip_close_notify, as the server won't
close the connection until the client sends a close_notify, but for the same
reason they don't need it (there is no race between receiving close_notify and
closing as the former is the cause of the later).
An alternative approach would be to make ssl_server2 keep the connection open
until it receives a close_notify. Unfortunately it creates problems for tests
where we simulate a lossy network, as the close_notify could be lost (and the
client can't retransmit it). We could modify udp_proxy with an option to never
drop alert messages, but when TLS 1.3 comes that would no longer work as the
type of messages will be encrypted.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
(Only the top-level ones, ie, for each call to eg asn1_get_mpi(), ensure
there's at least one test case that makes this call fail in one way, but don't
test the various ways to make asn1_get_mpi fail - that should be covered
elsewhere.)
- the new checks added by the previous commits needed exercising
- existing tests sometimes had wrong descriptions or where passing for the
wrong reason (eg with the "length mismatch" test, the function actually
failed before reaching the length check)
- while at it, add tests for the rest as well
The valid minimal-size key was generated with:
openssl genrsa 128 2>/dev/null | openssl rsa -outform der 2>/dev/null | xxd -p
Exercise the library functions with calloc returning NULL for a size
of 0. Make this a separate job with UBSan (and ASan) to detect
places where we try to dereference the result of calloc(0) or to do
things like
buf = calloc(size, 1);
if (buf == NULL && size != 0) return INSUFFICIENT_MEMORY;
memcpy(buf, source, size);
which has undefined behavior when buf is NULL at the memcpy call even
if size is 0.
This is needed because other test components jobs either use the system
malloc which returns non-NULL on Linux and FreeBSD, or the
memory_buffer_alloc malloc which returns NULL but does not give as
useful feedback with ASan (because the whole heap is a single C
object).
As the SSL programs, like ssl_client2 and ssl_server2, are dependent on
SSL and therefore about to be removed, the only consumer of query_config
is the query_compile_time_config test. As such, it makes sense to move
query_config to be next to what uses it.
Goals:
* Build with common compilers with common options, so that we don't
miss a (potentially useful) warning only triggered with certain
build options.
* A previous commit removed -O0 test jobs, leaving only the one with
-m32. We have inline assembly that is disabled with -O0, falling
back to generic C code. This commit restores a test that runs the
generic C code on a 64-bit platform.
Files deleted by us: keep them deleted.
```
git rm $(git status -s | sed -n 's/^DU //p')
```
Individual files with conflicts:
* `README.md`: keep the crypto version.
* `doxygen/input/doc_mainpage.h`: keep the crypto version (with an obsolete Mbed Crypto version number).
* `include/mbedtls/error.h`:
* `ERROR`: similar additions made through parallel commits, with only whitespace differences. Align with the tls version.
* `library/CMakeLists.txt`: keep the crypto version.
* `library/Makefile`: keep the crypto version.
* `scripts/generate_errors.pl`: keep the crypto version (the relevant changes were made through parallel commits).
* `tests/scripts/check-test-cases.py`:
* `Results`: keep the crypto version, which has both the new argument to the constructor (added in crypto only) and the class docstring (added through parallel commits).
* `tests/suites/helpers.function`:
* `ARRAY_LENGTH`, `ASSERT_ALLOC`: additions in the same location. Keep both, in indifferent order.
* `tests/suites/target_test.function`:
* `receive_uint32`: keep the crypto version which has an additional bug fix. The tls changes made in tls are irrelevant after this bug fix.
* `visualc/VS2010/mbedTLS.vcxproj`: run `scripts/generate_visualc_files.pl`.
Review of non-conflicting changes:
* `all.sh`: 1 change.
* zlib test components: don't add them.
* `include/CMakeLists.txt`: 1 change.
* `target_include_directories`: doesn't work as is (different target name). Don't take the change.
* All other non-conflicting changes: take them.
Let the caller decide what certificates and keys are loaded (EC/RSA)
instead of loading both for the server, and an unspecified one
for the client. Use only DER encoding.
For each of the crypto-only presets, run the build and check that the
resulting libmbedx509 and libmbedtls are empty.
Don't bother testing, because for each crypto-only preset, another
component builds that plus the x509 and tls parts and tests
everything.
We already have a specific component in all.sh for testing SSLv3, we don't
need to also test it in components that aren't specifically about it.
Previously config.py full enabled SSLv3, but it no longer does since it is
deprecated.
Change the encoding of key types, EC curve families and DH group
families to make the low-order bit a parity bit (with even parity).
This ensures that distinct key type values always have a Hamming
distance of at least 2, which makes it easier for implementations to
resist single bit flips.
All key types now have an encoding on 32 bits where the bottom 16 bits
are zero. Change to using 16 bits only.
Keep 32 bits for key types in storage, but move the significant
half-word from the top to the bottom.
Likewise, change EC curve and DH group families from 32 bits out of
which the top 8 and bottom 16 bits are zero, to 8 bits only.
Reorder psa_core_key_attributes_t to avoid padding.
Remove the values of curve encodings that are based on the TLS registry
and include the curve size, keeping only the new encoding that merely
encodes a curve family in 8 bits.
Keep the old constant names as aliases for the new values and
deprecate the old names.
Define constants for ECC curve families and DH group families. These
constants have 0x0000 in the lower 16 bits of the key type.
Support these constants in the implementation and in the PSA metadata
tests.
Switch the slot management and secure element driver HAL tests to the
new curve encodings. This requires SE driver code to become slightly
more clever when figuring out the bit-size of an imported EC key since
it now needs to take the data size into account.
Switch some documentation to the new encodings.
Remove the macro PSA_ECC_CURVE_BITS which can no longer be implemented.
Change the representation of psa_ecc_curve_t and psa_dh_group_t from
the IETF 16-bit encoding to a custom 24-bit encoding where the upper 8
bits represent a curve family and the lower 16 bits are the key size
in bits. Families are based on naming and mathematical similarity,
with sufficiently precise families that no two curves in a family have
the same bit size (for example SECP-R1 and SECP-R2 are two different
families).
As a consequence, the lower 16 bits of a key type value are always
either the key size or 0.
Don't rely on the bit size encoded in the PSA curve identifier, in
preparation for removing that.
For some inputs, the error code on EC key creation changes from
PSA_ERROR_INVALID_ARGUMENT to PSA_ERROR_NOT_SUPPORTED or vice versa.
There will be further such changes in subsequent commits.
Key types are now encoded through a category in the upper 4 bits (bits
28-31) and a type-within-category in the next 11 bits (bits 17-27),
with bit 16 unused and bits 0-15 only used for the EC curve or DH
group.
For symmetric keys, bits 20-22 encode the block size (0x0=stream,
0x3=8B, 0x4=16B).
The splitting of this test into two versions depending on whether SHA-1 was
allowed by the server was a mistake in
5d2511c4d4 - the test has nothing to do with
SHA-1 in the first place, as the server doesn't request a certificate from
the client so it doesn't matter if the server accepts SHA-1 or not.
psa_hash_compare is tested for good cases and invalid-signature cases
in hash_compute_compare. Also test invalid-argument cases. Also run a
few autonomous test cases with valid arguments.
While the whole script makes (often implicit) assumptions about the version of
GnuTLS used, generally speaking it should work out of the box with the version
packaged on our reference testing platform, which is Ubuntu 16.04 so far.
With the update from Jan 8 2020 (3.4.10-4ubuntu1.6), the patches for rejecting
SHA-1 in certificate signatures were backported, so we should avoid presenting
SHA-1 signed certificates to a GnuTLS peer in ssl-opt.sh.
Because two buffers were aliased too early in the code, it was possible that
after an allocation failure, free() would be called twice for the same pointer.
Previously mocked non-blocking read/write was returning 0 when buffer was empty/full. That was causing ERR_SSL_CONN_EOF error in tests which was using these mocked callbacks. Beside that non-blocking read/write was returning ERR_SSL_WANT_READ/_WRITE depending on block pattern set by test design. Such behavior forced to redesign of these functions so that they could be used in other tests
This error occurs when free space in the buffer is in the middle (the buffer has come full circle) and function mbedtls_test_buffer_put is called. Then the arguments for memcpy are calculated incorrectly and program ends with segmentation fault
Before, the string to parse may contain trailing garbage (there was
never more than one byte), and there was a separate argument
indicating the length of the content. Now, the string to parse is the
exact content, and the test code runs an extra test step with a
trailing byte added.
If there was a fatal error (bizarre behavior from the standard
library, or missing test data file), execute_tests did not close the
outcome file. Fix this.
Fix get_len_step when buffer_size==0. The intent of this test is to
ensure (via static or runtime buffer overflow analysis) that
mbedtls_asn1_get_len does not attempt to access beyond the end of the
buffer. When buffer_size is 0 (reached from get_len when parsing a
1-byte buffer), the buffer is buf[1..1] because allocating a 0-byte
buffer might yield a null pointer rather than a valid pointer. In this
case the end of the buffer is p==buf+1, not buf+buffer_size which is
buf+0.
The test passed because calling mbedtls_asn1_get_len(&p,end,...) with
end < p happens to work, but this is not guaranteed.
