Commit graph

1222 commits

Author SHA1 Message Date
Gilles Peskine 6213a00ec1 Storage format tests: cover algorithms for each key type
In the generated storage format test cases, cover all supported
algorithms for each key type. This is a step towards exercising
the key with all the algorithms it supports; a subsequent commit
will generate a policy that permits the specified algorithms.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-04-15 16:15:48 +02:00
Gilles Peskine 989c13dece Remove ad hoc is_valid_for_signature method
Use the new generic is_public method.

Impact on generated cases: there are new HMAC test cases for SIGN_HASH. It
was a bug that these test cases were previously not generated.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-04-15 16:15:44 +02:00
Gilles Peskine a16d8fcee9
Merge pull request #5697 from gilles-peskine-arm/psa-test-op-fail-2.28
Backport 2.28: PSA: systematically test operation failure
2022-04-15 10:52:50 +02:00
Manuel Pégourié-Gonnard b623832176 Fix compat.sh invocation in basic-built-test.sh
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2022-04-14 09:12:10 +02:00
Gilles Peskine 186331875a test_psa_crypto_config_accel_cipher: deactivate CMAC
We don't yet support all combinations of configurations. With all.sh as it
currently stands, component_test_psa_crypto_config_accel_cipher results in a
build where PSA_WANT_ALG_CMAC is disabled but CMAC operations succeed
nonetheless, going via the driver. With the systematic testing of
not-supported operations, this now results in a test failure.

The code in all.sh does not respect the principle documented in
df885c052c701b89c24ad5f305114f8f8b39d750:

> The PSA_WANT_* macros have to be the same as the ones
> used to build the Mbed TLS library the test driver
> library is supposed to be linked to as the PSA_WANT_*
> macros are used in the definition of structures and
> macros that are shared by the PSA crypto core,
> Mbed TLS drivers and the driver test library.

Disable PSA_WANT_ALG_CMAC completely in this test component. This is not
wrong and it makes the test component pass.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-04-12 17:15:56 +02:00
Gilles Peskine c2fc241e46 Test attempts to use a public key for a private-key operation
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-04-05 15:19:16 +02:00
Gilles Peskine 23cb12ef9f A key agreement algorithm can contain a key derivation
PSA_ALG_KEY_AGREEMENT(..., kdf) is a valid key derivation algorithm
when kdf is one.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-04-05 15:19:16 +02:00
Gilles Peskine d096445dfe In NOT_SUPPORTED test case descriptions, show what is not supported
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-04-05 15:19:16 +02:00
Gilles Peskine f8b6b503b4 Systematically generate test cases for operation setup failure
The test suite test_suite_psa_crypto_op_fail now runs a large number
of automatically generated test cases which attempt to perform a
one-shot operation or to set up a multi-part operation with invalid
parameters. The following cases are fully covered (based on the
enumeration of valid algorithms and key types):
* An algorithm is not supported.
* The key type is not compatible with the algorithm (for operations
  that use a key).
* The algorithm is not compatible for the operation.

Some test functions allow the library to return PSA_ERROR_NOT_SUPPORTED
where the test code generator expects PSA_ERROR_INVALID_ARGUMENT or vice
versa. This may be refined in the future.

Some corner cases with algorithms combining a key agreement with a key
derivation are not handled properly. This will be fixed in follow-up
commits.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-04-05 15:19:16 +02:00
Gilles Peskine a218047245 Generate test cases for hash operation failure
Test that hash operation functions fail when given a hash algorithm
that is not supported or an algorithm that is not a hash.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-04-05 15:19:16 +02:00
Gilles Peskine c05158bf86 New test suite for systematically generated operation failure tests
The new test suite psa_crypto_op_fail is intended for systematically
generated test cases that ensure that cryptographic operations with
invalid parameters fail as expected. I intend invalid parameters to
cover things like an invalid algorithm, an algorithm that is
incompatible with the operation, a key type that is incompatible with
the algorithm, etc.

This commit just creates the infrastructure. Subsequent commits will
add data generation and test code.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-04-05 15:19:16 +02:00
Gilles Peskine a296e48b2b Simplify key_for_usage_flags
Generate "with implication" and "without implication" usage test cases
separately.

The set of generated test cases is unchanged. The order, and the description
of "with implication" test cases, changes.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-04-05 15:19:16 +02:00
Gilles Peskine b5c4382bef check-generated-files.sh -u: don't update file timestamps
When running check-generated-files in update mode, all generated files were
regenerated. As a consequence,
```
tests/scripts/check-generated-files.sh -u && make
```
always caused most of the code to be rebuilt. Now, if a file hasn't changed,
preserve its original modification time (and other metadata), so the command
above doesn't rebuild anything that has actually not changed.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-04-05 15:19:16 +02:00
Gilles Peskine a8827d190f
Merge pull request #5542 from gilles-peskine-arm/config-baremetal-size-2.28
Backport 2.28: Disable debugging features in the primary code size measurement job
2022-04-04 16:13:05 +02:00
Dave Rodgman 52af769878 Update references to old Github organisation
Replace references to ARMmbed organisation with the new
org, Mbed-TLS, following project migration.

Signed-off-by: Dave Rodgman <dave.rodgman@arm.com>
2022-03-31 14:27:24 +01:00
Dave Rodgman ce514def84
Merge pull request #5563 from AndrzejKurek/timeless-2.28
Backport 2.28: Fix builds with MBEDTLS_HAVE_TIME disabled and test
2022-03-15 16:43:18 +00:00
David Horstmann 3cb5e9bf5a Use $PWD instead of $(pwd) for consistency
Change the new baremetal all.sh tests to use $PWD rather than
calling pwd again directly.

Signed-off-by: David Horstmann <david.horstmann@arm.com>
2022-03-04 15:25:42 -05:00
Daniel Axtens 1a021afd7d tests: add baremetal full config build
To be able to test utility programs for an absence of time.h, we need a
baremetal config that is not crypto only. Add one.

Signed-off-by: Daniel Axtens <dja@axtens.net>
2022-03-04 15:25:42 -05:00
Daniel Axtens 6f63cc7188 tests: prevent inclusion of time.h in baremetal compiles
baremetal compiles should not include time.h, as MBEDTLS_HAVE_TIME is
undefined. To test this, provide an overriding include directory that
has a time.h which throws a meaningful error if included.

Signed-off-by: Daniel Axtens <dja@axtens.net>
2022-03-04 15:25:42 -05:00
Gilles Peskine cfd4fae89d Add storage format checks to the interface checker
Expand abi_check.py to look for backward incompatibilities not only in
the interface exposed to application code (and to some extent driver
code), but also to the interface exposed via the storage format, which
is relevant when upgrading Mbed TLS on a device with a PSA keystore.

Strictly speaking, the storage format checks look for regressions in
the automatically generated storage format test data. Incompatible
changes that are not covered by the generated tests will also not be
covered by the interface checker.

A known defect in this commit is that the --brief output is not brief
for storage format checks.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-03-04 20:18:27 +01:00
Gilles Peskine df57835a76 Make mypy unconditional
Running mypy was optional for a transition period when it wasn't installed
on the CI. Now that it is, make it mandatory, to avoid silently skipping an
expected check if mypy doesn't work for some reason.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-02-28 16:11:58 +01:00
Ronald Cron 8e1ca4df2e
Merge pull request #5459 from gilles-peskine-arm/check_test_cases-list-2.28
Backport 2.28: check_test_cases.py --list
2022-02-15 13:52:37 +01:00
Gilles Peskine 60d9947bc4 Change our code size reference job to use baremetal_size
In build_arm_none_eabi_gcc_m0plus, use baremetal_size instead of baremetal
as the configuration, i.e. exclude debugging features. This job is the only
one switching to baremetal_size because it's our primary point of reference
for code size evolution, and which is the only job where we display the code
size built with -Os so it's presumably the only job for which we really care
about a meaningful code size report.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-02-15 12:55:18 +01:00
Manuel Pégourié-Gonnard 8b8760885e
Merge pull request #5465 from gilles-peskine-arm/cmake-test-suite-enumeration-2.28
Backport 2.28: CMake: generate the list of test suites automatically
2022-02-07 09:48:24 +01:00
Gilles Peskine d4c5c3d231 Remove obsolete calls to if_build_succeeded
This is now a no-op.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-02-04 00:37:49 +01:00
Gilles Peskine fdddb9de8f Remove obsolete variable restoration or unset at the end of a component
This is no longer useful now that components run in a subshell.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-02-04 00:36:23 +01:00
Gilles Peskine 717d55edbe Remove obsolete cd at the end of a component
This is no longer useful now that components run in a subshell.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-02-04 00:36:14 +01:00
Gilles Peskine ca9cfcaed9 Stop CMake out of source tests running on 16.04 (continued)
The race condition mentioned in the previous commit
"Stop CMake out of source tests running on 16.04"
has also been observed with test_cmake_as_subdirectory on 3.1 and can
presumably happen on 2.28 as well. So skip it on Ubuntu 16.04 as well.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-02-04 00:35:14 +01:00
Andrzej Kurek 19d6ab0fb8 Enable testing with PSA for config-mini-tls1_1
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-01-27 11:01:24 -05:00
Andrzej Kurek 19e83fa3a5 Restructure test-ref-configs to test with USE_PSA_CRYPTO turned on
Run some of the test configs twice, enabling MBEDTLS_USE_PSA_CRYPTO
and MBEDTLS_PSA_CRYPTO_C in one of the runs.
Add relevant comments in these configs.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-01-26 07:45:43 -05:00
Gilles Peskine efd696afb9 New option to list all test cases
Occasionally useful for diagnosing issues with test reports.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-01-25 13:32:32 +01:00
Gilles Peskine 0c2f8ee3f0 Move collect_available_test_cases to check_test_cases.py
No behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-01-25 13:32:25 +01:00
Paul Elliott 2ab9a7a57a Stop CMake out of source tests running on 16.04
Running the out of source CMake test on Ubuntu 16.04 using more than one
processor (as the CI does) can create a race condition whereby the build
fails to see a generated file, despite that file actually having been
generated. This problem appears to go away with 18.04 or newer, so make
the out of source tests not supported on Ubuntu 16.04

Signed-off-by: Paul Elliott <paul.elliott@arm.com>
2021-12-13 11:14:45 +00:00
Ronald Cron 620cbb9bf5
Merge pull request #5262 from xffbai/code-align-backport2.x
Backport 2.x: Fix (d)tls1_2 into (d)tls12 in version options
2021-12-09 16:26:24 +01:00
Manuel Pégourié-Gonnard c3319e73db
Merge pull request #5189 from gilles-peskine-arm/struct_reordering_2.x
Backport 2.x: Reorder structure fields to maximize usage of immediate offset access
2021-12-09 12:54:13 +01:00
Xiaofei Bai f40545d919 Fix (d)tls1_2 into (d)tls12 in version options
Signed-off-by: Xiaofei Bai <xiaofei.bai@arm.com>
2021-12-03 08:13:30 +00:00
Ronald Cron 0962370acf all.sh: psa: Add cipher acceleration test component
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2021-11-30 14:49:19 +01:00
Ronald Cron f7e483c0b4 all.sh: psa: Add hash acceleration test component
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2021-11-30 14:49:19 +01:00
Ronald Cron d2c53e60ca all.sh: psa: Add ECDSA and RSA signature acceleration component
Add ECDSA and RSA signature acceleration testing
with signature capabilitites removed from the
Mbed TLS library.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2021-11-30 14:49:19 +01:00
Ronald Cron b814bdabe4 Move to separately compiled PSA test driver library
This commit removes the test_psa_crypto_config_basic
all.sh component that can no longer work without
adapting it to the separately compiled test driver
library. This component is replaced by several
components in the following commits to test various
type of acceleration independently.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2021-11-30 14:49:19 +01:00
Gilles Peskine 27482f17f1 Run the dlopen test in shared library builds
Non-regression for the fix in https://github.com/ARMmbed/mbedtls/pull/5126:
libmbedtls and libmbedx509 did not declare their dependencies on libmbedx509
and libmbedcrypto when built with make.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-11-25 18:00:53 +01:00
Gilles Peskine 23b4096ecf Fix several bugs with multiline comments
Empty the current line if it's entirely inside a comment.

Don't incorrectly end a block comment at the second line if it doesn't
contain `*/`.

Recognize `/*` to start a multiline comment even if it isn't at the start of
the line.

When stripping off comments, consistently strip off `/*` and `*/`.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-11-17 20:45:39 +01:00
Gilles Peskine 44801627d2 Improve comment and string stripping
Make that part of the code more readable.

Add support for // line comments.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-11-17 20:45:06 +01:00
Gilles Peskine 4f04d619b5 Fix terminology in comment
In computing, brackets are []. () are called parentheses.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-11-17 20:39:56 +01:00
Gilles Peskine df30665a16 Move comment and string literal processing to a new function
No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-11-17 20:32:31 +01:00
Gilles Peskine c8fc67f341 Simplify some regex definitions
Use '|'.join([comma-separated list]) rather than r'...|' r'...|'. This way
there's less risk of forgetting a '|'. Pylint will yell if we forget a comma
between list elements.

Use match rather than search + mandatory start anchor for EXCLUSION_LINES.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-11-17 20:23:18 +01:00
Bence Szépkúti 2be65d5134 Remove expected failure from test_psa_compliance
Issue #5143 was fixed in PR #5192.

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2021-11-17 14:47:56 +01:00
Manuel Pégourié-Gonnard 394c65e0f0
Merge pull request #5169 from bensze01/test_psa_compliance_2.x
Backport 2.x: Run the PSA Compliance test suite in all.sh
2021-11-17 14:10:05 +01:00
Gilles Peskine b3f4dd5c81 Lift some code out of parse_identifiers
Make parse_identifiers less complex. Pylint was complaining that it had too
many local variables, and it had a point.

* Lift the constants identifier_regex and exclusion_lines to class
  constants (renamed to uppercase because they're constants).
* Lift the per-file loop into a new function parse_identifiers_in_file.

No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-11-16 21:26:08 +01:00
Gilles Peskine 7493c4017a Fix comment parsing
Fix cases like
```
/*short comment*/ /*long
 comment */
int mbedtls_foo;
```
where the previous code thought that the second line started outside of a
comment and ended inside of a comment.

I believe that the new code strips comments correctly. It also strips string
literals, just in case.

Fixes #5191.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-11-16 21:25:14 +01:00