Commit graph

1204 commits

Author SHA1 Message Date
Gilles Peskine fa37961413 Rename OMITTED_SYSTEMATIC_DEPENDENCIES
It isn't a set of dependencies, it's a set of symbols. So give it a
name that describes the symbol rather than a name that pretends it's a
collection of dependencies.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-02-01 13:22:38 +01:00
Gilles Peskine 0794572f96 More precise recognition of key policy negative tests
The test function asymmetric_signature_key_policy combines positive
and negative tests inside the code, so it doesn't take a status as its
last argument.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-02-01 13:22:38 +01:00
Gilles Peskine a87e178553 If there are no arguments, don't update the file content
This avoids having to bother with edge cases of the .data syntax.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-02-01 13:19:26 +01:00
Gilles Peskine 72d8e0ad04 Key policy negative tests don't actually require the algorithm
It doesn't make much difference in practice, but to keep closer to
what the current code does, run negative key policy tests even if the
algorithm for the operation attempt is not supported.

In particular, this allows the following test cases to run:
* "PSA key policy: agreement + KDF, wrong agreement algorithm"
* "PSA key policy: raw agreement, wrong algorithm"
Without this exception, those two test cases would never run, because
they would depend on PSA_ALG_WANT_FFDH. Since FFDH is not implemented
yet, it isn't enabled in any configuration. There's no alternative to
FFDH for these particular test cases because ECDH is the only key
agreement that is implemented in Mbed TLS so far.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-02-01 13:19:26 +01:00
Gilles Peskine 20987b9a46 Detect test cases that require something not to be supported
PSA_xxx_CATEGORY_yyy is used in metadata tests where it doesn't
involve any particular support, and elsewhere it's used as a value
that is definitely not supported but is in a plausible range. Such
symbols do not require any dependency.

If a test case is expects PSA_ERROR_NOT_SUPPORTED, its
dependencies (often including one negative dependency) cannot be
determined automatically, so leave that test case alone.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-02-01 13:19:26 +01:00
Gilles Peskine f032fa9d94 Special handling for some always-on or alias symbols
Some symbols don't require a dependency symbol:

* Modifiers such as truncated MAC
* Always-on features such as the raw data key type
* Aliases or special values such as RSA PKCS#1v1.5 raw

I'm not convinced that all of these warrant special handling in the
script, rather than having the expected symbol defined somewhere. But
for now I prefer to minimize changes to the header files.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-02-01 13:19:26 +01:00
Gilles Peskine 81dec0047f Remove classic crypto dependencies
Don't remove cipher-related dependencies because the corresponding
PSA_WANT_xxx dependencies are not implemented yet.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-02-01 13:19:26 +01:00
Gilles Peskine e4f539caf9 Don't add cipher-related PSA_WANT_xxx because they aren't implemented yet
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-02-01 13:19:26 +01:00
Gilles Peskine 9bbba5ea69 Add a function to remove classic crypto dependencies
A list of symbols to remove will be added in a subsequent commit.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-02-01 13:19:26 +01:00
Gilles Peskine 2d2e924401 Implement basic update of PSA_WANT dependencies
Remove any existing PSA_WANT_xxx dependency. Add PSA_WANT_xxx
dependencies based on the PSA_KEY_TYPE_xxx and PSA_ALG_xxx symbols
used in the test case arguments.

PSA_ECC_FAMILY_xxx and PSA_DH_GROUP_xxx are not implemented yet in the
PSA conditional inclusion mechanism in Mbed TLS, so this script
doesn't handle them yet.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-02-01 13:19:26 +01:00
Gilles Peskine 45e9e73e66 Add a possibility to keep manual dependencies for certain functions
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-02-01 13:19:26 +01:00
Gilles Peskine 82ebaa4d36 Implement stanza parsing in .data files
Parse the existing dependencies. For now, just write them back.
Subsequent commits will implement the dependency processing that is
the goal of this program.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-02-01 13:19:26 +01:00
Gilles Peskine bdffaeaf3a New script to determine PSA crypto test dependencies automatically
This commit only contains a framework to rewrite .data files. No
actual modification of the content is implemented yet.

For now, command line parsing is trivial: just a list of file names,
with no options.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-02-01 13:19:26 +01:00
Ronald Cron 336678bccc tests: psa: Test PSA client-only code
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2021-02-01 13:17:23 +01:00
Ronald Cron 88a8035860
Merge pull request #3953 from gilles-peskine-arm/python-mypy-mkdir
Python upscale: pass mypy and create library directory
2021-01-29 12:07:53 +01:00
Gilles Peskine 0370c17105 mypy: require at least version 0.780
0.780 works. The previous release, 0.770, does not.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-01-20 12:03:24 +01:00
Gilles Peskine 4738b96d75 Use $PYTHON when running mypy
Make sure to run mypy with the desired Python version.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-01-19 21:45:32 +01:00
Gilles Peskine c3b178768f Use can_mypy rather than just checking for mypy
As indicated in the comments in the can_mypy function, we don't just
need a mypy executable to be present, we need it to work.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-01-19 21:43:24 +01:00
Gilles Peskine bdde5d002c Use Python to check the version of pylint
This reduces dependencies, doesn't require maintainers to know awk,
and makes the version parsing more robust.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-01-19 21:42:05 +01:00
Gilles Peskine 6d82a7ef9f Say we're running pylint
Now that the script might additionally run mypy, it's more
user-friendly to indicate what's going on at the beginning as well.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-01-19 21:23:11 +01:00
Gilles Peskine 2991b5f6c0 Minor documentation fixes
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-01-19 21:23:11 +01:00
Gilles Peskine a222434952 Test SSL with non-deterministic ECDSA
In component_test_no_hmac_drbg, the fact that HMAC_DRBG is disabled
doesn't affect the SSL code, but the fact that deterministic ECDSA is
disabled does. So run some ECDSA-related SSL tests.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-01-13 20:31:24 +01:00
Gilles Peskine ba74904c48 SSL test programs: support HMAC_DRBG
Support HMAC_DRBG in ssl_client2 and ssl_server2, in addition to
CTR_DRBG. CTR_DRBG is still used if present, but it's now possible to
run the SSL test programs with CTR_DRBG disabled.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-01-13 20:30:03 +01:00
Gilles Peskine 1cc6a8ea15 Add --can-pylint and --can-mypy options
With just the option --can-pylint or --can-mypy, check whether the
requisite tool is available with an acceptable version and exit.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-01-06 17:12:18 +01:00
Gilles Peskine b13ed70b32 Check scripts/mbedtls_dev/*.py with pylint
mypy automatically checks the modules when it encounters them as
imports. Don't make it check them twice, because it would complain
about encountering them through different paths (via the command line
as scripts/mbedtls_dev/*.py and via imports as just mbedtls_dev/*.py).

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-01-06 17:12:10 +01:00
Gilles Peskine a51e1dbe76
Merge pull request #3895 from gilles-peskine-arm/psa-external-random
Alternative random generator support for PSA
2021-01-06 17:09:11 +01:00
Gilles Peskine 2adebc89da Move get_c_expression_values into a separate module
Create a directory mbedtls_dev intended to contain various Python
module for use by Python scripts located anywhere in the Mbed TLS
source tree.

Move get_c_expression_values and its auxiliary functions into a new
Python module mbedtls_dev.c_build_helper.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-12-11 00:35:19 +01:00
Gilles Peskine fc62211e3b Refactor and generalize run_c
Generalize the very ad hoc run_c function into a function to generate
a C program to print the value of a list of expressions. Refactor the
code into several functions to make it more manageable.

No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-12-11 00:35:11 +01:00
Gilles Peskine 45d350b9dc Allow tests/scripts/foo.py to import from scripts
Allow Python scripts in tests/scripts to import modules located in the
scripts directory. To do this, use
```
import scripts_path # pylint: disable=unused-import
```

Declare the scripts directory to pylint and to mypy.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-12-10 23:16:03 +01:00
Gilles Peskine 7be4551f23 Check typing of python scripts if mypy is available
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-12-10 21:32:32 +01:00
Gilles Peskine e6d0ac26ca mbedtls_test.py: Tell mypy to ignore mbed_host_tests
Since no typing stubs are available for mbed_host_tests.py, mypy
errors out on mbedtls_test.py with

    error: Skipping analyzing 'mbed_host_tests': found module but no type hints or library stubs

Ignore this import to get at least some benefit from mypy without
spending significant effort to write stubs.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-12-10 21:30:14 +01:00
Gilles Peskine 38b66dfc85 test_generate_test_code: remove Python 2 compatibility code
This makes the code cleaner.

As a bonus, mypy no longer gets confused.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-12-10 21:30:08 +01:00
Gilles Peskine ac9e7c0b6e check_files.py: pass mypy
Add enough type annotations to pass mypy 0.782 with Python 3.5. The
source code will still run normally under older Python versions.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-12-10 21:30:01 +01:00
Gilles Peskine 4aebb8d936 Test shebang lines
Executable scripts must have shebang (#!) line to be effectively
executable on most Unix-like systems. Enforce this, and conversely
enforce that files with a shebang line are executable.

Check that the specified interperter is consistent with the file
extension.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-12-10 21:29:55 +01:00
Gilles Peskine 15898eec23 Allow Python files not to be executable
.py files may be modules which are not standalone program, so allow
them not to be executable.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-12-10 21:29:49 +01:00
Manuel Pégourié-Gonnard 48c76c91f3
Merge pull request #3949 from bensze01/update_psa_values
Improve documentation in test_psa_constant_names.py
2020-12-10 11:19:30 +01:00
Bence Szépkúti 4af6560860 Improve documentation in test_psa_constant_names.py
Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-12-08 11:10:21 +01:00
Gilles Peskine 2b759626a9
Merge pull request #3948 from bensze01/update_psa_values
Update the values of PSA Crypto API macros to version 1.0.0
2020-12-08 11:03:33 +01:00
Bence Szépkúti 7e37bf97ee Update test_psa_constant_names.py for new values
Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-12-08 07:33:08 +01:00
John Durkop 1b7ee05461 Added tests to confirm hash support for crypto config
New tests have been added for all the hash algorithms to
confirm they compile correctly when using PSA_WANT and
accelerator guards.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-11-27 08:51:22 -08:00
Gilles Peskine c109b37b07 Test MBEDTLS_PSA_CRYPTO_EXTERNAL_RNG
Add two builds with MBEDTLS_PSA_CRYPTO_EXTERNAL_RNG to all.sh:
* full minus all DRBG (validates that PSA can work without any of the
  DRBG modules).
* with MBEDTLS_USE_PSA_CRYPTO and no CTR_DRBG (validates that PSA can
  work without CTR_DRBG, and that it works for USE_PSA_CRYPTO).

The goal is to exercise default/full, with/out USE_PSA_CRYPTO, and
with/out deterministic ECDSA (which requires HMAC_DRBG). The choice of
pairing is rather arbitrary.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-11-23 17:42:54 +01:00
Gilles Peskine 82e57d1611 PSA: support HMAC_DRBG
Support using HMAC_DRBG instead of CTR_DRBG in the PSA subsystem.

Use HMAC_DRBG if CTR_DRBG is available. Choose between SHA-256 and
SHA-512 based on availability.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-11-23 17:42:54 +01:00
Gilles Peskine 662deb38d6
Merge pull request #3547 from ronald-cron-arm/psa-openless
Openless PSA crypto APIs implementation
2020-11-20 18:48:33 +01:00
Gilles Peskine 9a68810405
Merge pull request #3830 from jdurkop/psa-crypto-config-phase2
Phase 2 support for MBDTLS_PSA_CRYPTO_CONFIG
2020-11-19 09:32:23 +01:00
John Durkop 07cc04a8ad Updates to PSA crypto library based on review comments
Moved new check_crypto_config.h file from include/psa to library
directory and the file is now included from *.c instead of the
crypto_config.h file. Fixed guards in PSA crypto library based
on review comments for new PSA crypto config features.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-11-18 08:09:49 -08:00
Gilles Peskine c70431451b
Merge pull request #3867 from gilles-peskine-arm/error-include-string
Fix several configuration-related issues
2020-11-17 20:32:59 +01:00
John Durkop 6ba40d1faa Corrected guards in PSA library based on review comments
Revised the placement of various new MBEDTLS_PSA_BUILTIN_xxx
guards based on review comments. Corrected guards in psa
test driver to use _ACCEL version instead of _BUILTIN version.
Updated check_config_psa.h to include additional dependency checks
for more algorithms. Renamed some of the new tests to be a little
more clear on the purpose.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-11-12 10:21:40 -08:00
Ronald Cron c3623dbc76 State PSA_CRYPTO_KEY_ID_ENCODES_OWNER and USE_PSA_CRYPTO incompatibility
Code under MBEDTLS_USE_PSA_CRYPTO define is PSA client
code intended to use key identifiers of type psa_key_id_t.
Thus the MBEDTLS_PSA_CRYPTO_KEY_ID_ENCODES_OWNER
configuration option is incompatible with
MBEDTLS_USE_PSA_CRYPTO.

State this in config.h and check_config.h.

As a consequence:
. remove MBEDTLS_PSA_CRYPTO_KEY_ID_ENCODES_OWNER from
  the full configuration, as MBEDTLS_USE_PSA_CRYPTO is
  part of it.

. add a new component in all.sh to keep testing the
  library when MBEDTLS_PSA_CRYPTO_KEY_ID_ENCODES_OWNER
  is set.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-11-10 16:00:41 +01:00
John Durkop 9814fa2b08 Minor updates from review comments
Updated macros in config_psa.h that used ECC_xxx to use KEY_TYPE_ECC_xxx
per comments from review. Implemented a check_config_psa.h to help with
dependency checking of features enabled in config_psa.h. Added
check_config_psa.h to visual studio project.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-11-10 06:11:41 -08:00
John Durkop 7fc75eac21 Enable all features in crypto_config.h
In order to pass existing tests like test_psa_crypto_config_basic
and test_psa_crypto_config_no_driver, all the new features need
to be enabled in the default crypto_config.h file. This change
enables those features by default and updates the other new
tests to compensate for everything being enabled by disabling
some features for some of the tests as needed.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-11-10 06:11:41 -08:00
John Durkop bd069d32e8 Enhanced testing for PSA crypto config features
Updated some of the test names to better reflect what they are testing.
Expanded the testing around RSA feature for PSA crypto config. Updated
the test script to support backing up and restoring the
include/psa/crypto_config.h file so that features can be individually
setup for each unique feature test.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-11-10 06:11:41 -08:00
John Durkop f4c4cb008c Added additional support for ECP for PSA_CRYPTO_CONFIG
The KEY_TYPE_ECC_KEY_PAIR and KEY_TYPE_ECC_PUBLIC_KEY were previously
being guarded by MBEDTLS_ECP_C in the PSA crypto library code. This change
moves it to the new MBEDTLS_PSA_BUILTIN_xxx and separates KEY_PAIR
and PUBLIC_KEY as needed. Tests have also been added to validate the new
settings.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-11-10 06:11:35 -08:00
John Durkop d032195278 Phase 2 support for MBEDTLS_PSA_CRYPTO_CONFIG
This phase adds in support for the following features being
added to the list of features that can be configured in the
include/psa/crypto_config.h header file using the PSA_WANT_ALG_xxx
macros: ECDH, HMAC, HKDF, and RSA. These changes include changes to
the PSA crypto library to use the appropriate new guards that
will allow the feature to be compiled in or out either using
new PSA_WANT_ALG_xxx or the previous MBEDTLS_xxx macros.

For HKDF and HMAC, most of the PSA library code did not have a
specific matching MBEDTLS_xxx macro for that feature, but was instead
using the generic dependent MBEDTLS_MD_C macro. The ECDH and RSA
features more closely aligned with a direct replacement with a similar
macro.

The new tests for RSA, HMAC, and HKDF would normally unset additional
dependent macros, but when attempting to implement that level of
testing it required removal of too many core features like MD_C, PK_C,
ECP_C and other low level features. This may point to additional phases of
work to complete the transition of these features to the new model.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-11-10 06:10:29 -08:00
Gilles Peskine 3809f5f70a Add a build with MBEDTLS_ERROR_STRERROR_DUMMY
Add a build with MBEDTLS_ERROR_STRERROR_DUMMY but not MBEDTLS_ERROR_C.
Previously, both options were enabled by default, but
MBEDTLS_ERROR_STRERROR_DUMMY only matters when MBEDTLS_ERROR_C is
enabled, so its effect was not tested.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-11-09 15:40:05 +01:00
Gilles Peskine 25fdebf0c6 Add missing configs from test-ref-configs.pl
Two sample configuration file were not being tested:
config-no-entropy.h and config-psa-crypto.h. Add them.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-11-09 15:17:07 +01:00
Gilles Peskine 7a78a1f47c Sort entries to make it easier to eyeball the list
No semantic change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-11-09 15:17:07 +01:00
Ronald Cron efcf52d521
Merge pull request #3614 from gilles-peskine-arm/ssl-opt-gnutls-printf-development
Fix printf escape errors in shell scripts
2020-11-04 17:55:42 +01:00
Gilles Peskine 2d0b7231f7
Merge pull request #3712 from jdurkop/psa-crypto-config-ecdsa-3670
Introduce MBEDTLS_PSA_CRYPTO_CONFIG and use it for ECDSA
2020-10-27 17:40:04 +01:00
John Durkop e7012c7725 Cleaned up test_psa_crypto_config_no_driver based on review comments
Removed comment referencing test driver header path and the inclusion of
the test driver directory from the build since it is not required
for that test.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-10-26 09:55:01 -07:00
John Durkop 8ac0b80e9b Rename test_psa_want_ecdsa_disabled_softare
Changed the test name from test_ to build_ to better reflect
what the test is currently performing.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-10-23 01:32:15 -07:00
John Durkop 4377bf7479 Added new PSA crypto config test with no test driver
Added a new test to all.sh to confirm that using MBEDTLS_PSA_CRYPTO_CONFIG
with no test driver and the library is configured with normal configurations
that the test works.

Minor updates to other PSA crypto tests to cleanup msg output for consistency.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-10-23 01:26:57 -07:00
John Durkop 28baa1f141 Fixed test_psa_want_ecdsa_disabled_software to use proper macros
Updated the test_psa_want_ecdsa_disabled_software to enable and disable
the correct macros to accomplish the desired test. The previous version
left out the disabling of additional macros to ensure items related
to MBEDTLS_ECDSA_C were also unset. The test was also missing the setting
of the accelerators MBEDTLS_PSA_ACCEL_ALG_ECDSA and DETERMINISTIC_ECDSA.

With the accelerators enabled the test portion had to be temporarily
disabled until the accelerator code is completed so the test will work
properly.

Updated the signature driver source to fix a compiler warning when
MBEDTLS_ECDSA_C is unset.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-10-23 00:51:52 -07:00
John Durkop 0ea39e0ee4 Correct use of MBEDTLS_PSA_BUILTIN_ALG_xxx in crypto library
The psa crypto library was generically using PSA_WANT_ALG_xxx, but
should have been using the correct MBEDTLS_PSA_BUILTIN_ALG_xxx
definition since that code is the builtin version. There were also
a couple of spots that needed to ensure the code block was enabled
for either ECDSA or DETERMINISTIC_ECDSA, not just one of them.

Fixed all the new ALG_ECDSA_DETERMINISTIC names to be
ALG_DETERMINISTIC_ECDSA instead.

Fixed test to use correct definitions.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-10-15 07:39:02 -07:00
John Durkop 7b453130e2 Updates to fix check-names and all.sh tests after review
Need to make sure the new MBEDTLS_PSA_ACCEL_XXX defines are not checked
since they should be defined by the PSA driver. Updated the list-macros.sh
script to modify the instances of those found to match the corresponding
MBEDTLS_PSA_BUILTIN_XXX that are defined in config_psa.h

Fixed definition of MBEDTLS_PSA_BUILTIN_ALG_ECDSA_DETERMINISTIC, name was
incorrect. Also fixed a missing space in the comments of crypto_config.h

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-10-12 21:56:26 -07:00
John Durkop dd544e1463 Fixed comments and test message output for new PSA tests
Provided a clearer comment for the description for ignoring
MBEDTLS_PSA_CRYPTO_CONFIG for full build setting.

Updated message output for test_psa_crypto_config_basic and
test_psa_want_ecdsa_disabled_software so it is more clear and
concise in what the test is doing. Removed inclusion of the
MBEDTLS_PSA_ACCEL_ALG_ECDSA since it should not have been used
for that particular tests.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-10-12 21:47:19 -07:00
Gilles Peskine ef94c4fcf4
Merge pull request #3474 from ronald-cron-arm/common-mbedtls_param_failed
Common mbedtls_param_failed()
2020-10-06 22:15:42 +02:00
John Durkop 816e043805 Update how test sets PSA_WANT_ALG_*
With the introduction of new tests for PSA crypto config features
the test needs to now be able to set and unset values from the
psa/crypto_config.h file. This change updates from passing the defines
as a build arguement to modifying the header files using config.py.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-10-05 21:18:06 -07:00
Cameron Nemo e18d09de37 fix(programs/psa): commit python3 generated file
Python should not be required for the build when the no_test target is
used. This commit adds the generated file to the source tree and the
check-generated-files script, and removes the generation from (c)make.

Fixes #3524

Signed-off-by: Cameron Nemo <cnemo@tutanota.com>
2020-09-24 20:57:46 -07:00
John Durkop d8959390c5 Add ECDSA support to PSA crypto configuration
Initial changes to PSA crypto core to support configuration
of ECDSA algorithm using PSA crypto configuration mechanism.
Guards using MBEDTLS_ECDSA_C and MBEDTLS_ECDSA_DETERMINISTIC have
been changed to be based off PSA_WANT_ALG_ECDSA and
PSA_WANT_ALG_ECDSA_DETERMINISTIC. Added new tests to all.sh to
confirm new settings are working properly. Current code does not
pass the tests since built in signature verification is not in place.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-09-24 05:01:17 -07:00
Gilles Peskine 6bf4f5fb65
Merge pull request #3585 from jdurkop/fix/undefined-reference-3294
Fix the build with MBEDTLS_USE_PSA_CRYPTO without ECDSA
2020-09-10 21:59:49 +02:00
John Durkop 619e09e705 Minor update to all.sh to change test order
Moved the new component_test_depends_pkalgs_psa to after the
component_test_depends_pkalgs test to be more consistent.

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-09-08 22:19:56 -07:00
John Durkop 2ec2eaac31 Fix test issues with depends-hashes
Needed to make additional fixes so that when MBEDTLS_USE_PSA_CRYPTO
is defined, the depends-hashes test will succeed. There are two
versions of the ecdsa_verify_wrap() function, one with
MBEDTLS_USE_PSA_CRYPTO and when when it is not enabled. The non PSA
version is not using the md_alg parameter since it is not required.
The PSA version was using that parameter to derive a different value
it needed for PSA_ALG_ECDSA. The arguement of PSA_ALG_ECDSA is
ignored for psa_sign_hash and psa_verify_hash. It is present because
it is used and must be a valid hash, not zero, for psa_sign_hash
(but not psa_verify_hash) with PSA_ALG_DETERMINISTIC_ECDSA, and it is
needed for psa_sign_message and psa_verify_message which are not
implemented yet. The local parameter now uses PSA_ALG_ECDSA_ANY for
the verify function to avoid using the md_alg parameter and avoids
returning incorrect error codes.

Fixes #3587

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-09-08 05:40:41 -07:00
Steven Cooreman f1720ea930 Fix macro naming to match inhouse style
Signed-off-by: Steven Cooreman <steven.cooreman@silabs.com>
2020-09-04 14:22:27 +02:00
Steven Cooreman 1cd39d5229 Use own define for building with test drivers
Trying to compile in the PSA accelerator test driver under MBEDTLS_TEST_HOOKS
turned out to be awkward regarding existing builds. We'll put it under a
custom (not in config.h) define instead, since it's something that only
should happen in test.

Signed-off-by: Steven Cooreman <steven.cooreman@silabs.com>
2020-09-04 14:22:27 +02:00
Steven Cooreman 55ae2176ab Add and splice in signature verification through driver
Signed-off-by: Steven Cooreman <steven.cooreman@silabs.com>
2020-09-04 14:22:27 +02:00
Steven Cooreman d57203d955 Add driver tests and run them through all.sh
Signed-off-by: Steven Cooreman <steven.cooreman@silabs.com>
2020-09-04 14:22:27 +02:00
Gilles Peskine aeedd74b42 Pass -std=c99 to arm-none-eabi-gcc
GCC up to 4.x defaults to C89. On our CI, we run the arm-none-eabi-gcc
version from Ubuntu 16.04 on Travis, and that's 4.9, so the gcc-arm
builds started failing on Travis when we introduced a C99 construct in
the configurations that we test on arm on Travis. Other builds, and
Jenkins CI, are not affected because they use GCC 5.x or newer.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-09-02 11:03:04 +02:00
Gilles Peskine 231befab51 Fix printf escape errors in shell scripts
Fix `printf "$foo"` which treats the value of `foo` as a printf format
rather than a string.

I used the following command to find potentially problematic lines:
```
git ls-files '*.sh' | xargs egrep 'printf +("?[^"]*|[^ ]*)\$'
```
The remaining ones are false positives for this regexp.

The errors only had minor consequences: the output of `ssl-opt.sh`
contained lines like
```
Renegotiation: gnutls server strict, client-initiated .................. ./tests/ssl-opt.sh: 741: printf: %S: invalid directive
PASS
```
and in case of failure the GnuTLS command containing a substring like
`--priority=NORMAL:%SAFE_RENEGOTIATION` was not included in the log
file. With the current tests, there was no risk of a test failure
going undetected.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-08-26 22:54:19 +02:00
Janos Follath d2ce916b58 Merge branch 'development-restricted' 2020-08-26 14:15:34 +01:00
Gilles Peskine 9e4d4387f0
Merge pull request #3433 from raoulstrackx/raoul/verify_crl_without_time
Always revoke certificate on CRL
2020-08-26 12:56:11 +02:00
Janos Follath d4ac4e037b
Merge pull request #736 from mpg/cf-varpos-copy-dev-restricted
Constant-flow copy of HMAC from variable position
2020-08-25 14:35:55 +01:00
John Durkop c14be901eb Add new test_depends_curves_psa to all.sh
Add new test (test_depends_curves_psa) to all.sh to confirm
that test is passing when MBEDTLS_USE_PSA_CRYPTO is defined.

Fix #3294

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-08-24 08:24:08 -07:00
John Durkop f35069a82d Fix undefined ref error when ECDSA not defined
Add guards in pk_wrap.c to ensure if ECDSA is not defined, errors
are returned.
Remove warnings in pk.c for unused variables.
Add new test (test_depends_pkalgs_psa) to all.sh to confirm
when USE_PSA_CRYPTO is defined that features are working properly.

Fix #3294

Signed-off-by: John Durkop <john.durkop@fermatsoftware.com>
2020-08-24 08:20:56 -07:00
Dan Handley abccfc1684 Merge development into development-restricted
* development:
  Update copyright notices to use Linux Foundation guidance
  Undef ASSERT before defining it to ensure that no previous definition has sneaked in through included files.
  Add ChangeLog entry for X.509 CN-type vulnerability
  Improve documentation of cn in x509_crt_verify()
  Fix comparison between different name types
  Add test: DNS names should not match IP addresses
  Remove obsolete buildbot reference in compat.sh
  Fix misuse of printf in shell script
  Fix added proxy command when IPv6 is used
  Simplify test syntax
  Fix logic error in setting client port
  ssl-opt.sh: include test name in log files
  ssl-opt.sh: remove old buildbot-specific condition
  ssl-opt.sh: add proxy to all DTLS tests

Signed-off-by: Dan Handley <dan.handley@arm.com>
2020-08-20 11:07:12 +01:00
Manuel Pégourié-Gonnard 73afa37507 Add option to test constant-flow with valgrind
Currently the new component in all.sh fails because
mbedtls_ssl_cf_memcpy_offset() is not actually constant flow - this is on
purpose to be able to verify that the new test works.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-08-20 10:22:41 +02:00
Bence Szépkúti 1e14827beb Update copyright notices to use Linux Foundation guidance
As a result, the copyright of contributors other than Arm is now
acknowledged, and the years of publishing are no longer tracked in the
source files.

Also remove the now-redundant lines declaring that the files are part of
MbedTLS.

This commit was generated using the following script:

# ========================
#!/bin/sh

# Find files
find '(' -path './.git' -o -path './3rdparty' ')' -prune -o -type f -print | xargs sed -bi '

# Replace copyright attribution line
s/Copyright.*Arm.*/Copyright The Mbed TLS Contributors/I

# Remove redundant declaration and the preceding line
$!N
/This file is part of Mbed TLS/Id
P
D
'
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-08-19 10:35:41 +02:00
Raoul Strackx a4e86141f1 Always revoke certificate on CRL
RFC5280 does not state that the `revocationDate` should be checked.

In addition, when no time source is available (i.e., when MBEDTLS_HAVE_TIME_DATE is not defined), `mbedtls_x509_time_is_past` always returns 0. This results in the CRL not being checked at all.

https://tools.ietf.org/html/rfc5280
Signed-off-by: Raoul Strackx <raoul.strackx@fortanix.com>
2020-08-17 09:05:03 +02:00
Manuel Pégourié-Gonnard c4af324a4b Merge branch 'development' into development-restricted
* development: (55 commits)
  Log change as bugfix
  Add changelog entry
  Clarify updates to the persistent state in storage
  With multiple applicable transparent drivers, the order is unspecified
  Minor clarifications
  Give some examples of purpsoses of pure-software transparent driver
  Fix typos
  Add a link to the PSA API specification
  Explain locations vs lifetimes
  Initialize key pointer in ecdh to NULL
  Add buffer zeroization when ecp_write_key fails
  Simplified key slot deletion
  Style fixes
  Use arc4random_buf instead of rand on NetBSD
  Apply review feedback
  Update open question section about public key storage
  Remove the paragraph about declaring application needs
  Change driver persistent data to a callback interface
  Rework and expand key management in opaque drivers
  Fix typos and copypasta
  ...
2020-08-14 10:11:21 +02:00
Gilles Peskine 9ec3648ab3
Merge pull request #3495 from mpg/check-generated-files
Add update option to check-generated-files
2020-08-13 11:24:23 +02:00
Manuel Pégourié-Gonnard 0b2112d304 Add comment on memsan + constant-flow testing 2020-07-27 09:33:49 +02:00
Gilles Peskine d3beca9e38 Test Everest with only Curve25519 enabled
tests/scripts/curves.pl tests the library with a single curve enabled.
This uses the legacy ECDH context and the default ECDH implementation.
For Curve25519, there is an alternative implementation, which is
Everest. Test this. This also tests the new ECDH context, which
Everest requires.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-07-22 03:17:24 +02:00
Gilles Peskine a2611604d4 curves.pl: test with each elliptic curve enabled
Previously curves.pl tested with all elliptic curves enabled except
one, for each curve. This catches tests that are missing dependencies
on one of the curve that they use, but does not catch misplaced
conditional directives around parts of the library.

Now, we additionally test with a single curve, for each curve. This
catches missing or extraneous guards around code that is specific to
one particular curve or to a class of curves.

Signed-off-by: Gilles Peskine <gilles.peskine@arm.com>
2020-07-22 03:17:24 +02:00
Gilles Peskine 7ab66a6bf1 Add missing dependencies for ECDH_xxx key exchanges
ECDH_ECDSA requires ECDSA and ECDH_RSA requires RSA.

Signed-off-by: Gilles Peskine <gilles.peskine@arm.com>
2020-07-22 03:17:23 +02:00
Manuel Pégourié-Gonnard 2774fc45ff Add -u option to check-generated-files.sh
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-07-16 10:54:38 +02:00
Manuel Pégourié-Gonnard 6240defd17 Add MBEDTLS_TEST_CONSTANT_FLOW_MEMSAN
This option allows to test the constant-flow nature of selected code, using
MemSan and the fundamental observation behind ctgrind that the set of
operations allowed on undefined memory by dynamic analysers is the same as the
set of operations allowed on secret data to avoid leaking it to a local
attacker via side channels, namely, any operation except branching and
dereferencing.

(This isn't the full story, as on some CPUs some instructions have variable
execution depending on the inputs, most notably division and on some cores
multiplication. However, testing that no branch or memory access depends on
secret data is already a good start.)

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-07-15 12:26:22 +02:00
Manuel Pégourié-Gonnard 65a6fa3e26 Make cf_hmac() STATIC_TESTABLE
The test function now depends on MBEDTLS_TEST_HOOKS, which is enabled by
config.py full, and since there are already components in all.sh exercising
the full config, this test function is sill exercised even with this new
dependency.

Since this is the first time a test function depends on MBEDTLS_TEST_HOOKS,
fix a bug in check-names.sh that wasn't apparent so far: headers from
library/*.h were not considered when looking for macro definitions. This
became apparent because MBEDTLS_STATIC_TESTABLE is defined in library/common.h
and started being used in library/ssl_msg.c, so was flagged as a likely typo.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-07-15 12:26:21 +02:00
Gilles Peskine 2426506fa0
Merge pull request #3458 from gilles-peskine-arm/analyze_outcomes-count_test_cases-1
Test outcome analysis: check that all available test cases have been executed
2020-07-03 15:12:44 +02:00
Gilles Peskine bbb3664957 Documentation improvements
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-07-03 09:32:50 +02:00
Ronald Cron a123614699 tests: Move mbedtls_param_failed() to test common code
This makes the implementation of mbedtls_param_failed()
for testing purpose available to programs. Thus removing
the ad-hoc implementations in programs.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-07-02 09:59:38 +02:00
Janos Follath be9a5752c2 Merge tag 'mbedtls-2.23.0' into merge-2.23.0-release-to-development
Mbed TLS 2.23.0
2020-07-01 11:23:17 +01:00
Gilles Peskine 961914df12
Merge pull request #3382 from stevew817/feature/volatile-keys-in-SE
Support volatile keys in external SE
2020-06-26 20:27:11 +02:00
Gilles Peskine 3d863f2631 Document the fields of TestCasesOutcomes
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-06-26 18:31:26 +02:00
Gilles Peskine 8d3c70a279 Check test case coverage
Check that every available test case in the test suites and ssl-opt.sh
has been executed at least once.

For the time being, only report a warning, because our coverage is
incomplete. Once we've updated all.sh to have full coverage, this
warning should become an error.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-06-26 18:29:34 +02:00
Gilles Peskine 15c2cbfed5 New script for test outcome analysis
This is a new script designed to analyze test outcomes collected
during a whole CI run.

This commit introduces the script, the code to read the outcome file,
and a very simple framework to report errors. It does not perform any
actual analysis yet.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-06-26 18:29:34 +02:00
Gilles Peskine 6f6ff3346d check_test_cases: move some functions into the logical class
With previous refactorings, some functions are now solely meant to be
called from other functions in a particular class. Move them into this
class.

No behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-06-26 18:29:30 +02:00
Gilles Peskine 78c45dbb0f check_test_cases: move "walk" functions into a class
Make the structure more Pythonic: use classes for abstraction and
refinement, rather than higher-order functions.

Convert walk(function, state, data) into instance.walk(data) where
instance has a method that implements function and state is a field of
instance.

No behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-06-26 18:29:25 +02:00
Gilles Peskine d34e9e450f check_test_cases: parametrize iteration functions by the action
Parametrize the code that iterates over test case descriptions by the
function to apply on each description.

No behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-06-25 16:19:39 +02:00
Gilles Peskine fb4f933f8e Rename Python scripts to use '_' and not '-'
You can't import a Python script whose name includes '-'.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-06-25 14:22:06 +02:00
Gilles Peskine 68a98516f6 basic-in-docker: call all.sh for sanity checks
Call all.sh for sanity checks, rather than maintain an explicit list.
This was done in .travis.yml in 3c7ffd7a40

Travis has diverged from basic-in-docker. This commit updates the
description of basic-in-docker to no longer refer to Travis. Alignment
with Travis may be desirable but that is beyond the scope of this commit.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-06-25 14:22:06 +02:00
Janos Follath 4d1884916b Merge branch 'development-restricted' into mbedtls-2.23.0r0 2020-06-25 09:17:25 +01:00
Gilles Peskine 04c6b61f43
Merge pull request #2639 from mpg/use-all-sh-checks-for-pre-push
Use all.sh and its component list in pre-push hook
2020-06-23 14:37:16 +02:00
Gilles Peskine c96d4e2739
Merge pull request #3410 from mpg/make-coverage-script-deterministic
Make basic-build-test.sh more deterministic
2020-06-22 12:30:48 +02:00
Manuel Pégourié-Gonnard 5430447a6e Adjust comments about SEED synchronisation
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-22 10:31:35 +02:00
Bence Szépkúti c7da1fe381 Add Apache-2.0 headers to all scripts
This commit was generated using the following script:

# ========================
#!/bin/sh

# Find scripts
find -path './.git' -prune -o '(' -name '*.gdb' -o -name '*.pl' -o -name '*.py' -o -name '*.sh' ')' -print | xargs sed -i '

# Remove Mbed TLS declaration if it occurs before the copyright line
1,/Copyright.*Arm/I {
  /This file is part of/,$ {
    /Copyright.*Arm/I! d
  }
}

# Convert non-standard header in scripts/abi_check.py to the format used in the other scripts
/"""/,/"""/ {

  # Cut copyright declaration
  /Copyright.*Arm/I {
    h
    N
    d
  }

  # Paste copyright declaration
  /"""/ {
    x
    /./ {
      s/^/# /    # Add #
      x          # Replace orignal buffer with Copyright declaration
      p          # Print original buffer, insert newline
      i\

      s/.*//     # Clear original buffer
    }
    x
  }
}

/Copyright.*Arm/I {

  # Print copyright declaration
  p

  # Read the two lines immediately following the copyright declaration
  N
  N

  # Insert Apache header if it is missing
  /SPDX/! {
    i\
# SPDX-License-Identifier: Apache-2.0\
#\
# Licensed under the Apache License, Version 2.0 (the "License"); you may\
# not use this file except in compliance with the License.\
# You may obtain a copy of the License at\
#\
# http://www.apache.org/licenses/LICENSE-2.0\
#\
# Unless required by applicable law or agreed to in writing, software\
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT\
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\
# See the License for the specific language governing permissions and\
# limitations under the License.

    # Insert Mbed TLS declaration if it is missing
    /This file is part of/! i\
#\
# This file is part of Mbed TLS (https://tls.mbed.org)
  }

  # Clear copyright declaration from buffer
  D
}
'
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-15 12:05:47 +02:00
Bence Szépkúti 700ee44545 Add missing copyright dates to scripts and sources
To find any files with a missing copyright declaration, use the following script:

# ========================
#!/bin/sh

# Find files with copyright declarations, and list their file extensions
exts=$(grep -Ril --exclude-dir .git --exclude-dir 3rdparty\
                 --exclude-dir programs/fuzz 'Copyright.*Arm' | sed '
  s/.*\///
  s/.*\./*./
  s/.*/-name "&"/
' | sort -u | sed -n '
  :l
    N
    $!bl
  s/\n/ -o /gp
')

# Find files with file extensions that ususally include copyright extensions,
# but don't include a copyright declaration themselves.
eval "find\
  '(' -path './.git' -o -path './3rdparty' -o -path './programs/fuzz' ')' -prune\
  -o ! -path './tests/data_files/format_pkcs12.fmt'\
     ! -path './programs/psa/psa_constant_names_generated.c'\
     '(' $exts ')' -print" | xargs grep -Li 'Copyright.*Arm'
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-15 12:05:46 +02:00
danh-arm 15fee93121
Merge pull request #3363 from bensze01/zeroize
Remove hardcoded line number from the zeroize test
2020-06-10 11:31:38 +01:00
Bence Szépkúti 5620d71d58 Remove hardcoded line number from the zeroize test
Instead, we insert a comment containing GDB_BREAK_HERE in the line we
want to break at, and let the gdb script search for it.

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-09 12:52:04 +02:00
Steven Cooreman 8335f41cda Enable figuring out number of cores when running on OS X
Signed-off-by: Steven Cooreman <steven.cooreman@silabs.com>
2020-06-08 18:20:08 +02:00
Manuel Pégourié-Gonnard e050191ef5 Make basic-build-test.sh deterministic
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-08 13:04:10 +02:00
Manuel Pégourié-Gonnard 304b099534 all.sh: clean up some uses of "local" variables
While pure sh doesn't have a concept of local variables, we can partially
emulate them by unsetting variables before we exit the function, and use the
convention of giving them lowercase names to distinguish from global
variables.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-08 11:01:59 +02:00
Manuel Pégourié-Gonnard f1f180a6a1 all.sh: keep dd output in non-quiet mode
Since dd prints everything on stderr, both normal status update and actual
errors when they occur, redirecting that to /dev/null is a trade-off that's
acceptable in quiet mode (typically used on a developer's machine and the
developer will re-run in non-quiet mode if anything fails without sufficient
detail in the output), but not that much in non-quiet mode.

For example, if our dd invocation fails because the disk in full on a CI
machine, we want the error to be reported at the time we invoke dd, and not
later when a seemingly unrelated test fails due to an incorrect seedfile.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-08 10:46:35 +02:00
Manuel Pégourié-Gonnard 21b3d12066
Merge pull request #3374 from danh-arm/dh/branch-cov
Enable branch coverage in basic_build_test.sh
2020-06-08 10:15:06 +02:00
Manuel Pégourié-Gonnard 53fb66db12 Add support for RESTARTABLE with internal RNG
Currently we draw pseudo-random numbers at the beginning and end of the main
loop. With ECP_RESTARTABLE, it's possible that between those two occasions we
returned from the multiplication function, hence lost our internal DRBG
context that lives in this function's stack frame. This would result in the
same pseudo-random numbers being used for blinding in multiple places. While
it's not immediately clear that this would give rise to an attack, it's also
absolutely not clear that it doesn't. So let's avoid that by using a DRBG
context that lives inside the restart context and persists across
return/resume cycles. That way the RESTARTABLE case uses exactly the
same pseudo-random numbers as the non-restartable case.

Testing and compile-time options:

- The case ECP_RESTARTABLE && !ECP_NO_INTERNAL_RNG is already tested by
  component_test_no_use_psa_crypto_full_cmake_asan.
- The case ECP_RESTARTABLE && ECP_NO_INTERNAL_RNG didn't have a pre-existing
  test so a component is added.

Testing and runtime options: when ECP_RESTARTABLE is enabled, the test suites
already contain cases where restart happens and cases where it doesn't
(because the operation is short enough or because restart is disabled (NULL
restart context)).

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-08 09:09:20 +02:00
Manuel Pégourié-Gonnard 1a3f9edc08 Add config.h option MBEDTLS_ECP_NO_INTERNAL_RNG
No effect so far, except on dependency checking, as the feature it's meant to
disable isn't implemented yet (so the descriptions in config.h and the
ChangeLog entry are anticipation for now).

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-08 09:09:20 +02:00
Manuel Pégourié-Gonnard 5b942dc45e Add test for dependencies on HMAC_DRBG in all.sh
Similarly to the recently-added tests for dependencies on CTR_DRBG:
constrained environments will probably want only one DRBG module, and we
should make sure that tests pass in such a configuration.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-05 09:29:51 +02:00
Janos Follath bba4c17b7a
Merge pull request #3315 from hanno-arm/tls13-experimental-macro
Add support for TLS 1.3 record protection routines
2020-06-04 15:51:54 +01:00
Manuel Pégourié-Gonnard 6abc20e0e3
Merge pull request #3378 from mpg/fix-ctr-drbg-deps
Fix undeclared dependencies on CTR_DRBG (and add test)
2020-06-03 10:55:52 +02:00
Manuel Pégourié-Gonnard a9119167e0 Make component_check_test_cases more -q frienly
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-02 11:52:14 +02:00
Manuel Pégourié-Gonnard dfb114a843 Make check_generate_test_code more -q friendly
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-02 11:52:14 +02:00
Manuel Pégourié-Gonnard 2b2bdaa793 Add a --quiet option to all.sh
The primary purpose is to use it to run all.sh -k -q in the pre-push hook, but
this can be useful in any circumstance where you're not interested in the full
output from each component and just want a short summary of which components
were run (and if any failed).

Note that only stdout from components is suppressed, stderr is preserved so
that errors are reported. This means components should avoid printing to
stderr in normal usage (ie in the absence of errors).

Currently all the `check_*` components obey this convention except:
- check_generate_test_code: unittest prints progress to stderr
- check_test_cases: lots of non-fatal warnings printed to stderr

These components will be fixed in follow-up commits.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-02 11:52:14 +02:00
Manuel Pégourié-Gonnard bf7ae6fb25 Silence dd invocation in all.sh
It brings no value and distracts us from the actual content.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-02 11:19:09 +02:00
Manuel Pégourié-Gonnard 817e368dfd Add test for building without CTR_DRBG
People who prefer to rely on HMAC_DRBG (for example because they use it for
deterministic ECDSA and don't want a second DRBG for code size reasons) should
be able to build and run the tests suites without CTR_DRBG.

Ideally we should make sure the level of testing (SSL) is the same regardless
of which DRBG modules is enabled, but that's a more significant piece of work.
For now, just ensure everything builds and `make test` passes.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-05-29 11:50:53 +02:00
Dan Handley a8b26c2ae4 Enable branch coverage in basic_build_test.sh
Enable branch coverage output in basic_build_test.sh. This
includes enabling branch coverage output to the lcov make target,
which is disabled by default.

Signed-off-by: Dan Handley <dan.handley@arm.com>
2020-05-28 17:51:46 +01:00
Gilles Peskine eca95db763 Finish the documentation of normalize_path
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:19:20 +02:00
Hanno Becker a711f6e277 Add 'build+unit test' test for experimental TLS 1.3 code to all.sh
Signed-off-by: Hanno Becker <hanno.becker@arm.com>
2020-05-28 10:32:23 +01:00
Gilles Peskine 30e0bb4a24 Run assemble_changelog.py in all.sh
Avoid nasty surprises where it would fail when we want to make a release.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:55:10 +02:00
Gilles Peskine 344da1cbd3 Some .pem files are openssl output and have tabs and that's ok
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:55:10 +02:00
Gilles Peskine d2df86f005 .dsw files are Visual Studio stuff
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:55:09 +02:00
Gilles Peskine 12b180a0b9 Permit empty files
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:55:09 +02:00
Gilles Peskine 3e2ee3cedc Check only files checked into Git
We're only interested in files that are committed and pushed to be
included in Mbed TLS, not in any other files that may be lying around.
So ask git for the list of file names.

This script is primarily intended to run on the CI, and there it runs
on a fresh Git checkout plus potentially some other checkouts or
leftovers from a previous part of the CI job. It should also run
reasonably well on developer machines, where there may be various
additional files. In both cases, git is available.

Ad hoc directory exclusions are no longer needed.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:53:32 +02:00
Gilles Peskine d4a853dbd7 Exclude binary files from text checks
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:53:32 +02:00
Gilles Peskine 0598db84c3 Regex mechanism for check-specific exemptions
Suffixes are convenient but not always sufficient.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:53:32 +02:00
Gilles Peskine c1d1b669db Check all files by default
Have an explicit list of exemptions for specific checks rather than
whitelisting files to check. Some checks, such as permissions, should
apply to all files.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:53:32 +02:00
Gilles Peskine 05a51a8a72 More accurate variable name
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:53:32 +02:00
Gilles Peskine d9f694960f
Merge pull request #3190 from gilles-peskine-arm/config-full-clarify-development
Clarify that the full config enables everything that can be tested together
2020-05-04 12:29:09 +02:00
Gilles Peskine 18487f62d8 all.sh: on arm builds (GCC or Arm Compiler), show the code size
Just show the code size in the logs, for human consumption.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 23:14:08 +02:00
Gilles Peskine 6e2fb86c1e all.sh: add a Cortex-M0+ build
It's pretty fast and adds a little variety.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 23:14:08 +02:00
Gilles Peskine 6537588d76 all.sh: build_arm_none_eabi_gcc: do optimize
Otherwise the bignum assembly code is not used.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 23:14:04 +02:00
Gilles Peskine 6d06134e93 Rename --arm-gcc-prefix to --arm-none-eabi-gcc-prefix
This is supposed to be for GCC (or a compiler with a compatible
command line interface) targeting arm-none-eabi, so name it
accordingly.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 18:41:35 +02:00
Gilles Peskine 8386ea22b2 all.sh: explain the testing around deprecated features
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 09:07:29 +02:00
Gilles Peskine af387e0ce1 check-files: support Windows .bat files
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-26 00:43:56 +02:00
Gilles Peskine 60f267bb1e all.sh: make the arm-gcc cross-compiler prefix configurable
Make it possible to use a compiler that isn't in $PATH, or that's
installed with a different name, or even a compiler for a different
target such as arm-linux-gnueabi.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-25 22:34:28 +02:00
Gilles Peskine 97bea01ff4 all.sh: run selftest in the full config and with ASan
Almost everything the selftest program does is in the test suites. But
just in case run the selftest program itself once in the full
configuration, and once in the default configuration with ASan, in
addition to running it out of box.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-25 22:34:23 +02:00
Gilles Peskine 30de2e84ef Make no_deprecated naming more consistent
Use "no_deprecated" both in the name of the configuration and in the
name of all.sh components, rather than a mixture of "no_deprecated"
and "non_deprecated".

Make all.sh component names more consistent.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-20 21:39:22 +02:00
Gilles Peskine 1093d9f9af all.sh: reorganize testing around deprecated features
build_deprecated combined the testing of deprecated features, and
testing of the build without deprecated features. Also, it violated the
component naming convention by being called build_xxx but running tests.

Replace it by:
* test_default_no_deprecated: check that you can remove deprecated
  features from the default build.
* test_full_no_deprecated: check that the library builds when deprecated
  features are disabled (and incidentally that the tests run).
* test_no_deprecated_warning: check that there are no warnings when
  deprecated features are disabled and MBEDTLS_DEPRECATED_WARNING
  is enabled.
* test_deprecated: test the deprecated features.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-20 15:46:56 +02:00
Gilles Peskine 3a584aecca Enable SSLv3 in the full config
It's deprecated, but not otherwise counter-indicated for the full
config: it doesn't conflict with anything and enabling it doesn't make
testing harder (especially since it defaults off in compat.sh).

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-20 15:39:36 +02:00
Gilles Peskine e094a18f6b Strict C99: check it in the full config
Ensure that there is a build with -pedantic in the full config, not
just in "exotic" configurations.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-20 15:39:36 +02:00
Gilles Peskine dc6d838a73 Enable MBEDTLS_PSA_CRYPTO_SE_C in config full
It started out as be experimental, but it is now robust enough not to
break the rest, so there's no reason to leave it out.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-20 15:39:32 +02:00
Gilles Peskine 5a2710e9af
Merge pull request #3120 from gilles-peskine-arm/check-windows-files
Check Windows files for sanity as well
2020-04-20 13:59:27 +02:00
Gilles Peskine 5c34ee14d3
Merge pull request #3118 from gilles-peskine-arm/pylint-up-to-2.4-development
Pass Pylint up to 2.4
2020-04-20 09:47:40 +02:00
Piotr Nowicki 9978e6ee14 Add tests for the ssl_context_info program
Signed-off-by: Piotr Nowicki <piotr.nowicki@arm.com>
2020-04-15 16:21:36 +02:00
Gilles Peskine 32e889dfc3 Document and fix the MBEDTLS_xxx_ALT logic for the full config
The intended logic around MBEDTLS_xxx_ALT is to exclude them from full
because they require the alternative implementation of one or more
library functions, except that MBEDTLS_PLATFORM_xxx_ALT are different:
they're alternative implementations of a platform function and they
have a built-in default, so they should be included in full. Document
this.

Fix a bug whereby MBEDTLS_PLATFORM_xxx_ALT didn't catch symbols where
xxx contains an underscore. As a consequence,
MBEDTLS_PLATFORM_GMTIME_R_ALT and MBEDTLS_PLATFORM_NV_SEED_ALT are now
enabled in the full config. Explicitly exclude
MBEDTLS_PLATFORM_SETUP_TEARDOWN_ALT because it behaves like the
non-platform ones, requiring an extra build-time dependency.
Explicitly exclude MBEDTLS_PLATFORM_NV_SEED_ALT from baremetal
because it requires MBEDTLS_ENTROPY_NV_SEED, and likewise explicitly
unset it from builds that unset MBEDTLS_ENTROPY_NV_SEED.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-14 21:23:48 +02:00
Gilles Peskine e0c84ac4d2 Pylint: explicitly note why we're doing an unchecked subprocess.run
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:23:22 +02:00
Gilles Peskine 8b022359e8 Pylint: minor code simplifications
Simplify the code in minor ways. Each of this changes fixes a warning
from Pylint 2.4 that doesn't appear with Pylint 1.7.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:23:22 +02:00
Gilles Peskine 184c096e95 Pylint: abide by useless-object-inheritance warnings
Inheriting from object is a remainder of Python 2 habits and is just
clutter in Python 3.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:23:22 +02:00
Gilles Peskine dd4c1c6fe7 mbedtls_test.py: drop compatibility with Python 2
Python 2 is no longer supported upstream. Actively drop compatibility
with Python 2.

Removing the inheritance of a class on object pacifies recent versions
of Pylint (useless-object-inheritance).

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:23:22 +02:00
Gilles Peskine aaee444c68 Document more methods in Python scripts
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:23:22 +02:00
Gilles Peskine c877c24ed0 Enable extra features that have tests
The "full" configuration excludes some deprecated or experimental
features. Enable the ones that have tests, don't have extra
requirements and don't turn off some other feature.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-10 11:33:52 +02:00
Gilles Peskine b3d0bab10b MBEDTLS_MEMORY_BACKTRACE is no longer included in the full config
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-10 11:33:52 +02:00
Gilles Peskine 40be51ffe3 Add a few echo statements to make the logs easier to follow
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-10 11:33:50 +02:00
Gilles Peskine bfcb6e16ab Create a large enough seedfile
The seedfile needs to have the size of the entropy accumulator, which
is 64 bytes (512 bits) since the entropy accumulator uses SHA-512 and
the seed size needs to be the same as the hash output (or larger).

We used to enable MBEDTLS_ENTROPY_FORCE_SHA256 in the full config, so
the entropy accumulator was 256 bits (32 bytes), and therefore a
32-byte seedfile worked. But we no longer turn on this option in the
full config, so the 32-byte seedfile no longer works.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-10 11:30:09 +02:00
Gilles Peskine 5757d54261 If 'make lcov' failed, exit immediately
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-10 11:30:02 +02:00
Gilles Peskine ca51b474dc Note that we keep going even if some tests fail
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-10 11:30:00 +02:00
Gilles Peskine 6d6ee98c14 Exit with a failure status if some tests failed
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-09 18:28:14 +02:00
Gilles Peskine d703a2ee19 In Windows files, detect CR without LF as well as LF without CR
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-01 13:37:16 +02:00
Gilles Peskine 545e13f4c1 Check that Windows files have Windows line endings
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-01 11:09:52 +02:00
Gilles Peskine 2c61873a0d Also check Windows files
Check Windows files for some issues, including permissions. Omit the
checks related to special characters (whitespace, line endings,
encoding) as appropriate.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-01 11:09:52 +02:00
Gilles Peskine 5308f12fc6 Sort the list for easier maintenance
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-01 11:09:50 +02:00
Gilles Peskine 6a45d1e4ce List each item on a separate line for easier maintenance
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-01 11:08:46 +02:00
Gilles Peskine 6e8d5a00b2 Clarify confusion between file names and suffixes of file names
To test a file name exactly, prepend a / to the base name.

files_to_check actually checks suffixes, not file names, so rename it
to extensions_to_check.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-01 11:06:17 +02:00
Gilles Peskine 13c95c4d74 Make check_python_files non-optional in all.sh
check_python_files was optional in all.sh because we used to have CI
machines where pylint wasn't available. But this had the downside that
check_python_files kept breaking because it wasn't checked in the CI.
Now our CI has pylint and check_python_files should not be optional.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-24 19:25:34 +01:00
Gilles Peskine 56e99d623d Make sure to use a Python 3 pylint
On some systems, such as Ubuntu up to 19.04, `pylint` is for Python 2
and `pylint3` is for Python 3, so we should not use `pylint` even if
it's available.

Use the Python module instead of the trivial shell wrapper. This way
we can make sure to use the correct Python version.

Fix #3111

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-24 19:23:41 +01:00
Gilles Peskine 5e7d6fd240 Merge 'mbedtls/development' into merge-crypto-unremoved-20200304
Merge the latest state of the target branch (mbedtls/development) into the
pull request to merge mbed-crypto into mbedtls.

Conflicts:

* ChangeLog: add/add conflict. Resolve by using the usual section order.
2020-03-23 18:02:07 +01:00
Gilles Peskine 7415f2fd12 Add comment to help syntax highlighting in editors
Restore this change which was accidentally reverted during the merge
of Mbed Crypto.
2020-03-23 17:55:32 +01:00
Gilles Peskine aae57bffd9 Remove remaining references to the crypto subdirectory 2020-03-23 17:55:31 +01:00
Gilles Peskine 4e616f893b Remove config-default.h
This file was the default mbedtls configuration in the mbedtls-psa
fork. It is no longer relevant.
2020-03-23 17:55:31 +01:00
Gilles Peskine a57a80e213 Restore full tls coverage to remaining scripts
Look for any other invocation of test scripts that was removed: look for a change that removes a line (`^-.*…`) containing one of the names of a test script (without its path because it may be used with a relative path). Look for `ssl-opt.sh` and `compat.sh`, as well any file in `tests/scripts` that only exists in tls.
```
git diff 'HEAD^{/^Merge}~1' HEAD --diff-filter=M -- . ':!library/error.c' ':!library/version_features.c' ':!programs/test/query_config.c' ':!visualc' ':!*.pdf' ':!*.der' | grep -E "^-.*($(comm -23 <(git ls-tree -r --name-only 'HEAD^{/^Merge}~1') <(git ls-tree -r --name-only $(git merge-base upstream-crypto/development 'HEAD^{/^Merge}^2')) | sed -n 's!^tests/scripts/!!p' | sed 's/\./\\./g' | tr '\n' '|')ssl-opt\.sh|compat\.sh)" | grep -v '^---'
```
This only turns up changes in `basic-in-docker.sh`.
2020-03-23 17:55:31 +01:00
Gilles Peskine 636c26ad75 Restore full tls coverage to all.sh
The merge of mbed-crypto removed some tls coverage. Restore it. Also
remove references to the `crypto` subdirectory brought by the mbedtls
side of the merge. In more detail:

* `tests/scripts/all.sh`:
    * `fuzz` in comments (×2): restore it.
    * `CTEST_OUTPUT_ON_FAILURE=1`: don't remove it.
    * `cd crypto` for `make clean`: don't restore it.
    * `cleanup`: do restore `programs/fuzz/Makefile`. Don't go into `crypto`. Keep only one copy of the calls to `rm` in `cmake_subproject`.
    * Comment legacy options: don't remove it.
    * `crypto/Makefile` and `pre_check_seedfile`: don't restore either. See below regarding the lack of need for `pre_check_seedfile`.
    * blank line in `pre_print_configuration`: restore it.
    * blank line before `#### Build and test`: restore it.
    * SSL tests in `component_test_full_cmake_gcc_asan` and zlib components: restore it.
    * `component_test_no_pem_no_fs` (×2): the merge placed two copies in different locations. Reconcile them: unset PSA storage like in crypto, and call `ssl-opt.sh` like in tls. Put the merged version at the tls location.
    * `component_test_everest`: do add it at the tls location.
    * `component_test_small_mbedtls_ssl_dtls_max_buffering`: restore the tls value.
    * `component_test_new_ecdh_context`…: move `component_test_new_ecdh_context` before `component_test_everest` and add a calls to `compat.sh` and `ssl-opt.sh` like in `component_test_everest`. Remove the redundant crypto-only `component_test_everest`. Don't remove `component_test_psa_collect_statuses`.
    * `component_test_full_cmake_clang`: don't remove `clang` in the `msg` call. Don't remove the call to `test_psa_constant_names.py`.
    * `component_test_full_make_gcc_o0`: remove it. It's subsumed by `component_test_gcc_opt`.
    * `component_build_deprecated`: don't remove anything.
    * `component_test_memory_buffer_allocator`: restore `ssl-opt.sh`.
    * `component_test_when_no_ciphersuites_have_mac`: restore it.
    * `component_test_platform_calloc_macro`: don't restore `unset MBEDTLS_MEMORY_BUFFER_ALLOC_C` which is now redundant. Don't restore explicit flags instead of `$ASAN_CFLAGS`.
    * `component_test_aes_fewer_tables`…: don't remove it.
    * `component_test_m32_o1`: restore SSL testing.
    * `component_test_m32_everest`: restore SSL testing.
    * `component_test_min_mpi_window_size`…: don't remove it.
    * `component_test_valgrind`: do restore the tls version of the comment.
    * `run_component`: don't remove the seedfile creation. This is better than `pre_check_seedfile` (see below).
    * `pre_check_seedfile`: don't restore it. `pre_check_seedfile` (from tls) creates a seedfile once and for all. This is not good enough if a component fails in such a way as to leave a broken seedfile, or if a component leaves a seedfile with a size that's wrong for the next component to run. Instead (from crypto), `run_component` creates a sufficiently large seedfile before each component.
2020-03-23 17:55:31 +01:00
Gilles Peskine b99bd39b4e Merge mbed-crypto into mbedtls: the merge commit
Merge `unremove-non-crypto` into `mbedtls/development`. The branch
`unremove-non-crypto` was obtained by starting from `mbed-crypto/development`,
then reverting many commits that removed X.509 and TLS functionality when Mbed
Crypto forked from Mbed TLS (the “unremoval”), then make a few tweaks to
facilitate the merge.

The unremoval step restored old versions of some tls files. If a file doesn't
exist in mbed-crypto, check out the mbedtls version, regardless of what
happened during the unremoval of tls files in the crypto tree. Also
unconditionally take the mbedtls version of a few files where the
modifications are completely project-specific and are not relevant in
mbed-crypto:

* `.github/issue_template.md`: completely different. We may want to reconcile
  them independently as a follow-up.
* `.travis.yml`: would only be reverted to an earlier tls version.
* `README.md`: completely different. We may want to reconcile them
  independently as a follow-up.
* `doxygen/input/doc_mainpage.h`: the changes in crypto were minimal and not
  relevant except as a stopgap as mbed-crypto did not have its own product
  versioning in the Doxygen documentation.
* `tests/.jenkins/Jenkinsfile`: completely different.
* `tests/data_files/Makefile`: there were no changes in mbed-crypto,
  but the unremoval step restored an old version.

Shell script for everything to do after the merge apart from the conflict
resolution:
```
tls_files=($(comm -23 <(git ls-tree -r --name-only HEAD) <(git ls-tree -r --name-only $(git merge-base upstream-crypto/development MERGE_HEAD))))
tls_files+=($tls_files .github/issue_template.md .travis.yml README.md doxygen/input/doc_mainpage.h tests/.jenkins/Jenkinsfile tests/data_files/Makefile)
git checkout --theirs HEAD -- $tls_files
git add -- $tls_files
```

Resolve the remaining conflicts:

* `library/CMakeLists.txt`:
    * Keep the TLS definition of `src_crypto`
    * `USE_SHARED_MBEDTLS_LIBRARY`: keep all three libraries, with both
      `include` and `crypto/include` in `target_include_directories`, all with
      version `2.21.0`.
* `programs/Makefile`:
    * Reconcile the APPS lists (add/add from a differently-formatted common
      ancestor): insert the `psa/*` from crypto into the tls list.
    * Keep the `fuzz` target defined only in tls version.
    * Keep the recipe (only in tls version) cleaning `ssl_pthread_server`
      stuff for the `clean` target.
* `scripts/config.py`:
    * `include_in_full`: add/add conflict. Keep both.
* `tests/scripts/all.sh`:
    * `component_test_no_use_psa_crypto_full_cmake_asan`: partially old
      version in crypto. Take the tls version.
    * `component_test_malloc_0_null` and more: take
      `component_test_malloc_0_null` from crypto (with `config.py` rather than
      `config.pl`, and with `$ASAN_FLAGS` rather than an explicit list), but
      add the call to `ssl-opt.sh` from tls. Take the other components from
      crypto.

With this commit, building and running the unit tests with both `make ` and
`cmake` work in the default configuration on Linux. Other platforms, build
systems and configurations are likely not to work, and there is some
regression in test coverage.

There is some loss of functionality because the unremoval step restored older
versions of tls content. This commit contains the latest tls version of
tls-only files, but some changes from the tls side in files that existed on
both sides have regressed. Most problematic changes are hunks that remove some
tls-specific feature and contain either a C preprocessor symbol identifying a
tls-specific module or option, or the name of a tls-specific file. Hunks
that remove a tls-specific preprocessor symbol can be identified with the
regular expression `^-.*MBEDTLS_(ERR_)?(PKCS11|X509|NET|SSL)_`.

Subsequent commits will revert a few parts of the patch from this merge commit
in order to restore the tls functionality that it removes, ensure that the
test coverage includes what was covered in either branch, and fix test
failures.
2020-03-23 17:54:46 +01:00
Gilles Peskine 3b46cd3f15 Invoke config.py instead of config.pl in reverted content
perl -i -pe 's/\bconfig\.pl/config.py/g' $(git grep -l -Fw config.pl
-- . '#!tests/scripts/test_config_script.py')
2020-03-19 14:23:45 +01:00
Gilles Peskine 69e8f7ffe3 Revert "all.sh: Remove dependency on TLS, NET, and X.509"
This reverts commit 9b90f2e294.

Conflicts:
* tests/scripts/all.sh: do the same changes, dancing around the new
  outcome file feature and components added in the same places.
  Make sure that the components that are getting added back are at the
  same locations as where they are now in mbedtls.
2020-03-19 14:23:45 +01:00
Gilles Peskine 920b77524d Revert "recursion.pl: Don't depend on X.509"
This reverts commit e23737c618.
2020-03-19 14:23:45 +01:00
Gilles Peskine 7dc97048d6 Revert "Remove tests that depend on TLS or X.509"
This reverts commit 9afb2e9921.

Conflicts:
* include/CMakeLists.txt
  * "Make config.h available" comment: there has been a change
    adjacent to where it was removed. Just re-add what was removed.
* tests/CMakeLists.txt:
  * compat.sh: there has been a change immediately before where it was
    removed. Just re-add what was removed.
2020-03-19 14:17:54 +01:00
Gilles Peskine 12230eb5c8 Revert "Remove irrelevant configs"
This reverts commit 1ad37309e4.

Conflicts:
* tests/scripts/test-ref-configs.pl:
    * config-ccm-psk-tls1_2.h: there has been an addition of
      "config-symmetric-only.h" at the place where the configurations
      that are added back were removed. Keep this configuration.
2020-03-19 14:04:59 +01:00
Gilles Peskine 9d28c426d1 Revert "check-names: Enable referencing Mbed TLS macros"
This reverts commit 7fcc7bc576.
2020-03-19 13:55:04 +01:00
Gilles Peskine 4e1174967a Revert "config: Remove TLS and NET options"
This reverts commit 1c66e48670.

Conflicts:
* include/mbedtls/check_config.h:
    * MBEDTLS_SSL_PROTO_SSL3: there has been an addition (of
      MBEDTLS_SHA512_NO_SHA384) at the place where it was removed. Re-add it
      after (alphabetical order).
    * MBEDTLS_ENABLE_WEAK_CIPHERSUITES: there has been an addition (of
      MBEDTLS_CTR_DRBG_USE_128_BIT_KEY) at the place where it was removed.
      Re-add it after (alphabetical order).
    * MBEDTLS_SSL_ALL_ALERT_MESSAGES: there has been an addition (of
      MBEDTLS_SHA512_SMALLER) at the place where it was removed. Re-add it
      after (alphabetical order).
* include/mbedtls/config.h:
    * MBEDTLS_ENABLE_WEAK_CIPHERSUITES: there has been an addition (of
      MBEDTLS_CTR_DRBG_USE_128_BIT_KEY) at the place where it was removed.
      Re-add it after (alphabetical order).
    * MBEDTLS_SSL_ALL_ALERT_MESSAGES: there has been an addition (of
      MBEDTLS_SHA512_SMALLER) at the place where it was removed. Re-add it
      after (alphabetical order).
* library/version_features.c: re-generate by running
  scripts/generate_features.pl.
* programs/test/query_config.c: re-generate by running
  scripts/generate_query_config.pl.
* scripts/config.pl: this file has been replaced by config.py. Port
  the reversed changes to config.py:
    * Revert removing three symbols from the list of symbols to
      exclude from full.
    * Revert removing one symbol (MBEDTLS_NET_C) from the list of symbols
      to exclude from baremetal.
* scripts/footprint.sh:
    * Re-add the line to unset MBEDTLS_NET_C, but with config.py instead of
      config.pl.
* tests/scripts/all.sh:
    * component_test_no_platform: re-add the line to unset MBEDTLS_NET_C, but
      with config.py instead of config.pl.
    * component_build_arm_none_eabi_gcc,
    component_build_arm_none_eabi_gcc_no_udbl_division,
    component_build_arm_none_eabi_gcc_no_64bit_multiplication,
    component_build_armcc: these components now use the baremetal
    configuration, so they do not need to turn off MBEDTLS_NET_C explicitly.
2020-03-19 13:55:04 +01:00
Gilles Peskine 252e391cca Revert "config: Remove X.509 options"
This reverts commit bb1f701212.

* include/mbedtls/check_config.h:
    * MBEDTLS_X509_RSASSA_PSS_SUPPORT: there has been an addition (of
      MBEDTLS_SHA512_NO_SHA384) at the place where it was removed.
      Re-add it before MBEDTLS_SHA512_NO_SHA384 to keep it grouped
      with MBEDTLS_RSA_C.

Conflicts:
* scripts/config.pl: this file has been replaced by config.py. Port
  the reversed changes to config.py:
    * Revert removing three symbols from the list of symbols to
      exclude from full.
2020-03-19 13:53:18 +01:00
Simon Butcher e30d03e4f4 Change the use of pylint to optionally use pylint3
Pylint when installed as a distro package can be installed as pylint3, whilst as
a PEP egg, it can be installed as pylint.

This commit changes the scripts to first use pylint if installed, and optionally
look for pylint3 if not installed. This is to allow a preference for the PEP
version over the distro version, assuming the PEP one is more likely to be
the correct one.

Signed-off-by: Simon Butcher <simon.butcher@arm.com>
2020-03-16 11:38:02 +00:00
Manuel Pégourié-Gonnard 21d1cbccda
Merge pull request #2262 from andresag01/iotssl-2544-deprecate-record-accel
Fix compilation failure when MBEDTLS_SSL_HW_RECORD_ACCEL is enabled
2020-03-16 10:37:16 +01:00
Jaeden Amero c31f970a46
Merge pull request #3075 from AndrzejKurek/variable-buffer-size
Variable buffer size
2020-03-10 21:46:35 +04:00
Gilles Peskine 84a63fad5b Revert "tests: Update generator with Mbed Crypto comments"
This reverts commit dfcf84aea5.
2020-03-04 15:39:14 +01:00
Piotr Nowicki 0937ed29b9
Add an acceptance test for memory usage after handshake
Signed-off-by: Piotr Nowicki <piotr.nowicki@arm.com>
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2020-03-04 09:28:35 -05:00
Manuel Pégourié-Gonnard 68192fcd3a Fix remaining occurrences of config.pl in all.sh
The .pl version is now a compat wrapper around the .py script. Better call the
.py script directly.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-03-04 10:53:13 +01:00
Darryl Green aad82f9bbb
Add variable buffer length tests to all.sh
Exercise the feature alone, with record splitting and DTLS connection ID.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
Signed-off-by: Darryl Green <darryl.green@arm.com>
2020-03-03 10:44:57 -05:00
Gilles Peskine 26e4fdc6cd Move MEMORY_BUFFER_ALLOC components to align the order with mbedtls
No code change. This commit just moves two functions to make the order
of component definitions match the one in mbedtls.
2020-03-02 21:15:04 +01:00
Manuel Pégourié-Gonnard dd8807f52d Add build with MBEDTLS_SSL_HW_RECORD_ACCEL to all.sh
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-02-26 09:56:27 +01:00
Gilles Peskine 9ab9621511 Move 3rdparty mentions to a separate line
This makes it easier to merge changes related to adding or removing
3rdparty items.

No semantic change.
2020-02-19 20:13:29 +01:00
Gilles Peskine 765d240ca6 Test component with malloc(0) returning NULL: run some ssl tests 2020-02-11 19:26:28 +01:00
Gilles Peskine c4ef7a9de3 Add a test component with malloc(0) returning NULL
Exercise the library functions with calloc returning NULL for a size
of 0. Make this a separate job with UBSan (and ASan) to detect
places where we try to dereference the result of calloc(0) or to do
things like

    buf = calloc(size, 1);
    if (buf == NULL && size != 0) return INSUFFICIENT_MEMORY;
    memcpy(buf, source, size);

which has undefined behavior when buf is NULL at the memcpy call even
if size is 0.

This is needed because other test components jobs either use the system
malloc which returns non-NULL on Linux and FreeBSD, or the
memory_buffer_alloc malloc which returns NULL but does not give as
useful feedback with ASan (because the whole heap is a single C
object).
2020-02-11 19:26:28 +01:00
Jaeden Amero 7cb47de12a query_config: Move to programs/test
As the SSL programs, like ssl_client2 and ssl_server2, are dependent on
SSL and therefore about to be removed, the only consumer of query_config
is the query_compile_time_config test. As such, it makes sense to move
query_config to be next to what uses it.
2020-02-11 19:26:27 +01:00
Manuel Pégourié-Gonnard 4c08dd4e71
Merge pull request #2852 from gilles-peskine-arm/2.19-fix-full-Os
Fix and test the full config with gcc and clang
2020-02-11 09:17:02 +01:00
Gilles Peskine 5da20cc569
Merge pull request #3023 from gilles-peskine-arm/config-crypto
Add crypto-only preset configurations
2020-02-05 11:17:56 +01:00
Gilles Peskine ec10bf1385 Test GCC and Clang with common build options
Goals:
* Build with common compilers with common options, so that we don't
  miss a (potentially useful) warning only triggered with certain
  build options.
* A previous commit removed -O0 test jobs, leaving only the one with
  -m32. We have inline assembly that is disabled with -O0, falling
  back to generic C code. This commit restores a test that runs the
  generic C code on a 64-bit platform.
2020-02-03 19:52:36 +01:00
Gilles Peskine 6ec0f0f6d0 Replace -O0 by -O1 or -Os in most components
Gcc skips some analyses when compiling with -O0, so we may miss
warnings about things like uninitialized variables.
2020-02-03 19:52:36 +01:00
Janos Follath 2fdb1af18c
Merge pull request #2236 from andresag01/iotssl-2156-deprecate-sslv3
Deprecate SSLv2 parsing and SSLv3
2020-02-03 15:11:33 +00:00
Gilles Peskine 6bb3915e96 Always use "-O1 -Werror" in crypto-only test builds
Pass -Werror because any compiler warning would be suspicious. Pass
-O1 because at -O0, gcc doesn't do as much analysis.
2020-02-03 11:59:20 +01:00
Gilles Peskine ec541fe0a1 Add test components for crypto-only builds
For each of the crypto-only presets, run the build and check that the
resulting libmbedx509 and libmbedtls are empty.

Don't bother testing, because for each crypto-only preset, another
component builds that plus the x509 and tls parts and tests
everything.
2020-01-31 15:14:18 +01:00
Manuel Pégourié-Gonnard 03035eb943 Stop testing ssl3 when it isn't enabled
We already have a specific component in all.sh for testing SSLv3, we don't
need to also test it in components that aren't specifically about it.

Previously config.py full enabled SSLv3, but it no longer does since it is
deprecated.
2020-01-31 12:40:36 +01:00
Gilles Peskine f65ed6f254 Change key types to a 16-bit encoding
All key types now have an encoding on 32 bits where the bottom 16 bits
are zero. Change to using 16 bits only.

Keep 32 bits for key types in storage, but move the significant
half-word from the top to the bottom.

Likewise, change EC curve and DH group families from 32 bits out of
which the top 8 and bottom 16 bits are zero, to 8 bits only.

Reorder psa_core_key_attributes_t to avoid padding.
2020-01-31 10:24:21 +01:00
Gilles Peskine 228abc5773 Define EC curve family constants
Define constants for ECC curve families and DH group families. These
constants have 0x0000 in the lower 16 bits of the key type.

Support these constants in the implementation and in the PSA metadata
tests.

Switch the slot management and secure element driver HAL tests to the
new curve encodings. This requires SE driver code to become slightly
more clever when figuring out the bit-size of an imported EC key since
it now needs to take the data size into account.

Switch some documentation to the new encodings.

Remove the macro PSA_ECC_CURVE_BITS which can no longer be implemented.
2020-01-31 10:15:32 +01:00
Gilles Peskine 1fb7aea9b3 Add command line option to hide warnings 2020-01-30 12:27:14 +01:00
Manuel Pégourié-Gonnard f2e2902c5a Add detection for zlib headers to all.sh 2020-01-24 10:44:13 +01:00
Manuel Pégourié-Gonnard 95e04490fa Add all.sh components with ZLIB enabled
ZLIB support is deprecated, but until it's removed it should still be tested.
2020-01-24 10:44:13 +01:00
Manuel Pégourié-Gonnard 20f236de37 Adjust depends-hashes.pl to test NO_SHA384 as well 2020-01-06 11:40:23 +01:00
Jaeden Amero 448d1cc854
Merge pull request #334 from dgreen-arm/fix-pylint-warnings
Fix some pylint warnings
2019-12-20 16:06:53 +00:00
Darryl Green fb5faa2582 Fix some pylint warnings
Add docstrings where they were missing and fix a too-long line
2019-12-20 15:14:59 +00:00
Darryl Green 1822061093 Fix some pylint warnings
Add docstrings where they were missing and fix a too-long line
2019-12-20 15:13:45 +00:00
Gilles Peskine 180850a229
Merge pull request #291 from gilles-peskine-arm/ctr_drbg-test_aes_128
Test MBEDTLS_CTR_DRBG_USE_128_BIT_KEY
2019-12-20 10:43:44 +01:00
Gilles Peskine f8210f2bd5 Test the block size for symmetric keys
Also insist on their category.

Fix a missing implementation of PSA_BLOCK_CIPHER_BLOCK_SIZE for
ChaCha20.
2019-12-12 09:00:27 +01:00
Gilles Peskine 325584889d Add option to show what values are tested
This is useful to inspect what the script does manually, in particular
to check that expected values do get tested. --keep-c provides the
same information but in a way that's harder to access.
2019-12-11 11:03:07 +01:00
Gilles Peskine 49af2d3a4f Support non-ASCII characters in headers
Filter out non-ASCII characters in automatically processed headers.

Do this in a way that minimizes the code change: keep manipulating
strings, but strip off non-ASCII characters when reading lines, which
should only remove characters in comments that we don't parse anyway.
2019-12-11 11:03:07 +01:00
Gilles Peskine 841b14be02 Add tests of deprecated PSA macros
When MBEDTLS_TEST_DEPRECATED is defined, run some additional tests to
validate deprecated PSA macros. We don't need to test deprecated
features extensively, but we should at least ensure that they don't
break the build.

Add some code to component_build_deprecated in all.sh to run these
tests with MBEDTLS_DEPRECATED_WARNING enabled. The tests are also
executed when MBEDTLS_DEPRECATED_WARNING and
MBEDTLS_DEPRECATED_REMOVED are both disabled.
2019-11-26 19:12:35 +01:00
Gilles Peskine 7a894f2142 Move backward compatibility aliases to their own header
Move backward compatibility aliases to a separate header. Reserve
crypto_extra.h for implementation-specific extensions that we intend
to keep supporting.

This is better documentation for users. New users should simply ignore
backward compatibility aliases, and old users can look at
crypto_compat.h to see what is deprecated without bothering about new
features appearing in crypto_extra.h.

This facilitates maintenance because scripts such as
generate_psa_constants that want to ignore backward compability
aliases can simply exclude crypto_compat.h from their parsing.
2019-11-26 18:20:58 +01:00
Gilles Peskine 8fa1348276 Enumerate metadata test functions explicitly
When gathering test cases from test_suite_psa_crypto_metadata, look up
the test function explicitly. This way test_psa_constant_names will
error out if we add a new test function that needs coverage here.

This change highlights an omission in the previous version:
asymmetric_signature_wildcard was silently ignored as a source of
algorithm expressions to test. Fix that.
2019-11-25 17:10:12 +01:00
Gilles Peskine 7961668738 Support key agreement
Key agreement algorithms were excluded back when they were constructed
with a macro conveying the key agreement itself taking the KDF as an
argument, because that was hard to support. Now the encoding has
changed and key agreement algorithms are constructed with
PSA_ALG_KEY_AGREEMENT taking two arguments, one that identifies the
raw key agreement and one that identifies the KDF. This is easy to
process, so add support.
2019-11-25 15:44:24 +01:00
Gilles Peskine 2bcfc714d2 Error out if a test case uses an unknown macro name
Insist that test cases must only use macro names that are declared in
a header. This may catch errors such as not parsing the intended
files.

Make this check easily overridden in a derived class.
2019-11-25 15:44:24 +01:00
Gilles Peskine 98a710c5b2 Fix the collection of ECC curves and DH groups
PSA_ECC_CURVE_xxx and PSA_DH_GROUP_xxx were not collected from
headers, only from test suites.
2019-11-25 15:44:23 +01:00
Gilles Peskine 8c8694c14d add_test_case_line: data-driven dispatch
No behavior change.
2019-11-25 15:44:23 +01:00
Gilles Peskine 84a45817a4 Allow gather_inputs to work with a derived Inputs class
No behavior change.
2019-11-25 15:44:23 +01:00
Gilles Peskine a5000f1dc6 Make a class for error data
No behavior change.
2019-11-25 15:44:23 +01:00
Gilles Peskine 2460933a6f Move test running and reporting functions into their own class
This makes the structure of the code more apparent.

No behavior change.
2019-11-25 15:44:23 +01:00
Gilles Peskine b86b6d32f9 Path options that affect run_c as separate arguments
No behavior change.
2019-11-25 15:44:23 +01:00
Gilles Peskine c231711dbc Move value collection into its own function
No behavior change.
2019-11-25 15:44:21 +01:00
Gilles Peskine ffe2d6e71b Move the type_word->name_set mapping into its own method
No behavior change.
2019-11-25 15:44:04 +01:00
Gilles Peskine 4408dfd0fc Minor docstring improvements
No behavior change.
2019-11-22 17:44:25 +01:00
Gilles Peskine 69f93b5040 Move the names of input files to global variables
No behavior change.
2019-11-21 16:49:50 +01:00
Gilles Peskine 8f5a5018e8 Describe options in alphabetical order
No behavior change.
2019-11-21 16:49:10 +01:00
Gilles Peskine 5a6dc895f2 Simplify expression normalization
No need to split lines, or remove whitespace after removing
whitespace. No behavior change.
2019-11-21 16:48:47 +01:00
Gilles Peskine 5a994c15f4 More readable code around expression generation
FOO(BAR) is an expression, not a name.
Pack expression generation into a method.
No behavior change.
2019-11-21 16:46:51 +01:00
Gilles Peskine 592f591c0d all.sh: test CTR_DRBG_USE_128_BIT_KEY and ENTROPY_FORCE_SHA256
Test MBEDTLS_CTR_DRBG_USE_128_BIT_KEY and MBEDTLS_ENTROPY_FORCE_SHA256
together and separately.
2019-11-21 13:54:00 +01:00