Commit graph

12410 commits

Author SHA1 Message Date
Gilles Peskine 2bcfc714d2 Error out if a test case uses an unknown macro name
Insist that test cases must only use macro names that are declared in
a header. This may catch errors such as not parsing the intended
files.

Make this check easily overridden in a derived class.
2019-11-25 15:44:24 +01:00
Gilles Peskine 98a710c5b2 Fix the collection of ECC curves and DH groups
PSA_ECC_CURVE_xxx and PSA_DH_GROUP_xxx were not collected from
headers, only from test suites.
2019-11-25 15:44:23 +01:00
Gilles Peskine 8c8694c14d add_test_case_line: data-driven dispatch
No behavior change.
2019-11-25 15:44:23 +01:00
Gilles Peskine 84a45817a4 Allow gather_inputs to work with a derived Inputs class
No behavior change.
2019-11-25 15:44:23 +01:00
Gilles Peskine a5000f1dc6 Make a class for error data
No behavior change.
2019-11-25 15:44:23 +01:00
Gilles Peskine 2460933a6f Move test running and reporting functions into their own class
This makes the structure of the code more apparent.

No behavior change.
2019-11-25 15:44:23 +01:00
Gilles Peskine b86b6d32f9 Path options that affect run_c as separate arguments
No behavior change.
2019-11-25 15:44:23 +01:00
Gilles Peskine c231711dbc Move value collection into its own function
No behavior change.
2019-11-25 15:44:21 +01:00
Gilles Peskine ffe2d6e71b Move the type_word->name_set mapping into its own method
No behavior change.
2019-11-25 15:44:04 +01:00
Gilles Peskine 4408dfd0fc Minor docstring improvements
No behavior change.
2019-11-22 17:44:25 +01:00
Gilles Peskine 69f93b5040 Move the names of input files to global variables
No behavior change.
2019-11-21 16:49:50 +01:00
Gilles Peskine 8f5a5018e8 Describe options in alphabetical order
No behavior change.
2019-11-21 16:49:10 +01:00
Gilles Peskine 5a6dc895f2 Simplify expression normalization
No need to split lines, or remove whitespace after removing
whitespace. No behavior change.
2019-11-21 16:48:47 +01:00
Gilles Peskine 5a994c15f4 More readable code around expression generation
FOO(BAR) is an expression, not a name.
Pack expression generation into a method.
No behavior change.
2019-11-21 16:46:51 +01:00
Gilles Peskine 592f591c0d all.sh: test CTR_DRBG_USE_128_BIT_KEY and ENTROPY_FORCE_SHA256
Test MBEDTLS_CTR_DRBG_USE_128_BIT_KEY and MBEDTLS_ENTROPY_FORCE_SHA256
together and separately.
2019-11-21 13:54:00 +01:00
Gilles Peskine 2ef377d56d all.sh: support variable seedfile size
The size of the seedfile used by the entropy module when
MBEDTLS_ENTROPY_NV_SEED is enabled is 32 byte when
MBEDTLS_ENTROPY_FORCE_SHA256 is enabled or MBEDTLS_SHA512_C is
disabled, and 64 bytes otherwise. A larger seedfile is ok on
entry (the code just grabs the first N bytes), but a smaller seedfile
is not ok. Therefore, if you run a component with a 32-byte seedfile
and then a component with a 64-byte seedfile, the second component
fails in the unit tests (up to test_suite_entropy which erases the
seedfile and creates a fresh one).

This is ok up to now because we only enable MBEDTLS_ENTROPY_NV_SEED
together with MBEDTLS_ENTROPY_FORCE_SHA256. But it prevents enabling
MBEDTLS_ENTROPY_NV_SEED without MBEDTLS_ENTROPY_FORCE_SHA256.

To fix this, unconditionally create a seedfile before each component.
2019-11-21 13:54:00 +01:00
Gilles Peskine 80a607171a config.pl full: exclude MBEDTLS_ENTROPY_FORCE_SHA256
This is a variant toggle, not an extra feature, so it should be tested
separately.

We test most of the effect of MBEDTLS_ENTROPY_FORCE_SHA256 (namely,
using SHA-256 in the entropy module) when we test the library with the
SHA512 module disabled (which we do at least via depends-hashes.pl).
This commit removes testing of the MBEDTLS_ENTROPY_FORCE_SHA256 option
itself, which should be added separately.
2019-11-21 13:52:09 +01:00
Gilles Peskine bbf67b98bb Remove selftest dependency in the test suite
The test suites should always run self-tests for all enabled features.
Otherwise we miss failing self-tests in CI runs, because we don't
always run the selftest program independently.

There was one spurious dependency to remove:
MBEDTLS_CTR_DRBG_USE_128_BIT_KEY for ctr_drbg, which was broken but
has now been fixed.
2019-11-21 13:49:20 +01:00
Gilles Peskine 02e79a4e43 MBEDTLS_CTR_DRBG_USE_128_BIT_KEY: add selftest data
In the CTR_DRBG module, add selftest data for when
MBEDTLS_CTR_DRBG_USE_128_BIT_KEY is enabled.

I generated the test data by running our own code. This is ok because
we have other tests that ensure that the algorithm is implemented
correctly.

This makes programs/self/selftest pass when
MBEDTLS_CTR_DRBG_USE_128_BIT_KEY is enabled.
2019-11-21 13:49:20 +01:00
Gilles Peskine 2e6cbcd931
Merge pull request #308 from tempesta-tech/ak-mpi_div_mpi-opt
Small performance improvement of mbedtls_mpi_div_mpi()
2019-11-19 11:32:29 +01:00
Jaeden Amero 004d9a7cf6
Merge pull request #322 from gilles-peskine-arm/merge-crypto-development-20191115
Update Mbed Crypto with latest Mbed TLS changes as of 2019-11-15
2019-11-18 13:59:11 +00:00
Gilles Peskine 24600e8290 Disable memory_buffer_alloc in the full config
Enabling MBEDTLS_MEMORY_BUFFER_ALLOC_C module together with
MBEDTLS_PLATFORM_MEMORY causes the library to use its own malloc
replacement. This makes memory management analyzers such as ASan
largely ineffective. We now test MBEDTLS_MEMORY_BUFFER_ALLOC_C
separately. Disable it in the "full" config.

This mirrors a change that was made in Mbed TLS on config.pl and had
not been ported to Mbed Crypto yet.

With this commit, config.py is aligned in Mbed Crypto and Mbed TLS.
2019-11-15 11:53:42 +01:00
Gilles Peskine 9afbfdc833 Merge remote-tracking branch 'myfork-public/development' into merge-crypto-development-20191115
First deal with deleted files.

* Files deleted by us: keep them deleted.
* Files deleted by them, whether modified by us or not: keep our version.

```
git rm $(git status -s | sed -n 's/^DU //p')
git reset -- $(git status -s | sed -n 's/^D  //p')
git checkout -- $(git status -s | sed -n 's/^ D //p')
git add -- $(git status -s | sed -n 's/^UD //p')
```

Individual files with conflicts:

* `3rdparty/everest/library/Hacl_Curve25519_joined.c`: spurious conflict because git mistakenly identified this file as a rename. Keep our version.
* `README.md`: conflict due to their change in a paragraph that doesn't exist in our version. Keep our version of this paragraph.
* `docs/architecture/Makefile`: near-identical additions. Adapt the definition of `all_markdown` and include the clean target.
* `doxygen/input/docs_mainpage.h`: conflict in the version number. Keep our version number.
* `include/mbedtls/config.h`: two delete/modify conflicts. Keep the removed chunks out.
* `library/CMakeLists.txt`: discard all their changes as they are not relevant.
* `library/Makefile`:
    * Discard the added chunk about the crypto submodule starting with `INCLUDING_FROM_MBEDTLS:=1`.
    * delete/modify: keep the removed chunk out.
    * library build: This is almost delete/modify. Their changes are mostly not applicable. Do keep the `libmbedcrypto.$(DLEXT): | libmbedcrypto.a` order dependency.
    * `.c.o`: `-o` was added on both sides but in a different place. Change to their place.
* `library/error.c`: to be regenerated.
* `library/version_features.c`: to be regenerated.
* `programs/Makefile`: Most of the changes are not relevant. The one relevant change is in the `clean` target for Windows; adapt it by removing `/S` from our version.
* `programs/test/query_config.c`: to be regenerated.
* `scripts/config.py`: added in parallel on both sides. Keep our version.
* `scripts/footprint.sh`: parallel changes. Keep our version.
* `scripts/generate_visualc_files.pl`: one delete/modify conflict. Keep the removed chunks out.
* `tests/Makefile`: discard all of their changes.
* `tests/scripts/all.sh`:
    * `pre_initialize_variables` add `append_outcome`: add it.
    * `pre_initialize_variables` add `ASAN_CFLAGS`: already there, keep our version.
    * `pre_parse_command_line` add `--no-append-outcome`: add it.
    * `pre_parse_command_line` add `--outcome-file`: add it.
    * `pre_print_configuration`: add `MBEDTLS_TEST_OUTCOME_FILE`.
    * Several changes in SSL-specific components: keep our version without them.
    * Several changes where `config.pl` was changed to `config.py` and there was an adjacent difference: keep our version.
    * Changes regarding the inclusion of `MBEDTLS_MEMORY_xxx`: ignore them here, they will be normalized in a subsequent commit.
    * `component_test_full_cmake_gcc_asan`: add it without the TLS tests.
    * `component_test_no_use_psa_crypto_full_cmake_asan`: keep the fixed `msg`, discard other changes.
    * `component_test_memory_buffer_allocator_backtrace`, `component_test_memory_buffer_allocator`: add them without the TLS tests.
    * `component_test_m32_everest`: added in parallel on both sides. Keep our version.
* `tests/scripts/check-names.sh`, `tests/scripts/list-enum-consts.pl`, `tests/scripts/list-identifiers.sh`, ``tests/scripts/list-macros.sh`: discard all of their changes.
* `tests/scripts/test-ref-configs.pl`: the change in the conflict is not relevant, so keep our version there.
* `visualc/VS2010/*.vcxproj`: to be regenerated.

Regenerate files:

```
scripts/generate_visualc_files.pl
git add visualc/VS2010/*.vcxproj
scripts/generate_errors.pl
git add library/error.c
scripts/generate_features.pl
git add library/version_features.c
scripts/generate_query_config.pl
git add programs/test/query_config.c
```

Rejected changes in non-conflicting files:

* `CMakeLists.txt`: discard their addition which has already been side-ported.
* `doxygen/mbedtls.doxyfile`: keep the version number change. Discard the changes related to `../crypto` paths.

Keep the following changes after examination:

* `.travis.yml`: all of their changes are relevant.
* `include/mbedtls/error.h`: do keep their changes. Even though Crypto doesn't use TLS errors, it must not encroach on TLS's allocated numbers.
* `tests/scripts/check-test-cases.py`: keep the code dealing with `ssl-opt.sh`. It works correctly when the file is not present.
2019-11-15 11:47:14 +01:00
Jaeden Amero 4ebb782578
Merge pull request #2926 from gilles-peskine-arm/pk_signature_max_size-tls-rm_521
Remove unused test data file
2019-11-15 09:43:06 +00:00
Gilles Peskine 43259ce31e Remove unused test data file
Since "Remove component designed to test MAX_SIGNATURE_SIZE",
secp521r1_prv.der is no longer used.

ec_521_prv.pem can be used for the same purpose.
2019-11-14 19:14:40 +01:00
Jaeden Amero 63ad668145
Merge pull request #321 from dgreen-arm/config_py
Replace config.pl by config.py
2019-11-14 16:56:12 +00:00
Jaeden Amero df8185eae3
Merge pull request #2923 from Patater/reduce-ram-pem-csr
x509write_csr: Reduce stack usage of mbedtls_x509write_csr_pem()
2019-11-14 16:39:48 +00:00
Gilles Peskine b9b5f493d3
Merge pull request #2920 from gilles-peskine-arm/pk_signature_max_size-tls
Use MBEDTLS_PK_SIGNATURE_MAX_SIZE
2019-11-14 10:22:32 +01:00
Gilles Peskine adc82f3535 Add set+get tests
The tests were not covering get for a symbol with a value. No symbol
has an uncommented value in the default config.h. (Actually there's
_CRT_SECURE_NO_DEPRECATE, but that's a bit of a hack that this script
is not expected to handle, so don't use it).

Add tests of "get FOO" after "set FOO" and "set FOO value", so that we
have coverage for "get FOO" when "FOO" has a value.
2019-11-13 14:33:34 +00:00
Gilles Peskine bc86f997ca Consolidate tests for set with/without values
We currently test setting a symbol with a value even if it didn't
originally had one and vice versa. So there's no need to have separate
lists of symbols to test with. Just test everything we want to test
with each symbol.
2019-11-13 14:33:34 +00:00
Gilles Peskine 61a90bd32d config.py testing: also test the get command 2019-11-13 14:33:34 +00:00
Gilles Peskine baf15df251 Compatibility redirect: add copyright notice 2019-11-13 14:33:34 +00:00
Gilles Peskine 518ce0beb3 Compatibility redirect: if python3 is not available, try python 2019-11-13 14:33:34 +00:00
Gilles Peskine 261742bd59 Fix config.py output when a symbol has acquired or lost a value
Normally a valueless symbol remains valueless and a symbol with a
value keeps having one. But just in case a symbol does get changed
from valueless to having a value, make sure there's a space between
the symbol and the value. And if a symbol gets changed from having a
value to valueless, strip trailing whitespace.

Add corresponding tests.

Also fix the case of a valueless symbol added with the set method,
which would have resulted in attempting to use None as a string. This
only happened with the Python API, not with the command line API.
2019-11-13 14:33:34 +00:00
Gilles Peskine 7b887cd14d Remove redundant test case 2019-11-13 14:33:34 +00:00
Gilles Peskine ea82042ff6 cmake: update interpreter requirement for the test suite generator
The test suite generator has been a Python script for a long time,
but tests/CMakeLists.txt still looked for Perl. The reference to
PYTHON_INTERP only worked due to a call to find_package(PythonInterp)
in the toplevel CMakeLists.txt, and cmake would not have printed the
expected error message if python was not available.
2019-11-13 14:33:34 +00:00
Gilles Peskine 9740929371 cmake: fix Python requirement
Perl is no longer needed.

Python must be version 3. Version 2 is not suitable.

The variable is PYTHONINTERP_FOUND, not PYTHON_FOUND.
2019-11-13 14:33:34 +00:00
Gilles Peskine aebf0027c0 Test script for config.py
Run config.py with various options and store the results in files.

This script also supports the now-removed config.pl.

This is a framework to run non-regression tests on config.py: run it
with the old version, run it with the new version, and compare the
output.

This is deliberately not a functional test suite so that we don't need
to maintain a set of known outputs. When something changes in
config.py (or config.h), run the script before, run it after, and
check manually whether any differences in the output are acceptable.
2019-11-13 14:33:34 +00:00
Gilles Peskine 7f04013099 Documentation improvements 2019-11-13 14:33:34 +00:00
Gilles Peskine 2552bc73d4 Fix "#define ... not found" error when using the default file name
Also make that error message end with a newline.
2019-11-13 14:33:34 +00:00
Gilles Peskine 63cdb2855f Fix "--force set" without a value sneaking a None in 2019-11-13 14:33:34 +00:00
Gilles Peskine a52f97d5a5 Fix --force requiring an argument 2019-11-13 14:33:34 +00:00
Gilles Peskine b6fa7970a6 Fix Config.unset() making the name known 2019-11-13 14:33:34 +00:00
Gilles Peskine 812f185bc8 Also search config.h near the script
By default, this script looks for include/mbedtls/config.h relative to
the current directory. This allows running config.py from outside the
build tree.

To support out-of-tree builds where config.h and config.py are in the
source tree and the current directory is in the build tree, also try
DIRECTORY_CONTAINING_SCRIPT/../include/mbedtls/config.h, and the
equivalent with symbolic links traversed.
2019-11-13 14:33:34 +00:00
Gilles Peskine 1854ec45af Report an error if switching to Python fails 2019-11-13 14:33:34 +00:00
Gilles Peskine 5d650c86b4 Fix 'config.py set' without --force
The `set` command can act on any known symbol.
2019-11-13 14:33:34 +00:00
Gilles Peskine a26ea87dde Fix encoding errors
config.h is encoded in UTF-8.
2019-11-13 14:33:34 +00:00
Gilles Peskine a47ab22852 Print help when invoked with no arguments 2019-11-13 14:33:34 +00:00
Gilles Peskine 04362a0ad6 Replace config.pl by a redirection to config.py
Keep config.pl in Perl in case people are running "perl config.pl".
2019-11-13 14:33:34 +00:00
Gilles Peskine 3bdd412d09 Invoke config.py instead of config.pl
git grep -Fl /config.pl | xargs sed -i -e 's!/config\.pl!/config.py!g'

Also:
* Change one comment in include/mbedtls/check_config.h.
* Change PERL to PYTHON in CMakeLists.txt.
2019-11-13 14:33:33 +00:00