Commit graph

535 commits

Author SHA1 Message Date
Jaeden Amero 2de07f1dd1 ssl: Don't access non-existent encrypt_then_mac field
When MBEDTLS_SSL_ENCRYPT_THEN_MAC is enabled, but not
MBEDTLS_SSL_SOME_MODES_USE_MAC, mbedtls_ssl_derive_keys() and
build_transforms() will attempt to use a non-existent `encrypt_then_mac`
field in the ssl_transform.

    Compile [ 93.7%]: ssl_tls.c
    [Error] ssl_tls.c@865,14: 'mbedtls_ssl_transform {aka struct mbedtls_ssl_transform}' ha
s no member named 'encrypt_then_mac'
    [ERROR] ./mbed-os/features/mbedtls/src/ssl_tls.c: In function 'mbedtls_ssl_derive_keys'
:
    ./mbed-os/features/mbedtls/src/ssl_tls.c:865:14: error: 'mbedtls_ssl_transform {aka str
uct mbedtls_ssl_transform}' has no member named 'encrypt_then_mac'
         transform->encrypt_then_mac = session->encrypt_then_mac;
                  ^~

Change mbedtls_ssl_derive_keys() and build_transforms() to only access
`encrypt_then_mac` if `encrypt_then_mac` is actually present.

Add a regression test to detect when we have regressions with
configurations that do not include any MAC ciphersuites.

Fixes d56ed2491b ("Reduce size of `ssl_transform` if no MAC ciphersuite is enabled")
2019-06-05 14:09:29 +01:00
Jaeden Amero 3b025ac52b test: Always use make clean by itself
When running make with parallelization, running both "clean" and "lib"
with a single make invocation can lead to each target building in
parallel. It's bad if lib is partially done building something, and then
clean deletes what was built. This can lead to errors later on in the
lib target.

    $ make -j9 clean lib
      CC    aes.c
      CC    aesni.c
      CC    arc4.c
      CC    aria.c
      CC    asn1parse.c
      CC    ./library/error.c
      CC    ./library/version.c
      CC    ./library/version_features.c
      AR    libmbedcrypto.a
    ar: aes.o: No such file or directory
    Makefile:120: recipe for target 'libmbedcrypto.a' failed
    make[2]: *** [libmbedcrypto.a] Error 1
    Makefile:152: recipe for target 'libmbedcrypto.a' failed
    make[1]: *** [libmbedcrypto.a] Error 2
    Makefile:19: recipe for target 'lib' failed
    make: *** [lib] Error 2
    make: *** Waiting for unfinished jobs....

To avoid this sort of trouble, always invoke clean by itself without
other targets throughout the library. Don't run clean in parallel with
other rules. The only place where clean was run in parallel with other
targets was in list-symbols.sh.
2019-05-31 12:47:07 +01:00
Jaeden Amero 2ab5cf658f Merge remote-tracking branch 'origin/pr/2403' into development
* origin/pr/2403: (24 commits)
  crypto: Update to Mbed Crypto 8907b019e7
  Create seedfile before running tests
  crypto: Update to Mbed Crypto 81f9539037
  ssl_cli.c : add explicit casting to unsigned char
  Generating visualc files - let Mbed TLS take precedence over crypto
  Add a link to the seedfile for out-of-tree cmake builds
  Adjust visual studio file generation to always use the crypto submodule
  all.sh: unparallelize mingw tests
  all.sh - disable parallelization for shared target tests
  config.pl: disable PSA_ITS_FILE and PSA_CRYPTO_STORAGE for baremetal
  all.sh: unset crypto storage define in a psa full config cmake asan test
  all.sh: unset FS_IO-dependent defines for tests that do not have it
  curves.pl - change test script to not depend on the implementation
  Export the submodule flag to sub-cmakes
  Disable MBEDTLS_ECP_RESTARTABLE in full config
  Export the submodule flag to sub-makes
  Force the usage of crypto submodule
  Fix crypto submodule usage in Makefile
  Documentation rewording
  Typo fixes in documentation
  ...
2019-05-23 09:08:55 +01:00
Gilles Peskine 3c8ccc068f Create seedfile before running tests
With MBEDTLS_USE_PSA_CRYPTO and MBEDTLS_ENTROPY_NV_SEED enabled, the
tests need a seedfile. Since test_suite_entropy is no longer there to
create it, and MBEDTLS_USE_PSA_CRYPTO is now enabled in the full
config, create tests/seedfile explicitly in basic-build-test.sh.
2019-05-23 03:01:35 -04:00
Andrzej Kurek 62faadd6a3 all.sh: unparallelize mingw tests
Sometimes, when building the shared target with crypto submodule, one could
get an "No rule to make target '../crypto/library/libmbedcrypto.so'" error.
This is due to two reasons - building in parallel and (probably) an
incomplete/incorrect list of dependencies for certain targets. The proposed
solution is to disable parallel builds with crypto submodule for now.
An issue has been raised here: https://github.com/ARMmbed/mbedtls/issues/2634
2019-05-23 03:01:35 -04:00
Andrzej Kurek 87615770c7 all.sh - disable parallelization for shared target tests
Sometimes, when building the shared target with crypto submodule, one could
get an "No rule to make target '../crypto/library/libmbedcrypto.so'" error.
This is due to two reasons - building in parallel and (probably) an
incomplete/incorrect list of dependencies for certain targets. The proposed
solution is to disable parallel builds with crypto submodule for now.
An issue has been raised here: https://github.com/ARMmbed/mbedtls/issues/2634
2019-05-23 03:01:35 -04:00
Andrzej Kurek 324b2f76ca all.sh: unset crypto storage define in a psa full config cmake asan test 2019-05-23 03:01:35 -04:00
Andrzej Kurek 73757080c1 all.sh: unset FS_IO-dependent defines for tests that do not have it 2019-05-23 03:01:35 -04:00
Andrzej Kurek 098b16ca83 curves.pl - change test script to not depend on the implementation
Currently the top-level makefile deploys commands to both Mbed TLS and
the submodule. Running make in the "tests" directory builds only the TLS tests.
The top level CMake on the other hand does not have the "tests" target defined,
so it also cannot be used, hence the raw "make".
2019-05-23 03:01:35 -04:00
Andrzej Kurek 346747cd24 Force the usage of crypto submodule
Remove all.sh tests exercising the optional usage of submodule
2019-05-23 03:01:35 -04:00
Manuel Pégourié-Gonnard 971dea3745 Enable USE_PSA_CRYPTO with config.pl full
Previously it was disabled as too experimental, which no longer holds. Also,
this option introduces new APIs, so it's not only about an internal
alternative (as the comment in config.pl used to state) - people who request a
full config should get all of the available APIs.

Adapt all.sh: now all builds with full config will also test this option, and
builds with the default config will test without it. Just to be sure, let's
have a build with full config minus this option.

Update documentation of MBEDTLS_USE_PSA_CRYPTO to reflect the status of the
new APIs it enables in Mbed TLS and why they're still opt-in.
2019-05-23 03:01:35 -04:00
Manuel Pégourié-Gonnard de7636e2dd Enable MBEDTLS_PSA_CRYPTO_C by default
Also enable it in scripts/config.pl full, as well as two storage options that
were only blacklisted from full config because they depended on
MBEDTLS_PSA_CRYPTO_C.
2019-05-23 03:01:35 -04:00
Manuel Pégourié-Gonnard d8167e85d6 Build from submodule by default (make, cmake)
Adapt tests in all.sh:
- tests with submodule enabled (default) no longer need to enable it
  explicitly, and no longer need runtime tests, as those are now handled by
all other test cases in this script
- tests with submodule disabled (old default) now need to disable it
  explicitly, and execute some runtime tests, as those are no longer tested
anywhere else in this script

Adapt documentation in Readme: remove the section "building with submodule"
and replace it with a new section before the other building sections.
Purposefully don't document how to build not from the submodule, as that
option is going away soon.
2019-05-23 03:01:35 -04:00
Gilles Peskine 9a52cf3d97 list-symbols.sh: if the build fails, print the build transcript
If "make clean lib" fails in list-symbols.sh, print the transcript
from running make.
2019-05-22 18:22:58 +02:00
Gilles Peskine 0c12e308af Document "check-names.sh -v" 2019-05-22 18:22:45 +02:00
Gilles Peskine 13f97dc164 all.sh: invoke check-names.sh in print-trace-on-exit mode 2019-05-15 17:55:33 +02:00
Gilles Peskine 36428d34c9 Print a command trace if the check-names.sh exits unexpectedly
We've observed that sometimes check-names.sh exits unexpectedly with
status 2 and no error message. The failure is not reproducible. This
commits makes the script print a trace if it exits unexpectedly.
2019-05-15 17:29:15 +02:00
Jaeden Amero 1cd7bea2b1 Merge remote-tracking branch 'origin/pr/2586' into development
* origin/pr/2586:
  all.sh: Require i686-w64-mingw32-gcc version >= 6
2019-04-24 11:23:54 +01:00
Jaeden Amero 9679cb48d5 Merge remote-tracking branch 'origin/pr/2549' into development
* origin/pr/2549:
  Document the scripts behaviour further
  Add --internal option to list-identifiers.sh
2019-04-24 11:22:24 +01:00
Darryl Green 1ae4886c82 Document the scripts behaviour further 2019-04-18 13:09:25 +01:00
Jaeden Amero 117b8a4516 all.sh: Require i686-w64-mingw32-gcc version >= 6
Require mingw gcc version 6 or greater in order to ensure
BCryptGenRandom() is available.
2019-04-17 15:23:17 +01:00
Jaeden Amero c41a3285de Merge remote-tracking branch 'origin/pr/2559' into development
* origin/pr/2559:
  Clarify comment mangled by an earlier refactoring
  Add an "out-of-box" component
  Run ssl-opt.sh on 32-bit runtime
2019-04-16 15:09:42 +01:00
Gilles Peskine f1349e4bfe Clarify comment mangled by an earlier refactoring 2019-04-10 18:41:53 +02:00
Gilles Peskine 7832c9fc3d Add an "out-of-box" component
Just run `make` and `make test`. And `selftest` for good measure.
2019-04-10 18:41:53 +02:00
Gilles Peskine 4b317616eb Run ssl-opt.sh on 32-bit runtime
Run ssl-opt.sh on x86_32 with ASan. This may detect bugs that only
show up on 32-bit platforms, for example due to size_t overflow.

For this component, turn off some memory management features that are
not useful, potentially slow, and may reduce ASan's effectiveness at
catching buffer overflows.
2019-04-10 18:41:53 +02:00
Andrzej Kurek 4a8d2dfdd6 all.sh: remove component_test_new_ecdh_context
Remove the ecdh_context component to have it only in the crypto repository
2019-04-08 06:20:00 -04:00
Andrzej Kurek cd9286f1ca Remove crypto-only related components from all.sh 2019-04-08 05:49:08 -04:00
Jaeden Amero 7c1e6193af Merge remote-tracking branch 'origin/pr/2464' into development
* origin/pr/2464:
  Allow main() to lack a docstring.
  Silence pylint
  check-files.py: readability improvement in permission check
  check-files.py: use class fields for class-wide constants
  check-files.py: clean up class structure
  abi_check.py: Document more methods
  check-files.py: document some classes and methods
  Fix pylint errors going uncaught
  Call pylint3, not pylint
  New, documented pylint configuration
2019-04-05 14:19:09 +01:00
Darryl Green 383d1fa6a5 Add --internal option to list-identifiers.sh
When doing ABI/API checking, its useful to have a list of all the
identifiers that are defined in the internal header files, as we
do not promise compatibility for them. This option allows for a
simple method of getting them for use with the ABI checking script.
2019-04-05 13:59:52 +01:00
Jaeden Amero 0ea33776ce Merge remote-tracking branch 'restricted/pr/552' into development
Ensure this merge passes tests by auto-generating query_config.c, adding
MBEDTLS_ECDH_LEGACY_CONTEXT to it.

* restricted/pr/552:
  Fix mbedtls_ecdh_get_params with new ECDH context
  Test undefining MBEDTLS_ECDH_LEGACY_CONTEXT in all.sh
  Define MBEDTLS_ECDH_LEGACY_CONTEXT in config.h
  Add changelog entry for mbedtls_ecdh_get_params robustness
  Fix ecdh_get_params with mismatching group
  Add test case for ecdh_get_params with mismatching group
  Add test case for ecdh_calc_secret
  Fix typo in documentation
2019-03-27 17:01:16 +00:00
Jaeden Amero 9714510736 all.sh: Generate seedfile for crypto submodule tests
When running tests from the crypto submodule, generate and use a
seedfile within the crypto/tests directory.
2019-03-19 15:45:09 +00:00
Jaeden Amero 0a9f9b20b6 Merge remote-tracking branch 'origin/pr/2317' into development
* origin/pr/2317:
  Update ChangeLog
  all.sh: Test MBEDTLS_MPI_WINDOW_SIZE=1
  Fix DEADCODE in mbedtls_mpi_exp_mod()
2019-03-05 16:33:42 +00:00
Jaeden Amero 41e1986ed0 Merge remote-tracking branch 'origin/pr/2293' into development
* origin/pr/2293:
  Declare test_suite_aes.ofb to CMake
  Add a facility to skip running some test suites
  run-test-suites: update the documentation
2019-03-05 16:30:37 +00:00
Simon Butcher 535ee4a35b Merge remote-tracking branch 'public/pr/2421' into development
* public/pr/2421: (68 commits)
  Fix unused variable warning in ssl_parse_certificate_coordinate()
  Add missing compile time guard in ssl_client2
  Update programs/ssl/query_config.c
  ssl_client2: Reset peer CRT info string on reconnect
  Add further debug statements on assertion failures
  Fix typo in documentation of ssl_parse_certificate_chain()
  Add debug output in case of assertion failure
  Fix typo in SSL ticket documentation
  Add config sanity check for !MBEDTLS_SSL_KEEP_PEER_CERTIFICATE
  ssl_client2: Zeroize peer CRT info buffer when reconnecting
  Reintroduce numerous ssl-opt.sh tests if !MBEDTLS_SSL_KEEP_PEER_CERT
  ssl_client2: Extract peer CRT info from verification callback
  Improve documentation of mbedtls_ssl_get_peer_cert()
  Improve documentation of MBEDTLS_SSL_KEEP_PEER_CERTIFICATE
  Fix indentation of Doxygen comment in ssl_internal.h
  Set peer CRT length only after successful allocation
  Remove question in comment about verify flags on cli vs. server
  Remove misleading and redundant guard around restartable ECC field
  Add test for !MBEDTLS_SSL_KEEP_PEER_CERTIFICATE to all.sh
  Free peer CRT chain immediately after verifying it
  ...
2019-03-01 12:46:07 +00:00
Andres Amaya Garcia 419bd0011f Adjust DES exclude lists in test scripts 2019-03-01 10:22:04 +01:00
Andres Amaya Garcia 2dadab7d3f Exclude 3DES tests in test scripts 2019-03-01 10:21:11 +01:00
Gilles Peskine e915d532a6 Silence pylint
Silence pylint in specific places where we're doing slightly unusual
or dodgy, but correct.
2019-02-27 11:03:25 +01:00
Gilles Peskine 23e64f226b check-files.py: readability improvement in permission check 2019-02-27 11:03:25 +01:00
Gilles Peskine 1e9698af4b check-files.py: use class fields for class-wide constants
In an issue tracker, heading and files_exemptions are class-wide
constants, so make them so instead of being per-instance fields.
2019-02-27 11:03:25 +01:00
Gilles Peskine 6ee576e0b5 check-files.py: clean up class structure
Line issue trackers are conceptually a subclass of file issue
trackers: they're file issue trackers where issues arise from checking
each line independently. So make it an actual subclass.

Pylint pointed out the design smell: there was an abstract method that
wasn't always overridden in concrete child classes.
2019-02-27 11:03:25 +01:00
Gilles Peskine 0d060ef328 check-files.py: document some classes and methods
Document all classes and longer methods.

Declare a static method as such. Pointed out by pylint.
2019-02-27 11:03:25 +01:00
Gilles Peskine aad2ebdf30 Fix pylint errors going uncaught
Make check-python-files.sh run pylint on all *.py files (in
directories where they are known to be present), rather than list
files explicitly.

Fix a bug whereby the return status of check-python-files.sh was only
based on the last file passing, i.e. errors in other files were
effectively ignored.

Make check-python-files.sh run pylint unconditionally. Since pylint3
is not critical, make all.sh to skip running check-python-files.sh if
pylint3 is not available.
2019-02-27 11:03:25 +01:00
Gilles Peskine b2c269eeee Call pylint3, not pylint
We use Python 3, so call Pylint for Python 3, not for Python 2.
2019-02-27 11:03:24 +01:00
Gilles Peskine 7f61575cba New, documented pylint configuration
The pylint configuration in .pylint was a modified version of the
output of `pylint --generate-rcfile` from an unknown version of
pylint. Replace it with a file that only contains settings that are
modified from the default, with an explanation of why each setting is
modified.

The new .pylintrc was written from scratch, based on the output of
pylint on the current version of the files and on a judgement of what
to silence generically, what to silence on a case-by-case basis and
what to fix.
2019-02-27 11:03:24 +01:00
Hanno Becker 545ced45f7 Add test for !MBEDTLS_SSL_KEEP_PEER_CERTIFICATE to all.sh 2019-02-26 14:38:09 +00:00
Jaeden Amero 86016a03a1 Merge remote-tracking branch 'origin/pr/2338' into development 2019-02-22 12:55:30 +00:00
Gilles Peskine e023c80477 Test undefining MBEDTLS_ECDH_LEGACY_CONTEXT in all.sh
Test that the library works with the new context format.
2019-02-22 13:04:23 +01:00
Jaeden Amero 415620c1f2 Merge remote-tracking branch 'origin/pr/2105' into development
Additional work done as part of merge:
    - Run ./tests/scripts/check-generated-files.sh and check in the
      resulting changes to programs/ssl/query_config.c
2019-02-22 10:33:15 +00:00
Jaeden Amero d9516b508a Merge remote-tracking branch 'origin/pr/2407' into development 2019-02-22 10:32:44 +00:00
Hanno Becker 241b524964 Disable restartable ECC in full config PSA test in all.sh 2019-02-22 10:26:30 +00:00