Commit graph

1197 commits

Author SHA1 Message Date
Gilles Peskine fb4f933f8e Rename Python scripts to use '_' and not '-'
You can't import a Python script whose name includes '-'.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-06-25 14:22:06 +02:00
Gilles Peskine 68a98516f6 basic-in-docker: call all.sh for sanity checks
Call all.sh for sanity checks, rather than maintain an explicit list.
This was done in .travis.yml in 3c7ffd7a40

Travis has diverged from basic-in-docker. This commit updates the
description of basic-in-docker to no longer refer to Travis. Alignment
with Travis may be desirable but that is beyond the scope of this commit.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-06-25 14:22:06 +02:00
Janos Follath 4d1884916b Merge branch 'development-restricted' into mbedtls-2.23.0r0 2020-06-25 09:17:25 +01:00
Gilles Peskine 04c6b61f43
Merge pull request #2639 from mpg/use-all-sh-checks-for-pre-push
Use all.sh and its component list in pre-push hook
2020-06-23 14:37:16 +02:00
Gilles Peskine c96d4e2739
Merge pull request #3410 from mpg/make-coverage-script-deterministic
Make basic-build-test.sh more deterministic
2020-06-22 12:30:48 +02:00
Manuel Pégourié-Gonnard 5430447a6e Adjust comments about SEED synchronisation
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-22 10:31:35 +02:00
Bence Szépkúti c7da1fe381 Add Apache-2.0 headers to all scripts
This commit was generated using the following script:

# ========================
#!/bin/sh

# Find scripts
find -path './.git' -prune -o '(' -name '*.gdb' -o -name '*.pl' -o -name '*.py' -o -name '*.sh' ')' -print | xargs sed -i '

# Remove Mbed TLS declaration if it occurs before the copyright line
1,/Copyright.*Arm/I {
  /This file is part of/,$ {
    /Copyright.*Arm/I! d
  }
}

# Convert non-standard header in scripts/abi_check.py to the format used in the other scripts
/"""/,/"""/ {

  # Cut copyright declaration
  /Copyright.*Arm/I {
    h
    N
    d
  }

  # Paste copyright declaration
  /"""/ {
    x
    /./ {
      s/^/# /    # Add #
      x          # Replace orignal buffer with Copyright declaration
      p          # Print original buffer, insert newline
      i\

      s/.*//     # Clear original buffer
    }
    x
  }
}

/Copyright.*Arm/I {

  # Print copyright declaration
  p

  # Read the two lines immediately following the copyright declaration
  N
  N

  # Insert Apache header if it is missing
  /SPDX/! {
    i\
# SPDX-License-Identifier: Apache-2.0\
#\
# Licensed under the Apache License, Version 2.0 (the "License"); you may\
# not use this file except in compliance with the License.\
# You may obtain a copy of the License at\
#\
# http://www.apache.org/licenses/LICENSE-2.0\
#\
# Unless required by applicable law or agreed to in writing, software\
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT\
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\
# See the License for the specific language governing permissions and\
# limitations under the License.

    # Insert Mbed TLS declaration if it is missing
    /This file is part of/! i\
#\
# This file is part of Mbed TLS (https://tls.mbed.org)
  }

  # Clear copyright declaration from buffer
  D
}
'
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-15 12:05:47 +02:00
Bence Szépkúti 700ee44545 Add missing copyright dates to scripts and sources
To find any files with a missing copyright declaration, use the following script:

# ========================
#!/bin/sh

# Find files with copyright declarations, and list their file extensions
exts=$(grep -Ril --exclude-dir .git --exclude-dir 3rdparty\
                 --exclude-dir programs/fuzz 'Copyright.*Arm' | sed '
  s/.*\///
  s/.*\./*./
  s/.*/-name "&"/
' | sort -u | sed -n '
  :l
    N
    $!bl
  s/\n/ -o /gp
')

# Find files with file extensions that ususally include copyright extensions,
# but don't include a copyright declaration themselves.
eval "find\
  '(' -path './.git' -o -path './3rdparty' -o -path './programs/fuzz' ')' -prune\
  -o ! -path './tests/data_files/format_pkcs12.fmt'\
     ! -path './programs/psa/psa_constant_names_generated.c'\
     '(' $exts ')' -print" | xargs grep -Li 'Copyright.*Arm'
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-15 12:05:46 +02:00
danh-arm 15fee93121
Merge pull request #3363 from bensze01/zeroize
Remove hardcoded line number from the zeroize test
2020-06-10 11:31:38 +01:00
Bence Szépkúti 5620d71d58 Remove hardcoded line number from the zeroize test
Instead, we insert a comment containing GDB_BREAK_HERE in the line we
want to break at, and let the gdb script search for it.

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-09 12:52:04 +02:00
Steven Cooreman 8335f41cda Enable figuring out number of cores when running on OS X
Signed-off-by: Steven Cooreman <steven.cooreman@silabs.com>
2020-06-08 18:20:08 +02:00
Manuel Pégourié-Gonnard e050191ef5 Make basic-build-test.sh deterministic
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-08 13:04:10 +02:00
Manuel Pégourié-Gonnard 304b099534 all.sh: clean up some uses of "local" variables
While pure sh doesn't have a concept of local variables, we can partially
emulate them by unsetting variables before we exit the function, and use the
convention of giving them lowercase names to distinguish from global
variables.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-08 11:01:59 +02:00
Manuel Pégourié-Gonnard f1f180a6a1 all.sh: keep dd output in non-quiet mode
Since dd prints everything on stderr, both normal status update and actual
errors when they occur, redirecting that to /dev/null is a trade-off that's
acceptable in quiet mode (typically used on a developer's machine and the
developer will re-run in non-quiet mode if anything fails without sufficient
detail in the output), but not that much in non-quiet mode.

For example, if our dd invocation fails because the disk in full on a CI
machine, we want the error to be reported at the time we invoke dd, and not
later when a seemingly unrelated test fails due to an incorrect seedfile.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-08 10:46:35 +02:00
Manuel Pégourié-Gonnard 21b3d12066
Merge pull request #3374 from danh-arm/dh/branch-cov
Enable branch coverage in basic_build_test.sh
2020-06-08 10:15:06 +02:00
Manuel Pégourié-Gonnard 53fb66db12 Add support for RESTARTABLE with internal RNG
Currently we draw pseudo-random numbers at the beginning and end of the main
loop. With ECP_RESTARTABLE, it's possible that between those two occasions we
returned from the multiplication function, hence lost our internal DRBG
context that lives in this function's stack frame. This would result in the
same pseudo-random numbers being used for blinding in multiple places. While
it's not immediately clear that this would give rise to an attack, it's also
absolutely not clear that it doesn't. So let's avoid that by using a DRBG
context that lives inside the restart context and persists across
return/resume cycles. That way the RESTARTABLE case uses exactly the
same pseudo-random numbers as the non-restartable case.

Testing and compile-time options:

- The case ECP_RESTARTABLE && !ECP_NO_INTERNAL_RNG is already tested by
  component_test_no_use_psa_crypto_full_cmake_asan.
- The case ECP_RESTARTABLE && ECP_NO_INTERNAL_RNG didn't have a pre-existing
  test so a component is added.

Testing and runtime options: when ECP_RESTARTABLE is enabled, the test suites
already contain cases where restart happens and cases where it doesn't
(because the operation is short enough or because restart is disabled (NULL
restart context)).

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-08 09:09:20 +02:00
Manuel Pégourié-Gonnard 1a3f9edc08 Add config.h option MBEDTLS_ECP_NO_INTERNAL_RNG
No effect so far, except on dependency checking, as the feature it's meant to
disable isn't implemented yet (so the descriptions in config.h and the
ChangeLog entry are anticipation for now).

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-08 09:09:20 +02:00
Manuel Pégourié-Gonnard 5b942dc45e Add test for dependencies on HMAC_DRBG in all.sh
Similarly to the recently-added tests for dependencies on CTR_DRBG:
constrained environments will probably want only one DRBG module, and we
should make sure that tests pass in such a configuration.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-05 09:29:51 +02:00
Janos Follath bba4c17b7a
Merge pull request #3315 from hanno-arm/tls13-experimental-macro
Add support for TLS 1.3 record protection routines
2020-06-04 15:51:54 +01:00
Manuel Pégourié-Gonnard 6abc20e0e3
Merge pull request #3378 from mpg/fix-ctr-drbg-deps
Fix undeclared dependencies on CTR_DRBG (and add test)
2020-06-03 10:55:52 +02:00
Manuel Pégourié-Gonnard a9119167e0 Make component_check_test_cases more -q frienly
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-02 11:52:14 +02:00
Manuel Pégourié-Gonnard dfb114a843 Make check_generate_test_code more -q friendly
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-02 11:52:14 +02:00
Manuel Pégourié-Gonnard 2b2bdaa793 Add a --quiet option to all.sh
The primary purpose is to use it to run all.sh -k -q in the pre-push hook, but
this can be useful in any circumstance where you're not interested in the full
output from each component and just want a short summary of which components
were run (and if any failed).

Note that only stdout from components is suppressed, stderr is preserved so
that errors are reported. This means components should avoid printing to
stderr in normal usage (ie in the absence of errors).

Currently all the `check_*` components obey this convention except:
- check_generate_test_code: unittest prints progress to stderr
- check_test_cases: lots of non-fatal warnings printed to stderr

These components will be fixed in follow-up commits.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-02 11:52:14 +02:00
Manuel Pégourié-Gonnard bf7ae6fb25 Silence dd invocation in all.sh
It brings no value and distracts us from the actual content.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-02 11:19:09 +02:00
Manuel Pégourié-Gonnard 817e368dfd Add test for building without CTR_DRBG
People who prefer to rely on HMAC_DRBG (for example because they use it for
deterministic ECDSA and don't want a second DRBG for code size reasons) should
be able to build and run the tests suites without CTR_DRBG.

Ideally we should make sure the level of testing (SSL) is the same regardless
of which DRBG modules is enabled, but that's a more significant piece of work.
For now, just ensure everything builds and `make test` passes.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-05-29 11:50:53 +02:00
Dan Handley a8b26c2ae4 Enable branch coverage in basic_build_test.sh
Enable branch coverage output in basic_build_test.sh. This
includes enabling branch coverage output to the lcov make target,
which is disabled by default.

Signed-off-by: Dan Handley <dan.handley@arm.com>
2020-05-28 17:51:46 +01:00
Gilles Peskine eca95db763 Finish the documentation of normalize_path
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:19:20 +02:00
Hanno Becker a711f6e277 Add 'build+unit test' test for experimental TLS 1.3 code to all.sh
Signed-off-by: Hanno Becker <hanno.becker@arm.com>
2020-05-28 10:32:23 +01:00
Gilles Peskine 30e0bb4a24 Run assemble_changelog.py in all.sh
Avoid nasty surprises where it would fail when we want to make a release.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:55:10 +02:00
Gilles Peskine 344da1cbd3 Some .pem files are openssl output and have tabs and that's ok
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:55:10 +02:00
Gilles Peskine d2df86f005 .dsw files are Visual Studio stuff
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:55:09 +02:00
Gilles Peskine 12b180a0b9 Permit empty files
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:55:09 +02:00
Gilles Peskine 3e2ee3cedc Check only files checked into Git
We're only interested in files that are committed and pushed to be
included in Mbed TLS, not in any other files that may be lying around.
So ask git for the list of file names.

This script is primarily intended to run on the CI, and there it runs
on a fresh Git checkout plus potentially some other checkouts or
leftovers from a previous part of the CI job. It should also run
reasonably well on developer machines, where there may be various
additional files. In both cases, git is available.

Ad hoc directory exclusions are no longer needed.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:53:32 +02:00
Gilles Peskine d4a853dbd7 Exclude binary files from text checks
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:53:32 +02:00
Gilles Peskine 0598db84c3 Regex mechanism for check-specific exemptions
Suffixes are convenient but not always sufficient.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:53:32 +02:00
Gilles Peskine c1d1b669db Check all files by default
Have an explicit list of exemptions for specific checks rather than
whitelisting files to check. Some checks, such as permissions, should
apply to all files.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:53:32 +02:00
Gilles Peskine 05a51a8a72 More accurate variable name
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-27 21:53:32 +02:00
Gilles Peskine d9f694960f
Merge pull request #3190 from gilles-peskine-arm/config-full-clarify-development
Clarify that the full config enables everything that can be tested together
2020-05-04 12:29:09 +02:00
Gilles Peskine 18487f62d8 all.sh: on arm builds (GCC or Arm Compiler), show the code size
Just show the code size in the logs, for human consumption.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 23:14:08 +02:00
Gilles Peskine 6e2fb86c1e all.sh: add a Cortex-M0+ build
It's pretty fast and adds a little variety.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 23:14:08 +02:00
Gilles Peskine 6537588d76 all.sh: build_arm_none_eabi_gcc: do optimize
Otherwise the bignum assembly code is not used.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 23:14:04 +02:00
Gilles Peskine 6d06134e93 Rename --arm-gcc-prefix to --arm-none-eabi-gcc-prefix
This is supposed to be for GCC (or a compiler with a compatible
command line interface) targeting arm-none-eabi, so name it
accordingly.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 18:41:35 +02:00
Gilles Peskine 8386ea22b2 all.sh: explain the testing around deprecated features
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 09:07:29 +02:00
Gilles Peskine af387e0ce1 check-files: support Windows .bat files
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-26 00:43:56 +02:00
Gilles Peskine 60f267bb1e all.sh: make the arm-gcc cross-compiler prefix configurable
Make it possible to use a compiler that isn't in $PATH, or that's
installed with a different name, or even a compiler for a different
target such as arm-linux-gnueabi.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-25 22:34:28 +02:00
Gilles Peskine 97bea01ff4 all.sh: run selftest in the full config and with ASan
Almost everything the selftest program does is in the test suites. But
just in case run the selftest program itself once in the full
configuration, and once in the default configuration with ASan, in
addition to running it out of box.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-25 22:34:23 +02:00
Gilles Peskine 30de2e84ef Make no_deprecated naming more consistent
Use "no_deprecated" both in the name of the configuration and in the
name of all.sh components, rather than a mixture of "no_deprecated"
and "non_deprecated".

Make all.sh component names more consistent.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-20 21:39:22 +02:00
Gilles Peskine 1093d9f9af all.sh: reorganize testing around deprecated features
build_deprecated combined the testing of deprecated features, and
testing of the build without deprecated features. Also, it violated the
component naming convention by being called build_xxx but running tests.

Replace it by:
* test_default_no_deprecated: check that you can remove deprecated
  features from the default build.
* test_full_no_deprecated: check that the library builds when deprecated
  features are disabled (and incidentally that the tests run).
* test_no_deprecated_warning: check that there are no warnings when
  deprecated features are disabled and MBEDTLS_DEPRECATED_WARNING
  is enabled.
* test_deprecated: test the deprecated features.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-20 15:46:56 +02:00
Gilles Peskine 3a584aecca Enable SSLv3 in the full config
It's deprecated, but not otherwise counter-indicated for the full
config: it doesn't conflict with anything and enabling it doesn't make
testing harder (especially since it defaults off in compat.sh).

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-20 15:39:36 +02:00
Gilles Peskine e094a18f6b Strict C99: check it in the full config
Ensure that there is a build with -pedantic in the full config, not
just in "exotic" configurations.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-20 15:39:36 +02:00
Gilles Peskine dc6d838a73 Enable MBEDTLS_PSA_CRYPTO_SE_C in config full
It started out as be experimental, but it is now robust enough not to
break the rest, so there's no reason to leave it out.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-20 15:39:32 +02:00
Gilles Peskine 5a2710e9af
Merge pull request #3120 from gilles-peskine-arm/check-windows-files
Check Windows files for sanity as well
2020-04-20 13:59:27 +02:00
Gilles Peskine 5c34ee14d3
Merge pull request #3118 from gilles-peskine-arm/pylint-up-to-2.4-development
Pass Pylint up to 2.4
2020-04-20 09:47:40 +02:00
Piotr Nowicki 9978e6ee14 Add tests for the ssl_context_info program
Signed-off-by: Piotr Nowicki <piotr.nowicki@arm.com>
2020-04-15 16:21:36 +02:00
Gilles Peskine 32e889dfc3 Document and fix the MBEDTLS_xxx_ALT logic for the full config
The intended logic around MBEDTLS_xxx_ALT is to exclude them from full
because they require the alternative implementation of one or more
library functions, except that MBEDTLS_PLATFORM_xxx_ALT are different:
they're alternative implementations of a platform function and they
have a built-in default, so they should be included in full. Document
this.

Fix a bug whereby MBEDTLS_PLATFORM_xxx_ALT didn't catch symbols where
xxx contains an underscore. As a consequence,
MBEDTLS_PLATFORM_GMTIME_R_ALT and MBEDTLS_PLATFORM_NV_SEED_ALT are now
enabled in the full config. Explicitly exclude
MBEDTLS_PLATFORM_SETUP_TEARDOWN_ALT because it behaves like the
non-platform ones, requiring an extra build-time dependency.
Explicitly exclude MBEDTLS_PLATFORM_NV_SEED_ALT from baremetal
because it requires MBEDTLS_ENTROPY_NV_SEED, and likewise explicitly
unset it from builds that unset MBEDTLS_ENTROPY_NV_SEED.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-14 21:23:48 +02:00
Gilles Peskine e0c84ac4d2 Pylint: explicitly note why we're doing an unchecked subprocess.run
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:23:22 +02:00
Gilles Peskine 8b022359e8 Pylint: minor code simplifications
Simplify the code in minor ways. Each of this changes fixes a warning
from Pylint 2.4 that doesn't appear with Pylint 1.7.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:23:22 +02:00
Gilles Peskine 184c096e95 Pylint: abide by useless-object-inheritance warnings
Inheriting from object is a remainder of Python 2 habits and is just
clutter in Python 3.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:23:22 +02:00
Gilles Peskine dd4c1c6fe7 mbedtls_test.py: drop compatibility with Python 2
Python 2 is no longer supported upstream. Actively drop compatibility
with Python 2.

Removing the inheritance of a class on object pacifies recent versions
of Pylint (useless-object-inheritance).

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:23:22 +02:00
Gilles Peskine aaee444c68 Document more methods in Python scripts
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:23:22 +02:00
Gilles Peskine c877c24ed0 Enable extra features that have tests
The "full" configuration excludes some deprecated or experimental
features. Enable the ones that have tests, don't have extra
requirements and don't turn off some other feature.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-10 11:33:52 +02:00
Gilles Peskine b3d0bab10b MBEDTLS_MEMORY_BACKTRACE is no longer included in the full config
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-10 11:33:52 +02:00
Gilles Peskine 40be51ffe3 Add a few echo statements to make the logs easier to follow
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-10 11:33:50 +02:00
Gilles Peskine bfcb6e16ab Create a large enough seedfile
The seedfile needs to have the size of the entropy accumulator, which
is 64 bytes (512 bits) since the entropy accumulator uses SHA-512 and
the seed size needs to be the same as the hash output (or larger).

We used to enable MBEDTLS_ENTROPY_FORCE_SHA256 in the full config, so
the entropy accumulator was 256 bits (32 bytes), and therefore a
32-byte seedfile worked. But we no longer turn on this option in the
full config, so the 32-byte seedfile no longer works.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-10 11:30:09 +02:00
Gilles Peskine 5757d54261 If 'make lcov' failed, exit immediately
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-10 11:30:02 +02:00
Gilles Peskine ca51b474dc Note that we keep going even if some tests fail
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-10 11:30:00 +02:00
Gilles Peskine 6d6ee98c14 Exit with a failure status if some tests failed
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-09 18:28:14 +02:00
Gilles Peskine d703a2ee19 In Windows files, detect CR without LF as well as LF without CR
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-01 13:37:16 +02:00
Gilles Peskine 545e13f4c1 Check that Windows files have Windows line endings
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-01 11:09:52 +02:00
Gilles Peskine 2c61873a0d Also check Windows files
Check Windows files for some issues, including permissions. Omit the
checks related to special characters (whitespace, line endings,
encoding) as appropriate.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-01 11:09:52 +02:00
Gilles Peskine 5308f12fc6 Sort the list for easier maintenance
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-01 11:09:50 +02:00
Gilles Peskine 6a45d1e4ce List each item on a separate line for easier maintenance
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-01 11:08:46 +02:00
Gilles Peskine 6e8d5a00b2 Clarify confusion between file names and suffixes of file names
To test a file name exactly, prepend a / to the base name.

files_to_check actually checks suffixes, not file names, so rename it
to extensions_to_check.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-01 11:06:17 +02:00
Gilles Peskine 13c95c4d74 Make check_python_files non-optional in all.sh
check_python_files was optional in all.sh because we used to have CI
machines where pylint wasn't available. But this had the downside that
check_python_files kept breaking because it wasn't checked in the CI.
Now our CI has pylint and check_python_files should not be optional.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-24 19:25:34 +01:00
Gilles Peskine 56e99d623d Make sure to use a Python 3 pylint
On some systems, such as Ubuntu up to 19.04, `pylint` is for Python 2
and `pylint3` is for Python 3, so we should not use `pylint` even if
it's available.

Use the Python module instead of the trivial shell wrapper. This way
we can make sure to use the correct Python version.

Fix #3111

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-24 19:23:41 +01:00
Gilles Peskine 5e7d6fd240 Merge 'mbedtls/development' into merge-crypto-unremoved-20200304
Merge the latest state of the target branch (mbedtls/development) into the
pull request to merge mbed-crypto into mbedtls.

Conflicts:

* ChangeLog: add/add conflict. Resolve by using the usual section order.
2020-03-23 18:02:07 +01:00
Gilles Peskine 7415f2fd12 Add comment to help syntax highlighting in editors
Restore this change which was accidentally reverted during the merge
of Mbed Crypto.
2020-03-23 17:55:32 +01:00
Gilles Peskine aae57bffd9 Remove remaining references to the crypto subdirectory 2020-03-23 17:55:31 +01:00
Gilles Peskine 4e616f893b Remove config-default.h
This file was the default mbedtls configuration in the mbedtls-psa
fork. It is no longer relevant.
2020-03-23 17:55:31 +01:00
Gilles Peskine a57a80e213 Restore full tls coverage to remaining scripts
Look for any other invocation of test scripts that was removed: look for a change that removes a line (`^-.*…`) containing one of the names of a test script (without its path because it may be used with a relative path). Look for `ssl-opt.sh` and `compat.sh`, as well any file in `tests/scripts` that only exists in tls.
```
git diff 'HEAD^{/^Merge}~1' HEAD --diff-filter=M -- . ':!library/error.c' ':!library/version_features.c' ':!programs/test/query_config.c' ':!visualc' ':!*.pdf' ':!*.der' | grep -E "^-.*($(comm -23 <(git ls-tree -r --name-only 'HEAD^{/^Merge}~1') <(git ls-tree -r --name-only $(git merge-base upstream-crypto/development 'HEAD^{/^Merge}^2')) | sed -n 's!^tests/scripts/!!p' | sed 's/\./\\./g' | tr '\n' '|')ssl-opt\.sh|compat\.sh)" | grep -v '^---'
```
This only turns up changes in `basic-in-docker.sh`.
2020-03-23 17:55:31 +01:00
Gilles Peskine 636c26ad75 Restore full tls coverage to all.sh
The merge of mbed-crypto removed some tls coverage. Restore it. Also
remove references to the `crypto` subdirectory brought by the mbedtls
side of the merge. In more detail:

* `tests/scripts/all.sh`:
    * `fuzz` in comments (×2): restore it.
    * `CTEST_OUTPUT_ON_FAILURE=1`: don't remove it.
    * `cd crypto` for `make clean`: don't restore it.
    * `cleanup`: do restore `programs/fuzz/Makefile`. Don't go into `crypto`. Keep only one copy of the calls to `rm` in `cmake_subproject`.
    * Comment legacy options: don't remove it.
    * `crypto/Makefile` and `pre_check_seedfile`: don't restore either. See below regarding the lack of need for `pre_check_seedfile`.
    * blank line in `pre_print_configuration`: restore it.
    * blank line before `#### Build and test`: restore it.
    * SSL tests in `component_test_full_cmake_gcc_asan` and zlib components: restore it.
    * `component_test_no_pem_no_fs` (×2): the merge placed two copies in different locations. Reconcile them: unset PSA storage like in crypto, and call `ssl-opt.sh` like in tls. Put the merged version at the tls location.
    * `component_test_everest`: do add it at the tls location.
    * `component_test_small_mbedtls_ssl_dtls_max_buffering`: restore the tls value.
    * `component_test_new_ecdh_context`…: move `component_test_new_ecdh_context` before `component_test_everest` and add a calls to `compat.sh` and `ssl-opt.sh` like in `component_test_everest`. Remove the redundant crypto-only `component_test_everest`. Don't remove `component_test_psa_collect_statuses`.
    * `component_test_full_cmake_clang`: don't remove `clang` in the `msg` call. Don't remove the call to `test_psa_constant_names.py`.
    * `component_test_full_make_gcc_o0`: remove it. It's subsumed by `component_test_gcc_opt`.
    * `component_build_deprecated`: don't remove anything.
    * `component_test_memory_buffer_allocator`: restore `ssl-opt.sh`.
    * `component_test_when_no_ciphersuites_have_mac`: restore it.
    * `component_test_platform_calloc_macro`: don't restore `unset MBEDTLS_MEMORY_BUFFER_ALLOC_C` which is now redundant. Don't restore explicit flags instead of `$ASAN_CFLAGS`.
    * `component_test_aes_fewer_tables`…: don't remove it.
    * `component_test_m32_o1`: restore SSL testing.
    * `component_test_m32_everest`: restore SSL testing.
    * `component_test_min_mpi_window_size`…: don't remove it.
    * `component_test_valgrind`: do restore the tls version of the comment.
    * `run_component`: don't remove the seedfile creation. This is better than `pre_check_seedfile` (see below).
    * `pre_check_seedfile`: don't restore it. `pre_check_seedfile` (from tls) creates a seedfile once and for all. This is not good enough if a component fails in such a way as to leave a broken seedfile, or if a component leaves a seedfile with a size that's wrong for the next component to run. Instead (from crypto), `run_component` creates a sufficiently large seedfile before each component.
2020-03-23 17:55:31 +01:00
Gilles Peskine b99bd39b4e Merge mbed-crypto into mbedtls: the merge commit
Merge `unremove-non-crypto` into `mbedtls/development`. The branch
`unremove-non-crypto` was obtained by starting from `mbed-crypto/development`,
then reverting many commits that removed X.509 and TLS functionality when Mbed
Crypto forked from Mbed TLS (the “unremoval”), then make a few tweaks to
facilitate the merge.

The unremoval step restored old versions of some tls files. If a file doesn't
exist in mbed-crypto, check out the mbedtls version, regardless of what
happened during the unremoval of tls files in the crypto tree. Also
unconditionally take the mbedtls version of a few files where the
modifications are completely project-specific and are not relevant in
mbed-crypto:

* `.github/issue_template.md`: completely different. We may want to reconcile
  them independently as a follow-up.
* `.travis.yml`: would only be reverted to an earlier tls version.
* `README.md`: completely different. We may want to reconcile them
  independently as a follow-up.
* `doxygen/input/doc_mainpage.h`: the changes in crypto were minimal and not
  relevant except as a stopgap as mbed-crypto did not have its own product
  versioning in the Doxygen documentation.
* `tests/.jenkins/Jenkinsfile`: completely different.
* `tests/data_files/Makefile`: there were no changes in mbed-crypto,
  but the unremoval step restored an old version.

Shell script for everything to do after the merge apart from the conflict
resolution:
```
tls_files=($(comm -23 <(git ls-tree -r --name-only HEAD) <(git ls-tree -r --name-only $(git merge-base upstream-crypto/development MERGE_HEAD))))
tls_files+=($tls_files .github/issue_template.md .travis.yml README.md doxygen/input/doc_mainpage.h tests/.jenkins/Jenkinsfile tests/data_files/Makefile)
git checkout --theirs HEAD -- $tls_files
git add -- $tls_files
```

Resolve the remaining conflicts:

* `library/CMakeLists.txt`:
    * Keep the TLS definition of `src_crypto`
    * `USE_SHARED_MBEDTLS_LIBRARY`: keep all three libraries, with both
      `include` and `crypto/include` in `target_include_directories`, all with
      version `2.21.0`.
* `programs/Makefile`:
    * Reconcile the APPS lists (add/add from a differently-formatted common
      ancestor): insert the `psa/*` from crypto into the tls list.
    * Keep the `fuzz` target defined only in tls version.
    * Keep the recipe (only in tls version) cleaning `ssl_pthread_server`
      stuff for the `clean` target.
* `scripts/config.py`:
    * `include_in_full`: add/add conflict. Keep both.
* `tests/scripts/all.sh`:
    * `component_test_no_use_psa_crypto_full_cmake_asan`: partially old
      version in crypto. Take the tls version.
    * `component_test_malloc_0_null` and more: take
      `component_test_malloc_0_null` from crypto (with `config.py` rather than
      `config.pl`, and with `$ASAN_FLAGS` rather than an explicit list), but
      add the call to `ssl-opt.sh` from tls. Take the other components from
      crypto.

With this commit, building and running the unit tests with both `make ` and
`cmake` work in the default configuration on Linux. Other platforms, build
systems and configurations are likely not to work, and there is some
regression in test coverage.

There is some loss of functionality because the unremoval step restored older
versions of tls content. This commit contains the latest tls version of
tls-only files, but some changes from the tls side in files that existed on
both sides have regressed. Most problematic changes are hunks that remove some
tls-specific feature and contain either a C preprocessor symbol identifying a
tls-specific module or option, or the name of a tls-specific file. Hunks
that remove a tls-specific preprocessor symbol can be identified with the
regular expression `^-.*MBEDTLS_(ERR_)?(PKCS11|X509|NET|SSL)_`.

Subsequent commits will revert a few parts of the patch from this merge commit
in order to restore the tls functionality that it removes, ensure that the
test coverage includes what was covered in either branch, and fix test
failures.
2020-03-23 17:54:46 +01:00
Gilles Peskine 3b46cd3f15 Invoke config.py instead of config.pl in reverted content
perl -i -pe 's/\bconfig\.pl/config.py/g' $(git grep -l -Fw config.pl
-- . '#!tests/scripts/test_config_script.py')
2020-03-19 14:23:45 +01:00
Gilles Peskine 69e8f7ffe3 Revert "all.sh: Remove dependency on TLS, NET, and X.509"
This reverts commit 9b90f2e294.

Conflicts:
* tests/scripts/all.sh: do the same changes, dancing around the new
  outcome file feature and components added in the same places.
  Make sure that the components that are getting added back are at the
  same locations as where they are now in mbedtls.
2020-03-19 14:23:45 +01:00
Gilles Peskine 920b77524d Revert "recursion.pl: Don't depend on X.509"
This reverts commit e23737c618.
2020-03-19 14:23:45 +01:00
Gilles Peskine 7dc97048d6 Revert "Remove tests that depend on TLS or X.509"
This reverts commit 9afb2e9921.

Conflicts:
* include/CMakeLists.txt
  * "Make config.h available" comment: there has been a change
    adjacent to where it was removed. Just re-add what was removed.
* tests/CMakeLists.txt:
  * compat.sh: there has been a change immediately before where it was
    removed. Just re-add what was removed.
2020-03-19 14:17:54 +01:00
Gilles Peskine 12230eb5c8 Revert "Remove irrelevant configs"
This reverts commit 1ad37309e4.

Conflicts:
* tests/scripts/test-ref-configs.pl:
    * config-ccm-psk-tls1_2.h: there has been an addition of
      "config-symmetric-only.h" at the place where the configurations
      that are added back were removed. Keep this configuration.
2020-03-19 14:04:59 +01:00
Gilles Peskine 9d28c426d1 Revert "check-names: Enable referencing Mbed TLS macros"
This reverts commit 7fcc7bc576.
2020-03-19 13:55:04 +01:00
Gilles Peskine 4e1174967a Revert "config: Remove TLS and NET options"
This reverts commit 1c66e48670.

Conflicts:
* include/mbedtls/check_config.h:
    * MBEDTLS_SSL_PROTO_SSL3: there has been an addition (of
      MBEDTLS_SHA512_NO_SHA384) at the place where it was removed. Re-add it
      after (alphabetical order).
    * MBEDTLS_ENABLE_WEAK_CIPHERSUITES: there has been an addition (of
      MBEDTLS_CTR_DRBG_USE_128_BIT_KEY) at the place where it was removed.
      Re-add it after (alphabetical order).
    * MBEDTLS_SSL_ALL_ALERT_MESSAGES: there has been an addition (of
      MBEDTLS_SHA512_SMALLER) at the place where it was removed. Re-add it
      after (alphabetical order).
* include/mbedtls/config.h:
    * MBEDTLS_ENABLE_WEAK_CIPHERSUITES: there has been an addition (of
      MBEDTLS_CTR_DRBG_USE_128_BIT_KEY) at the place where it was removed.
      Re-add it after (alphabetical order).
    * MBEDTLS_SSL_ALL_ALERT_MESSAGES: there has been an addition (of
      MBEDTLS_SHA512_SMALLER) at the place where it was removed. Re-add it
      after (alphabetical order).
* library/version_features.c: re-generate by running
  scripts/generate_features.pl.
* programs/test/query_config.c: re-generate by running
  scripts/generate_query_config.pl.
* scripts/config.pl: this file has been replaced by config.py. Port
  the reversed changes to config.py:
    * Revert removing three symbols from the list of symbols to
      exclude from full.
    * Revert removing one symbol (MBEDTLS_NET_C) from the list of symbols
      to exclude from baremetal.
* scripts/footprint.sh:
    * Re-add the line to unset MBEDTLS_NET_C, but with config.py instead of
      config.pl.
* tests/scripts/all.sh:
    * component_test_no_platform: re-add the line to unset MBEDTLS_NET_C, but
      with config.py instead of config.pl.
    * component_build_arm_none_eabi_gcc,
    component_build_arm_none_eabi_gcc_no_udbl_division,
    component_build_arm_none_eabi_gcc_no_64bit_multiplication,
    component_build_armcc: these components now use the baremetal
    configuration, so they do not need to turn off MBEDTLS_NET_C explicitly.
2020-03-19 13:55:04 +01:00
Gilles Peskine 252e391cca Revert "config: Remove X.509 options"
This reverts commit bb1f701212.

* include/mbedtls/check_config.h:
    * MBEDTLS_X509_RSASSA_PSS_SUPPORT: there has been an addition (of
      MBEDTLS_SHA512_NO_SHA384) at the place where it was removed.
      Re-add it before MBEDTLS_SHA512_NO_SHA384 to keep it grouped
      with MBEDTLS_RSA_C.

Conflicts:
* scripts/config.pl: this file has been replaced by config.py. Port
  the reversed changes to config.py:
    * Revert removing three symbols from the list of symbols to
      exclude from full.
2020-03-19 13:53:18 +01:00
Simon Butcher e30d03e4f4 Change the use of pylint to optionally use pylint3
Pylint when installed as a distro package can be installed as pylint3, whilst as
a PEP egg, it can be installed as pylint.

This commit changes the scripts to first use pylint if installed, and optionally
look for pylint3 if not installed. This is to allow a preference for the PEP
version over the distro version, assuming the PEP one is more likely to be
the correct one.

Signed-off-by: Simon Butcher <simon.butcher@arm.com>
2020-03-16 11:38:02 +00:00
Manuel Pégourié-Gonnard 21d1cbccda
Merge pull request #2262 from andresag01/iotssl-2544-deprecate-record-accel
Fix compilation failure when MBEDTLS_SSL_HW_RECORD_ACCEL is enabled
2020-03-16 10:37:16 +01:00
Jaeden Amero c31f970a46
Merge pull request #3075 from AndrzejKurek/variable-buffer-size
Variable buffer size
2020-03-10 21:46:35 +04:00
Gilles Peskine 84a63fad5b Revert "tests: Update generator with Mbed Crypto comments"
This reverts commit dfcf84aea5.
2020-03-04 15:39:14 +01:00
Piotr Nowicki 0937ed29b9
Add an acceptance test for memory usage after handshake
Signed-off-by: Piotr Nowicki <piotr.nowicki@arm.com>
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2020-03-04 09:28:35 -05:00
Manuel Pégourié-Gonnard 68192fcd3a Fix remaining occurrences of config.pl in all.sh
The .pl version is now a compat wrapper around the .py script. Better call the
.py script directly.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-03-04 10:53:13 +01:00
Darryl Green aad82f9bbb
Add variable buffer length tests to all.sh
Exercise the feature alone, with record splitting and DTLS connection ID.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
Signed-off-by: Darryl Green <darryl.green@arm.com>
2020-03-03 10:44:57 -05:00
Gilles Peskine 26e4fdc6cd Move MEMORY_BUFFER_ALLOC components to align the order with mbedtls
No code change. This commit just moves two functions to make the order
of component definitions match the one in mbedtls.
2020-03-02 21:15:04 +01:00
Manuel Pégourié-Gonnard dd8807f52d Add build with MBEDTLS_SSL_HW_RECORD_ACCEL to all.sh
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-02-26 09:56:27 +01:00
Gilles Peskine 9ab9621511 Move 3rdparty mentions to a separate line
This makes it easier to merge changes related to adding or removing
3rdparty items.

No semantic change.
2020-02-19 20:13:29 +01:00
Gilles Peskine 765d240ca6 Test component with malloc(0) returning NULL: run some ssl tests 2020-02-11 19:26:28 +01:00
Gilles Peskine c4ef7a9de3 Add a test component with malloc(0) returning NULL
Exercise the library functions with calloc returning NULL for a size
of 0. Make this a separate job with UBSan (and ASan) to detect
places where we try to dereference the result of calloc(0) or to do
things like

    buf = calloc(size, 1);
    if (buf == NULL && size != 0) return INSUFFICIENT_MEMORY;
    memcpy(buf, source, size);

which has undefined behavior when buf is NULL at the memcpy call even
if size is 0.

This is needed because other test components jobs either use the system
malloc which returns non-NULL on Linux and FreeBSD, or the
memory_buffer_alloc malloc which returns NULL but does not give as
useful feedback with ASan (because the whole heap is a single C
object).
2020-02-11 19:26:28 +01:00
Jaeden Amero 7cb47de12a query_config: Move to programs/test
As the SSL programs, like ssl_client2 and ssl_server2, are dependent on
SSL and therefore about to be removed, the only consumer of query_config
is the query_compile_time_config test. As such, it makes sense to move
query_config to be next to what uses it.
2020-02-11 19:26:27 +01:00
Manuel Pégourié-Gonnard 4c08dd4e71
Merge pull request #2852 from gilles-peskine-arm/2.19-fix-full-Os
Fix and test the full config with gcc and clang
2020-02-11 09:17:02 +01:00
Gilles Peskine 5da20cc569
Merge pull request #3023 from gilles-peskine-arm/config-crypto
Add crypto-only preset configurations
2020-02-05 11:17:56 +01:00
Gilles Peskine ec10bf1385 Test GCC and Clang with common build options
Goals:
* Build with common compilers with common options, so that we don't
  miss a (potentially useful) warning only triggered with certain
  build options.
* A previous commit removed -O0 test jobs, leaving only the one with
  -m32. We have inline assembly that is disabled with -O0, falling
  back to generic C code. This commit restores a test that runs the
  generic C code on a 64-bit platform.
2020-02-03 19:52:36 +01:00
Gilles Peskine 6ec0f0f6d0 Replace -O0 by -O1 or -Os in most components
Gcc skips some analyses when compiling with -O0, so we may miss
warnings about things like uninitialized variables.
2020-02-03 19:52:36 +01:00
Janos Follath 2fdb1af18c
Merge pull request #2236 from andresag01/iotssl-2156-deprecate-sslv3
Deprecate SSLv2 parsing and SSLv3
2020-02-03 15:11:33 +00:00
Gilles Peskine 6bb3915e96 Always use "-O1 -Werror" in crypto-only test builds
Pass -Werror because any compiler warning would be suspicious. Pass
-O1 because at -O0, gcc doesn't do as much analysis.
2020-02-03 11:59:20 +01:00
Gilles Peskine ec541fe0a1 Add test components for crypto-only builds
For each of the crypto-only presets, run the build and check that the
resulting libmbedx509 and libmbedtls are empty.

Don't bother testing, because for each crypto-only preset, another
component builds that plus the x509 and tls parts and tests
everything.
2020-01-31 15:14:18 +01:00
Manuel Pégourié-Gonnard 03035eb943 Stop testing ssl3 when it isn't enabled
We already have a specific component in all.sh for testing SSLv3, we don't
need to also test it in components that aren't specifically about it.

Previously config.py full enabled SSLv3, but it no longer does since it is
deprecated.
2020-01-31 12:40:36 +01:00
Gilles Peskine f65ed6f254 Change key types to a 16-bit encoding
All key types now have an encoding on 32 bits where the bottom 16 bits
are zero. Change to using 16 bits only.

Keep 32 bits for key types in storage, but move the significant
half-word from the top to the bottom.

Likewise, change EC curve and DH group families from 32 bits out of
which the top 8 and bottom 16 bits are zero, to 8 bits only.

Reorder psa_core_key_attributes_t to avoid padding.
2020-01-31 10:24:21 +01:00
Gilles Peskine 228abc5773 Define EC curve family constants
Define constants for ECC curve families and DH group families. These
constants have 0x0000 in the lower 16 bits of the key type.

Support these constants in the implementation and in the PSA metadata
tests.

Switch the slot management and secure element driver HAL tests to the
new curve encodings. This requires SE driver code to become slightly
more clever when figuring out the bit-size of an imported EC key since
it now needs to take the data size into account.

Switch some documentation to the new encodings.

Remove the macro PSA_ECC_CURVE_BITS which can no longer be implemented.
2020-01-31 10:15:32 +01:00
Gilles Peskine 1fb7aea9b3 Add command line option to hide warnings 2020-01-30 12:27:14 +01:00
Manuel Pégourié-Gonnard f2e2902c5a Add detection for zlib headers to all.sh 2020-01-24 10:44:13 +01:00
Manuel Pégourié-Gonnard 95e04490fa Add all.sh components with ZLIB enabled
ZLIB support is deprecated, but until it's removed it should still be tested.
2020-01-24 10:44:13 +01:00
Manuel Pégourié-Gonnard 20f236de37 Adjust depends-hashes.pl to test NO_SHA384 as well 2020-01-06 11:40:23 +01:00
Jaeden Amero 448d1cc854
Merge pull request #334 from dgreen-arm/fix-pylint-warnings
Fix some pylint warnings
2019-12-20 16:06:53 +00:00
Darryl Green fb5faa2582 Fix some pylint warnings
Add docstrings where they were missing and fix a too-long line
2019-12-20 15:14:59 +00:00
Darryl Green 1822061093 Fix some pylint warnings
Add docstrings where they were missing and fix a too-long line
2019-12-20 15:13:45 +00:00
Gilles Peskine 180850a229
Merge pull request #291 from gilles-peskine-arm/ctr_drbg-test_aes_128
Test MBEDTLS_CTR_DRBG_USE_128_BIT_KEY
2019-12-20 10:43:44 +01:00
Gilles Peskine f8210f2bd5 Test the block size for symmetric keys
Also insist on their category.

Fix a missing implementation of PSA_BLOCK_CIPHER_BLOCK_SIZE for
ChaCha20.
2019-12-12 09:00:27 +01:00
Gilles Peskine 325584889d Add option to show what values are tested
This is useful to inspect what the script does manually, in particular
to check that expected values do get tested. --keep-c provides the
same information but in a way that's harder to access.
2019-12-11 11:03:07 +01:00
Gilles Peskine 49af2d3a4f Support non-ASCII characters in headers
Filter out non-ASCII characters in automatically processed headers.

Do this in a way that minimizes the code change: keep manipulating
strings, but strip off non-ASCII characters when reading lines, which
should only remove characters in comments that we don't parse anyway.
2019-12-11 11:03:07 +01:00
Gilles Peskine 841b14be02 Add tests of deprecated PSA macros
When MBEDTLS_TEST_DEPRECATED is defined, run some additional tests to
validate deprecated PSA macros. We don't need to test deprecated
features extensively, but we should at least ensure that they don't
break the build.

Add some code to component_build_deprecated in all.sh to run these
tests with MBEDTLS_DEPRECATED_WARNING enabled. The tests are also
executed when MBEDTLS_DEPRECATED_WARNING and
MBEDTLS_DEPRECATED_REMOVED are both disabled.
2019-11-26 19:12:35 +01:00
Gilles Peskine 7a894f2142 Move backward compatibility aliases to their own header
Move backward compatibility aliases to a separate header. Reserve
crypto_extra.h for implementation-specific extensions that we intend
to keep supporting.

This is better documentation for users. New users should simply ignore
backward compatibility aliases, and old users can look at
crypto_compat.h to see what is deprecated without bothering about new
features appearing in crypto_extra.h.

This facilitates maintenance because scripts such as
generate_psa_constants that want to ignore backward compability
aliases can simply exclude crypto_compat.h from their parsing.
2019-11-26 18:20:58 +01:00
Gilles Peskine 8fa1348276 Enumerate metadata test functions explicitly
When gathering test cases from test_suite_psa_crypto_metadata, look up
the test function explicitly. This way test_psa_constant_names will
error out if we add a new test function that needs coverage here.

This change highlights an omission in the previous version:
asymmetric_signature_wildcard was silently ignored as a source of
algorithm expressions to test. Fix that.
2019-11-25 17:10:12 +01:00
Gilles Peskine 7961668738 Support key agreement
Key agreement algorithms were excluded back when they were constructed
with a macro conveying the key agreement itself taking the KDF as an
argument, because that was hard to support. Now the encoding has
changed and key agreement algorithms are constructed with
PSA_ALG_KEY_AGREEMENT taking two arguments, one that identifies the
raw key agreement and one that identifies the KDF. This is easy to
process, so add support.
2019-11-25 15:44:24 +01:00
Gilles Peskine 2bcfc714d2 Error out if a test case uses an unknown macro name
Insist that test cases must only use macro names that are declared in
a header. This may catch errors such as not parsing the intended
files.

Make this check easily overridden in a derived class.
2019-11-25 15:44:24 +01:00
Gilles Peskine 98a710c5b2 Fix the collection of ECC curves and DH groups
PSA_ECC_CURVE_xxx and PSA_DH_GROUP_xxx were not collected from
headers, only from test suites.
2019-11-25 15:44:23 +01:00
Gilles Peskine 8c8694c14d add_test_case_line: data-driven dispatch
No behavior change.
2019-11-25 15:44:23 +01:00
Gilles Peskine 84a45817a4 Allow gather_inputs to work with a derived Inputs class
No behavior change.
2019-11-25 15:44:23 +01:00
Gilles Peskine a5000f1dc6 Make a class for error data
No behavior change.
2019-11-25 15:44:23 +01:00
Gilles Peskine 2460933a6f Move test running and reporting functions into their own class
This makes the structure of the code more apparent.

No behavior change.
2019-11-25 15:44:23 +01:00
Gilles Peskine b86b6d32f9 Path options that affect run_c as separate arguments
No behavior change.
2019-11-25 15:44:23 +01:00
Gilles Peskine c231711dbc Move value collection into its own function
No behavior change.
2019-11-25 15:44:21 +01:00
Gilles Peskine ffe2d6e71b Move the type_word->name_set mapping into its own method
No behavior change.
2019-11-25 15:44:04 +01:00
Gilles Peskine 4408dfd0fc Minor docstring improvements
No behavior change.
2019-11-22 17:44:25 +01:00
Gilles Peskine 69f93b5040 Move the names of input files to global variables
No behavior change.
2019-11-21 16:49:50 +01:00
Gilles Peskine 8f5a5018e8 Describe options in alphabetical order
No behavior change.
2019-11-21 16:49:10 +01:00
Gilles Peskine 5a6dc895f2 Simplify expression normalization
No need to split lines, or remove whitespace after removing
whitespace. No behavior change.
2019-11-21 16:48:47 +01:00
Gilles Peskine 5a994c15f4 More readable code around expression generation
FOO(BAR) is an expression, not a name.
Pack expression generation into a method.
No behavior change.
2019-11-21 16:46:51 +01:00
Gilles Peskine 592f591c0d all.sh: test CTR_DRBG_USE_128_BIT_KEY and ENTROPY_FORCE_SHA256
Test MBEDTLS_CTR_DRBG_USE_128_BIT_KEY and MBEDTLS_ENTROPY_FORCE_SHA256
together and separately.
2019-11-21 13:54:00 +01:00
Gilles Peskine 2ef377d56d all.sh: support variable seedfile size
The size of the seedfile used by the entropy module when
MBEDTLS_ENTROPY_NV_SEED is enabled is 32 byte when
MBEDTLS_ENTROPY_FORCE_SHA256 is enabled or MBEDTLS_SHA512_C is
disabled, and 64 bytes otherwise. A larger seedfile is ok on
entry (the code just grabs the first N bytes), but a smaller seedfile
is not ok. Therefore, if you run a component with a 32-byte seedfile
and then a component with a 64-byte seedfile, the second component
fails in the unit tests (up to test_suite_entropy which erases the
seedfile and creates a fresh one).

This is ok up to now because we only enable MBEDTLS_ENTROPY_NV_SEED
together with MBEDTLS_ENTROPY_FORCE_SHA256. But it prevents enabling
MBEDTLS_ENTROPY_NV_SEED without MBEDTLS_ENTROPY_FORCE_SHA256.

To fix this, unconditionally create a seedfile before each component.
2019-11-21 13:54:00 +01:00
Gilles Peskine 24600e8290 Disable memory_buffer_alloc in the full config
Enabling MBEDTLS_MEMORY_BUFFER_ALLOC_C module together with
MBEDTLS_PLATFORM_MEMORY causes the library to use its own malloc
replacement. This makes memory management analyzers such as ASan
largely ineffective. We now test MBEDTLS_MEMORY_BUFFER_ALLOC_C
separately. Disable it in the "full" config.

This mirrors a change that was made in Mbed TLS on config.pl and had
not been ported to Mbed Crypto yet.

With this commit, config.py is aligned in Mbed Crypto and Mbed TLS.
2019-11-15 11:53:42 +01:00
Gilles Peskine 9afbfdc833 Merge remote-tracking branch 'myfork-public/development' into merge-crypto-development-20191115
First deal with deleted files.

* Files deleted by us: keep them deleted.
* Files deleted by them, whether modified by us or not: keep our version.

```
git rm $(git status -s | sed -n 's/^DU //p')
git reset -- $(git status -s | sed -n 's/^D  //p')
git checkout -- $(git status -s | sed -n 's/^ D //p')
git add -- $(git status -s | sed -n 's/^UD //p')
```

Individual files with conflicts:

* `3rdparty/everest/library/Hacl_Curve25519_joined.c`: spurious conflict because git mistakenly identified this file as a rename. Keep our version.
* `README.md`: conflict due to their change in a paragraph that doesn't exist in our version. Keep our version of this paragraph.
* `docs/architecture/Makefile`: near-identical additions. Adapt the definition of `all_markdown` and include the clean target.
* `doxygen/input/docs_mainpage.h`: conflict in the version number. Keep our version number.
* `include/mbedtls/config.h`: two delete/modify conflicts. Keep the removed chunks out.
* `library/CMakeLists.txt`: discard all their changes as they are not relevant.
* `library/Makefile`:
    * Discard the added chunk about the crypto submodule starting with `INCLUDING_FROM_MBEDTLS:=1`.
    * delete/modify: keep the removed chunk out.
    * library build: This is almost delete/modify. Their changes are mostly not applicable. Do keep the `libmbedcrypto.$(DLEXT): | libmbedcrypto.a` order dependency.
    * `.c.o`: `-o` was added on both sides but in a different place. Change to their place.
* `library/error.c`: to be regenerated.
* `library/version_features.c`: to be regenerated.
* `programs/Makefile`: Most of the changes are not relevant. The one relevant change is in the `clean` target for Windows; adapt it by removing `/S` from our version.
* `programs/test/query_config.c`: to be regenerated.
* `scripts/config.py`: added in parallel on both sides. Keep our version.
* `scripts/footprint.sh`: parallel changes. Keep our version.
* `scripts/generate_visualc_files.pl`: one delete/modify conflict. Keep the removed chunks out.
* `tests/Makefile`: discard all of their changes.
* `tests/scripts/all.sh`:
    * `pre_initialize_variables` add `append_outcome`: add it.
    * `pre_initialize_variables` add `ASAN_CFLAGS`: already there, keep our version.
    * `pre_parse_command_line` add `--no-append-outcome`: add it.
    * `pre_parse_command_line` add `--outcome-file`: add it.
    * `pre_print_configuration`: add `MBEDTLS_TEST_OUTCOME_FILE`.
    * Several changes in SSL-specific components: keep our version without them.
    * Several changes where `config.pl` was changed to `config.py` and there was an adjacent difference: keep our version.
    * Changes regarding the inclusion of `MBEDTLS_MEMORY_xxx`: ignore them here, they will be normalized in a subsequent commit.
    * `component_test_full_cmake_gcc_asan`: add it without the TLS tests.
    * `component_test_no_use_psa_crypto_full_cmake_asan`: keep the fixed `msg`, discard other changes.
    * `component_test_memory_buffer_allocator_backtrace`, `component_test_memory_buffer_allocator`: add them without the TLS tests.
    * `component_test_m32_everest`: added in parallel on both sides. Keep our version.
* `tests/scripts/check-names.sh`, `tests/scripts/list-enum-consts.pl`, `tests/scripts/list-identifiers.sh`, ``tests/scripts/list-macros.sh`: discard all of their changes.
* `tests/scripts/test-ref-configs.pl`: the change in the conflict is not relevant, so keep our version there.
* `visualc/VS2010/*.vcxproj`: to be regenerated.

Regenerate files:

```
scripts/generate_visualc_files.pl
git add visualc/VS2010/*.vcxproj
scripts/generate_errors.pl
git add library/error.c
scripts/generate_features.pl
git add library/version_features.c
scripts/generate_query_config.pl
git add programs/test/query_config.c
```

Rejected changes in non-conflicting files:

* `CMakeLists.txt`: discard their addition which has already been side-ported.
* `doxygen/mbedtls.doxyfile`: keep the version number change. Discard the changes related to `../crypto` paths.

Keep the following changes after examination:

* `.travis.yml`: all of their changes are relevant.
* `include/mbedtls/error.h`: do keep their changes. Even though Crypto doesn't use TLS errors, it must not encroach on TLS's allocated numbers.
* `tests/scripts/check-test-cases.py`: keep the code dealing with `ssl-opt.sh`. It works correctly when the file is not present.
2019-11-15 11:47:14 +01:00
Gilles Peskine adc82f3535 Add set+get tests
The tests were not covering get for a symbol with a value. No symbol
has an uncommented value in the default config.h. (Actually there's
_CRT_SECURE_NO_DEPRECATE, but that's a bit of a hack that this script
is not expected to handle, so don't use it).

Add tests of "get FOO" after "set FOO" and "set FOO value", so that we
have coverage for "get FOO" when "FOO" has a value.
2019-11-13 14:33:34 +00:00
Gilles Peskine bc86f997ca Consolidate tests for set with/without values
We currently test setting a symbol with a value even if it didn't
originally had one and vice versa. So there's no need to have separate
lists of symbols to test with. Just test everything we want to test
with each symbol.
2019-11-13 14:33:34 +00:00
Gilles Peskine 61a90bd32d config.py testing: also test the get command 2019-11-13 14:33:34 +00:00
Gilles Peskine 261742bd59 Fix config.py output when a symbol has acquired or lost a value
Normally a valueless symbol remains valueless and a symbol with a
value keeps having one. But just in case a symbol does get changed
from valueless to having a value, make sure there's a space between
the symbol and the value. And if a symbol gets changed from having a
value to valueless, strip trailing whitespace.

Add corresponding tests.

Also fix the case of a valueless symbol added with the set method,
which would have resulted in attempting to use None as a string. This
only happened with the Python API, not with the command line API.
2019-11-13 14:33:34 +00:00
Gilles Peskine 7b887cd14d Remove redundant test case 2019-11-13 14:33:34 +00:00
Gilles Peskine aebf0027c0 Test script for config.py
Run config.py with various options and store the results in files.

This script also supports the now-removed config.pl.

This is a framework to run non-regression tests on config.py: run it
with the old version, run it with the new version, and compare the
output.

This is deliberately not a functional test suite so that we don't need
to maintain a set of known outputs. When something changes in
config.py (or config.h), run the script before, run it after, and
check manually whether any differences in the output are acceptable.
2019-11-13 14:33:34 +00:00
Gilles Peskine 3bdd412d09 Invoke config.py instead of config.pl
git grep -Fl /config.pl | xargs sed -i -e 's!/config\.pl!/config.py!g'

Also:
* Change one comment in include/mbedtls/check_config.h.
* Change PERL to PYTHON in CMakeLists.txt.
2019-11-13 14:33:33 +00:00
Gilles Peskine 2e86a206f7 Remove component designed to test MAX_SIGNATURE_SIZE
MBEDTLS_PK_SIGNATURE_MAX_SIZE is tested in Mbed Crypto. Its effect on
Mbed TLS is also tested via the X.509 tests. The case of
MBEDTLS_MPI_MAX_SIZE < MBEDTLS_ECDSA_MAX_LEN, for which this component
was added as a regression test, is covered by config-suite-b.h which
is tested via test-ref-configs.pl.
2019-11-13 15:32:11 +01:00
Gilles Peskine cb0101ff33
Merge pull request #298 from gilles-peskine-arm/config-symmetric-only
Test a build without any asymmetric cryptography
2019-11-12 19:37:13 +01:00
Gilles Peskine 1a9bd94549 Disable MBEDTLS_MEMORY_BUFFER_ALLOC_C after config.pl full
Enabling memory_buffer_alloc is slow and makes ASan ineffective. We
have a patch pending to remove it from the full config. In the
meantime, disable it explicitly.
2019-10-31 16:11:34 +01:00
Gilles Peskine ce35cb3cc7 'make test' must fail if Asan fails
When running 'make test' with GNU make, if a test suite program
displays "PASSED", this was automatically counted as a pass. This
would in particular count as passing:
* A test suite with the substring "PASSED" in a test description.
* A test suite where all the test cases succeeded, but the final
  cleanup failed, in particular if a sanitizer reported a memory leak.

Use the test executable's return status instead to determine whether
the test suite passed. It's always 0 on PASSED unless the executable's
cleanup code fails, and it's never 0 on any failure.

Fix ARMmbed/mbed-crypto#303
2019-10-21 20:10:21 +02:00
Gilles Peskine 5ca393f7b8 Asan make builds: avoid sanitizer recovery
Some sanitizers default to displaying an error message and recovering.
This could result in a test being recorded as passing despite a
complaint from the sanitizer. Turn off sanitizer recovery to avoid
this risk.
2019-10-21 20:10:12 +02:00
Gilles Peskine 2558d30f93 Use UBsan in addition to Asan with 'make test'
When building with make with the address sanitizer enabled, also
enable the undefined behavior sanitizer.
2019-10-21 20:09:22 +02:00
Gilles Peskine 8fd5942229 Unify ASan options in make builds
Use a common set of options when building with Asan without CMake.
2019-10-21 20:09:13 +02:00
Gilles Peskine 8b5389f360 'make test' must fail if Asan fails
When running 'make test' with GNU make, if a test suite program
displays "PASSED", this was automatically counted as a pass. This
would in particular count as passing:
* A test suite with the substring "PASSED" in a test description.
* A test suite where all the test cases succeeded, but the final
  cleanup failed, in particular if a sanitizer reported a memory leak.

Use the test executable's return status instead to determine whether
the test suite passed. It's always 0 on PASSED unless the executable's
cleanup code fails, and it's never 0 on any failure.

Fix ARMmbed/mbed-crypto#303
2019-10-21 19:17:42 +02:00
Gilles Peskine bfeed663d2 Asan make builds: avoid sanitizer recovery
Some sanitizers default to displaying an error message and recovering.
This could result in a test being recorded as passing despite a
complaint from the sanitizer. Turn off sanitizer recovery to avoid
this risk.
2019-10-21 19:08:01 +02:00
Gilles Peskine 004206c7f5 Unify ASan options in make builds
Use a common set of options when building with Asan without CMake.
2019-10-21 17:13:45 +02:00
Gilles Peskine dd61a2e3b8 Use the intended configuration in component_test_se_full 2019-10-21 15:58:20 +02:00
Jaeden Amero 719ae74253
Merge pull request #2884 from AndrzejKurek/iotssl-2886-fix-parallel-make-jobs-2
Fix parallel make jobs for shared target
2019-10-18 14:18:55 +01:00
Gilles Peskine 581bfcfc96 Create seedfile in test-ref-configs.pl
config-symmetric-only.h enables MBEDTLS_ENTROPY_NV_SEED so it needs a
seedfile. Create it in test-ref-configs.pl so that the script works on
its own, even if it is not invoked by all.sh.
2019-10-11 18:21:08 +02:00
Gilles Peskine fec306452b Add a reference configuration without any asymmetric cryptography
Add a reference configuration with most symmetric cryptographic
algorithms enabled, but without any asymmetric cryptography. This
checks that we don't have spurious unexpected dependencies on
asymmetric-only modules such as bignum.

Keep HAVE_ASM disabled because it's platform-specific.

Keep HAVEGE disabled because it's untested and not recommended.

Keep MEMORY_BUFFER_ALLOC out because it isn't related to cryptography
at all and it makes memory sanitizers ineffective.

Keep THREADING disabled because it requires special build options.
2019-10-11 18:21:08 +02:00
Andrzej Kurek 232e8f959a Enable parallel shared target tests
Now that the dependency issues for shared target are fixed, the feature can be
enabled in tests.
2019-10-07 09:24:43 -04:00
Jaeden Amero 230b87a1ea Merge remote-tracking branch 'origin/pr/2843' into development
* origin/pr/2843: (26 commits)
  Make hyperlink a hyperlink in every markdown flavor
  Update the crypto submodule to be the same as development
  Document test case descriptions
  Restore MBEDTLS_TEST_OUTCOME_FILE after test_default_out_of_box
  ssl-opt.sh: Fix some test case descriptions
  Reject non-ASCII characters in test case descriptions
  Process input files as binary
  Factor description-checking code into a common function
  Fix cosmetic error in warnings
  Fix regex matching run_test calls in ssl-opt.sh
  all.sh: run check-test-cases.py
  Better information messages for quick checks
  Fix configuration short name in key-exchanges.pl
  Make test case descriptions unique
  New test script check-test-cases.py
  Document the test outcome file
  Create infrastructure for architecture documents in Markdown
  all.sh --outcome-file creates an outcome file
  Set meaningful test configuration names when running tests
  ssl-opt: remove semicolons from test case descriptions
  ...
2019-10-02 18:01:32 +01:00
Gilles Peskine 717cd76e8a Restore MBEDTLS_TEST_OUTCOME_FILE after test_default_out_of_box
Since components run in the main process, unsetting
MBEDTLS_TEST_OUTCOME_FILE unset it in subsequent components as well.
To avoid this, save and restore the value.

(Making each component run in a subshell would be a better solution,
but it would be a much bigger change.)
2019-09-27 20:24:33 +02:00
Gilles Peskine 57870e8c67 Reject non-ASCII characters in test case descriptions
Don't require that all the tools we use to process test outcomes are
Unicode-clean.
2019-09-24 19:21:20 +02:00
Gilles Peskine f12ad58a1d Process input files as binary
Don't die if there's a non-ASCII character and we're running in an
ASCII environment.
2019-09-24 19:21:20 +02:00
Gilles Peskine 32b9421f12 Factor description-checking code into a common function
Behavior change: some error messages are slightly different.
2019-09-24 19:21:20 +02:00
Gilles Peskine 283df2e90c Fix cosmetic error in warnings 2019-09-24 19:21:20 +02:00
Gilles Peskine 168858f52d Fix regex matching run_test calls in ssl-opt.sh
No descriptions were processed before due to bugs in the regex.

Support \" inside double-quoted strings.
2019-09-24 19:21:19 +02:00
Gilles Peskine 895868bc82 all.sh: run check-test-cases.py 2019-09-24 19:21:19 +02:00
Gilles Peskine 600bb694ac Better information messages for quick checks
Call them "check" rather than "test" to distinguish them from tests
that build and run code, and for consistency with the component names.
2019-09-24 19:21:19 +02:00
Gilles Peskine a9478bab08 Fix configuration short name in key-exchanges.pl
This is testing with $kex, not without $kex, so use $kex, not "-$kex".

In test-ref-configs.pl, use $conf rather than "$conf". This is purely
a matter of Perl coding style.
2019-09-24 19:21:19 +02:00
Gilles Peskine ba94b58127 New test script check-test-cases.py
This script checks test case descriptions in test_suite_*.data and
ssl-opt.sh.

It reports the following issues:
* Error: forbidden character in a test case description.
* Error: Duplicate test description.
* Warning: Test description is too long.
2019-09-24 19:21:19 +02:00
Gilles Peskine 67ffdafde6 all.sh --outcome-file creates an outcome file
By default, remove the outcome file before starting. With
--append-outcome, append to the existing outcome file if there is one.
2019-09-24 19:21:19 +02:00
Gilles Peskine 31b0a3c351 Add a test component with malloc(0) returning NULL
Exercise the library functions with calloc returning NULL for a size
of 0. Make this a separate job with UBSan (and ASan) to detect
places where we try to dereference the result of calloc(0) or to do
things like

    buf = calloc(size, 1);
    if (buf == NULL && size != 0) return INSUFFICIENT_MEMORY;
    memcpy(buf, source, size);

which has undefined behavior when buf is NULL at the memcpy call even
if size is 0.

This is needed because other test components jobs either use the system
malloc which returns non-NULL on Linux and FreeBSD, or the
memory_buffer_alloc malloc which returns NULL but does not give as
useful feedback with ASan (because the whole heap is a single C
object).
2019-09-24 18:28:26 +02:00
Gilles Peskine c893235ea1
Merge pull request #115 from gilles-peskine-arm/psa-error-compatibility_aliases
Improve how generate_psa_constants handles compatibility aliases
2019-09-19 16:20:08 +02:00
Gilles Peskine 16a25e005d Add set+get tests
The tests were not covering get for a symbol with a value. No symbol
has an uncommented value in the default config.h. (Actually there's
_CRT_SECURE_NO_DEPRECATE, but that's a bit of a hack that this script
is not expected to handle, so don't use it).

Add tests of "get FOO" after "set FOO" and "set FOO value", so that we
have coverage for "get FOO" when "FOO" has a value.
2019-09-19 12:19:24 +02:00
Gilles Peskine fd7ad33ee9 Consolidate tests for set with/without values
We currently test setting a symbol with a value even if it didn't
originally had one and vice versa. So there's no need to have separate
lists of symbols to test with. Just test everything we want to test
with each symbol.
2019-09-19 12:18:23 +02:00
Gilles Peskine 9004a1768b Set meaningful test configuration names when running tests
Set MBEDTLS_TEST_PLATFORM and MBEDTLS_TEST_CONFIGURATION to meaningful
values in all.sh. These environment variables are used when
writing an outcome file, which happens if MBEDTLS_TEST_OUTCOME_FILE is
also set.

When running one of the try-multiple-configuration scripts, set
MBEDTLS_TEST_CONFIGURATION to a value that uniquely describes the
configuration.
2019-09-18 17:44:29 +02:00
Gilles Peskine a708dae94b Add comment to help syntax highlighting in editors 2019-09-18 17:44:29 +02:00
Gilles Peskine 61695e70f8 config.py testing: also test the get command 2019-09-13 15:17:57 +02:00
Gilles Peskine f686042554 Fix config.py output when a symbol has acquired or lost a value
Normally a valueless symbol remains valueless and a symbol with a
value keeps having one. But just in case a symbol does get changed
from valueless to having a value, make sure there's a space between
the symbol and the value. And if a symbol gets changed from having a
value to valueless, strip trailing whitespace.

Add corresponding tests.

Also fix the case of a valueless symbol added with the set method,
which would have resulted in attempting to use None as a string. This
only happened with the Python API, not with the command line API.
2019-09-13 11:04:24 +02:00
Gilles Peskine e3734bd13a Remove redundant test case 2019-09-13 11:04:24 +02:00
Gilles Peskine 878acd6490 Test script for config.py
Run config.py with various options and store the results in files.

This script also supports the now-removed config.pl.

This is a framework to run non-regression tests on config.py: run it
with the old version, run it with the new version, and compare the
output.

This is deliberately not a functional test suite so that we don't need
to maintain a set of known outputs. When something changes in
config.py (or config.h), run the script before, run it after, and
check manually whether any differences in the output are acceptable.
2019-09-13 11:04:24 +02:00
Gilles Peskine 5d46f6a89b Invoke config.py instead of config.pl
git grep -Fl /config.pl | xargs sed -i -e 's!/config\.pl!/config.py!g'

Also:
* Change one comment in include/mbedtls/check_config.h.
* Change PERL to PYTHON in CMakeLists.txt.
2019-09-13 11:04:23 +02:00
Jaeden Amero 62be28b525 Merge remote-tracking branch 'origin/pr/2469' into development
* origin/pr/2469:
  Enable MBEDTLS_MEMORY_DEBUG in memory buffer alloc test in all.sh
  Remove unnecessary memory buffer alloc unsets
  Disable DTLS proxy tests for MEMORY_BUFFER_ALLOC test
  all.sh: restructure memory allocator tests
  Add missing dependency in memory buffer alloc set in all.sh
  Don't set MBEDTLS_MEMORY_DEBUG through `scripts/config.pl full`
  Add cfg dep MBEDTLS_MEMORY_DEBUG->MBEDTLS_MEMORY_BUFFER_ALLOC_C
  Fix memory leak in CSR test suite on failure
  Fix a memory leak in x509write test suite
  Add all.sh run with full config and ASan enabled
  Add all.sh run with MBEDTLS_MEMORY_BUFFER_ALLOC_C enabled
  Update documentation of exceptions for `config.pl full`
  Adapt all.sh to removal of buffer allocator from full config
  Disable memory buffer allocator in full config
  Check dependencies of MBEDTLS_MEMORY_BACKTRACE in check_config.h
2019-09-12 16:45:55 +01:00
Andrzej Kurek 9f409f6aec Enable MBEDTLS_MEMORY_DEBUG in memory buffer alloc test in all.sh 2019-09-10 04:28:55 -04:00
Gilles Peskine 75cc771d3a Run psa_collect_statuses.py in all.sh
Since it needs a slightly different build, even if that's only for the
tests, make it its own component.
2019-09-06 19:47:17 +02:00
Gilles Peskine 51681556cf PSA return status coverage script
Add infrastructure to run unit tests and collect the return values for
every PSA API function that returns psa_status_t.

    ./tests/scripts/psa_collect_statuses.py >statuses.txt
2019-09-06 19:28:47 +02:00
Gilles Peskine 1983512803 Add backward compatibility alias for PSA_ERROR_CORRUPTION_DETECTED
This was renamed from PSA_ERROR_TAMPERING_DETECTED. Add a backward
compatibility alias in case somebody was already using it.
2019-09-06 17:48:56 +02:00
Gilles Peskine 5c196fb599 Readability improvements
No indented semantic change.
2019-09-06 17:35:51 +02:00
Andrzej Kurek 4b3a45e190 Remove unnecessary memory buffer alloc unsets
This define is turned off by default
2019-09-06 07:47:56 -04:00
Andrzej Kurek 1e56d2c3de Disable DTLS proxy tests for MEMORY_BUFFER_ALLOC test 2019-09-06 07:44:37 -04:00
Andrzej Kurek 69f20aae77 all.sh: restructure memory allocator tests
Run basic tests and ssl-opt with memory backtrace disabled, then
run basic tests only with it enabled.
2019-09-06 07:44:37 -04:00
Hanno Becker d7064202ea Add missing dependency in memory buffer alloc set in all.sh 2019-09-06 07:44:37 -04:00
Hanno Becker 0163551aa0 Add all.sh run with full config and ASan enabled 2019-09-06 07:44:37 -04:00
Hanno Becker 0fb9ba2760 Add all.sh run with MBEDTLS_MEMORY_BUFFER_ALLOC_C enabled
With the removal of MBEDTLS_MEMORY_BUFFER_ALLOC_C from the
full config, there are no tests for it remaining in all.sh.
This commit adds a build as well as runs of `make test` and
`ssl-opt.sh` with MBEDTLS_MEMORY_BUFFER_ALLOC_C enabled to all.sh.
2019-09-06 07:44:37 -04:00
Unknown 790c281f51 Adapt all.sh to removal of buffer allocator from full config
Previously, numerous all.sh tests manually disabled the buffer allocator
or memory backtracting after setting a full config as the starting point.

With the removal of MBEDTLS_MEMORY_BACKTRACE and MBEDTLS_MEMORY_BUFFER_ALLOC_C
from full configs, this is no longer necessary.
2019-09-06 07:44:37 -04:00
Jaeden Amero c9c4ca3f40 Merge remote-tracking branch 'origin/development' into development-restricted
* origin/development:
  Fix copypasta in msg
  When not using PSA crypto, disable it
  Disable MEMORY_BUFFER_ALLOC with ASan
  Remove config.pl calls with no effect
  ssl-opt.sh: wait for proxy to start before running the script further
  Adapt ChangeLog
  Fix mpi_bigendian_to_host() on bigendian systems
2019-09-05 18:14:14 +01:00
Unknown f094b53e8e all.sh: disable MEMORY_BUFFER_ALLOC in cmake asan build
Enabling MBEDTLS_MEMORY_BUFFER_ALLOC_C bypasses ASan leak checks
because system calloc() and free() aren't used.
2019-09-05 09:34:57 -04:00
Jaeden Amero ba7f4d1484 Merge remote-tracking branch 'origin/pr/2771' into development
* origin/pr/2771:
  Fix copypasta in msg
  When not using PSA crypto, disable it
  Disable MEMORY_BUFFER_ALLOC with ASan
  Remove config.pl calls with no effect
2019-09-05 14:23:55 +01:00
Gilles Peskine dc3a179995 Fix copypasta in msg 2019-09-03 14:11:36 +02:00
Gilles Peskine 6ce30722d0 When not using PSA crypto, disable it
In the test with the full config without MBEDTLS_USE_PSA_CRYPTO, don't
build MBEDTLS_PSA_CRYPTO_C, since it isn't supposed to be used.
2019-09-03 14:11:36 +02:00
Gilles Peskine 751bb4c0e1 Disable MEMORY_BUFFER_ALLOC with ASan
MBEDTLS_MEMORY_BUFFER_ALLOC_C makes ASan mostly ineffective since it
hides allocations. So disable it when testing with ASan.
2019-09-03 14:11:36 +02:00
Gilles Peskine c6f1c84663 Remove config.pl calls with no effect
When MBEDTLS_MEMORY_BUFFER_ALLOC_C is disabled, other
MBEDTLS_MEMORY_xxx options have no effect, so don't bother unsetting
them explicitly.
2019-09-03 14:11:36 +02:00
Jaeden Amero 4cf0e7e4d2 Merge remote-tracking branch 'origin/development' into development-restricted
* origin/development: (42 commits)
  Handle deleting non-existant files on Windows
  Update submodule
  Use 3rdparty headers from the submodule
  Add Everest components to all.sh
  3rdparty: Add config checks for Everest
  Fix macros in benchmark.c
  Update generated files
  3rdparty: Fix inclusion order of CMakeLists.txt
  Fix trailing whitespace
  ECDH: Fix inclusion of platform.h for proper use of MBEDTLS_ERR_PLATFORM_FEATURE_UNSUPPORTED
  ECDH: Fix use of ECDH API in full handshake benchmark
  ECDH: Removed unnecessary calls to mbedtls_ecp_group_load in ECDH benchmark
  ECDH: Fix Everest x25519 make_public
  Fix file permissions
  3rdparty: Rename THIRDPARTY_OBJECTS
  3rdparty: Update description of MBEDTLS_ECDH_VARIANT_EVEREST_ENABLED
  3rdparty: Fix Makefile coding conventions
  ECDSA: Refactor return value checks for mbedtls_ecdsa_can_do
  Add a changelog entry for Everest ECDH (X25519)
  Document that curve lists can include partially-supported curves
  ...
2019-08-30 16:24:18 +01:00
Gilles Peskine 0c6b79979c Add Everest components to all.sh
Test a native build and a 32-bit build. For variety, the native build
is with CMake and clang, and the 32-bit build is with GNU make and
gcc.
2019-08-29 16:12:38 +01:00
Christoph M. Wintersteiger 37eb90617a 3rdparty: Fix Makefile coding conventions 2019-08-29 16:12:38 +01:00
Christoph M. Wintersteiger 9b33e7d7d7 ECDH: Exclude FStar and Hacl* from exported symbol checks 2019-08-29 16:12:38 +01:00
Christoph M. Wintersteiger 346932a099 Fix preprocessor directive recognition in list-enum-consts.pl 2019-08-29 16:12:38 +01:00
Christoph M. Wintersteiger e0e8eb3114 ECDH: Add #ifdef filter to tests/scripts/list-enum-consts.pl
This allows the use of #ifdef ... #endif in enum definitions (e.g.,
mbedtls_ecdh_variant in ecdh.h).
2019-08-29 16:12:38 +01:00
Jaeden Amero f0716542c4
Merge pull request #140 from yanesca/everest_integration
Everest integration
2019-08-29 16:02:49 +01:00
Jaeden Amero f1cdceae0d Merge remote-tracking branch 'origin/development' into development-restricted
* origin/development: (114 commits)
  Don't redefine calloc and free
  Add changelog entry to record checking
  Fix compiler warning
  Add debug messages
  Remove duplicate entries from ChangeLog
  Fix parameter name in doxygen
  Add missing guards for mac usage
  Improve reability and debugability of large if
  Fix a typo in a comment
  Fix MSVC warning
  Fix compile error in reduced configurations
  Avoid duplication of session format header
  Implement config-checking header to context s11n
  Provide serialisation API only if it's enabled
  Fix compiler warning: comparing signed to unsigned
  Actually reset the context on save as advertised
  Re-use buffer allocated by handshake_init()
  Enable serialisation tests in ssl-opt.sh
  Change requirements for setting timer callback
  Add setting of forced fields when deserializing
  ...
2019-08-27 10:09:10 +01:00
Manuel Pégourié-Gonnard eef4c753f1 Adapt buffering test to new ticket size
The size of the ticket used in this test dropped from 192 to 143 bytes, so
move all sizes used in this test down 50 bytes. Also, we now need to adapt the
server response size as the default size would otherwise collide with the new
mtu value.
2019-08-23 12:50:17 +03:00
Unknown 712f7a804e Add a check for double underscores in identifiers in check-names.sh 2019-08-21 03:34:00 -04:00
Unknown e2e19959d7 Rename AEAD macro to not use double underscores
This pattern (identifiers containing a double underscore anywhere in them)
is reserved.
2019-08-21 03:33:04 -04:00
Gilles Peskine 09a24b3dda Add Everest components to all.sh
Test a native build and a 32-bit build. For variety, the native build
is with CMake and clang, and the 32-bit build is with GNU make and
gcc.
2019-08-19 13:37:46 +01:00
Christoph M. Wintersteiger 02174b90dc 3rdparty: Fix Makefile coding conventions 2019-08-19 13:37:46 +01:00
Christoph M. Wintersteiger 8a0f5bb3c1 Make check-names.sh find the right names in 3rdparty
Essentially adds the Everest .h and .c files to the various variables. This
should be generalized at some point, but there is no infrastructure for this
yet.
2019-08-19 13:36:44 +01:00
Christoph M. Wintersteiger 1083a25a29 ECDH: Exclude FStar and Hacl* from exported symbol checks 2019-08-19 13:36:44 +01:00
Christoph M. Wintersteiger 7cc4c68eb6 Fix preprocessor directive recognition in list-enum-consts.pl 2019-08-19 13:36:44 +01:00
Christoph M. Wintersteiger 6817b9e819 ECDH: Add #ifdef filter to tests/scripts/list-enum-consts.pl
This allows the use of #ifdef ... #endif in enum definitions (e.g.,
mbedtls_ecdh_variant in ecdh.h).
2019-08-19 13:36:44 +01:00
Jaeden Amero 89e7655691 Merge remote-tracking branch 'crypto/pr/212' into development
* crypto/pr/212: (337 commits)
  Make TODO comments consistent
  Fix PSA tests
  Fix psa_generate_random for >1024 bytes
  Add tests to generate more random than MBEDTLS_CTR_DRBG_MAX_REQUEST
  Fix double free in psa_generate_key when psa_generate_random fails
  Fix copypasta in test data
  Avoid a lowercase letter in a macro name
  Correct some comments
  Fix PSA init/deinit in mbedtls_xxx tests when using PSA
  Make psa_calculate_key_bits return psa_key_bits_t
  Adjust secure element code to the new ITS interface
  More refactoring: consolidate attribute validation
  Fix policy validity check on key creation.
  Add test function for import with a bad policy
  Test key creation with an invalid type (0 and nonzero)
  Remove "allocated" flag from key slots
  Take advantage of psa_core_key_attributes_t internally #2
  Store the key size in the slot in memory
  Take advantage of psa_core_key_attributes_t internally: key loading
  Switch storage functions over to psa_core_key_attributes_t
  ...
2019-08-16 11:02:31 +01:00
Jaeden Amero 58f11d010b Merge branch 'development' into development-restricted
* development:
  Update crypto to a repo with latest crypto
  Update Mbed Crypto
  tls: Remove duplicate psa_util.h include
  Remove unused cryptography test files
  Remove crypto C files
  Remove files sourced from Mbed Crypto
  config: Fix Doxygen link to MBEDTLS_PARAM_FAILED
  Use mbedtls-based path for includes
  check-names: Consider crypto-sourced header files
2019-08-16 10:14:40 +01:00
Jaeden Amero 9d20e1f2c4 Merge remote-tracking branch 'tls/development' into development
Resolve conflicts by performing the following actions:
- Reject changes to ChangeLog, as Mbed Crypto doesn't have one
- Reject changes to tests/compat.sh, as Mbed Crypto doesn't have it
- Reject changes to programs/fuzz/onefile.c, as Mbed Crypto doesn't have
  it
- Resolve minor whitespace differences in library/ecdsa.c by taking the
  version from Mbed TLS upstream.

* origin/development:
  Honor MBEDTLS_CONFIG_FILE in fuzz tests
  Test that a shared library build produces a dynamically linked executable
  Test that the shared library build with CMake works
  Add a test of MBEDTLS_CONFIG_FILE
  Exclude DTLS 1.2 only with older OpenSSL
  Document the rationale for the armel build
  Switch armel build to -Os
  Add a build on ARMv5TE in ARM mode
  Add changelog entry for ARM assembly fix
  bn_mul.h: require at least ARMv6 to enable the ARM DSP code
  Adapt ChangeLog
  ECP restart: Don't calculate address of sub ctx if ctx is NULL
2019-08-15 15:49:46 +01:00
Jaeden Amero 78d9d0c1e9 check-names: Consider crypto-sourced header files
Many identifiers come from Mbed Crypto. Teach check-names.sh to look in
the crypto submodule for identifiers, to avoid incorrect test results.
2019-08-15 15:24:26 +01:00
Gilles Peskine 4017d882dd Merge remote-tracking branch 'upstream-restricted/development-proposed' into development-restricted-proposed 2019-08-14 18:38:42 +02:00
Gilles Peskine 61fc108d25 Merge remote-tracking branch 'upstream-public/pr/2728' into development 2019-08-14 16:00:58 +02:00
Gilles Peskine 56c0161b68 Test that a shared library build produces a dynamically linked executable 2019-08-13 17:54:26 +02:00
Gilles Peskine cf74050fea Test that the shared library build with CMake works 2019-08-13 17:54:26 +02:00
Gilles Peskine abf9b4dee8 Add a test of MBEDTLS_CONFIG_FILE
configs/README.txt documents that you can use an alternative
configuration file by defining the preprocessor symbol
MBEDTLS_CONFIG_FILE. Test this.
2019-08-13 17:54:26 +02:00
Gilles Peskine 2c897d76ff Document the rationale for the armel build
Call the component xxx_arm5vte, because that's what it does. Explain
"armel", and more generally why this component exists, in a comment.
2019-08-09 16:05:05 +02:00
Gilles Peskine 8a52af9b77 Switch armel build to -Os
Without any -O option, the default is -O0, and then the assembly code
is not used, so this would not be a non-regression test for the
assembly code that doesn't build.
2019-08-08 16:09:02 +02:00
Gilles Peskine 93e4e03f94 Add a build on ARMv5TE in ARM mode
Non-regression test for
"bn_mul.h: require at least ARMv6 to enable the ARM DSP code"
2019-08-05 11:41:29 +02:00
Gilles Peskine 72c8c5b352 Merge remote-tracking branch 'upstream-crypto/development' into psa-api-1.0-beta-merge_development_20190801
Conflict resolution:
* `scripts/config.pl`:
  Take the exclusion of `MBEDTLS_PSA_CRYPTO_SE_C` from the API branch.
  Take the removal of `MBEDTLS_PSA_CRYPTO_STORAGE_ITS_C` (obsolete) from
  the development branch.
* `tests/scripts/all.sh`:
  Multiple instances of factoring a sequence of `config.pl` calls into
  a mere `config.pl baremetal` in the development branch, and a change in
  the composition of `baremetal` in the API branch. In each case, take the
  version from development.
* `tests/suites/test_suite_psa_crypto_slot_management.function`:
  A function became non-static in development and disappeared in the API
  branch. Keep the version from the API branch. Functions need to be
  non-static if they're defined but unused in some configurations,
  which is not the case for any function in this file at the moment.
* `tests/suites/test_suite_psa_crypto.function`:
  Consecutive changes in the two branches, reconciled.
2019-07-31 17:47:49 +02:00
Jaeden Amero 8dd1690993 Merge remote-tracking branch 'tls/development' into development
Resolve conflicts by performing the following operations:
- Reject changes related to building a crypto submodule, since Mbed
  Crypto is the crypto submodule.
- Reject X.509, NET, and SSL changes.
- Reject changes to README, as Mbed Crypto is a different project from
  Mbed TLS, with a different README.
- Avoid adding mention of ssl-opt.sh in a comment near some modified
  code in include/CMakeLists.txt (around where ENABLE_TESTING as added).
- Align config.pl in Mbed TLS with config.pl in Mbed Crypto where PSA
  options are concerned, to make future merging easier. There is no
  reason for the two to be different in this regard, now that Mbed TLS
  always depends on Mbed Crypto. Remaining differences are only the
  PSA_CRYPTO_KEY_FILE_ID_ENCODES_OWNER option and the absence of X.509,
  NET, and SSL related options in Mbed Crypto's config.pl.
- Align config.h in Mbed Crypto with Mbed TLS's copy, with a few notable
  exceptions:
  - Leave CMAC on by default.
  - Leave storage on by default (including ITS emulation).
  - Avoid documenting the PSA Crypto API as is in beta stage in
    documentation for MBEDTLS_PSA_CRYPTO_C.
  The only remaining differences are a lack of X.509, NET, and SSL
  options in Mbed Crypto's config.h, as well as an additional
  Mbed-Crypto-specific PSA_CRYPTO_KEY_FILE_ID_ENCODES_OWNER option.
  Documentation for the check params feature and related macros is also
  updated to match Mbed TLS's description.
- Reject tests/data_files/Makefile changes to generate DER versions of
  CRTs and keys, as none of those are used by Mbed Crypto tests.
- Add the "no PEM and no filesystem" test to all.sh, without ssl-opt.sh
  run, as Mbed Crypto doesn't have ssl-opt.sh. Also remove use of PSA
  Crypto storage and ITS emulation, since those depend on filesystem
  support.
- Reject addition of test when no ciphersuites have MAC to all.sh, as
  the option being tested, MBEDTLS_SSL_SOME_MODES_USE_MAC, is not
  present in Mbed Crypto.
- Use baremetal config in all.sh, as Mbed Crypto's baremetal
  configuration does exclude the net module (as it doesn't exist in Mbed
  Crypto)
- Reject cmake_subproject_build changes, continuing to link only
  libmbedcrypto.
- Reject changes to visualc and associated templates. Mbed Crypto
  doesn't need additional logic to handle submodule-sourced headers.
- Avoid adding fuzzers from Mbed TLS. The only relevant fuzzers are the
  privkey and pubkey fuzzers, but non-trivial work would be required to
  integrate those into Mbed Crypto (more than is comfortable in a merge
  commit).
- Reject addition of Docker wrappers for compat.sh and ssl-opt.sh, as
  those are not present in Mbed Crypto.
- Remove calls to SSL-related scripts from basic-in-docker.sh

Fix test errors by performing the following:
- Avoid using a link that Doxygen can't seem to resolve in Mbed Crypto,
  but can resolve in Mbed TLS. In documentation for
  MBEDTLS_CHECK_PARAMS, don't attempt to link to MBEDTLS_PARAM_FAILED.

* origin/development: (339 commits)
  Do not build fuzz on windows
  No booleans and import config
  Removing space before opening parenthesis
  Style corrections
  Syntax fix
  Fixes warnings from MSVC
  Add a linker flag to enable gcov in basic-build-test.sh
  Update crypto submodule to a revision with the HAVEGE header changes
  Test with MBEDTLS_ECP_RESTARTABLE
  Allow TODO in code
  Use the docstring in the command line help
  Split _abi_compliance_command into smaller functions
  Record the commits that were compared
  Document how to build the typical argument for -s
  Allow running /somewhere/else/path/to/abi_check.py
  tests: Limit each log to 10 GiB
  Warn if VLAs are used
  Remove redundant compiler flag
  Consistently spell -Wextra
  Fix parsing issue when int parameter is in base 16
  ...
2019-07-31 10:37:53 +01:00
Gilles Peskine f96aefe3ad Test with secure element support
Test with default config + SE with Clang and with full config + SE
with GCC, for variety. Full+Clang+Asan has known issues so don't do
that.
2019-07-24 14:58:38 +02:00
Gilles Peskine 1d04b05fae Dear check-names, where you accept struct, also accept union. 2019-07-23 17:38:41 +02:00
Jaeden Amero e637061dec Merge remote-tracking branch 'origin/development' into development-restricted
* origin/development: (36 commits)
  Do not build fuzz on windows
  No booleans and import config
  Removing space before opening parenthesis
  Style corrections
  Syntax fix
  Fixes warnings from MSVC
  Add a linker flag to enable gcov in basic-build-test.sh
  tests: Limit each log to 10 GiB
  Fix parsing issue when int parameter is in base 16
  checks MBEDTLS_PEM_PARSE_C
  Restore programs/fuzz/Makefile after in-tree cmake
  Move fuzz directory to programs
  Documentation for corpus generation
  Restore tests/fuzz/Makefile after in-tree cmake
  Adding ifdefs to avoid warnings for unused globals
  Adds LDFLAGS fsanitize=address
  Refactor receive_uint32()
  Refactor get_byte function
  Make the script portable to both pythons
  Update the test encoding to support python3
  ...
2019-07-12 10:15:49 +01:00
Jaeden Amero 8306508250 Merge remote-tracking branch 'origin/pr/2660' into development
* origin/pr/2660:
  Fix parsing issue when int parameter is in base 16
  Refactor receive_uint32()
  Refactor get_byte function
  Make the script portable to both pythons
  Update the test encoding to support python3
  update the test script
2019-07-11 16:17:38 +01:00
Jaeden Amero 072959f5c2 Merge remote-tracking branch 'origin/pr/1622' into development
* origin/pr/1622: (29 commits)
  Do not build fuzz on windows
  No booleans and import config
  Removing space before opening parenthesis
  Style corrections
  Syntax fix
  Fixes warnings from MSVC
  Add a linker flag to enable gcov in basic-build-test.sh
  checks MBEDTLS_PEM_PARSE_C
  Restore programs/fuzz/Makefile after in-tree cmake
  Move fuzz directory to programs
  Documentation for corpus generation
  Restore tests/fuzz/Makefile after in-tree cmake
  Adding ifdefs to avoid warnings for unused globals
  Adds LDFLAGS fsanitize=address
  Ignore compiled object files and executables
  Also clean the fuzz subdirectory
  copyediting README.md
  Protecting client/server fuzz targts with ifdefs
  Makefile support 1
  Fuzz README and direct compilation
  ...
2019-07-11 16:17:18 +01:00
Jaeden Amero c510b1ab29 Merge remote-tracking branch 'origin/development' into development-restricted
* origin/development: (33 commits)
  Test with MBEDTLS_ECP_RESTARTABLE
  Allow TODO in code
  Use the docstring in the command line help
  Split _abi_compliance_command into smaller functions
  Record the commits that were compared
  Document how to build the typical argument for -s
  Allow running /somewhere/else/path/to/abi_check.py
  Warn if VLAs are used
  Remove redundant compiler flag
  Consistently spell -Wextra
  Update Mbed Crypto to contain mbed-crypto#152
  Improve compatibility with firewalled networks
  Dockerfile: apt -> apt-get
  Change Docker container to bionic
  Clean up file prologue comments
  Add docker-based test scripts
  ChangeLog: Add ChangeLog entry for #2681
  Allow declarations after statements
  CMake: Add a subdirectory build regression test
  README: Enable builds as a CMake subproject
  ...
2019-07-10 08:46:54 +01:00
Jaeden Amero 74a87f8b29 Merge remote-tracking branch 'origin/pr/2738' into development
* origin/pr/2738:
  Test with MBEDTLS_ECP_RESTARTABLE
2019-07-10 07:55:25 +01:00
Jaeden Amero 5db519b59a Merge remote-tracking branch 'origin/pr/2730' into development
* origin/pr/2730:
  Allow TODO in code
  Use the docstring in the command line help
2019-07-10 07:55:25 +01:00
Jaeden Amero 0b8b5e3393 Merge remote-tracking branch 'origin/pr/2706' into development
* origin/pr/2706:
  Update Mbed Crypto to contain mbed-crypto#152
  CMake: Add a subdirectory build regression test
  README: Enable builds as a CMake subproject
  ChangeLog: Enable builds as a CMake subproject
  Remove use of CMAKE_SOURCE_DIR
2019-07-10 07:54:49 +01:00
Philippe Antoine 702c65922f Add a linker flag to enable gcov in basic-build-test.sh 2019-07-09 17:44:53 +02:00
Jaeden Amero 0f220ec73b Test with MBEDTLS_ECP_RESTARTABLE
We accidentally disabled testing with MBEDTLS_ECP_RESTARTABLE. Re-enable
testing with restartable ECP when MBEDTLS_USE_PSA_CRYPTO is not set.

Fixes 971dea3745 ("Enable USE_PSA_CRYPTO with config.pl full")
2019-07-05 15:14:57 +01:00
Gilles Peskine 55b49ee10f Allow TODO in code
Don't reject TODO in code. Fix #2587
2019-07-04 19:39:06 +02:00
Gilles Peskine 7dfcfceb49 Use the docstring in the command line help 2019-07-04 19:39:06 +02:00
Ron Eldor 5131f771ef Fix parsing issue when int parameter is in base 16
Fix error `ValueError: invalid literal for int() with base 10:` that
is caused when a parameter is given in base 16. Use relevant base
when calling `int()` function.
2019-07-02 11:02:29 +03:00
Jaeden Amero ee6f9b2a9f
Merge pull request #152 from Patater/cmake-subproject-fix
Enable use of Mbed TLS and Mbed Crypto as a CMake subproject
2019-07-02 08:39:11 +01:00
Peter Kolbus 718c74ca85 Improve compatibility with firewalled networks
* Allow specifying MBEDTLS_DOCKER_REGISTRY for organizations that have
  a mirroring proxy of Docker Hub
* Specify "--network host" during build to ensure use of the host's
  DNS resolution.
2019-06-29 17:45:34 -05:00
Peter Kolbus 49c2435a40 Change Docker container to bionic
Commit 117b8a4516 requires version 6+
of i686-w64-mingw32-gcc to run the mingw builds, but Ubuntu Xenial (16.04)
supplies 5.3.1. Change the Docker container to Ubuntu Bionic (18.04) to
pick up a version that will run the tests.
2019-06-29 17:45:34 -05:00
Peter Kolbus 4225b1a966 Clean up file prologue comments
Update new files to include the same legalese as in aes.h. Editorial
touchups in Dockerfile and ssl-opt-in-docker.sh.
2019-06-29 17:45:34 -05:00
Peter Kolbus e4e2d3a396 Add docker-based test scripts
Enable running tests under Docker. This makes it easier to spin up an
environment with all dependencies (especially the multiple versions of
openssl and gnutls needed).
* tests/docker/xenial/Dockerfile: Definition for the docker image,
  including local builds for openssl and gnutls.
* tests/scripts/docker_env.sh: New helper script that creates the Docker
  image and has a function to run a command in the Docker container.
* tests/docker/all-in-docker.sh: Wrapper for all.sh under Docker.
* tests/docker/basic-in-docker.sh: Script that runs the same commands as
  .travis.yml, in Docker.
* tests/ssl-opt-in-docker.sh: Wrapper to run ssl-opt.sh in Docker.
* tests/compat-in-docker.sh: Wrapper to run compat.sh in Docker.
* tests/make-in-docker.sh: Wrapper to run make in Docker.

Change-Id: Ie092b1deed24c24c3859754535589523ce1d0a58
2019-06-29 17:45:34 -05:00
Philippe Antoine 5dece6da2c Restore programs/fuzz/Makefile after in-tree cmake 2019-06-27 16:55:07 +02:00
k-stachowiak 5559b31b6b Disable optimizations for the full+make+gcc all.sh component 2019-06-27 11:28:11 +02:00
Gilles Peskine a8ade16ffd Gate secure element support by a separate config option
Secure element support has its own source file, and in addition
requires many hooks in other files. This is a nontrivial amount of
code, so make it optional (but default on).
2019-06-26 20:01:35 +02:00
k-stachowiak 0291cb7180 Add an alternarive full build component to all.sh 2019-06-26 15:52:12 +02:00
Jaeden Amero e8451f2274 CMake: Add a subdirectory build regression test
If we have a regression with the "build Mbed Crypto as a subdirectory
with CMake" feature and fail to build, fail the test.
2019-06-26 12:46:53 +01:00
Philippe Antoine 1c582c3b8b Restore tests/fuzz/Makefile after in-tree cmake
Save and restore the Makefile in tests/fuzz like the other makefiles.
2019-06-25 21:55:21 +02:00
Philippe Antoine cd2c127f75 Adds LDFLAGS fsanitize=address 2019-06-25 21:50:07 +02:00
Jaeden Amero ab83fdf944 CMake: Add a subdirectory build regression test
If we have a regression with the "build Mbed TLS as a subdirectory with
CMake" feature and fail to build, fail the test.
2019-06-25 15:21:24 +01:00
Ron Eldor 64e45950de Make the script portable to both pythons
Make the script work for python3 and for python2
2019-06-25 14:56:43 +03:00
Ron Eldor 5075f4df18 Update the test encoding to support python3
Since Python3 handles encoding differently than Python2,
a change in the way the data is encoded and sent to the target is needed.
1. Change the test data to be sent as hex string
2. Convert the characters to binary bytes.

This is done because the mbed tools translate the encoding differently
(mbed-greentea, and mbed-htrunner)
2019-06-25 14:56:01 +03:00
Ron Eldor 33908e8429 update the test script
Update `mbedtls_test.py` script to work with Python 3.7.
resolves #2653
2019-06-25 14:55:43 +03:00
Jaeden Amero bd3a7464b7 Merge remote-tracking branch 'restricted/pr/573' into development-restricted
* restricted/pr/573:
  Remove redundant config.pl call
  Add a test for signing content with a long ECDSA key
  Add documentation notes about the required size of the signature buffers
  Add missing MBEDTLS_ECP_C dependencies in check_config.h
  Change size of preallocated buffer for pk_sign() calls
2019-06-24 11:40:33 +01:00
Jaeden Amero fd0f65459c Merge remote-tracking branch 'origin/pr/2697' into development
* origin/pr/2697:
  Update crypto submodule
  Add all.sh component that exercises invalid_param checks
  Remove mbedtls_param_failed from programs
  Make it easier to define MBEDTLS_PARAM_FAILED as assert
  Make test suites compatible with #include <assert.h>
  Pass -m32 to the linker as well
  Don't systematically rebuild programs
2019-06-21 13:21:05 +01:00
Jaeden Amero ca4e4a8b2a Merge remote-tracking branch 'origin/pr/2614' into development
* origin/pr/2614:
  Use 'config.pl baremetal' in all.sh
2019-06-21 13:15:07 +01:00
Gilles Peskine adcde5e997 Add all.sh component that exercises invalid_param checks
With the change to the full config, there were no longer any tests
that exercise invalid-parameter behavior. The test suite exercises
invalid-parameter behavior by calling TEST_INVALID_PARAM and friends,
relying on the test suite's mbedtls_check_param function. This
function is only enabled if MBEDTLS_CHECK_PARAMS is defined but not
MBEDTLS_CHECK_PARAMS_ASSERT.

Add a component to all.sh that enables MBEDTLS_CHECK_PARAMS but
disables MBEDTLS_CHECK_PARAMS_ASSERT and doesn't define
MBEDTLS_PARAM_FAILED. This way, the xxx_invalid_param() tests do run.
Since sample programs don't provide a mbedtls_check_param function,
this component doesn't build the sample programs.
2019-06-14 13:09:13 +02:00
Jaeden Amero e1b02df515 Merge remote-tracking branch 'origin/pr/2260' into development
* origin/pr/2260:
  Update crypto submodule
  Remove heading spaces in tests/data_files/Makefile
  Re-generate library/certs.c from script
  Add new line at the end of test-ca2.key.enc
  Use strict syntax to annotate origin of test data in certs.c
  Add run to all.sh exercising !MBEDTLS_PEM_PARSE_C + !MBEDTLS_FS_IO
  Allow DHM self test to run without MBEDTLS_PEM_PARSE_C
  ssl-opt.sh: Auto-skip tests that use files if MBEDTLS_FS_IO unset
  Document origin of hardcoded certificates in library/certs.c
  Adapt ChangeLog
  Rename server1.der to server1.crt.der
  Add DER encoded files to git tree
  Add build instructions to generate DER versions of CRTs and keys
  Document "none" value for ca_path/ca_file in ssl_client2/ssl_server2
  ssl_server2: Skip CA setup if `ca_path` or `ca_file` argument "none"
  ssl_client2: Skip CA setup if `ca_path` or `ca_file` argument "none"
  Correct white spaces in ssl_server2 and ssl_client2
  Adapt ssl_client2 to parse DER encoded test CRTs if PEM is disabled
  Adapt ssl_server2 to parse DER encoded test CRTs if PEM is disabled
2019-06-14 08:46:48 +01:00
k-stachowiak 69969ef088 Remove redundant config.pl call 2019-06-13 11:53:17 +02:00
k-stachowiak c1955559ad Add a test for signing content with a long ECDSA key
Due to the way the current PK API works, it may have not been clear
for the library clients, how big output buffers they should pass
to the signing functions. Depending on the key type they depend on
MPI or EC specific compile-time constants.

Inside the library, there were places, where it was assumed that
the MPI size will always be enough, even for ECDSA signatures.
However, for very small sizes of the MBEDTLS_MPI_MAX_SIZE and
sufficiently large key, the EC signature could exceed the MPI size
and cause a stack overflow.

This test establishes both conditions -- small MPI size and the use
of a long ECDSA key -- and attempts to sign an arbitrary file.
This can cause a stack overvlow if the signature buffers are not
big enough, therefore the test is performed for an ASan build.
2019-06-10 11:46:18 +02:00
Gilles Peskine 5d26e7cbfe Pass -m32 to the linker as well
For unit tests and sample programs, CFLAGS=-m32 is enough to get a
32-bit build, because these programs are all compiled directly
from *.c to the executable in one shot. But with makefile rules that
first build object files and then link them, LDFLAGS=-m32 is also
needed.
2019-06-07 18:15:37 +02:00
Jaeden Amero 6b1683dd5d all.sh: Perform targeted EtM tests for MAC-less configs
When testing a configuration where no ciphersuites have MAC, via
component_test_when_no_ciphersuites_have_mac(), perform a targeted test
of only encrypt-then-MAC tests within ssl-opt.sh.
2019-06-05 14:42:50 +01:00
Jaeden Amero 2de07f1dd1 ssl: Don't access non-existent encrypt_then_mac field
When MBEDTLS_SSL_ENCRYPT_THEN_MAC is enabled, but not
MBEDTLS_SSL_SOME_MODES_USE_MAC, mbedtls_ssl_derive_keys() and
build_transforms() will attempt to use a non-existent `encrypt_then_mac`
field in the ssl_transform.

    Compile [ 93.7%]: ssl_tls.c
    [Error] ssl_tls.c@865,14: 'mbedtls_ssl_transform {aka struct mbedtls_ssl_transform}' ha
s no member named 'encrypt_then_mac'
    [ERROR] ./mbed-os/features/mbedtls/src/ssl_tls.c: In function 'mbedtls_ssl_derive_keys'
:
    ./mbed-os/features/mbedtls/src/ssl_tls.c:865:14: error: 'mbedtls_ssl_transform {aka str
uct mbedtls_ssl_transform}' has no member named 'encrypt_then_mac'
         transform->encrypt_then_mac = session->encrypt_then_mac;
                  ^~

Change mbedtls_ssl_derive_keys() and build_transforms() to only access
`encrypt_then_mac` if `encrypt_then_mac` is actually present.

Add a regression test to detect when we have regressions with
configurations that do not include any MAC ciphersuites.

Fixes d56ed2491b ("Reduce size of `ssl_transform` if no MAC ciphersuite is enabled")
2019-06-05 14:09:29 +01:00
Hanno Becker a545be2622 Add run to all.sh exercising !MBEDTLS_PEM_PARSE_C + !MBEDTLS_FS_IO 2019-06-03 17:46:56 +01:00
Gilles Peskine a3b93ff893 Make docstring style consistent
Use PEP 257 indented docstring style, mostly: always with """, with the
terminating """ on a separate line if the docstring is more than one
line, and with all lines indented to the opening """.

This commit does not change the text to keep the first paragraph single-line.
2019-06-03 11:23:56 +02:00
Jaeden Amero 3b025ac52b test: Always use make clean by itself
When running make with parallelization, running both "clean" and "lib"
with a single make invocation can lead to each target building in
parallel. It's bad if lib is partially done building something, and then
clean deletes what was built. This can lead to errors later on in the
lib target.

    $ make -j9 clean lib
      CC    aes.c
      CC    aesni.c
      CC    arc4.c
      CC    aria.c
      CC    asn1parse.c
      CC    ./library/error.c
      CC    ./library/version.c
      CC    ./library/version_features.c
      AR    libmbedcrypto.a
    ar: aes.o: No such file or directory
    Makefile:120: recipe for target 'libmbedcrypto.a' failed
    make[2]: *** [libmbedcrypto.a] Error 1
    Makefile:152: recipe for target 'libmbedcrypto.a' failed
    make[1]: *** [libmbedcrypto.a] Error 2
    Makefile:19: recipe for target 'lib' failed
    make: *** [lib] Error 2
    make: *** Waiting for unfinished jobs....

To avoid this sort of trouble, always invoke clean by itself without
other targets throughout the library. Don't run clean in parallel with
other rules. The only place where clean was run in parallel with other
targets was in list-symbols.sh.
2019-05-31 12:47:07 +01:00
Gilles Peskine 54f544581a Pacify Pylint
Pass Pylint by cleaning up the code where possible and silencing
Pylint where I know better.

No behavior change.
2019-05-29 09:58:59 +02:00
Gilles Peskine 42a0a0aeea Obey Python naming and method structure conventions
* Rename internal methods and fields to start with an underscore.
* Rename global constants to uppercase.
* Change methods that don't use self to be class methods or static
  methods as appropriate.

No behavior change in this commit.
2019-05-29 09:58:59 +02:00
Gilles Peskine 2938268fb4 Merge remote-tracking branch 'upstream-crypto/development' into psa-api-1.0-beta-merge_development_20190524
Conflicts:
* library/ssl_cli.c, library/ssl_tls.c:
  Removed on the development branch. Keep them removed.
* include/psa/crypto_extra.h, library/psa_crypto_storage.c,
  tests/suites/test_suite_psa_crypto.data,
  tests/suites/test_suite_psa_crypto.function,
  tests/suites/test_suite_psa_crypto_persistent_key.data,
  tests/suites/test_suite_psa_crypto_slot_management.data,
  tests/suites/test_suite_psa_crypto_slot_management.function:
  Modified on the development branch only to implement the enrollment
  algorithm, which has been reimplemented on the API branch.
  Keep the API branch.
2019-05-29 09:57:29 +02:00
Jaeden Amero 2ab5cf658f Merge remote-tracking branch 'origin/pr/2403' into development
* origin/pr/2403: (24 commits)
  crypto: Update to Mbed Crypto 8907b019e7
  Create seedfile before running tests
  crypto: Update to Mbed Crypto 81f9539037
  ssl_cli.c : add explicit casting to unsigned char
  Generating visualc files - let Mbed TLS take precedence over crypto
  Add a link to the seedfile for out-of-tree cmake builds
  Adjust visual studio file generation to always use the crypto submodule
  all.sh: unparallelize mingw tests
  all.sh - disable parallelization for shared target tests
  config.pl: disable PSA_ITS_FILE and PSA_CRYPTO_STORAGE for baremetal
  all.sh: unset crypto storage define in a psa full config cmake asan test
  all.sh: unset FS_IO-dependent defines for tests that do not have it
  curves.pl - change test script to not depend on the implementation
  Export the submodule flag to sub-cmakes
  Disable MBEDTLS_ECP_RESTARTABLE in full config
  Export the submodule flag to sub-makes
  Force the usage of crypto submodule
  Fix crypto submodule usage in Makefile
  Documentation rewording
  Typo fixes in documentation
  ...
2019-05-23 09:08:55 +01:00
Gilles Peskine 3c8ccc068f Create seedfile before running tests
With MBEDTLS_USE_PSA_CRYPTO and MBEDTLS_ENTROPY_NV_SEED enabled, the
tests need a seedfile. Since test_suite_entropy is no longer there to
create it, and MBEDTLS_USE_PSA_CRYPTO is now enabled in the full
config, create tests/seedfile explicitly in basic-build-test.sh.
2019-05-23 03:01:35 -04:00
Andrzej Kurek 62faadd6a3 all.sh: unparallelize mingw tests
Sometimes, when building the shared target with crypto submodule, one could
get an "No rule to make target '../crypto/library/libmbedcrypto.so'" error.
This is due to two reasons - building in parallel and (probably) an
incomplete/incorrect list of dependencies for certain targets. The proposed
solution is to disable parallel builds with crypto submodule for now.
An issue has been raised here: https://github.com/ARMmbed/mbedtls/issues/2634
2019-05-23 03:01:35 -04:00
Andrzej Kurek 87615770c7 all.sh - disable parallelization for shared target tests
Sometimes, when building the shared target with crypto submodule, one could
get an "No rule to make target '../crypto/library/libmbedcrypto.so'" error.
This is due to two reasons - building in parallel and (probably) an
incomplete/incorrect list of dependencies for certain targets. The proposed
solution is to disable parallel builds with crypto submodule for now.
An issue has been raised here: https://github.com/ARMmbed/mbedtls/issues/2634
2019-05-23 03:01:35 -04:00
Andrzej Kurek 324b2f76ca all.sh: unset crypto storage define in a psa full config cmake asan test 2019-05-23 03:01:35 -04:00
Andrzej Kurek 73757080c1 all.sh: unset FS_IO-dependent defines for tests that do not have it 2019-05-23 03:01:35 -04:00
Andrzej Kurek 098b16ca83 curves.pl - change test script to not depend on the implementation
Currently the top-level makefile deploys commands to both Mbed TLS and
the submodule. Running make in the "tests" directory builds only the TLS tests.
The top level CMake on the other hand does not have the "tests" target defined,
so it also cannot be used, hence the raw "make".
2019-05-23 03:01:35 -04:00
Andrzej Kurek 346747cd24 Force the usage of crypto submodule
Remove all.sh tests exercising the optional usage of submodule
2019-05-23 03:01:35 -04:00
Manuel Pégourié-Gonnard 971dea3745 Enable USE_PSA_CRYPTO with config.pl full
Previously it was disabled as too experimental, which no longer holds. Also,
this option introduces new APIs, so it's not only about an internal
alternative (as the comment in config.pl used to state) - people who request a
full config should get all of the available APIs.

Adapt all.sh: now all builds with full config will also test this option, and
builds with the default config will test without it. Just to be sure, let's
have a build with full config minus this option.

Update documentation of MBEDTLS_USE_PSA_CRYPTO to reflect the status of the
new APIs it enables in Mbed TLS and why they're still opt-in.
2019-05-23 03:01:35 -04:00
Manuel Pégourié-Gonnard de7636e2dd Enable MBEDTLS_PSA_CRYPTO_C by default
Also enable it in scripts/config.pl full, as well as two storage options that
were only blacklisted from full config because they depended on
MBEDTLS_PSA_CRYPTO_C.
2019-05-23 03:01:35 -04:00
Manuel Pégourié-Gonnard d8167e85d6 Build from submodule by default (make, cmake)
Adapt tests in all.sh:
- tests with submodule enabled (default) no longer need to enable it
  explicitly, and no longer need runtime tests, as those are now handled by
all other test cases in this script
- tests with submodule disabled (old default) now need to disable it
  explicitly, and execute some runtime tests, as those are no longer tested
anywhere else in this script

Adapt documentation in Readme: remove the section "building with submodule"
and replace it with a new section before the other building sections.
Purposefully don't document how to build not from the submodule, as that
option is going away soon.
2019-05-23 03:01:35 -04:00
Gilles Peskine 9a52cf3d97 list-symbols.sh: if the build fails, print the build transcript
If "make clean lib" fails in list-symbols.sh, print the transcript
from running make.
2019-05-22 18:22:58 +02:00
Gilles Peskine 0c12e308af Document "check-names.sh -v" 2019-05-22 18:22:45 +02:00
Gilles Peskine dcaefae849 Parametrize Diffie-Hellman keys by a group identifier
Parametrize finite-field Diffie-Hellman key types with a DH group
identifier, in the same way elliptic curve keys are parametrized with
an EC curve identifier.

Define the DH groups from the TLS registry (these are the groups from
RFC 7919).

Replicate the macro definitions and the metadata tests from elliptic
curve identifiers to DH group identifiers.

Define PSA_DH_GROUP_CUSTOM as an implementation-specific extension for
which domain parameters are used to specify the group.
2019-05-16 19:51:10 +02:00
Gilles Peskine 13f97dc164 all.sh: invoke check-names.sh in print-trace-on-exit mode 2019-05-15 17:55:33 +02:00
Gilles Peskine 36428d34c9 Print a command trace if the check-names.sh exits unexpectedly
We've observed that sometimes check-names.sh exits unexpectedly with
status 2 and no error message. The failure is not reproducible. This
commits makes the script print a trace if it exits unexpectedly.
2019-05-15 17:29:15 +02:00
Jaeden Amero 7b3603c6d8 Merge remote-tracking branch 'tls/development' into development
Resolve merge conflicts by performing the following actions:

- Reject changes to deleted files.
- Reject changes to generate_errors.pl and generate_visualc_files.pl.
  Don't add an 'include-crypto' option which would attempt to use the
  non-existent crypto submodule.
- list-identifiers.sh had the `--internal` option added to it, which
  lists identifiers only in internal headers. Add PSA-specific internal
  headers to list-identifiers.sh.

* origin/development: (40 commits)
  Document the scripts behaviour further
  Use check_output instead of Popen
  all.sh: Require i686-w64-mingw32-gcc version >= 6
  generate_visualc_files.pl: add mbedtls source shadowing by crypto
  generate_errors.pl: refactor and simplify the code
  Start unused variable with underscore
  Correct documentation
  generate_errors.pl: typo fix
  revert changes to generate_features.pl and generate_query_config.pl
  Check that the report directory is a directory
  Use namespaces instead of full classes
  Fix pylint issues
  Don't put abi dumps in subfolders
  Add verbose switch to silence all output except the final report
  Fetch the remote crypto branch, rather than cloning it
  Prefix internal functions with underscore
  Add RepoVersion class to make handling of many arguments easier
  Reduce indentation levels
  Improve documentation
  Use optional arguments for setting repositories
  ...
2019-05-01 14:12:43 +01:00
Manuel Pégourié-Gonnard df2bfcfe57 Use 'config.pl baremetal' in all.sh 2019-04-29 12:44:12 +02:00
Jaeden Amero dfcf84aea5 tests: Update generator with Mbed Crypto comments
Update comments in the top of the test code generator script with the
name of the parent project.
2019-04-25 11:46:21 +01:00
Jaeden Amero bb1f701212 config: Remove X.509 options
Note that this fails check-names.sh because options that TLS and X.509
files use are no longer present in config.h.
2019-04-25 11:46:21 +01:00
Jaeden Amero 1c66e48670 config: Remove TLS and NET options
Remove TLS and NET options from config files and scripts.

Note that this fails check-names.sh because options that TLS and NET
files use are no longer present in config.h.
2019-04-25 11:46:21 +01:00
Jaeden Amero 7fcc7bc576 check-names: Enable referencing Mbed TLS macros
Add a macro whitelist which enables us to refer to Mbed TLS macros that
are no longer present in Mbed Crypto.
2019-04-25 11:46:21 +01:00
Jaeden Amero 1ad37309e4 Remove irrelevant configs
Remove configuration files that are TLS-specific or otherwise not
relevant to a pure-crypto library.
2019-04-25 11:46:21 +01:00
Jaeden Amero 1cd7bea2b1 Merge remote-tracking branch 'origin/pr/2586' into development
* origin/pr/2586:
  all.sh: Require i686-w64-mingw32-gcc version >= 6
2019-04-24 11:23:54 +01:00
Jaeden Amero 9679cb48d5 Merge remote-tracking branch 'origin/pr/2549' into development
* origin/pr/2549:
  Document the scripts behaviour further
  Add --internal option to list-identifiers.sh
2019-04-24 11:22:24 +01:00
Darryl Green 1ae4886c82 Document the scripts behaviour further 2019-04-18 13:09:25 +01:00
Gilles Peskine 882e57ecba psa_constant_names: support key agreement algorithms 2019-04-18 09:42:21 +02:00
Jaeden Amero 117b8a4516 all.sh: Require i686-w64-mingw32-gcc version >= 6
Require mingw gcc version 6 or greater in order to ensure
BCryptGenRandom() is available.
2019-04-17 15:23:17 +01:00
Jaeden Amero 521dbc67da Merge remote-tracking branch 'tls/development' into development
Merge Mbed TLS at f790a6cbee into Mbed Crypto.

Resolve conflicts by performing the following:
    - Reject changes to README.md
    - Don't add crypto as a submodule
    - Remove test/ssl_cert_test from programs/Makefile
    - Add cipher.nist_kw test to tests/CMakeLists.txt
    - Reject removal of crypto-specific all.sh tests
    - Reject update to SSL-specific portion of component_test_valgrind
      in all.sh
    - Reject addition of ssl-opt.sh testing to component_test_m32_o1 in
      all.sh

* tls/development: (87 commits)
  Call mbedtls_cipher_free() to reset a cipher context
  Don't call mbedtls_cipher_setkey twice
  Update crypto submodule
  Minor fixes in get certificate policies oid test
  Add certificate policy oid x509 extension
  cpp_dummy_build: Add missing header psa_util.h
  Clarify comment mangled by an earlier refactoring
  Add an "out-of-box" component
  Run ssl-opt.sh on 32-bit runtime
  Don't use debug level 1 for informational messages
  Skip uncritical unsupported extensions
  Give credit to OSS-Fuzz for #2404
  all.sh: remove component_test_new_ecdh_context
  Remove crypto-only related components from all.sh
  Remove ssl_cert_test sample app
  Make CRT callback tests more robust
  Rename constant in client2.c
  Document and test flags in x509_verify
  Fix style issues and a typo
  Fix a rebase error
  ...
2019-04-17 12:12:24 +01:00
Jaeden Amero c41a3285de Merge remote-tracking branch 'origin/pr/2559' into development
* origin/pr/2559:
  Clarify comment mangled by an earlier refactoring
  Add an "out-of-box" component
  Run ssl-opt.sh on 32-bit runtime
2019-04-16 15:09:42 +01:00
Gilles Peskine f1349e4bfe Clarify comment mangled by an earlier refactoring 2019-04-10 18:41:53 +02:00
Gilles Peskine 7832c9fc3d Add an "out-of-box" component
Just run `make` and `make test`. And `selftest` for good measure.
2019-04-10 18:41:53 +02:00
Gilles Peskine 4b317616eb Run ssl-opt.sh on 32-bit runtime
Run ssl-opt.sh on x86_32 with ASan. This may detect bugs that only
show up on 32-bit platforms, for example due to size_t overflow.

For this component, turn off some memory management features that are
not useful, potentially slow, and may reduce ASan's effectiveness at
catching buffer overflows.
2019-04-10 18:41:53 +02:00
Andrzej Kurek 4a8d2dfdd6 all.sh: remove component_test_new_ecdh_context
Remove the ecdh_context component to have it only in the crypto repository
2019-04-08 06:20:00 -04:00
Andrzej Kurek cd9286f1ca Remove crypto-only related components from all.sh 2019-04-08 05:49:08 -04:00
Jaeden Amero 7c1e6193af Merge remote-tracking branch 'origin/pr/2464' into development
* origin/pr/2464:
  Allow main() to lack a docstring.
  Silence pylint
  check-files.py: readability improvement in permission check
  check-files.py: use class fields for class-wide constants
  check-files.py: clean up class structure
  abi_check.py: Document more methods
  check-files.py: document some classes and methods
  Fix pylint errors going uncaught
  Call pylint3, not pylint
  New, documented pylint configuration
2019-04-05 14:19:09 +01:00
Darryl Green 383d1fa6a5 Add --internal option to list-identifiers.sh
When doing ABI/API checking, its useful to have a list of all the
identifiers that are defined in the internal header files, as we
do not promise compatibility for them. This option allows for a
simple method of getting them for use with the ABI checking script.
2019-04-05 13:59:52 +01:00
Jaeden Amero c70a3c76bf Merge remote-tracking branch 'tls/development' into development
Resolve conflicts actions:
 - Reject path changes to config.h
 - Reject submodule-related changes in build scripts (Makefile,
   CMakeLists.txt)
 - Add oid test suite to list of tests in tests/CMakeLists.txt,
   rejecting any test filtering related changes (which TLS uses to avoid
   duplicating crypto tests)
 - Add legacy ECDH test to all.sh without including
   all.sh tests that depend on SSL
2019-03-28 16:02:25 +00:00
Jaeden Amero 0ea33776ce Merge remote-tracking branch 'restricted/pr/552' into development
Ensure this merge passes tests by auto-generating query_config.c, adding
MBEDTLS_ECDH_LEGACY_CONTEXT to it.

* restricted/pr/552:
  Fix mbedtls_ecdh_get_params with new ECDH context
  Test undefining MBEDTLS_ECDH_LEGACY_CONTEXT in all.sh
  Define MBEDTLS_ECDH_LEGACY_CONTEXT in config.h
  Add changelog entry for mbedtls_ecdh_get_params robustness
  Fix ecdh_get_params with mismatching group
  Add test case for ecdh_get_params with mismatching group
  Add test case for ecdh_calc_secret
  Fix typo in documentation
2019-03-27 17:01:16 +00:00
Jaeden Amero 9714510736 all.sh: Generate seedfile for crypto submodule tests
When running tests from the crypto submodule, generate and use a
seedfile within the crypto/tests directory.
2019-03-19 15:45:09 +00:00
Gilles Peskine e435f23019 Remove psa_crypto_storage_file
Now that we have ITS over files, we no longer need a direct backend
for key storage over files. Remove psa_crypto_storage_file and its
tests.

Switch MBEDTLS_PSA_CRYPTO_STORAGE_ITS_C and MBEDTLS_PSA_ITS_FILE_C on
by default. This preserves functionality and test coverage in the
default configuration, but forgets any key previously stored using the
file backend.
2019-03-15 11:14:51 +01:00
Jaeden Amero 9afb2e9921 Remove tests that depend on TLS or X.509 2019-03-11 16:49:26 +00:00
Jaeden Amero e23737c618 recursion.pl: Don't depend on X.509 2019-03-11 16:48:36 +00:00
Jaeden Amero 03c60de0e0 query_config: Move to programs/test
As the SSL programs, like ssl_client2 and ssl_server2, are dependent on
SSL and therefore about to be removed, the only consumer of query_config
is the query_compile_time_config test. As such, it makes sense to move
query_config to be next to what uses it.
2019-03-11 16:48:36 +00:00
Jaeden Amero 9b90f2e294 all.sh: Remove dependency on TLS, NET, and X.509 2019-03-11 16:48:35 +00:00
Jaeden Amero 2b9eb0bd6c Merge remote-tracking branch 'tls/development' into development
* origin/development: (113 commits)
  Update query_config.c
  Fix failure in SSLv3 per-version suites test
  Adjust DES exclude lists in test scripts
  Clarify 3DES changes in ChangeLog
  Fix documentation for 3DES removal
  Exclude 3DES tests in test scripts
  Fix wording of ChangeLog and 3DES_REMOVE docs
  Reduce priority of 3DES ciphersuites
  Fix unused variable warning in ssl_parse_certificate_coordinate()
  Update the crypto submodule to a78c958
  Fix ChangeLog entry to correct release version
  Fix typo in x509write test data
  Add ChangeLog entry for unused bits in bitstrings
  Improve docs for named bitstrings and their usage
  Add tests for (named) bitstring to suite_asn1write
  Add new function mbedtls_asn1_write_named_bitstring()
  Add missing compile time guard in ssl_client2
  Update programs/ssl/query_config.c
  ssl_client2: Reset peer CRT info string on reconnect
  Add further debug statements on assertion failures
  ...
2019-03-07 12:02:18 +00:00
Jaeden Amero 0a9f9b20b6 Merge remote-tracking branch 'origin/pr/2317' into development
* origin/pr/2317:
  Update ChangeLog
  all.sh: Test MBEDTLS_MPI_WINDOW_SIZE=1
  Fix DEADCODE in mbedtls_mpi_exp_mod()
2019-03-05 16:33:42 +00:00
Jaeden Amero 41e1986ed0 Merge remote-tracking branch 'origin/pr/2293' into development
* origin/pr/2293:
  Declare test_suite_aes.ofb to CMake
  Add a facility to skip running some test suites
  run-test-suites: update the documentation
2019-03-05 16:30:37 +00:00
Simon Butcher 535ee4a35b Merge remote-tracking branch 'public/pr/2421' into development
* public/pr/2421: (68 commits)
  Fix unused variable warning in ssl_parse_certificate_coordinate()
  Add missing compile time guard in ssl_client2
  Update programs/ssl/query_config.c
  ssl_client2: Reset peer CRT info string on reconnect
  Add further debug statements on assertion failures
  Fix typo in documentation of ssl_parse_certificate_chain()
  Add debug output in case of assertion failure
  Fix typo in SSL ticket documentation
  Add config sanity check for !MBEDTLS_SSL_KEEP_PEER_CERTIFICATE
  ssl_client2: Zeroize peer CRT info buffer when reconnecting
  Reintroduce numerous ssl-opt.sh tests if !MBEDTLS_SSL_KEEP_PEER_CERT
  ssl_client2: Extract peer CRT info from verification callback
  Improve documentation of mbedtls_ssl_get_peer_cert()
  Improve documentation of MBEDTLS_SSL_KEEP_PEER_CERTIFICATE
  Fix indentation of Doxygen comment in ssl_internal.h
  Set peer CRT length only after successful allocation
  Remove question in comment about verify flags on cli vs. server
  Remove misleading and redundant guard around restartable ECC field
  Add test for !MBEDTLS_SSL_KEEP_PEER_CERTIFICATE to all.sh
  Free peer CRT chain immediately after verifying it
  ...
2019-03-01 12:46:07 +00:00
Andres Amaya Garcia 419bd0011f Adjust DES exclude lists in test scripts 2019-03-01 10:22:04 +01:00
Andres Amaya Garcia 2dadab7d3f Exclude 3DES tests in test scripts 2019-03-01 10:21:11 +01:00
Jaeden Amero a9d6ba2510 Merge remote-tracking branch 'tls/development' into development
Additional work done as part of merge:
    - Run ./tests/scripts/check-generated-files.sh and check in the
      resulting changes to programs/ssl/query_config.c
2019-02-27 15:15:53 +00:00
Gilles Peskine e915d532a6 Silence pylint
Silence pylint in specific places where we're doing slightly unusual
or dodgy, but correct.
2019-02-27 11:03:25 +01:00
Gilles Peskine 23e64f226b check-files.py: readability improvement in permission check 2019-02-27 11:03:25 +01:00
Gilles Peskine 1e9698af4b check-files.py: use class fields for class-wide constants
In an issue tracker, heading and files_exemptions are class-wide
constants, so make them so instead of being per-instance fields.
2019-02-27 11:03:25 +01:00
Gilles Peskine 6ee576e0b5 check-files.py: clean up class structure
Line issue trackers are conceptually a subclass of file issue
trackers: they're file issue trackers where issues arise from checking
each line independently. So make it an actual subclass.

Pylint pointed out the design smell: there was an abstract method that
wasn't always overridden in concrete child classes.
2019-02-27 11:03:25 +01:00
Gilles Peskine 0d060ef328 check-files.py: document some classes and methods
Document all classes and longer methods.

Declare a static method as such. Pointed out by pylint.
2019-02-27 11:03:25 +01:00
Gilles Peskine aad2ebdf30 Fix pylint errors going uncaught
Make check-python-files.sh run pylint on all *.py files (in
directories where they are known to be present), rather than list
files explicitly.

Fix a bug whereby the return status of check-python-files.sh was only
based on the last file passing, i.e. errors in other files were
effectively ignored.

Make check-python-files.sh run pylint unconditionally. Since pylint3
is not critical, make all.sh to skip running check-python-files.sh if
pylint3 is not available.
2019-02-27 11:03:25 +01:00
Gilles Peskine b2c269eeee Call pylint3, not pylint
We use Python 3, so call Pylint for Python 3, not for Python 2.
2019-02-27 11:03:24 +01:00
Gilles Peskine 7f61575cba New, documented pylint configuration
The pylint configuration in .pylint was a modified version of the
output of `pylint --generate-rcfile` from an unknown version of
pylint. Replace it with a file that only contains settings that are
modified from the default, with an explanation of why each setting is
modified.

The new .pylintrc was written from scratch, based on the output of
pylint on the current version of the files and on a judgement of what
to silence generically, what to silence on a case-by-case basis and
what to fix.
2019-02-27 11:03:24 +01:00
Hanno Becker 545ced45f7 Add test for !MBEDTLS_SSL_KEEP_PEER_CERTIFICATE to all.sh 2019-02-26 14:38:09 +00:00
Jaeden Amero 86016a03a1 Merge remote-tracking branch 'origin/pr/2338' into development 2019-02-22 12:55:30 +00:00