Commit graph

2780 commits

Author SHA1 Message Date
Ronald Cron afbe3ee87c tests: nist_kw: Prepare to char* to data_t* type change
In preparation of changing the type of some parameters
of mbedtls_nist_kw_wrap/unwrap() from `char *` to `data_t`
to get rid of the calls to mbedtls_test_unhexify():

- Change the name of parameters and local variables to
  clarify which ones are related to the outputs of the
  library functions under test and which ones are
  related to the expected values of those outputs.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-06-26 09:48:11 +02:00
Ronald Cron 7e8f1af0a6 tests: chacha20: Prepare to char* to data_t* type change
In preparation of changing the type of some parameters of
test_chacha20() from `char *` to `data_t` to get rid of the
calls to mbedtls_test_unhexify():

- Reduce the size of output[] buffer to 375 as its content
  is "ASCII expended" into a buffer of 751 bytes.
- Align naming of variables to store and check the
  output of mbedtls_chacha20_crypt(). No *dst* variables
  anynore, only *output* variables.
- Use two different buffers to store the expected output
  of mbedtls_chacha20_crypt() (expected_output_str[]) and
  the ASCII string representation of the output of
  mbedtls_chacha20_crypt() (output_string[]). Both were
  stored in dst_str[] before.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-06-26 09:47:54 +02:00
Ronald Cron d7a4df8341 tests: hkdf: Prepare to char* to data_t* type change
In preparation of changing the type of some parameters
of test_hkdf() from `char *` to `data_t` to get rid of the
calls to mbedtls_test_unhexify():

- Align naming of variables related to the expected okm
- Rename `okm_hex[]` to `okm_string[]`
- Added TEST_ASSERT( expected_okm_len <= sizeof( okm ) ) to check
  that the okm[] buffer is large enough for the okm output.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-06-26 09:47:25 +02:00
Ronald Cron 6d1f0ad7b6 tests: aria: Prepare to char* to data_t* type change
In preparation of changing the type of some parameters
of some test functions from `char *` to `data_t` to get
rid of the calls to mbedtls_test_unhexify():

- Align the name of source data length local variable
  with the name of the local variable containing the
  source data, respectively src_str and src_str_len.
- Change the type of length, index local variables
  from int to size_t.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-06-26 09:46:50 +02:00
Ronald Cron 5b13a86e11 tests: Reformating due to *hexify functions renaming
Command to find the files in which lines have gone
larger than 79 characters due to the renaming:

grep '.\{80\}' \
`git diff-tree --no-commit-id --name-only -r HEAD` \
| grep hexify

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-06-26 09:38:21 +02:00
Ronald Cron a0c9ff3e52 tests: Add mbedtls_test_ prefix to *hexify functions
Add mbedtls_test_ prefix to hexify() and unhexify()
test helper functions.

Command to change *.function files:
find . -name "*.function" -exec awk -i inplace \
    '{sub(/(un|)hexify\>/,"mbedtls_test_&")}1' {} \;

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-06-26 09:15:43 +02:00
Janos Follath f69b919844 Merge branch 'mbedtls-2.16-restricted' into mbedtls-2.16.7r0 2020-06-25 09:19:21 +01:00
Manuel Pégourié-Gonnard 805b1461b8 all.sh: clean up some uses of "local" variables
While pure sh doesn't have a concept of local variables, we can partially
emulate them by unsetting variables before we exit the function, and use the
convention of giving them lowercase names to distinguish from global
variables.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-23 11:42:59 +02:00
Manuel Pégourié-Gonnard d2b5ad871d Use all.sh in pre-push hook
The list in the pre-push hook was redundant with the list of `check_*`
components in all.sh, and unsurprisingly it was outdated.

Missing components were:

- check_recursion
- check_changelog
- check_test_cases
- check_python_files
- check_generate_test_code

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-23 11:37:04 +02:00
Manuel Pégourié-Gonnard 1ee685018a Make check_generate_test_code more -q friendly
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-23 11:36:32 +02:00
Manuel Pégourié-Gonnard de4ad2da38 Add a --quiet option to all.sh
The primary purpose is to use it to run all.sh -k -q in the pre-push hook, but
this can be useful in any circumstance where you're not interested in the full
output from each component and just want a short summary of which components
were run (and if any failed).

Note that only stdout from components is suppressed, stderr is preserved so
that errors are reported. This means components should avoid printing to
stderr in normal usage (ie in the absence of errors).

Currently all the `check_*` components obey this convention except:
- check_generate_test_code: unittest prints progress to stderr
- check_test_cases: lots of non-fatal warnings printed to stderr

These components will be fixed in follow-up commits.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-23 11:36:30 +02:00
Manuel Pégourié-Gonnard 1bff6847fb Adjust comments about SEED synchronisation
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-22 10:53:44 +02:00
Manuel Pégourié-Gonnard c2400d38fe Make basic-build-test.sh deterministic
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-22 10:53:42 +02:00
Manuel Pégourié-Gonnard 2df5857dbe Remove SHA-1 as a fallback option
- it's 2020, there shouldn't be too many systems out there where SHA-1 is the
  only available hash option, so its usefulness is limited
- OTOH testing configurations without SHA-2 reveal bugs that are not easy to
  fix in a fully compatible way

So overall, the benefit/cost ratio is not good enough to justify keeping SHA-1
as a fallback option here.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-18 12:17:59 +02:00
Manuel Pégourié-Gonnard 72177e362b Add fall-back to hash-based KDF for internal ECP DRBG
The dependency on a DRBG module was perhaps a bit strict for LTS branches, so
let's have an option that works with no DRBG when at least one SHA module is
present.

This changes the internal API of ecp_drbg_seed() by adding the size of the
MPI as a parameter. Re-computing the size from the number of limbs doesn't
work too well here as we're writing out to a fixed-size buffer and for some
curves (P-521) that would round up too much. Using mbedtls_mpi_get_len() is
not entirely satisfactory either as it would mean using a variable-length
encoding, with could open side channels.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-16 12:51:42 +02:00
Manuel Pégourié-Gonnard 047986c2f8 Add support for RESTARTABLE with internal RNG
Currently we draw pseudo-random numbers at the beginning and end of the main
loop. With ECP_RESTARTABLE, it's possible that between those two occasions we
returned from the multiplication function, hence lost our internal DRBG
context that lives in this function's stack frame. This would result in the
same pseudo-random numbers being used for blinding in multiple places. While
it's not immediately clear that this would give rise to an attack, it's also
absolutely not clear that it doesn't. So let's avoid that by using a DRBG
context that lives inside the restart context and persists across
return/resume cycles. That way the RESTARTABLE case uses exactly the
same pseudo-random numbers as the non-restartable case.

Testing and compile-time options:

- The case ECP_RESTARTABLE && !ECP_NO_INTERNAL_RNG is already tested by
  component_test_no_use_psa_crypto_full_cmake_asan.
- The case ECP_RESTARTABLE && ECP_NO_INTERNAL_RNG didn't have a pre-existing
  test so a component is added.

Testing and runtime options: when ECP_RESTARTABLE is enabled, the test suites
already contain cases where restart happens and cases where it doesn't
(because the operation is short enough or because restart is disabled (NULL
restart context)).

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-16 10:52:32 +02:00
Manuel Pégourié-Gonnard 23983f30de Add config.h option MBEDTLS_ECP_NO_INTERNAL_RNG
No effect so far, except on dependency checking, as the feature it's meant to
disable isn't implemented yet (so the descriptions in config.h and the
ChangeLog entry are anticipation for now).

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-16 10:51:42 +02:00
Bence Szépkúti f744bd72ee Update license headers to Apache-2.0 OR GPL-2.0-or-later
This will allow us to ship the LTS branches in a single archive

This commit was generated using the following script:

# ========================
#!/bin/sh

header1='\ *  SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later\
 *\
 *  This file is provided under the Apache License 2.0, or the\
 *  GNU General Public License v2.0 or later.\
 *\
 *  **********\
 *  Apache License 2.0:\
 *\
 *  Licensed under the Apache License, Version 2.0 (the "License"); you may\
 *  not use this file except in compliance with the License.\
 *  You may obtain a copy of the License at\
 *\
 *  http://www.apache.org/licenses/LICENSE-2.0\
 *\
 *  Unless required by applicable law or agreed to in writing, software\
 *  distributed under the License is distributed on an "AS IS" BASIS, WITHOUT\
 *  WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\
 *  See the License for the specific language governing permissions and\
 *  limitations under the License.\
 *\
 *  **********\
 *\
 *  **********\
 *  GNU General Public License v2.0 or later:\
 *\
 *  This program is free software; you can redistribute it and/or modify\
 *  it under the terms of the GNU General Public License as published by\
 *  the Free Software Foundation; either version 2 of the License, or\
 *  (at your option) any later version.\
 *\
 *  This program is distributed in the hope that it will be useful,\
 *  but WITHOUT ANY WARRANTY; without even the implied warranty of\
 *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\
 *  GNU General Public License for more details.\
 *\
 *  You should have received a copy of the GNU General Public License along\
 *  with this program; if not, write to the Free Software Foundation, Inc.,\
 *  51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.\
 *\
 *  **********'

find -path './.git' -prune -o '(' -name '*.c' -o -name '*.cpp' -o -name '*.fmt' -o -name '*.h' ')' -print | xargs sed -i "
# Normalize the first line of the copyright headers (no text on the first line of a block comment)
/^\/\*.*Copyright.*Arm/I s/\/\*/&\n */

# Insert new copyright header
/SPDX-License-Identifier/ i\
$header1

# Delete old copyright header
/SPDX-License-Identifier/,$ {
  # Delete lines until the one preceding the mbedtls declaration
  N
  1,/This file is part of/ {
    /This file is part of/! D
  }
}
"

# Format copyright header for inclusion into scripts
header2=$(echo "$header1" | sed 's/^\\\? \* \?/#/')

find -path './.git' -prune -o '(' -name '*.gdb' -o -name '*.pl' -o -name '*.py' -o -name '*.sh' ')' -print | xargs sed -i "
# Insert new copyright header
/SPDX-License-Identifier/ i\
$header2

# Delete old copyright header
/SPDX-License-Identifier/,$ {
  # Delete lines until the one preceding the mbedtls declaration
  N
  1,/This file is part of/ {
    /This file is part of/! D
  }
}
"
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-15 12:48:48 +02:00
Bence Szépkúti 51b41d5422 Add Apache-2.0 headers to all scripts
This commit was generated using the following script:

# ========================
#!/bin/sh

# Find scripts
find -path './.git' -prune -o '(' -name '*.gdb' -o -name '*.pl' -o -name '*.py' -o -name '*.sh' ')' -print | xargs sed -i '

# Remove Mbed TLS declaration if it occurs before the copyright line
1,/Copyright.*Arm/I {
  /This file is part of/,$ {
    /Copyright.*Arm/I! d
  }
}

# Convert non-standard header in scripts/abi_check.py to the format used in the other scripts
/"""/,/"""/ {

  # Cut copyright declaration
  /Copyright.*Arm/I {
    h
    N
    d
  }

  # Paste copyright declaration
  /"""/ {
    x
    /./ {
      s/^/# /    # Add #
      x          # Replace orignal buffer with Copyright declaration
      p          # Print original buffer, insert newline
      i\

      s/.*//     # Clear original buffer
    }
    x
  }
}

/Copyright.*Arm/I {

  # Print copyright declaration
  p

  # Read the two lines immediately following the copyright declaration
  N
  N

  # Insert Apache header if it is missing
  /SPDX/! {
    i\
# SPDX-License-Identifier: Apache-2.0\
#\
# Licensed under the Apache License, Version 2.0 (the "License"); you may\
# not use this file except in compliance with the License.\
# You may obtain a copy of the License at\
#\
# http://www.apache.org/licenses/LICENSE-2.0\
#\
# Unless required by applicable law or agreed to in writing, software\
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT\
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\
# See the License for the specific language governing permissions and\
# limitations under the License.

    # Insert Mbed TLS declaration if it is missing
    /This file is part of/! i\
#\
# This file is part of Mbed TLS (https://tls.mbed.org)
  }

  # Clear copyright declaration from buffer
  D
}
'
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-15 12:18:56 +02:00
Bence Szépkúti 468a76f88a Add copyright dates to all scripts
To find any files with a missing copyright declaration, use the following script:

# ========================
#!/bin/sh

# Find files with copyright declarations, and list their file extensions
exts=$(grep -Ril --exclude-dir .git 'Copyright.*Arm' | sed '
  s/.*\./-name "*./
  s/$/"/
' | sort -u | sed -n '
  :l
    N
    $!bl
  s/\n/ -o /gp
')

# Find files with file extensions that ususally include copyright extensions, but don't include a copyright declaration themselves.
eval "find -path './.git' -prune -o ! -path './tests/data_files/format_pkcs12.fmt' '(' $exts ')' -print" | xargs grep -Li 'Copyright.*Arm'
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-15 12:18:56 +02:00
Bence Szépkúti cd6fd060ec Remove hardcoded line number from the zeroize test
Instead, we insert a comment containing GDB_BREAK_HERE in the line we
want to break at, and let the gdb script search for it.

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-10 11:13:41 +02:00
Bence Szépkúti 74fc3be396 Remove superfluous include from zeroize.c
This brings zeroize.c and test_zeroize.gdb in sync with development.

The include was introduced in 3b0c43063 (#2710).
Reverts ff8ae1115 from the same pull request.

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-10 11:13:13 +02:00
Manuel Pégourié-Gonnard 471ae59783
Merge pull request #3396 from danh-arm/dh/branch-cov-2.16
Backport 2.16: Enable branch coverage in basic_build_test.sh
2020-06-08 10:15:17 +02:00
Manuel Pégourié-Gonnard 3a562d75fb
Merge pull request #3401 from mpg/fix-hmac-drbg-deps-2.16
[Backport 2.16] Fix undeclared dependencies on HMAC_DRBG
2020-06-05 11:50:11 +02:00
Manuel Pégourié-Gonnard 62b293df5b
Merge pull request #3352 from gilles-peskine-arm/fix-ecp-mul-memory-leak-2.16
Backport 2.16: Fix potential memory leak in EC multiplication
2020-06-05 11:44:02 +02:00
Manuel Pégourié-Gonnard c5243c198c Add test for dependencies on HMAC_DRBG in all.sh
Similarly to the recently-added tests for dependencies on CTR_DRBG:
constrained environments will probably want only one DRBG module, and we
should make sure that tests pass in such a configuration.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-05 09:47:28 +02:00
Dan Handley 8cb1981acd Enable branch coverage in basic_build_test.sh
Enable branch coverage output in basic_build_test.sh. This
includes enabling branch coverage output to the lcov make target,
which is disabled by default.

Signed-off-by: Dan Handley <dan.handley@arm.com>
2020-06-04 16:31:59 +01:00
Manuel Pégourié-Gonnard 49d3dd0afc
Merge pull request #3384 from mpg/fix-ctr-drbg-deps-2.16
[Backport 2.16] Fix undeclared dependencies on CTR_DRBG (and add test)
2020-06-03 10:56:03 +02:00
Manuel Pégourié-Gonnard 014ff5b0d6 Add test for building without CTR_DRBG
People who prefer to rely on HMAC_DRBG (for example because they use it for
deterministic ECDSA and don't want a second DRBG for code size reasons) should
be able to build and run the tests suites without CTR_DRBG.

Ideally we should make sure the level of testing (SSL) is the same regardless
of which DRBG modules is enabled, but that's a more significant piece of work.
For now, just ensure everything builds and `make test` passes.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-03 09:54:19 +02:00
Manuel Pégourié-Gonnard 503b79e866
Merge pull request #3375 from gilles-peskine-arm/basic-build-test-status-2.16
Backport 2.16: Fix failure detection in basic-build-test.sh
2020-06-03 09:41:29 +02:00
Manuel Pégourié-Gonnard f2c6e340de Fix undeclared deps on MBEDTLS_CTR_DRBG in tests
While at it, declare deps on ENTROPY as well.

A non-regression test will be added in a follow-up commit.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-02 12:16:07 +02:00
Gilles Peskine f25cd6ba5f MBEDTLS_MEMORY_BACKTRACE is no longer included in the full config
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 19:06:33 +02:00
Gilles Peskine d9701ae21e Create a seedfile explicitly
Running the entropy unit test creates a suitable seedfile, but this
only works due to the happy accident that no prior unit test needs one
(specifically, test_suite_entropy runs before test_suite_rsa). So
create one explicitly, both for robustness and to keep the script
closer to the version in development where the explicit seedfile
creation is required.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 19:02:52 +02:00
Gilles Peskine ed1f673525 If 'make lcov' failed, exit immediately
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 19:01:02 +02:00
Gilles Peskine 8bfe12b1ff Note that we keep going even if some tests fail
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 19:00:55 +02:00
Gilles Peskine f54a5de66a Exit with a failure status if some tests failed
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 19:00:45 +02:00
Gilles Peskine b6484875cc Finish the documentation of normalize_path
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:31:39 +02:00
Gilles Peskine 3c23c82dac Run assemble_changelog.py in all.sh
Avoid nasty surprises where it would fail when we want to make a release.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:31:39 +02:00
Gilles Peskine 43c74d20de Some .pem files are openssl output and have tabs and that's ok
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:31:24 +02:00
Gilles Peskine 401825148f .dsw files are Visual Studio stuff
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:31:24 +02:00
Gilles Peskine 66de311e0b Permit empty files
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:31:24 +02:00
Gilles Peskine cd29f862f6 Normalize line endings
Convert all text files to Unix line endings unless they're Windows
stuff.

Make sure that all text files have a trailing newline.

Remove whitespace at the end of lines.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:31:24 +02:00
Gilles Peskine ce5d854dca Check only files checked into Git
We're only interested in files that are committed and pushed to be
included in Mbed TLS, not in any other files that may be lying around.
So ask git for the list of file names.

This script is primarily intended to run on the CI, and there it runs
on a fresh Git checkout plus potentially some other checkouts or
leftovers from a previous part of the CI job. It should also run
reasonably well on developer machines, where there may be various
additional files. In both cases, git is available.

Ad hoc directory exclusions are no longer needed.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:26:45 +02:00
Gilles Peskine ffaef81586 Exclude binary files from text checks
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:26:45 +02:00
Gilles Peskine e6f1f245f5 Regex mechanism for check-specific exemptions
Suffixes are convenient but not always sufficient.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:26:45 +02:00
Gilles Peskine 125f89b893 Check all files by default
Have an explicit list of exemptions for specific checks rather than
whitelisting files to check. Some checks, such as permissions, should
apply to all files.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:26:45 +02:00
Gilles Peskine e856ba11a1 More accurate variable name
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:25:35 +02:00
Jonas c98ca37cb9 Add test cases to check rng failure
Signed-off-by: Jonas <jonas.lejeune4420@gmail.com>
2020-05-25 13:55:09 +02:00
Gilles Peskine 8b34fef0a8
Merge pull request #2710 from k-stachowiak/unified-exit-in-examples-2.16
Backport 2.16: Unify the example programs' termination
2020-05-12 10:46:36 +02:00
k-stachowiak ff8ae11159 Adjust the zeroization test to the new length of zeroize.c 2020-05-11 22:16:42 +02:00
Gilles Peskine 0f593416e0
Merge pull request #3247 from gilles-peskine-arm/travis-rationalize-2.16
Backport 2.16: Rationalize Travis builds
2020-05-04 12:10:55 +02:00
Gilles Peskine e6c0c7d369 all.sh: on arm builds (GCC or Arm Compiler), show the code size
Just show the code size in the logs, for human consumption.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 23:19:44 +02:00
Gilles Peskine dac156b631 all.sh: add a Cortex-M0+ build
It's pretty fast and adds a little variety.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 23:19:44 +02:00
Gilles Peskine fcccfbc64a all.sh: build_arm_none_eabi_gcc: do optimize
Otherwise the bignum assembly code is not used.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 23:18:24 +02:00
Gilles Peskine a3c6c8a945 Rename --arm-gcc-prefix to --arm-none-eabi-gcc-prefix
This is supposed to be for GCC (or a compiler with a compatible
command line interface) targeting arm-none-eabi, so name it
accordingly.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 18:43:21 +02:00
Gilles Peskine 77b1f3051a all.sh: reorganize testing around deprecated features
build_deprecated combined the testing of deprecated features, and
testing of the build without deprecated features. Also, it violated the
component naming convention by being called build_xxx but running tests.

Replace it by:
* test_default_no_deprecated: check that you can remove deprecated
  features from the default build.
* test_full_deprecated_warning: check that enabling DEPRECATED_WARNING
  doesn't cause any warning from our own code.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 18:24:08 +02:00
Gilles Peskine 313bb50db3 Strict C99: check it in the full config
Ensure that there is a build with -pedantic in the full config, not
just in "exotic" configurations.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-28 15:02:51 +02:00
Gilles Peskine 5ac979309b Strict C99: make sure that fileno() is declared
<stdio.h> only declares the non-ISO-C function fileno() if an
appropriate POSIX symbol is defined or if using a compiler such as GCC
in non-pedantic mode. Define the appropriate POSIX symbol.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-28 15:02:50 +02:00
Gilles Peskine 1e557b7f17 Fix build failure with MBEDTLS_PLATFORM_NV_SEED_ALT
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-28 15:02:50 +02:00
Gilles Peskine df4f7c1274 Document and fix the MBEDTLS_xxx_ALT logic for the full config
The intended logic around MBEDTLS_xxx_ALT is to exclude them from full
because they require the alternative implementation of one or more
library functions, except that MBEDTLS_PLATFORM_xxx_ALT are different:
they're alternative implementations of a platform function and they
have a built-in default, so they should be included in full. Document
this.

Fix a bug whereby MBEDTLS_PLATFORM_xxx_ALT didn't catch symbols where
xxx contains an underscore. As a consequence,
MBEDTLS_PLATFORM_GMTIME_R_ALT and MBEDTLS_PLATFORM_NV_SEED_ALT are now
enabled in the full config. Explicitly exclude
MBEDTLS_PLATFORM_SETUP_TEARDOWN_ALT because it behaves like the
non-platform ones, requiring an extra build-time dependency.
Explicitly exclude MBEDTLS_PLATFORM_NV_SEED_ALT from baremetal
because it requires MBEDTLS_ENTROPY_NV_SEED, and likewise explicitly
unset it from builds that unset MBEDTLS_ENTROPY_NV_SEED.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-28 15:02:50 +02:00
Gilles Peskine 7c8c9c9531 check-files: support Windows .bat files
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-27 09:14:42 +00:00
Gilles Peskine 67c3c3fc1b all.sh: make the arm-gcc cross-compiler prefix configurable
Make it possible to use a compiler that isn't in $PATH, or that's
installed with a different name, or even a compiler for a different
target such as arm-linux-gnueabi.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-27 09:14:42 +00:00
Gilles Peskine 5bd9f56d15 all.sh: run selftest in the full config and with ASan
Almost everything the selftest program does is in the test suites. But
just in case run the selftest program itself once in the full
configuration, and once in the default configuration with ASan, in
addition to running it out of box.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-27 09:14:42 +00:00
Gilles Peskine ff873432d4
Merge pull request #3207 from mpg/check-bounds-of-test-intex-in-suites-2.16
[Backport 2.16] Add lower bound check to function index lookup
2020-04-20 15:19:47 +02:00
Gilles Peskine 15316fdb94
Merge pull request #3169 from gilles-peskine-arm/check-windows-files-2.16
Backport 2.16: Check Windows files for sanity as well
2020-04-20 13:59:18 +02:00
k-stachowiak 531b00bb39 Make negative function_id value impossible in test suites
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-04-20 10:04:25 +02:00
Gilles Peskine 12c299caea
Merge pull request #3184 from gilles-peskine-arm/pylint-up-to-2.4-2.16
Backport 2.16: Pass Pylint up to 2.4
2020-04-20 09:47:46 +02:00
Jaeden Amero da1d437389
Merge pull request #3197 from piotr-now/max_pathlen_overflow_mbedtls-2.16
Backport 2.16: Guard from undefined behaviour in case of an INT_MAX max_pathlen
2020-04-17 14:24:49 +01:00
Andrzej Kurek acf7f2ce93 Guard from undefined behaviour in case of an INT_MAX max_pathlen
When parsing a certificate with the basic constraints extension
the max_pathlen that was read from it was incremented regardless
of its value. However, if the max_pathlen is equal to INT_MAX (which
is highly unlikely), an undefined behaviour would occur.
This commit adds a check to ensure that such value is not accepted
as valid. Relevant tests for INT_MAX and INT_MAX-1 are also introduced.
Certificates added in this commit were generated using the
test_suite_x509write, function test_x509_crt_check. Input data taken
from the "Certificate write check Server1 SHA1" test case, so the generated
files are like the "server1.crt", but with the "is_ca" field set to 1 and
max_pathlen as described by the file name.

Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
Signed-off-by: Piotr Nowicki <piotr.nowicki@arm.com>
2020-04-17 11:29:20 +02:00
Gilles Peskine a8bc32872a
Merge pull request #3164 from ronald-cron-arm/unmet-dependencies-buffer-overflow-fix-2.16
[backport 2.16] Unmet dependencies buffer overflow fix
2020-04-17 10:08:24 +02:00
Gilles Peskine 399b82f986 Pylint: minor code simplifications
Simplify the code in minor ways. Each of this changes fixes a warning
from Pylint 2.4 that doesn't appear with Pylint 1.7.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:51:08 +02:00
Gilles Peskine 5d1dfd4108 Pylint: abide by useless-object-inheritance warnings
Inheriting from object is a remainder of Python 2 habits and is just
clutter in Python 3.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:50:51 +02:00
Gilles Peskine a1bb3f86e9 mbedtls_test.py: drop compatibility with Python 2
Python 2 is no longer supported upstream. Actively drop compatibility
with Python 2.

Removing the inheritance of a class on object pacifies recent versions
of Pylint (useless-object-inheritance).

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:50:51 +02:00
Gilles Peskine af67f8db77 Document more methods in Python scripts
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:50:49 +02:00
Gilles Peskine 8c3ad4bfcb Make check_python_files non-optional in all.sh
check_python_files was optional in all.sh because we used to have CI
machines where pylint wasn't available. But this had the downside that
check_python_files kept breaking because it wasn't checked in the CI.
Now our CI has pylint and check_python_files should not be optional.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:38:21 +02:00
Gilles Peskine c373000307 Make sure to use a Python 3 pylint
On some systems, such as Ubuntu up to 19.04, `pylint` is for Python 2
and `pylint3` is for Python 3, so we should not use `pylint` even if
it's available.

Use the Python module instead of the trivial shell wrapper. This way
we can make sure to use the correct Python version.

Fix #3111

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 20:38:21 +02:00
Janos Follath c04703c58c Bump version to Mbed TLS 2.16.6
Signed-off-by: Janos Follath <janos.follath@arm.com>
2020-04-08 17:17:27 +01:00
Gilles Peskine 368ccd416b In Windows files, detect CR without LF as well as LF without CR
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-06 16:28:53 +02:00
Gilles Peskine 0d5b016709 Check that Windows files have Windows line endings
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-06 16:28:52 +02:00
Gilles Peskine cecc726b91 Also check Windows files
Check Windows files for some issues, including permissions. Omit the
checks related to special characters (whitespace, line endings,
encoding) as appropriate.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-06 16:28:50 +02:00
Gilles Peskine 1978b68a2f Sort the list for easier maintenance
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-06 16:28:45 +02:00
Gilles Peskine eb9929e6e1 List each item on a separate line for easier maintenance
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-06 16:27:14 +02:00
Gilles Peskine d69f51b216 Clarify confusion between file names and suffixes of file names
To test a file name exactly, prepend a / to the base name.

files_to_check actually checks suffixes, not file names, so rename it
to extensions_to_check.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-06 16:20:09 +02:00
Ronald Cron b19ad118dd unit tests: Indicate missing unmet dependencies
The identifiers of the unmet dependencies of a test case are
stored in a buffer of fixed size that can be potentially too
small to store all the unmet dependencies. Indicate in test
reports if some unmet dependencies are missing.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-04-06 11:58:27 +02:00
Ronald Cron 69cc630750 unit tests: Fix potential buffer overflow
Fix potential buffer overflow when tracking the unmet dependencies
of a test case. The identifiers of unmet dependencies are stored
in an array of fixed size. Ensure that we don't overrun the array.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-04-06 11:58:27 +02:00
Gilles Peskine bde4d3045b Prefer unsigned types for non-negative numbers
Use size_t for some variables that are array indices.
Use unsigned for some variables that are counts of "small" things.

This is a backport of commit 3c1c8ea3e7.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-04-06 11:57:36 +02:00
Ronald Cron 07c83f27ad unit tests: Backport ARRAY_LENGTH macro
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-04-06 10:23:05 +02:00
Gilles Peskine 5e65619669
Merge pull request #3142 from mpg/fix-reconnect-2.16
[Backport 2.16] Improve testing of DTLS client hard reconnect
2020-04-02 19:21:16 +02:00
Manuel Pégourié-Gonnard 901e87e5d1
Merge pull request #3124 from AndrzejKurek/typo_asn1_tests
[mbedtls-2.16] Typo fix in test_suite_asn1write.function
2020-04-01 12:27:10 +02:00
Gilles Peskine e8caf2634b Since unmet_dependencies stores integers, represent them as int
Since unmet_dependencies only ever contains strings that are integers
written out in decimal, store the integer instead. Do this
unconditionally since it doesn't cost any extra memory.

This commit saves a little memory and more importantly avoids a gotcha
with uninitialized pointers which caused a bug on development (the
array was only initialized in verbose mode).

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-31 10:57:32 +02:00
Manuel Pégourié-Gonnard 34cbf106f3 Adjust timeout of tests with "no resend" assertions
There are currently 4 tests in ssl-opt.sh with either -C "resend" or -S
"resend", that is, asserting that no retransmission will occur. They sometimes
fail on loaded CI machines as one side doesn't send a message fast enough,
causing the other side to retransmit, causing the test to fail.

(For the "reconnect" test there was an other issue causing random failures,
fixed in a previous commit, but even after that fix the test would still
sometimes randomly fail, even if much more rarely.)

While it's a hard problem to fix in a general and perfect way, in practice the
probability of failures can be drastically reduced by making the timeout
values much larger.

For some tests, where retransmissions are actually expected, this would have
the negative effect of increasing the average running time of the test, as
each side would wait for longer before it starts retransmission, so we have a
trade-off between average running time and probability of spurious failures.

But for tests where retransmission is not expected, there is no such trade-off
as the expected running time of the test (assuming the code is correct most of
the time) is not impacted by the timeout value. So the only negative effect of
increasing the timeout value is on the worst-case running time on the test,
which is much less important, as test should only fail quite rarely.

This commit addresses the easy case of tests that don't expect retransmission
by increasing the value of their timeout range to 10s-20s. This value
corresponds to the value used for tests that assert `-S "autoreduction"` which
are in the same case and where the current value seems acceptable so far.

It also represents an increase, compared to the values before this commit, of
a factor 20 for the "reconnect" tests which were frequently observed to fail
in the CI, and of a factor 10 for the first two "DTLS proxy" tests, which were
observed to fail much less frequently, so hopefully the new values are enough
to reduce the probability of spurious failures to an acceptable level.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-03-31 09:37:31 +02:00
Manuel Pégourié-Gonnard b85ce9eaeb Add negative test for hard reconnect cookie check
The server must check client reachability (we chose to do that by checking a
cookie) before destroying the existing association (RFC 6347 section 4.2.8).
Let's make sure we do, by having a proxy-in-the-middle inject a ClientHello -
the server should notice, but not destroy the connection.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-03-31 09:37:28 +02:00
Andrzej Kurek 5c15847086
Typo fix in test_suite_asn1write.function
This commit fixes issue #2782, spotted by tmarti2.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2020-03-25 07:48:11 -04:00
Manuel Pégourié-Gonnard 98a879a806 Improve some test names in ssl-opt.sh
- "Default" should only be used for tests that actually use the defaults (ie,
  not passing options on the command line, except maybe debug/dtls)
- All tests in the "Encrypt then MAC" group should start with that string as a
  common prefix

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-03-24 10:57:23 +01:00
Simon Butcher 47f728718f Change the use of pylint to optionally use pylint3
Pylint when installed as a distro package can be installed as pylint3, whilst as
a PEP egg, it can be installed as pylint.

This commit changes the scripts to first use pylint if installed, and optionally
look for pylint3 if not installed. This is to allow a preference for the PEP
version over the distro version, assuming the PEP one is more likely to be
the correct one.

Signed-off-by: Simon Butcher <simon.butcher@arm.com>
2020-03-16 13:53:06 +00:00
Manuel Pégourié-Gonnard acfa36890c
Merge pull request #2439 from andresag01/mbedtls-2.16-iotssl-2544-deprecate-record-accel
Backport 2.16: Fix compilation failure when MBEDTLS_SSL_HW_RECORD_ACCEL is enabled
2020-03-16 10:37:24 +01:00
Andres Amaya Garcia 9f3bdb87e5 Add test for MBEDTLS_SSL_HW_RECORD_ACCEL in all.sh
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-02-26 10:14:28 +01:00
Manuel Pégourié-Gonnard 498e632b08 Fix possible close_notify/ClientHello confusion
The ssl-opt.sh test cases using session resumption tend to fail occasionally
on the CI due to a race condition in how ssl_server2 and ssl_client2 handle
the reconnection cycle.

The server does the following in order:
- S1 send application data
- S2 send a close_notify alert
- S3 close the client socket
- S4 wait for a "new connection" (actually a new datagram)
- S5 start a handshake

The client does the following in order:
- C1 wait for and read application data from the server
- C2 send a close_notify alert
- C3 close the server socket
- C4 reset session data and re-open a server socket
- C5 start a handshake

If the client has been able to send the close_notify (C2) and if has been
delivered to the server before if closes the client socket (S3), when the
server reaches S4, the datagram that we start the new connection will be the
ClientHello and everything will be fine.

However if S3 wins the race and happens before the close_notify is delivered,
in S4 the close_notify is what will be seen as the first datagram in a new
connection, and then in S5 this will rightfully be rejected as not being a
valid ClientHello and the server will close the connection (and go wait for
another one). The client will then fail to read from the socket and exit
non-zero and the ssl-opt.sh harness will correctly report this as a failure.

In order to avoid this race condition in test using ssl_client2 and
ssl_server2, this commits introduces a new command-line option
skip_close_notify to ssl_client2 and uses it in all ssl-opt.sh tests that use
session resumption with DTLS and ssl_server2.

This works because ssl_server2 knows how many messages it expects in each
direction and in what order, and closes the connection after that rather than
relying on close_notify (which is also why there was a race in the first
place).

Tests that use another server (in practice there are two of them, using
OpenSSL as a server) wouldn't work with skip_close_notify, as the server won't
close the connection until the client sends a close_notify, but for the same
reason they don't need it (there is no race between receiving close_notify and
closing as the former is the cause of the later).

An alternative approach would be to make ssl_server2 keep the connection open
until it receives a close_notify. Unfortunately it creates problems for tests
where we simulate a lossy network, as the close_notify could be lost (and the
client can't retransmit it). We could modify udp_proxy with an option to never
drop alert messages, but when TLS 1.3 comes that would no longer work as the
type of messages will be encrypted.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-02-26 09:33:05 +01:00
Janos Follath 334cf59930 Bump version to Mbed TLS 2.16.5 2020-02-19 12:03:00 +00:00
Manuel Pégourié-Gonnard 96ed13e21b Test each failure mode of pk_parse_key_pkcs1_der()
(Only the top-level ones, ie, for each call to eg asn1_get_mpi(), ensure
there's at least one test case that makes this call fail in one way, but don't
test the various ways to make asn1_get_mpi fail - that should be covered
elsewhere.)

- the new checks added by the previous commits needed exercising
- existing tests sometimes had wrong descriptions or where passing for the
  wrong reason (eg with the "length mismatch" test, the function actually
failed before reaching the length check)
- while at it, add tests for the rest as well

The valid minimal-size key was generated with:

openssl genrsa 128 2>/dev/null | openssl rsa -outform der 2>/dev/null | xxd -p
2020-02-18 10:49:06 +01:00