Commit graph

2737 commits

Author SHA1 Message Date
Gilles Peskine a586099fd3
Merge pull request #3475 from gilles-peskine-arm/rename-check_files-2.16
Backport 2.16: Rename Python scripts to use '_' and not '-'
2020-07-03 15:12:52 +02:00
Gilles Peskine ee69477890 Rename Python scripts to use '_' and not '-'
You can't import a Python script whose name includes '-'.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-07-02 12:06:46 +02:00
Janos Follath 15a7cd1052 Merge tag 'mbedtls-2.16.7' into merge-2.16.7-release-to-mbedtls-2.16
Mbed TLS 2.16.7
2020-07-01 11:34:02 +01:00
Janos Follath ef5f8fc52a Bump version to Mbed TLS 2.16.7
Executed "./scripts/bump_version.sh --version 2.16.7"

Signed-off-by: Janos Follath <janos.follath@arm.com>
2020-06-26 12:35:53 +01:00
Ronald Cron 14a5645cbf tests: Get rid of mbedtls_test_unhexify() in unit test code
In test functions calling mbedtls_test_unhexify(), change the
type of the associated parameters from `char*` to `data_t`.

That way the `unhexify` operation is done by the test
framework and not by the unit test code.

Use for the new parameters of type data_t the name of the
local variable that used to store the `unhexify` version of
the `char*` parameter.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-06-26 10:19:21 +02:00
Ronald Cron 1d5ef2919b tests: ccm: Prepare to char* to data_t* type change
In preparation of changing the type of some parameters
of mbedtls_ccm_star_encrypt_and_tag/auth_decrypt from
`char *` to `data_t` to get rid of the calls to
mbedtls_test_unhexify():

- Change the name of parameters and local variables to
  clarify which ones are related to the outputs of the
  library functions under test and which ones are
  related to the expected values of those outputs.

- Use two different buffers to store the plain and cipher
  text as expected by the library functions.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-06-26 10:15:06 +02:00
Ronald Cron b2eb38d391 tests: aes.ofb: Prepare to char* to data_t* type change
In preparation of changing the type of some parameters
of aes_encrypt_ofb() from `char *` to `data_t` to get rid
of the calls to mbedtls_test_unhexify():

- Change the name of parameters and local variables to
  clarify which ones are related to the outputs of the
  library functions under test and which ones are
  related to the expected values of those outputs.

- Add assertion on fragment_size parameter

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-06-26 09:48:55 +02:00
Ronald Cron afbe3ee87c tests: nist_kw: Prepare to char* to data_t* type change
In preparation of changing the type of some parameters
of mbedtls_nist_kw_wrap/unwrap() from `char *` to `data_t`
to get rid of the calls to mbedtls_test_unhexify():

- Change the name of parameters and local variables to
  clarify which ones are related to the outputs of the
  library functions under test and which ones are
  related to the expected values of those outputs.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-06-26 09:48:11 +02:00
Ronald Cron 7e8f1af0a6 tests: chacha20: Prepare to char* to data_t* type change
In preparation of changing the type of some parameters of
test_chacha20() from `char *` to `data_t` to get rid of the
calls to mbedtls_test_unhexify():

- Reduce the size of output[] buffer to 375 as its content
  is "ASCII expended" into a buffer of 751 bytes.
- Align naming of variables to store and check the
  output of mbedtls_chacha20_crypt(). No *dst* variables
  anynore, only *output* variables.
- Use two different buffers to store the expected output
  of mbedtls_chacha20_crypt() (expected_output_str[]) and
  the ASCII string representation of the output of
  mbedtls_chacha20_crypt() (output_string[]). Both were
  stored in dst_str[] before.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-06-26 09:47:54 +02:00
Ronald Cron d7a4df8341 tests: hkdf: Prepare to char* to data_t* type change
In preparation of changing the type of some parameters
of test_hkdf() from `char *` to `data_t` to get rid of the
calls to mbedtls_test_unhexify():

- Align naming of variables related to the expected okm
- Rename `okm_hex[]` to `okm_string[]`
- Added TEST_ASSERT( expected_okm_len <= sizeof( okm ) ) to check
  that the okm[] buffer is large enough for the okm output.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-06-26 09:47:25 +02:00
Ronald Cron 6d1f0ad7b6 tests: aria: Prepare to char* to data_t* type change
In preparation of changing the type of some parameters
of some test functions from `char *` to `data_t` to get
rid of the calls to mbedtls_test_unhexify():

- Align the name of source data length local variable
  with the name of the local variable containing the
  source data, respectively src_str and src_str_len.
- Change the type of length, index local variables
  from int to size_t.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-06-26 09:46:50 +02:00
Ronald Cron 5b13a86e11 tests: Reformating due to *hexify functions renaming
Command to find the files in which lines have gone
larger than 79 characters due to the renaming:

grep '.\{80\}' \
`git diff-tree --no-commit-id --name-only -r HEAD` \
| grep hexify

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-06-26 09:38:21 +02:00
Ronald Cron a0c9ff3e52 tests: Add mbedtls_test_ prefix to *hexify functions
Add mbedtls_test_ prefix to hexify() and unhexify()
test helper functions.

Command to change *.function files:
find . -name "*.function" -exec awk -i inplace \
    '{sub(/(un|)hexify\>/,"mbedtls_test_&")}1' {} \;

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-06-26 09:15:43 +02:00
Janos Follath f69b919844 Merge branch 'mbedtls-2.16-restricted' into mbedtls-2.16.7r0 2020-06-25 09:19:21 +01:00
Manuel Pégourié-Gonnard 805b1461b8 all.sh: clean up some uses of "local" variables
While pure sh doesn't have a concept of local variables, we can partially
emulate them by unsetting variables before we exit the function, and use the
convention of giving them lowercase names to distinguish from global
variables.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-23 11:42:59 +02:00
Manuel Pégourié-Gonnard d2b5ad871d Use all.sh in pre-push hook
The list in the pre-push hook was redundant with the list of `check_*`
components in all.sh, and unsurprisingly it was outdated.

Missing components were:

- check_recursion
- check_changelog
- check_test_cases
- check_python_files
- check_generate_test_code

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-23 11:37:04 +02:00
Manuel Pégourié-Gonnard 1ee685018a Make check_generate_test_code more -q friendly
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-23 11:36:32 +02:00
Manuel Pégourié-Gonnard de4ad2da38 Add a --quiet option to all.sh
The primary purpose is to use it to run all.sh -k -q in the pre-push hook, but
this can be useful in any circumstance where you're not interested in the full
output from each component and just want a short summary of which components
were run (and if any failed).

Note that only stdout from components is suppressed, stderr is preserved so
that errors are reported. This means components should avoid printing to
stderr in normal usage (ie in the absence of errors).

Currently all the `check_*` components obey this convention except:
- check_generate_test_code: unittest prints progress to stderr
- check_test_cases: lots of non-fatal warnings printed to stderr

These components will be fixed in follow-up commits.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-23 11:36:30 +02:00
Manuel Pégourié-Gonnard 1bff6847fb Adjust comments about SEED synchronisation
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-22 10:53:44 +02:00
Manuel Pégourié-Gonnard c2400d38fe Make basic-build-test.sh deterministic
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-22 10:53:42 +02:00
Manuel Pégourié-Gonnard 2df5857dbe Remove SHA-1 as a fallback option
- it's 2020, there shouldn't be too many systems out there where SHA-1 is the
  only available hash option, so its usefulness is limited
- OTOH testing configurations without SHA-2 reveal bugs that are not easy to
  fix in a fully compatible way

So overall, the benefit/cost ratio is not good enough to justify keeping SHA-1
as a fallback option here.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-18 12:17:59 +02:00
Manuel Pégourié-Gonnard 72177e362b Add fall-back to hash-based KDF for internal ECP DRBG
The dependency on a DRBG module was perhaps a bit strict for LTS branches, so
let's have an option that works with no DRBG when at least one SHA module is
present.

This changes the internal API of ecp_drbg_seed() by adding the size of the
MPI as a parameter. Re-computing the size from the number of limbs doesn't
work too well here as we're writing out to a fixed-size buffer and for some
curves (P-521) that would round up too much. Using mbedtls_mpi_get_len() is
not entirely satisfactory either as it would mean using a variable-length
encoding, with could open side channels.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-16 12:51:42 +02:00
Manuel Pégourié-Gonnard 047986c2f8 Add support for RESTARTABLE with internal RNG
Currently we draw pseudo-random numbers at the beginning and end of the main
loop. With ECP_RESTARTABLE, it's possible that between those two occasions we
returned from the multiplication function, hence lost our internal DRBG
context that lives in this function's stack frame. This would result in the
same pseudo-random numbers being used for blinding in multiple places. While
it's not immediately clear that this would give rise to an attack, it's also
absolutely not clear that it doesn't. So let's avoid that by using a DRBG
context that lives inside the restart context and persists across
return/resume cycles. That way the RESTARTABLE case uses exactly the
same pseudo-random numbers as the non-restartable case.

Testing and compile-time options:

- The case ECP_RESTARTABLE && !ECP_NO_INTERNAL_RNG is already tested by
  component_test_no_use_psa_crypto_full_cmake_asan.
- The case ECP_RESTARTABLE && ECP_NO_INTERNAL_RNG didn't have a pre-existing
  test so a component is added.

Testing and runtime options: when ECP_RESTARTABLE is enabled, the test suites
already contain cases where restart happens and cases where it doesn't
(because the operation is short enough or because restart is disabled (NULL
restart context)).

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-16 10:52:32 +02:00
Manuel Pégourié-Gonnard 23983f30de Add config.h option MBEDTLS_ECP_NO_INTERNAL_RNG
No effect so far, except on dependency checking, as the feature it's meant to
disable isn't implemented yet (so the descriptions in config.h and the
ChangeLog entry are anticipation for now).

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-16 10:51:42 +02:00
Bence Szépkúti f744bd72ee Update license headers to Apache-2.0 OR GPL-2.0-or-later
This will allow us to ship the LTS branches in a single archive

This commit was generated using the following script:

# ========================
#!/bin/sh

header1='\ *  SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later\
 *\
 *  This file is provided under the Apache License 2.0, or the\
 *  GNU General Public License v2.0 or later.\
 *\
 *  **********\
 *  Apache License 2.0:\
 *\
 *  Licensed under the Apache License, Version 2.0 (the "License"); you may\
 *  not use this file except in compliance with the License.\
 *  You may obtain a copy of the License at\
 *\
 *  http://www.apache.org/licenses/LICENSE-2.0\
 *\
 *  Unless required by applicable law or agreed to in writing, software\
 *  distributed under the License is distributed on an "AS IS" BASIS, WITHOUT\
 *  WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\
 *  See the License for the specific language governing permissions and\
 *  limitations under the License.\
 *\
 *  **********\
 *\
 *  **********\
 *  GNU General Public License v2.0 or later:\
 *\
 *  This program is free software; you can redistribute it and/or modify\
 *  it under the terms of the GNU General Public License as published by\
 *  the Free Software Foundation; either version 2 of the License, or\
 *  (at your option) any later version.\
 *\
 *  This program is distributed in the hope that it will be useful,\
 *  but WITHOUT ANY WARRANTY; without even the implied warranty of\
 *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\
 *  GNU General Public License for more details.\
 *\
 *  You should have received a copy of the GNU General Public License along\
 *  with this program; if not, write to the Free Software Foundation, Inc.,\
 *  51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.\
 *\
 *  **********'

find -path './.git' -prune -o '(' -name '*.c' -o -name '*.cpp' -o -name '*.fmt' -o -name '*.h' ')' -print | xargs sed -i "
# Normalize the first line of the copyright headers (no text on the first line of a block comment)
/^\/\*.*Copyright.*Arm/I s/\/\*/&\n */

# Insert new copyright header
/SPDX-License-Identifier/ i\
$header1

# Delete old copyright header
/SPDX-License-Identifier/,$ {
  # Delete lines until the one preceding the mbedtls declaration
  N
  1,/This file is part of/ {
    /This file is part of/! D
  }
}
"

# Format copyright header for inclusion into scripts
header2=$(echo "$header1" | sed 's/^\\\? \* \?/#/')

find -path './.git' -prune -o '(' -name '*.gdb' -o -name '*.pl' -o -name '*.py' -o -name '*.sh' ')' -print | xargs sed -i "
# Insert new copyright header
/SPDX-License-Identifier/ i\
$header2

# Delete old copyright header
/SPDX-License-Identifier/,$ {
  # Delete lines until the one preceding the mbedtls declaration
  N
  1,/This file is part of/ {
    /This file is part of/! D
  }
}
"
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-15 12:48:48 +02:00
Bence Szépkúti 51b41d5422 Add Apache-2.0 headers to all scripts
This commit was generated using the following script:

# ========================
#!/bin/sh

# Find scripts
find -path './.git' -prune -o '(' -name '*.gdb' -o -name '*.pl' -o -name '*.py' -o -name '*.sh' ')' -print | xargs sed -i '

# Remove Mbed TLS declaration if it occurs before the copyright line
1,/Copyright.*Arm/I {
  /This file is part of/,$ {
    /Copyright.*Arm/I! d
  }
}

# Convert non-standard header in scripts/abi_check.py to the format used in the other scripts
/"""/,/"""/ {

  # Cut copyright declaration
  /Copyright.*Arm/I {
    h
    N
    d
  }

  # Paste copyright declaration
  /"""/ {
    x
    /./ {
      s/^/# /    # Add #
      x          # Replace orignal buffer with Copyright declaration
      p          # Print original buffer, insert newline
      i\

      s/.*//     # Clear original buffer
    }
    x
  }
}

/Copyright.*Arm/I {

  # Print copyright declaration
  p

  # Read the two lines immediately following the copyright declaration
  N
  N

  # Insert Apache header if it is missing
  /SPDX/! {
    i\
# SPDX-License-Identifier: Apache-2.0\
#\
# Licensed under the Apache License, Version 2.0 (the "License"); you may\
# not use this file except in compliance with the License.\
# You may obtain a copy of the License at\
#\
# http://www.apache.org/licenses/LICENSE-2.0\
#\
# Unless required by applicable law or agreed to in writing, software\
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT\
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\
# See the License for the specific language governing permissions and\
# limitations under the License.

    # Insert Mbed TLS declaration if it is missing
    /This file is part of/! i\
#\
# This file is part of Mbed TLS (https://tls.mbed.org)
  }

  # Clear copyright declaration from buffer
  D
}
'
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-15 12:18:56 +02:00
Bence Szépkúti 468a76f88a Add copyright dates to all scripts
To find any files with a missing copyright declaration, use the following script:

# ========================
#!/bin/sh

# Find files with copyright declarations, and list their file extensions
exts=$(grep -Ril --exclude-dir .git 'Copyright.*Arm' | sed '
  s/.*\./-name "*./
  s/$/"/
' | sort -u | sed -n '
  :l
    N
    $!bl
  s/\n/ -o /gp
')

# Find files with file extensions that ususally include copyright extensions, but don't include a copyright declaration themselves.
eval "find -path './.git' -prune -o ! -path './tests/data_files/format_pkcs12.fmt' '(' $exts ')' -print" | xargs grep -Li 'Copyright.*Arm'
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-15 12:18:56 +02:00
Bence Szépkúti cd6fd060ec Remove hardcoded line number from the zeroize test
Instead, we insert a comment containing GDB_BREAK_HERE in the line we
want to break at, and let the gdb script search for it.

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-10 11:13:41 +02:00
Bence Szépkúti 74fc3be396 Remove superfluous include from zeroize.c
This brings zeroize.c and test_zeroize.gdb in sync with development.

The include was introduced in 3b0c43063 (#2710).
Reverts ff8ae1115 from the same pull request.

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-10 11:13:13 +02:00
Manuel Pégourié-Gonnard 471ae59783
Merge pull request #3396 from danh-arm/dh/branch-cov-2.16
Backport 2.16: Enable branch coverage in basic_build_test.sh
2020-06-08 10:15:17 +02:00
Manuel Pégourié-Gonnard 3a562d75fb
Merge pull request #3401 from mpg/fix-hmac-drbg-deps-2.16
[Backport 2.16] Fix undeclared dependencies on HMAC_DRBG
2020-06-05 11:50:11 +02:00
Manuel Pégourié-Gonnard 62b293df5b
Merge pull request #3352 from gilles-peskine-arm/fix-ecp-mul-memory-leak-2.16
Backport 2.16: Fix potential memory leak in EC multiplication
2020-06-05 11:44:02 +02:00
Manuel Pégourié-Gonnard c5243c198c Add test for dependencies on HMAC_DRBG in all.sh
Similarly to the recently-added tests for dependencies on CTR_DRBG:
constrained environments will probably want only one DRBG module, and we
should make sure that tests pass in such a configuration.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-05 09:47:28 +02:00
Dan Handley 8cb1981acd Enable branch coverage in basic_build_test.sh
Enable branch coverage output in basic_build_test.sh. This
includes enabling branch coverage output to the lcov make target,
which is disabled by default.

Signed-off-by: Dan Handley <dan.handley@arm.com>
2020-06-04 16:31:59 +01:00
Manuel Pégourié-Gonnard 49d3dd0afc
Merge pull request #3384 from mpg/fix-ctr-drbg-deps-2.16
[Backport 2.16] Fix undeclared dependencies on CTR_DRBG (and add test)
2020-06-03 10:56:03 +02:00
Manuel Pégourié-Gonnard 014ff5b0d6 Add test for building without CTR_DRBG
People who prefer to rely on HMAC_DRBG (for example because they use it for
deterministic ECDSA and don't want a second DRBG for code size reasons) should
be able to build and run the tests suites without CTR_DRBG.

Ideally we should make sure the level of testing (SSL) is the same regardless
of which DRBG modules is enabled, but that's a more significant piece of work.
For now, just ensure everything builds and `make test` passes.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-03 09:54:19 +02:00
Manuel Pégourié-Gonnard 503b79e866
Merge pull request #3375 from gilles-peskine-arm/basic-build-test-status-2.16
Backport 2.16: Fix failure detection in basic-build-test.sh
2020-06-03 09:41:29 +02:00
Manuel Pégourié-Gonnard f2c6e340de Fix undeclared deps on MBEDTLS_CTR_DRBG in tests
While at it, declare deps on ENTROPY as well.

A non-regression test will be added in a follow-up commit.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-02 12:16:07 +02:00
Gilles Peskine f25cd6ba5f MBEDTLS_MEMORY_BACKTRACE is no longer included in the full config
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 19:06:33 +02:00
Gilles Peskine d9701ae21e Create a seedfile explicitly
Running the entropy unit test creates a suitable seedfile, but this
only works due to the happy accident that no prior unit test needs one
(specifically, test_suite_entropy runs before test_suite_rsa). So
create one explicitly, both for robustness and to keep the script
closer to the version in development where the explicit seedfile
creation is required.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 19:02:52 +02:00
Gilles Peskine ed1f673525 If 'make lcov' failed, exit immediately
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 19:01:02 +02:00
Gilles Peskine 8bfe12b1ff Note that we keep going even if some tests fail
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 19:00:55 +02:00
Gilles Peskine f54a5de66a Exit with a failure status if some tests failed
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 19:00:45 +02:00
Gilles Peskine b6484875cc Finish the documentation of normalize_path
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:31:39 +02:00
Gilles Peskine 3c23c82dac Run assemble_changelog.py in all.sh
Avoid nasty surprises where it would fail when we want to make a release.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:31:39 +02:00
Gilles Peskine 43c74d20de Some .pem files are openssl output and have tabs and that's ok
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:31:24 +02:00
Gilles Peskine 401825148f .dsw files are Visual Studio stuff
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:31:24 +02:00
Gilles Peskine 66de311e0b Permit empty files
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:31:24 +02:00
Gilles Peskine cd29f862f6 Normalize line endings
Convert all text files to Unix line endings unless they're Windows
stuff.

Make sure that all text files have a trailing newline.

Remove whitespace at the end of lines.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:31:24 +02:00
Gilles Peskine ce5d854dca Check only files checked into Git
We're only interested in files that are committed and pushed to be
included in Mbed TLS, not in any other files that may be lying around.
So ask git for the list of file names.

This script is primarily intended to run on the CI, and there it runs
on a fresh Git checkout plus potentially some other checkouts or
leftovers from a previous part of the CI job. It should also run
reasonably well on developer machines, where there may be various
additional files. In both cases, git is available.

Ad hoc directory exclusions are no longer needed.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:26:45 +02:00