Commit graph

441 commits

Author SHA1 Message Date
Manuel Pégourié-Gonnard e4a5c05d49 Adjust comments about SEED synchronisation
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-22 10:54:38 +02:00
Manuel Pégourié-Gonnard 54d95b1722 Make basic-build-test.sh deterministic
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-22 10:54:38 +02:00
Bence Szépkúti 4e9f71227a Update license headers to Apache-2.0 OR GPL-2.0-or-later
This will allow us to ship the LTS branches in a single archive

This commit was generated using the following script:

# ========================
#!/bin/sh

header1='\ *  SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later\
 *\
 *  This file is provided under the Apache License 2.0, or the\
 *  GNU General Public License v2.0 or later.\
 *\
 *  **********\
 *  Apache License 2.0:\
 *\
 *  Licensed under the Apache License, Version 2.0 (the "License"); you may\
 *  not use this file except in compliance with the License.\
 *  You may obtain a copy of the License at\
 *\
 *  http://www.apache.org/licenses/LICENSE-2.0\
 *\
 *  Unless required by applicable law or agreed to in writing, software\
 *  distributed under the License is distributed on an "AS IS" BASIS, WITHOUT\
 *  WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\
 *  See the License for the specific language governing permissions and\
 *  limitations under the License.\
 *\
 *  **********\
 *\
 *  **********\
 *  GNU General Public License v2.0 or later:\
 *\
 *  This program is free software; you can redistribute it and/or modify\
 *  it under the terms of the GNU General Public License as published by\
 *  the Free Software Foundation; either version 2 of the License, or\
 *  (at your option) any later version.\
 *\
 *  This program is distributed in the hope that it will be useful,\
 *  but WITHOUT ANY WARRANTY; without even the implied warranty of\
 *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\
 *  GNU General Public License for more details.\
 *\
 *  You should have received a copy of the GNU General Public License along\
 *  with this program; if not, write to the Free Software Foundation, Inc.,\
 *  51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.\
 *\
 *  **********'

find -path './.git' -prune -o '(' -name '*.c' -o -name '*.cpp' -o -name '*.fmt' -o -name '*.h' ')' -print | xargs sed -i "
# Normalize the first line of the copyright headers (no text on the first line of a block comment)
/^\/\*.*Copyright.*Arm/I s/\/\*/&\n */

# Insert new copyright header
/SPDX-License-Identifier/ i\
$header1

# Delete old copyright header
/SPDX-License-Identifier/,$ {
  # Delete lines until the one preceding the mbedtls declaration
  N
  1,/This file is part of/ {
    /This file is part of/! D
  }
}
"

# Format copyright header for inclusion into scripts
header2=$(echo "$header1" | sed 's/^\\\? \* \?/#/')

find -path './.git' -prune -o '(' -name '*.gdb' -o -name '*.pl' -o -name '*.py' -o -name '*.sh' ')' -print | xargs sed -i "
# Insert new copyright header
/SPDX-License-Identifier/ i\
$header2

# Delete old copyright header
/SPDX-License-Identifier/,$ {
  # Delete lines until the one preceding the mbedtls declaration
  N
  1,/This file is part of/ {
    /This file is part of/! D
  }
}
"
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-15 12:56:41 +02:00
Bence Szépkúti 09b4f19f94 Add Apache-2.0 headers to all scripts
This commit was generated using the following script:

# ========================
#!/bin/sh

# Find scripts
find -path './.git' -prune -o '(' -name '*.gdb' -o -name '*.pl' -o -name '*.py' -o -name '*.sh' ')' -print | xargs sed -i '

# Remove Mbed TLS declaration if it occurs before the copyright line
1,/Copyright.*Arm/I {
  /This file is part of/,$ {
    /Copyright.*Arm/I! d
  }
}

# Convert non-standard header in scripts/abi_check.py to the format used in the other scripts
/"""/,/"""/ {

  # Cut copyright declaration
  /Copyright.*Arm/I {
    h
    N
    d
  }

  # Paste copyright declaration
  /"""/ {
    x
    /./ {
      s/^/# /    # Add #
      x          # Replace orignal buffer with Copyright declaration
      p          # Print original buffer, insert newline
      i\

      s/.*//     # Clear original buffer
    }
    x
  }
}

/Copyright.*Arm/I {

  # Print copyright declaration
  p

  # Read the two lines immediately following the copyright declaration
  N
  N

  # Insert Apache header if it is missing
  /SPDX/! {
    i\
# SPDX-License-Identifier: Apache-2.0\
#\
# Licensed under the Apache License, Version 2.0 (the "License"); you may\
# not use this file except in compliance with the License.\
# You may obtain a copy of the License at\
#\
# http://www.apache.org/licenses/LICENSE-2.0\
#\
# Unless required by applicable law or agreed to in writing, software\
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT\
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\
# See the License for the specific language governing permissions and\
# limitations under the License.

    # Insert Mbed TLS declaration if it is missing
    /This file is part of/! i\
#\
# This file is part of Mbed TLS (https://tls.mbed.org)
  }

  # Clear copyright declaration from buffer
  D
}
'
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-08 17:46:40 +02:00
Bence Szépkúti b7246ad938 Add copyright dates to all scripts
To find any files with a missing copyright declaration, use the following script:

# ========================
#!/bin/sh

# Find files with copyright declarations, and list their file extensions
exts=$(grep -Ril --exclude-dir .git 'Copyright.*Arm' | sed '
  s/.*\./-name "*./
  s/$/"/
' | sort -u | sed -n '
  :l
    N
    $!bl
  s/\n/ -o /gp
')

# Find files with file extensions that ususally include copyright extensions,
# but don't include a copyright declaration themselves.
eval "find -path './.git' -prune -o\
  ! -path './tests/data_files/format_pkcs12.fmt'\
  ! -path './programs/psa/psa_constant_names_generated.c'\
  '(' $exts ')' -print" | xargs grep -Li 'Copyright.*Arm'
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2020-06-08 17:26:24 +02:00
Manuel Pégourié-Gonnard 9967bfe35b
Merge pull request #3397 from danh-arm/dh/branch-cov-2.7
Backport 2.7: Enable branch coverage in basic_build_test.sh
2020-06-08 10:15:26 +02:00
Manuel Pégourié-Gonnard cdfa2f983b Add test for dependencies on HMAC_DRBG in all.sh
Similarly to the recently-added tests for dependencies on CTR_DRBG:
constrained environments will probably want only one DRBG module, and we
should make sure that tests pass in such a configuration.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-05 09:52:39 +02:00
Dan Handley aba9e22d25 Enable branch coverage in basic_build_test.sh
Enable branch coverage output in basic_build_test.sh. This
includes enabling branch coverage output to the lcov make target,
which is disabled by default.

Signed-off-by: Dan Handley <dan.handley@arm.com>
2020-06-04 16:41:02 +01:00
Manuel Pégourié-Gonnard 7e384f11f5
Merge pull request #3385 from mpg/fix-ctr-drbg-deps-2.7
[Backport 2.7] Fix undeclared dependencies on CTR_DRBG (and add test)
2020-06-03 10:56:13 +02:00
Manuel Pégourié-Gonnard c98fde5ca8 Add test for building without CTR_DRBG
People who prefer to rely on HMAC_DRBG (for example because they use it for
deterministic ECDSA and don't want a second DRBG for code size reasons) should
be able to build and run the tests suites without CTR_DRBG.

Ideally we should make sure the level of testing (SSL) is the same regardless
of which DRBG modules is enabled, but that's a more significant piece of work.
For now, just ensure everything builds and `make test` passes.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-06-03 09:57:08 +02:00
Manuel Pégourié-Gonnard 801318d47e
Merge pull request #3376 from gilles-peskine-arm/basic-build-test-status-2.7
Backport 2.7: Fix failure detection in basic-build-test.sh
2020-06-03 09:41:34 +02:00
Gilles Peskine 39b610227b MBEDTLS_MEMORY_BACKTRACE is no longer included in the full config
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 20:47:34 +02:00
Gilles Peskine 9cc203a7ea Create a seedfile explicitly
Running the entropy unit test creates a suitable seedfile, but this
only works due to the happy accident that no prior unit test needs one
(specifically, test_suite_entropy runs before test_suite_rsa). So
create one explicitly, both for robustness and to keep the script
closer to the version in development where the explicit seedfile
creation is required.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 20:47:34 +02:00
Gilles Peskine b07541d9f0 If 'make lcov' failed, exit immediately
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 20:47:34 +02:00
Gilles Peskine 27c36f050f Note that we keep going even if some tests fail
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 20:47:34 +02:00
Gilles Peskine dd7ea389ff Exit with a failure status if some tests failed
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 20:47:34 +02:00
Gilles Peskine 14b559a4c3 Finish the documentation of normalize_path
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:34:20 +02:00
Gilles Peskine c3189259e7 Run assemble_changelog.py in all.sh
Avoid nasty surprises where it would fail when we want to make a release.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:34:20 +02:00
Gilles Peskine 8fa5be5224 Some .pem files are openssl output and have tabs and that's ok
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:34:20 +02:00
Gilles Peskine e7e149f469 .dsw files are Visual Studio stuff
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:34:20 +02:00
Gilles Peskine be76c19801 Permit empty files
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:34:20 +02:00
Gilles Peskine 4bda3254cf Check only files checked into Git
We're only interested in files that are committed and pushed to be
included in Mbed TLS, not in any other files that may be lying around.
So ask git for the list of file names.

This script is primarily intended to run on the CI, and there it runs
on a fresh Git checkout plus potentially some other checkouts or
leftovers from a previous part of the CI job. It should also run
reasonably well on developer machines, where there may be various
additional files. In both cases, git is available.

Ad hoc directory exclusions are no longer needed.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:33:56 +02:00
Gilles Peskine 986a06de94 Exclude binary files from text checks
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:33:18 +02:00
Gilles Peskine b4805ece78 Regex mechanism for check-specific exemptions
Suffixes are convenient but not always sufficient.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:33:18 +02:00
Gilles Peskine ab55bc71fe Check all files by default
Have an explicit list of exemptions for specific checks rather than
whitelisting files to check. Some checks, such as permissions, should
apply to all files.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:33:18 +02:00
Gilles Peskine 45137617db More accurate variable name
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-28 18:33:18 +02:00
Gilles Peskine b0483152fb
Merge pull request #3248 from gilles-peskine-arm/travis-rationalize-2.7
Backport 2.7: Rationalize Travis builds
2020-05-04 12:10:57 +02:00
Gilles Peskine 81b60fb7bd all.sh: on arm builds (GCC or Arm Compiler), show the code size
Just show the code size in the logs, for human consumption.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 23:31:04 +02:00
Gilles Peskine 4fb7a2f23f all.sh: add a Cortex-M0+ build
It's pretty fast and adds a little variety.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 23:31:04 +02:00
Gilles Peskine 009908b562 all.sh: build_arm_none_eabi_gcc: do optimize
Otherwise the bignum assembly code is not used.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 23:31:04 +02:00
Gilles Peskine a27af6f023 all.sh: backport component_build_arm_none_eabi_gcc_arm5vte
Backport this component that was added to development after 2.7.0.
It's easier to keep the 2.7 branch closer to the other maintained
branches.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 23:31:03 +02:00
Gilles Peskine f1709bb4ad Rename --arm-gcc-prefix to --arm-none-eabi-gcc-prefix
This is supposed to be for GCC (or a compiler with a compatible
command line interface) targeting arm-none-eabi, so name it
accordingly.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 23:31:03 +02:00
Gilles Peskine adaaddbe3b all.sh: reorganize testing around deprecated features
build_deprecated combined the testing of deprecated features, and
testing of the build without deprecated features. Also, it violated the
component naming convention by being called build_xxx but running tests.

Replace it by:
* test_default_no_deprecated: check that you can remove deprecated
  features from the default build.
* test_full_deprecated_warning: check that enabling DEPRECATED_WARNING
  doesn't cause any warning from our own code.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 18:47:37 +02:00
Gilles Peskine a21c5e9988 Document and fix the MBEDTLS_xxx_ALT logic for the full config
The intended logic around MBEDTLS_xxx_ALT is to exclude them from full
because they require the alternative implementation of one or more
library functions, except that MBEDTLS_PLATFORM_xxx_ALT are different:
they're alternative implementations of a platform function and they
have a built-in default, so they should be included in full. Document
this.

Fix a bug whereby MBEDTLS_PLATFORM_xxx_ALT didn't catch symbols where
xxx contains an underscore. As a consequence, MBEDTLS_PLATFORM_NV_SEED_ALT
is now enabled in the full config. Explicitly exclude
MBEDTLS_PLATFORM_SETUP_TEARDOWN_ALT because it behaves like the
non-platform ones, requiring an extra build-time dependency.
Explicitly exclude MBEDTLS_PLATFORM_NV_SEED_ALT from baremetal
because it requires MBEDTLS_ENTROPY_NV_SEED, and likewise explicitly
unset it from builds that unset MBEDTLS_ENTROPY_NV_SEED.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-30 09:16:36 +02:00
Gilles Peskine 86e5816dda check-files: support Windows .bat files
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-27 11:27:39 +02:00
Gilles Peskine cd7b0422ef all.sh: make the arm-gcc cross-compiler prefix configurable
Make it possible to use a compiler that isn't in $PATH, or that's
installed with a different name, or even a compiler for a different
target such as arm-linux-gnueabi.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-27 11:24:40 +02:00
Gilles Peskine bfda033f3d all.sh: run selftest in the full config and with ASan
Almost everything the selftest program does is in the test suites. But
just in case run the selftest program itself once in the full
configuration, and once in the default configuration with ASan, in
addition to running it out of box.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-27 11:23:11 +02:00
Gilles Peskine 5d32e64ad0
Merge pull request #3170 from gilles-peskine-arm/check-windows-files-2.7
Backport 2.7: Check Windows files for sanity as well
2020-04-20 13:59:22 +02:00
Gilles Peskine b5847d20d3 Pylint: abide by useless-object-inheritance warnings
Inheriting from object is a remainder of Python 2 habits and is just
clutter in Python 3.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 21:08:26 +02:00
Gilles Peskine 558e26dbda Document more methods in Python scripts
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-11 21:07:43 +02:00
Gilles Peskine 70ef5c6c9a In Windows files, detect CR without LF as well as LF without CR
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-06 16:39:06 +02:00
Gilles Peskine 783da6345e Check that Windows files have Windows line endings
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-06 16:39:04 +02:00
Gilles Peskine 227dfd435e Also check Windows files
Check Windows files for some issues, including permissions. Omit the
checks related to special characters (whitespace, line endings,
encoding) as appropriate.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-06 16:39:03 +02:00
Gilles Peskine 318f15ef92 Sort the list for easier maintenance
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-06 16:39:02 +02:00
Gilles Peskine c7153226f6 List each item on a separate line for easier maintenance
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-06 16:39:00 +02:00
Gilles Peskine c251e0d75a Clarify confusion between file names and suffixes of file names
To test a file name exactly, prepend a / to the base name.

files_to_check actually checks suffixes, not file names, so rename it
to extensions_to_check.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-06 16:38:58 +02:00
Andres Amaya Garcia bb13e3b8e4 Add test for MBEDTLS_SSL_HW_RECORD_ACCEL in all.sh
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-02-26 10:26:02 +01:00
Gilles Peskine fa0e8b51c4 Test GCC and Clang with common build options
Goals:
* Build with common compilers with common options, so that we don't
  miss a (potentially useful) warning only triggered with certain
  build options.
* A previous commit removed -O0 test jobs, leaving only the one with
  -m32. We have inline assembly that is disabled with -O0, falling
  back to generic C code. This commit restores a test that runs the
  generic C code on a 64-bit platform.
2020-02-03 20:06:31 +01:00
Gilles Peskine c9247122e3 Replace -O0 by -O1 or -Os in most components
Gcc skips some analyses when compiling with -O0, so we may miss
warnings about things like uninitialized variables.
2020-02-03 20:06:31 +01:00
Manuel Pégourié-Gonnard 2150fb22c8 Add detection for zlib headers to all.sh 2020-01-29 09:51:56 +01:00