Commit graph

184 commits

Author SHA1 Message Date
Gilles Peskine c780095901 Delete $OUT_OF_SOURCE_DIR under --force even without Yotta
The deletion of "$OUT_OF_SOURCE_DIR" had mistakenly been lumped
together with Yotta.
2019-01-10 18:29:37 +01:00
Gilles Peskine c9663b1685 Fix sometimes-spurious warning about changed config.h
After backing up and restoring config.h, `git diff-files` may report
it as potentially-changed because it isn't sure whether the index is
up to date. To avoid this, make sure that the git index is up-to-date.

This fixes the warning about changed config.h that you might get when
you run all.sh twice in succession, yet if you run `git status` or
`git diff` everything seems up to date and you no longer get the
warning because these git commands update the index.

https://stackoverflow.com/questions/36367190/git-diff-files-output-changes-after-git-status
2019-01-10 18:29:37 +01:00
Gilles Peskine 2906a0ae8a all.sh: Update the maintainer documentation 2019-01-10 18:29:33 +01:00
Gilles Peskine 541fb1e33b all.sh: only check tools that are going to be used
Don't require openssl, mingw, etc. if we aren't going to run a
component that uses them.
2019-01-10 18:28:17 +01:00
Gilles Peskine 53084872ab all.sh: only look for armcc if it is used
Only look for armcc if component_build_armcc or component_build_yotta
is to be executed, instead of requiring the option --no-armcc.

You can still pass --no-armcc, but it's no longer required when
listing components to run. With no list of components or an exclude
list on the command line, --no-armcc is equivalent to having
build_armcc in the exclude list.

Omit the yotta pre-checks if the build_yotta component is not going to
be executed. This makes --no-yotta equivalent to specifying a list of
components to run that doesn't include build_yotta.
2019-01-10 18:28:17 +01:00
Gilles Peskine b3241cbea7 Add command line options to list available components 2019-01-10 18:28:17 +01:00
Gilles Peskine eb39b9b729 Minor cleanups to component name gathering
Bring the code in line with the version in Mbed TLS 2.16+.
2019-01-10 18:28:17 +01:00
Gilles Peskine 3fbdd21ca5 Add conditional component inclusion facility
Add a conditional execution facility: if a function support_xxx exists
and returns false then component_xxx is not executed (except when the
command line lists an explicit set of components to execute).

Use this facility for the 64-bit-specific or amd64-specific components.
2019-01-10 18:28:17 +01:00
Gilles Peskine 7120f77889 all.sh: fix MAKEFLAGS setting
MAKEFLAGS was set to -j if it was already set, instead of being set if
not previously set as intended. So now all.sh will do parallel builds
if invoked without MAKEFLAGS in the environment.
2019-01-10 18:28:17 +01:00
Gilles Peskine 770ad7e2c9 all.sh: don't insist on Linux; always run Valgrind
Don't bail out of all.sh if the OS isn't Linux. We only expect
everything to pass on a recent Linux x86_64, but it's useful to call
all.sh to run some components on any platform.

In all.sh, always run both MemorySanitizer and Valgrind. Valgrind is
slower than ASan and MSan but finds some things that they don't.

Run MSan unconditionally, not just on Linux/x86_64. MSan is supported
on some other OSes and CPUs these days.

Use `all.sh --except test_memsan` if you want to omit MSan because it
isn't supported on your platform. Use `all.sh --except test_memcheck`
if you want to omit Valgrind because it's too slow.

Portability: ecognize amd64 (FreeBSD arch string) as well as x86_64
(Linux arch string) for `uname -m`. The `make` utility must still
be GNU make.
2019-01-10 18:28:17 +01:00
Gilles Peskine 4e7b323fd8 Use CMAKE_BUILD_TYPE to do Asan builds
Use `cmake -D CMAKE_BUILD_TYPE=Asan` rather than manually setting
`-fsanitize=address`. This lets cmake determine the necessary compiler
and linker flags.

With UNSAFE_BUILD on, force -Wno-error. This is necessary to build
with MBEDTLS_TEST_NULL_ENTROPY.
2019-01-10 18:28:17 +01:00
Gilles Peskine 72adb432bc Back up and restore config.h systematically
In all.sh, always save config.h before running a component, instead of
doing it manually in each component that requires it (except when we
forget, which has happened). This would break a script that requires
config.h.bak not to exist, but we don't have any of those.
2019-01-10 18:28:17 +01:00
Gilles Peskine 11ddca6b74 Add the current component name to msg output and the final report 2019-01-10 18:28:17 +01:00
Gilles Peskine 6e9842315a Add --except mode: run all components except a list
Allow the list to use wildcards, e.g. you can run the sanity checks with
all.sh --except "test_*" "build_*"
2019-01-10 18:28:17 +01:00
Gilles Peskine 91bd8b78ed all.sh: with non-option arguments, run only these components 2019-01-10 18:28:17 +01:00
Gilles Peskine 3484ed8797 Move test-ref-configs into its own component 2019-01-10 18:28:16 +01:00
Gilles Peskine 2f300dbb2f Remove duplicate component for RSA_NO_CRT 2019-01-10 18:28:16 +01:00
Gilles Peskine 1a2ca72ddc Break up the tests into components
Split the long list of tests into individual functions. Each time the
test code called the `cleanup` function, I start a new function called
`component_xxx`.

Run all the components by enumerating the `component_xxx` functions.
After running each component, call `cleanup`.

A few sanity checks didn't have calls to `cleanup` because they didn't
need them. I put them into separate components anyway, so there are
now a few extra harmless calls to `cleanup`.
2019-01-10 18:28:16 +01:00
Gilles Peskine 57db6ff938 Move the code into functions. No behavior change.
Move almost all the code of this script into functions. There is no
intended behavior change. The goal of this commit is to make
subsequent improvements easier to follow.

A very large number of lines have been reintended. To see what's going
on, ignore whitespace differences (e.g. diff -w).

I followed the following rules:

* Minimize the amount of code that gets moved.
* Don't change anything to what gets executed or displayed.
* Almost all the code must end up in a function.

This commit is in preparation for breaking up the sequence of tests
into individual components that can run independently.
2019-01-08 22:04:31 +01:00
Simon Butcher c37966239c Merge remote-tracking branch 'public/pr/2080' into mbedtls-2.7-restricted-proposed 2018-11-07 13:34:27 +00:00
Hanno Becker 72311b468d Add test for MBEDTLS_PLATFORM_{CALLOC/FREE}_MACRO to all.sh
This commit adds a test to tests/scripts/all.sh exercising an
ASan build of the default configuration with

MBEDTLS_PLATFORM_MEMORY enabled,
MBEDTLS_PLATFORM_CALLOC_MACRO set to std calloc
MBEDTLS_PLATFORM_FREE_MACRO   set to std free

(This should functionally be indistinguishable from a default build)
2018-10-11 11:10:09 +01:00
Gilles Peskine b46c59e2a7 In keep-going mode, don't hard-fail on some tests
Add if_build_succeeded in front of the invocation of some test runs
where it was missing.
2018-09-27 10:12:58 +02:00
Gilles Peskine 732826d265 In keep-going mode, don't hard-fail on some auxiliary script
Add record_status in front of the invocation of several scripts where
it was missing.
2018-09-27 10:08:31 +02:00
Simon Butcher d7126d7009 Merge remote-tracking branch 'public/pr/779' into mbedtls-2.7 2018-07-24 13:38:44 +01:00
Simon Butcher bd40916dfa Merge remote-tracking branch 'public/pr/1872' into mbedtls-2.7 2018-07-24 12:12:43 +01:00
Simon Butcher 948f264302 Add additional i386 tests to all.sh
Added an additional i386 test to all.sh, to allow one test with -O0 which
compiles out inline assembly, and one to test with -01 which includes the inline
assembly.
2018-07-23 13:41:25 +01:00
Jaeden Amero 5113bdec6e all.sh: Return error on keep-going failure
When calling all.sh from a script and using "--keep-going", errors were
sometimes missed due to all.sh always returning 0 "success" return code.
Return 1 if there is any failure encountered during a "keep-going" run.
2018-07-23 10:24:31 +01:00
Simon Butcher 7c6b84102d Expand i386 all.sh tests to full config ASan builds
The i386 test builds were only building the default configuration and had
no address sanitisation. This commit expands the test configuration to the full
configuration in all.sh and builds with ASan for when the test suites are
executed.
2018-07-20 21:34:04 +01:00
Simon Butcher 0e342f77fc Merge remote-tracking branch 'public/pr/1390' into mbedtls-2.7 2018-06-27 11:11:34 +01:00
Simon Butcher 8c83673eb2 Merge remote-tracking branch 'public/pr/1708' into mbedtls-2.7 2018-06-12 17:26:55 +01:00
Simon Butcher ee3a3d4a72 Merge remote-tracking branch 'public/pr/1470' into mbedtls-2.7 2018-06-11 11:30:33 +01:00
Darryl Green bd38c3b89f Add check-files.py to all.sh 2018-06-05 11:57:12 +01:00
Jaeden Amero 11d5551d0a Merge remote-tracking branch 'upstream-public/pr/1487' into mbedtls-2.7-proposed 2018-05-04 11:06:21 +01:00
Gilles Peskine 763da6e550 all.sh --keep-going: properly handle multiple-builds scripts
In keep-going mode, if a multiple-builds script fails, record its
status and keep going.
2018-03-23 02:18:33 +01:00
Gilles Peskine 2cfeb887b4 Merge tag 'mbedtls-2.7.2' into iotssl-1381-x509-verify-refactor-2.7-restricted
Conflict resolution:

* ChangeLog
* tests/data_files/Makefile: concurrent additions, order irrelevant
* tests/data_files/test-ca.opensslconf: concurrent additions, order irrelevant
* tests/scripts/all.sh: one comment change conflicted with a code
  addition. In addition some of the additions in the
  iotssl-1381-x509-verify-refactor-restricted branch need support for
  keep-going mode, this will be added in a subsequent commit.
2018-03-23 02:12:44 +01:00
Gilles Peskine 0114ffc76b all.sh: Verify out-of-tree testing with CMake
Run a test case in ssl-opt.sh to validate that testing works in an
out-of-tree CMake build.
2018-03-21 12:29:20 +01:00
Gilles Peskine a71d64c74f all.sh: fix cleanup happening during an out-of-tree build 2018-03-21 12:29:15 +01:00
Gilles Peskine 31b07e2833 all.sh: be more conservative when cleaning up CMake artefacts
Only delete things that we expect to find, to avoid deleting other
things that people might have lying around in their build tree.
Explicitly skip .git to avoid e.g. accidentally matching a branch
name.
2018-03-21 12:29:08 +01:00
Gilles Peskine 19ceb7104c all.sh: add opposites to all boolean options
All options can now be overridden by a subsequent option, e.g.
"all.sh --foo --no-foo" is equivalent to "all.sh --no-foo". This
allows making wrapper scripts with default options and occasionally
overriding those options when running the wrapper script.
2018-03-21 08:48:40 +01:00
Gilles Peskine 54933e95bd all.sh: option parsing: reduce vertical spread
Only whitespace changes.
2018-03-21 08:48:33 +01:00
Gilles Peskine 53038ebecc all.sh: with --no-armcc, don't call armcc from output_env.sh
When not running armcc, don't try to invoke armcc at all, not even to
report its version.
2018-03-21 08:48:26 +01:00
Gilles Peskine 955d70459d Merge remote-tracking branch 'upstream-restricted/pr/460' into mbedtls-2.7-restricted-proposed 2018-03-13 17:24:33 +01:00
Gilles Peskine 4848b97bc7 Merge remote-tracking branch 'upstream-public/pr/1249' into mbedtls-2.7-proposed 2018-03-11 00:48:17 +01:00
Hanno Becker 69d45cce5d Add a run with RSA_NO_CRT to all.sh 2018-03-09 10:46:23 +00:00
Manuel Pégourié-Gonnard 4d9e7cb66b Fix usage of CFLAGS with cmake in all.sh
With cmake, CFLAGS has to be set when invoking cmake, not make (which totally
ignores the value of CFLAGS when it runs and only keeps the one from cmake).

Also, in that case the flags were either redundant (-Werror etc) or wrong
(-std=c99 -pedantic) as some parts of the library will not build with
-pedantic (see the other -pedantic tests, which are correct, for what needs to
be disabled).
2018-03-05 13:00:03 +01:00
Manuel Pégourié-Gonnard 1ddd682843 Fix depends_on:pk_alg in test suites 2018-03-05 12:58:51 +01:00
Manuel Pégourié-Gonnard 056eab5ff0 Fix usage of {curves,key-exchanges}.pl in all.sh 2018-03-05 12:58:22 +01:00
Manuel Pégourié-Gonnard 483a776019 Fix missing depends_on:SHA/MD in x509 tests 2018-03-05 12:57:36 +01:00
Gilles Peskine b4ef45b4f5 all.sh: build with -O -Werror in the full config
Build with MBEDTLS_DEPRECATED_REMOVED and MBEDTLS_DEPRECATED_WARNING
separately.

Do these builds with `-O -Werror -Wall -Wextra` to catch a maximum of
issues while we're at it. Do one with gcc and one with clang for
variety. This caught an uninitialized variable warning in cmac.c that
builds without -O didn't catch.
2018-03-01 22:23:50 +01:00
Gilles Peskine 0afe624fff all.sh: add test with MBEDTLS_DEPRECATED_REMOVED 2018-02-21 19:28:12 +01:00