Commit graph

12904 commits

Author SHA1 Message Date
Gilles Peskine 3ab121a603 hkdf_expand: explicitly initialize t
t is never used uninitialized, since the first loop iteration reads 0
bytes of it and then writes hash_len bytes, and subsequent iterations
read and write hash_len bytes. However this is somewhat fragile, and
it would be legitimate for a static analyzer to be unsure.

Initialize t explicitly, to make the code clearer and more robust, at
negligible cost.

Reported by Vasily Evseenko in
https://github.com/ARMmbed/mbedtls/pull/2942
with a slightly different fix.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-02 19:54:51 +02:00
Gilles Peskine 60d6516c60 Fix whitespace style
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-04-02 19:50:00 +02:00
Manuel Pégourié-Gonnard 6f8aaba524
Merge pull request #3131 from AndrzejKurek/makefile-remove-priority-dependency
makefile: remove unneeded priority dependency from library makefile
2020-03-27 13:06:16 +01:00
Gilles Peskine 1596c5e9cb
Merge pull request #3133 from gilles-peskine-arm/changelog-date-pylint
assemble_changelog.py: Reformatting to pacify pylint
2020-03-27 11:29:37 +01:00
Gilles Peskine ac0f0860f1 Reformatting to pacify pylint
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-27 10:56:45 +01:00
Gilles Peskine 4457a0a489
Merge pull request #3128 from gilles-peskine-arm/changelog-date
Sort changelog entries by their merge date
2020-03-27 10:42:43 +01:00
Gilles Peskine 42f384c186 Mbed Crypto isn't a separate project anymore
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-27 09:23:38 +01:00
Gilles Peskine 98a53aa399 Follow renames when looking for the creation of a file
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 22:50:11 +01:00
Gilles Peskine 13dc634282 Simplify the matching of the last line
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 22:49:44 +01:00
Gilles Peskine 7fa3eb7d44 Simplify sorting
Calling sort_key explicitly is marginally faster, but less readable.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 22:41:32 +01:00
Gilles Peskine 28af958ea4 Documentation improvements
Document that git is needed.

Be clearer about the entry sort key being an entry sort key, not just
a merge order. Be clearer about what "merge order" means.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 22:39:18 +01:00
Andrzej Kurek 98911cdd3a
makefile: remove unneeded priority dependency from library makefile
This change was first introduced in 8af3923 - see this commit for more background.
After the removal of crypto directory, there are no targets that require a 
crypto library with the directory prefix, so there's also no need for the priority
dependency to be declared. This commit removes it.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2020-03-26 07:18:42 -04:00
Gilles Peskine 8f46bbf46f Sort changelog entries by their merge date
Changelog entry files were listed in reverse alphabetical order of the
file name, by happenstance. Now, changelog entry files are listed in
the order in which the changes were merged. More precisely: look for
the git commit where the entry file was created, and look where this
commit was merged into the current branch. List older merges first.
List never-merged commits in date order after all the merged ones.
List never-committed files in file timestamp order after all the
committed ones.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 11:58:00 +01:00
Gilles Peskine 27a1facd48 Factor out list_files_to_merge
No behavior changes.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-26 11:57:54 +01:00
Manuel Pégourié-Gonnard 2451ac698d
Merge pull request #3126 from gilles-peskine-arm/merge-crypto-stragglers-20200325
Merge mbed-crypto stragglers 2020-03-25
2020-03-25 22:12:30 +01:00
Gilles Peskine 854770b84e Temporarily remove ChangeLog.md
All the content that is currently in ChangeLog.md is the result of
merging Mbed Crypto, and that content is already present in
ChangeLog. Since we aren't using ChangeLog.md yet, remove it for now.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-25 17:13:19 +01:00
Gilles Peskine 5d324b203a Merge remote-tracking branch 'upstream-crypto/development' into HEAD
Conflicts:
* .github/issue_template.md, .github/pull_request_template.md:
  The changes in Mbed Crypto are not relevant to Mbed TLS. Keep the
  mbedtls versions.
2020-03-25 15:47:43 +01:00
Gilles Peskine 8bad3ab45f
Merge pull request #300 from gilles-peskine-arm/assemble-changelog
Changelog assembly script
2020-03-25 15:17:04 +01:00
Jaeden Amero dfd5ea239d
Merge pull request #3117 from AndrzejKurek/null_buffer_tests
Increase test coverage by adding AES and CAMELLIA empty buffer tests
2020-03-25 18:04:48 +04:00
Manuel Pégourié-Gonnard 8b0187e6f4
Merge pull request #392 from mpg/no-new-issue-or-pr
Have issue and PR templates redirect to Mbed TLS
2020-03-25 09:55:33 +01:00
Manuel Pégourié-Gonnard cdcb187eb2 Have issue and PR templates redirect to Mbed TLS
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-03-25 09:31:23 +01:00
Gilles Peskine d414c32a16
Merge pull request #3113 from mpg/ssl-opt-names
Improve some test names in ssl-opt.sh
2020-03-24 19:48:09 +01:00
Andrzej Kurek 1b7a780e65
Increase test coverage by adding AES and CAMELLIA empty buffer tests
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2020-03-24 13:18:58 -04:00
Gilles Peskine 7a054d1def
Merge pull request #3094 from gilles-peskine-arm/merge-crypto-docs
Merge documentation from Mbed Crypto
2020-03-24 14:19:34 +01:00
Gilles Peskine 37b61814cb
Merge pull request #3085 from gilles-peskine-arm/merge-crypto-unremoved-20200304
Merge of Mbed Crypto

Merge Mbed Crypto back into Mbed TLS. See https://github.com/ARMmbed/mbed-crypto/issues/374 for more details.
2020-03-24 13:56:15 +01:00
Manuel Pégourié-Gonnard 9e2c80fa24 Improve some test names in ssl-opt.sh
- "Default" should only be used for tests that actually use the defaults (ie,
  not passing options on the command line, except maybe debug/dtls)
- All tests in the "Encrypt then MAC" group should start with that string as a
  common prefix

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2020-03-24 10:53:39 +01:00
Gilles Peskine 3ed2de9f54 Refresh generated files 2020-03-23 21:54:07 +01:00
Gilles Peskine f03bd81be0 Place MBEDTLS_SSL_VARIABLE_BUFFER_LENGTH in alphabetical order
The automatic merge of include/mbedtls/config.h had placed
MBEDTLS_SSL_VARIABLE_BUFFER_LENGTH before
MBEDTLS_SSL_TRUNCATED_HMAC_COMPAT.
2020-03-23 18:13:58 +01:00
Gilles Peskine 5e7d6fd240 Merge 'mbedtls/development' into merge-crypto-unremoved-20200304
Merge the latest state of the target branch (mbedtls/development) into the
pull request to merge mbed-crypto into mbedtls.

Conflicts:

* ChangeLog: add/add conflict. Resolve by using the usual section order.
2020-03-23 18:02:07 +01:00
Gilles Peskine 5ec51f923a Future-proof make clean under Windows
In Windows cmd, `del foo` succeeds if `foo` doesn't exist, but
`del *.ext` fails if `*.ext` doesn't match any files. Therefore we use
the idiom `if exist *.ext del *.ext` to delete files matching
wildcards.

We've accidentally used `if exist $(some_list) del $(some_list)`, and
that's wrong, because it's only syntactically correct if
`$(some_list)` contains exactly one element. As long as `$(some_list)`
contains actual file names and not wildcards, just use `del $(some_list)`.
2020-03-23 17:55:32 +01:00
Gilles Peskine a647f1296e Delete third-party objects in cleanup 2020-03-23 17:55:32 +01:00
Gilles Peskine 049fefff90 Add dependencies on !MBEDTLS_SHA512_NO_SHA384 to x509/ssl unit tests
Add dependencies on !MBEDTLS_SHA512_NO_SHA384 to X.509 and SSL unit
tests that use SHA-384 (identified based on having a description that
contains "SHA384" or "SHA-384").
2020-03-23 17:55:32 +01:00
Gilles Peskine 6eb45f99f0 Add missing dependency on MBEDTLS_CIPHER_MODE_CBC 2020-03-23 17:55:32 +01:00
Gilles Peskine 19e841e19f Fix warning about a potentially initialized variable
The variable drop_len is only used under conditions where it has been
initialized, but this is somewhat fragile and not obvious for a
compiler to see.
2020-03-23 17:55:32 +01:00
Gilles Peskine eccd888717 Rename identifiers containing double-underscore
Rename identifiers containing double-underscore (`__`) to avoid `__`.
The reason to avoid double-underscore is that all identifiers
containing double-underscore are reserved in C++. Rename all such
identifiers that appear in any public header, including ssl_internal.h
which is in principle private but in practice is installed with the
public headers.

This commit makes check-names.sh pass.

```
perl -i -pe 's/\bMBEDTLS_SSL__ECP_RESTARTABLE\b/MBEDTLS_SSL_ECP_RESTARTABLE_ENABLED/g; s/\bMBEDTLS_KEY_EXCHANGE_(_\w+)_(_\w+)\b/MBEDTLS_KEY_EXCHANGE${1}${2}/g' include/mbedtls/*.h library/*.c programs/*/*.c scripts/data_files/rename-1.3-2.0.txt tests/suites/*.function
```
2020-03-23 17:55:32 +01:00
Gilles Peskine 2f084fe02c Add changelog entries about the crypto submodule
Add a changelog entry for an already-released version to indicate when
the crypto submodule became mandatory.

Add a changelog entry for the removal of the crypto submodule.
2020-03-23 17:55:32 +01:00
Gilles Peskine 7415f2fd12 Add comment to help syntax highlighting in editors
Restore this change which was accidentally reverted during the merge
of Mbed Crypto.
2020-03-23 17:55:32 +01:00
Gilles Peskine 0a968f7c7f Fix the shared library build with CMake
Fix an error in the earlier merge.
2020-03-23 17:55:32 +01:00
Gilles Peskine 55d5353b3a Remove lingering build code referencing the crypto submodule
Remove code guarded by `USE_CRYPTO_SUBMODULE`. It's dead now that
crypto can no longer be a submodule.

In `library/Makefile`:

* Replace `$(CRYPTO_INCLUDE)` with the single include directory
  `-I../include`.
* Remove references to `$(OBJS_CRYPTO)` when it's in addition to the
  local objects (`*.o`) since `$(OBJS_CRYPTO)` is now a subset of the
  local objects.
* Merge modules that were duplicated between the mbedtls and the
  mbed-crypto repositories back into the single list for `OBJS_CRYPTO`.
2020-03-23 17:55:32 +01:00
Gilles Peskine f8c2965d11 Remove the crypto submodule
All crypto files are now in the main tree.
2020-03-23 17:55:31 +01:00
Gilles Peskine aae57bffd9 Remove remaining references to the crypto subdirectory 2020-03-23 17:55:31 +01:00
Gilles Peskine 18a9d5b425 Use an indirect reference to detect the 3rdparty directory 2020-03-23 17:55:31 +01:00
Gilles Peskine 37940d923e Restore linking against all libraries in the cmake_subproject test 2020-03-23 17:55:31 +01:00
Gilles Peskine 4973196e5a Restore fuzz to the cmake build 2020-03-23 17:55:31 +01:00
Gilles Peskine d2e77b51a1 Restore the library SO version in Makefile 2020-03-23 17:55:31 +01:00
Gilles Peskine 4e616f893b Remove config-default.h
This file was the default mbedtls configuration in the mbedtls-psa
fork. It is no longer relevant.
2020-03-23 17:55:31 +01:00
Gilles Peskine b22d0cfa99 Re-generate automatically generated files
```
scripts/generate_features.pl
scripts/generate_errors.pl
scripts/generate_query_config.pl
scripts/generate_visualc_files.pl
```
2020-03-23 17:55:31 +01:00
Gilles Peskine d3d02900f2 Restore X509 and SSL functionality
Look for changes that remove X509/SSL functionality.
```
git diff 'HEAD^{/^Merge}~1' HEAD --diff-filter=M -- . ':!library/error.c' ':!library/version_features.c' ':!programs/test/query_config.c' ':!visualc' ':!*.pdf' ':!*.der' | grep -E "^-.*MBEDTLS_(ERR_)?(PKCS11|X509|NET|SSL)_"
```

All of these removals are in `config.h` or `check_config.h`. Selectively revert the differences in these two files.

```
git diff 'HEAD^{/^Merge}~1' 'HEAD^{/^Merge}' include/mbedtls/config.h include/mbedtls/check_config.h | git apply -p1 -R
```

* `include/mbedtls/check_config.h`:
    * ARIA for GCM: don't remove it.
    * `MBEDTLS_PSA_CRYPTO_SE_C`: don't remove it.
    * `MBEDTLS_SHA512_NO_SHA384`: don't remove it.
    * `MBEDTLS_SSL_DTLS_CONNECTION_ID`: restore it.
* `include/mbedtls/config.h`:
    * warning about `MBEDTLS_ECDSA_SIGN_ALT`: don't remove it.
    * `MBEDTLS_CTR_DRBG_USE_128_BIT_KEY` full paragraph: don't remove it.
    * `MBEDTLS_PSA_CRYPTO_KEY_FILE_ID_ENCODES_OWNER`: don't remove it.
    * `MBEDTLS_SHA512_SMALLER`…: don't remove it.
    * `MBEDTLS_SSL_RECORD_CHECKING`: restore it.
    * `MBEDTLS_SSL_CONTEXT_SERIALIZATION`: restore it.
    * `MBEDTLS_USE_PSA_CRYPTO` note: don't restore the tls version.
    * `MBEDTLS_USE_PSA_CRYPTO` warning: restore the tls version.
    * `MBEDTLS_CMAC_C`: restore it to being disabled by default. It's a minor API change in Mbed TLS because it changes the layout of `mbedtls_cipher_context_t`.
    * `MBEDTLS_CTR_DRBG_C`: don't restore the older version of the description from tls.
    * `MBEDTLS_GCM_C`: don't restore the older description from tls.
    * `MBEDTLS_PSA_CRYPTO_C`: don't restore `crypto/`.
    * `MBEDTLS_PSA_CRYPTO_SE_C`: don't remove it.
    * `MBEDTLS_PSA_CRYPTO_STORAGE_C`: don't restore `crypto/`. Don't disable it by default.
    * `MBEDTLS_PSA_ITS_FILE_C`: don't restore, like for ``MBEDTLS_PSA_CRYPTO_STORAGE_C``.
    * `MBEDTLS_CTR_DRBG_USE_128_BIT_KEY` single line: don't restore it since there is now a full paragraph in the proper section above.
    * `MBEDTLS_SSL_CID_IN_LEN_MAX`…: restore it.
    * `MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_CERTIFICATES`: restore the version without a space to format the line like the others.
2020-03-23 17:55:31 +01:00
Gilles Peskine a57a80e213 Restore full tls coverage to remaining scripts
Look for any other invocation of test scripts that was removed: look for a change that removes a line (`^-.*…`) containing one of the names of a test script (without its path because it may be used with a relative path). Look for `ssl-opt.sh` and `compat.sh`, as well any file in `tests/scripts` that only exists in tls.
```
git diff 'HEAD^{/^Merge}~1' HEAD --diff-filter=M -- . ':!library/error.c' ':!library/version_features.c' ':!programs/test/query_config.c' ':!visualc' ':!*.pdf' ':!*.der' | grep -E "^-.*($(comm -23 <(git ls-tree -r --name-only 'HEAD^{/^Merge}~1') <(git ls-tree -r --name-only $(git merge-base upstream-crypto/development 'HEAD^{/^Merge}^2')) | sed -n 's!^tests/scripts/!!p' | sed 's/\./\\./g' | tr '\n' '|')ssl-opt\.sh|compat\.sh)" | grep -v '^---'
```
This only turns up changes in `basic-in-docker.sh`.
2020-03-23 17:55:31 +01:00
Gilles Peskine 636c26ad75 Restore full tls coverage to all.sh
The merge of mbed-crypto removed some tls coverage. Restore it. Also
remove references to the `crypto` subdirectory brought by the mbedtls
side of the merge. In more detail:

* `tests/scripts/all.sh`:
    * `fuzz` in comments (×2): restore it.
    * `CTEST_OUTPUT_ON_FAILURE=1`: don't remove it.
    * `cd crypto` for `make clean`: don't restore it.
    * `cleanup`: do restore `programs/fuzz/Makefile`. Don't go into `crypto`. Keep only one copy of the calls to `rm` in `cmake_subproject`.
    * Comment legacy options: don't remove it.
    * `crypto/Makefile` and `pre_check_seedfile`: don't restore either. See below regarding the lack of need for `pre_check_seedfile`.
    * blank line in `pre_print_configuration`: restore it.
    * blank line before `#### Build and test`: restore it.
    * SSL tests in `component_test_full_cmake_gcc_asan` and zlib components: restore it.
    * `component_test_no_pem_no_fs` (×2): the merge placed two copies in different locations. Reconcile them: unset PSA storage like in crypto, and call `ssl-opt.sh` like in tls. Put the merged version at the tls location.
    * `component_test_everest`: do add it at the tls location.
    * `component_test_small_mbedtls_ssl_dtls_max_buffering`: restore the tls value.
    * `component_test_new_ecdh_context`…: move `component_test_new_ecdh_context` before `component_test_everest` and add a calls to `compat.sh` and `ssl-opt.sh` like in `component_test_everest`. Remove the redundant crypto-only `component_test_everest`. Don't remove `component_test_psa_collect_statuses`.
    * `component_test_full_cmake_clang`: don't remove `clang` in the `msg` call. Don't remove the call to `test_psa_constant_names.py`.
    * `component_test_full_make_gcc_o0`: remove it. It's subsumed by `component_test_gcc_opt`.
    * `component_build_deprecated`: don't remove anything.
    * `component_test_memory_buffer_allocator`: restore `ssl-opt.sh`.
    * `component_test_when_no_ciphersuites_have_mac`: restore it.
    * `component_test_platform_calloc_macro`: don't restore `unset MBEDTLS_MEMORY_BUFFER_ALLOC_C` which is now redundant. Don't restore explicit flags instead of `$ASAN_CFLAGS`.
    * `component_test_aes_fewer_tables`…: don't remove it.
    * `component_test_m32_o1`: restore SSL testing.
    * `component_test_m32_everest`: restore SSL testing.
    * `component_test_min_mpi_window_size`…: don't remove it.
    * `component_test_valgrind`: do restore the tls version of the comment.
    * `run_component`: don't remove the seedfile creation. This is better than `pre_check_seedfile` (see below).
    * `pre_check_seedfile`: don't restore it. `pre_check_seedfile` (from tls) creates a seedfile once and for all. This is not good enough if a component fails in such a way as to leave a broken seedfile, or if a component leaves a seedfile with a size that's wrong for the next component to run. Instead (from crypto), `run_component` creates a sufficiently large seedfile before each component.
2020-03-23 17:55:31 +01:00