107450 Commits

Author SHA1 Message Date
Christian Heimes
54f2898fe7
bpo-40645: Implement HMAC in C (GH-20129)
The internal module ``_hashlib`` wraps and exposes OpenSSL's HMAC API. The
new code will be used in Python 3.10 after the internal implementation
details of the pure Python HMAC module are no longer part of the public API.

The code is based on a patch by Petr Viktorin for RHEL and Python 3.6.

Co-Authored-By: Petr Viktorin <encukou@gmail.com>
2020-05-17 13:49:10 +02:00
Zackery Spytz
46545000c2
bpo-39976: Add **other_popen_kwargs to subprocess docs (GH-20145)
Patch by Zackery Spytz.
2020-05-17 03:52:47 -07:00
Ned Deily
fa098b6bc8
bpo-35569: add Erlend to Misc/ACKS (GH-20146) 2020-05-17 02:57:25 -04:00
Erlend Egeberg Aasland
9a45bfe6f4
bpo-35569: Expose RFC 3542 IPv6 socket options on macOS (GH-19526) 2020-05-17 02:32:46 -04:00
Chris Jerdonek
d7184d3dbd
bpo-29587: Add another test for the gen.throw() fix. (GH-19859) 2020-05-16 21:14:48 -07:00
Lysandros Nikolaou
2c8cd06afe
bpo-40334: Improvements to error-handling code in the PEG parser (GH-20003)
The following improvements are implemented in this commit:
- `p->error_indicator` is set, in case malloc or realloc fail.
- Avoid memory leaks in the case that realloc fails.
- Call `PyErr_NoMemory()` instead of `PyErr_Format()`, because it requires no memory.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-17 04:19:23 +01:00
Pablo Galindo
6341fc7257
bpo-38870: Use subTest in test_unparse for better error reporting (GH-20141) 2020-05-17 03:53:57 +01:00
Pablo Galindo
af8e5f84d9
Use subTest in test_exceptions for better error reporting (GH-20140) 2020-05-17 01:22:00 +01:00
Christian Heimes
837f9e42e3
bpo-40645: Deprecated internal details of hmac.HMAC (GH-20132) 2020-05-17 01:05:40 +02:00
Batuhan Taskaya
dff92bb31f
bpo-38870: Implement round tripping support for typed AST in ast.unparse (GH-17797) 2020-05-17 00:04:12 +01:00
Batuhan Taskaya
e966af7cff
bpo-38870: Correctly handle empty docstrings in ast.unparse (GH-18768)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-16 23:49:07 +01:00
Batuhan Taskaya
d5a980a607
bpo-40165: Suppress stderr when checking if test_stty_match should be skipped (GH-19325) 2020-05-16 23:38:02 +01:00
Batuhan Taskaya
cae2275949
bpo-40211: Clarify os.preadv and os.pwritev are supported on AIX 7.1+ (GH-19401) 2020-05-16 23:36:40 +01:00
Batuhan Taskaya
31245d19f2
bpo-39740: Early declare devpoll_methods to support old compilers (GH-19281) 2020-05-16 23:36:14 +01:00
Batuhan Taskaya
25160cdc47
bpo-38870: Don't put unnecessary parentheses on class declarations in ast.parse (GH-20134) 2020-05-16 22:53:25 +01:00
Batuhan Taskaya
ce4a753dcb
bpo-38870: Do not separate factor prefixes in ast.unparse (GH-20133) 2020-05-16 22:46:11 +01:00
Christian Heimes
d5b3f6b7f9
bpo-37630: Use SHA3 and SHAKE XOF from OpenSSL (GH-16049)
OpenSSL 1.1.1 comes with SHA3 and SHAKE builtin.

Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: @tiran
2020-05-16 13:27:06 -07:00
Paul Ganssle
b17e49e0de
bpo-40503: Add documentation and what's new entry for zoneinfo (GH-20006)
This adds the documentation for the `zoneinfo` module added in PEP 615: https://www.python.org/dev/peps/pep-0615/

The implementation itself was GH-19909: https://github.com/python/cpython/pull/19909

bpo-40503: https://bugs.python.org/issue40503

Co-authored-by: Victor Stinner <vstinner@python.org>
2020-05-16 12:14:58 -04:00
Paul Ganssle
1b97b9b0ad
bpo-24416: Return named tuple from date.isocalendar() (GH-20113)
{date, datetime}.isocalendar() now return a private custom named tuple object
IsoCalendarDate rather than a simple tuple.

In order to leave IsocalendarDate as a private class and to improve what
backwards compatibility is offered for pickling the result of a
datetime.isocalendar() call, add a __reduce__ method to the named tuples that
reduces them to plain tuples. (This is the part of this PR most likely to cause
problems — if it causes major issues, switching to a strucseq or equivalent
would be prudent).

The pure python implementation of IsoCalendarDate uses positional-only
arguments, since it is private and only constructed by position anyway; the
equivalent change in the argument clinic on the C side would require us to move
the forward declaration of the type above the clinic import for whatever
reason, so it seems preferable to hold off on that for now.

bpo-24416: https://bugs.python.org/issue24416

Original PR by Dong-hee Na with only minor alterations by Paul Ganssle.

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2020-05-16 10:02:59 -04:00
Dong-hee Na
aa92a7cf21
bpo-39305: Update nntplib to merge nntplib.NNTP and nntplib._NNTPBase (GH-19817) 2020-05-16 19:31:54 +09:00
Hai Shi
372fa3ead5
bpo-40275: lazy import modules in test.support (GH-20128)
Automerge-Triggered-By: @vstinner
2020-05-16 03:01:39 -07:00
Batuhan Taskaya
4541086232
bpo-40192: Use thread_cputime for time.thread_time to improve resolution (GH-19381)
On AIX, time.thread_time() is now implemented with thread_cputime()
which has nanosecond resolution, rather than
clock_gettime(CLOCK_THREAD_CPUTIME_ID) which has a resolution of 10 ms.
2020-05-16 11:39:09 +02:00
Paul Ganssle
62972d9d73
bpo-40503: PEP 615: Tests and implementation for zoneinfo (GH-19909)
This is the initial implementation of PEP 615, the zoneinfo module,
ported from the standalone reference implementation (see
https://www.python.org/dev/peps/pep-0615/#reference-implementation for a
link, which has a more detailed commit history).

This includes (hopefully) all functional elements described in the PEP,
but documentation is found in a separate PR. This includes:

1. A pure python implementation of the ZoneInfo class
2. A C accelerated implementation of the ZoneInfo class
3. Tests with 100% branch coverage for the Python code (though C code
   coverage is less than 100%).
4. A compile-time configuration option on Linux (though not on Windows)

Differences from the reference implementation:

- The module is arranged slightly differently: the accelerated module is
  `_zoneinfo` rather than `zoneinfo._czoneinfo`, which also necessitates
  some changes in the test support function. (Suggested by Victor
  Stinner and Steve Dower.)
- The tests are arranged slightly differently and do not include the
  property tests. The tests live at test/test_zoneinfo/test_zoneinfo.py
  rather than test/test_zoneinfo.py or test/test_zoneinfo/__init__.py
  because we may do some refactoring in the future that would likely
  require this separation anyway; we may:
        - include the property tests
        - automatically run all the tests against both pure Python and C,
          rather than manually constructing C and Python test classes (similar
          to the way this works with test_datetime.py, which generates C
          and Python test cases from datetimetester.py).
- This includes a compile-time configuration option on Linux (though not
  on Windows); added with much help from Thomas Wouters.
- Integration into the CPython build system is obviously different from
  building a standalone zoneinfo module wheel.
- This includes configuration to install the tzdata package as part of
  CI, though only on the coverage jobs. Introducing a PyPI dependency as
  part of the CI build was controversial, and this is seen as less of a
  major change, since the coverage jobs already depend on pip and PyPI.

Additional changes that were introduced as part of this PR, most / all of
which were backported to the reference implementation:

- Fixed reference and memory leaks

    With much debugging help from Pablo Galindo

- Added smoke tests ensuring that the C and Python modules are built

    The import machinery can be somewhat fragile, and the "seamlessly falls
    back to pure Python" nature of this module makes it so that a problem
    building the C extension or a failure to import the pure Python version
    might easily go unnoticed.

- Adjustments to zoneinfo.__dir__

    Suggested by Petr Viktorin.

- Slight refactorings as suggested by Steve Dower.

- Removed unnecessary if check on std_abbr

    Discovered this because of a missing line in branch coverage.
2020-05-16 10:20:06 +02:00
Christian Heimes
6e8cda91d9
bpo-40457: Support OpenSSL without TLS 1.0/1.1 (GH-19862)
OpenSSL can be build without support for TLS 1.0 and 1.1. The ssl module
now correctly adheres to OPENSSL_NO_TLS1 and OPENSSL_NO_TLS1_1 flags.

Also update multissltest to test with latest OpenSSL and LibreSSL
releases.

Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: @tiran
2020-05-15 18:33:05 -07:00
Zackery Spytz
6b6092f533
bpo-39075: types.SimpleNamespace no longer sorts attributes in its repr (GH-19430) 2020-05-15 18:27:54 -07:00
Chris Jerdonek
1ce5841eca
bpo-31033: Add a msg argument to Future.cancel() and Task.cancel() (GH-19979) 2020-05-15 16:55:50 -07:00
Chris Jerdonek
fe1176e882
Add Andrew York to ACKS for GH-19622. (GH-20105)
This updates ACKS for commit 003708bcf8f2c58d4b65f68318acf164d713e008
contributed by Andrew York.
2020-05-15 15:19:39 -07:00
Christian Heimes
9b60e55db2
bpo-40637: Add option to disable builtin hashes (GH-20121)
Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: @tiran
2020-05-15 14:54:53 -07:00
Shantanu
a2b3cdd661
bpo-40445: Update compileall.compile_dir docs (GH-19806) 2020-05-15 14:28:23 -07:00
Gregory P. Smith
6a5d3ff676
bpo-40636: Clarify the zip built-in docstring. (GH-20118)
Clarify the zip built-in docstring.

This puts much simpler text up front along with an example.

As it was, the zip built-in docstring was technically correct.  But too
technical for the reader who shouldn't _need_ to know about `__next__` and
`StopIteration` as most people do not need to understand the internal
implementation details of the iterator protocol in their daily life.

This is a documentation only change, intended to be backported to 3.8; it is
only tangentially related to PEP-618 which might offer new behavior options
in the future.

Wording based a bit more on enumerate per Brandt's suggestion.

This gets rid of the legacy wording paragraph which seems too tied to
implementation details of the iterator protocol which isn't relevant here.

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2020-05-15 14:26:00 -07:00
Christian Heimes
938717fd04
bpo-40163: Fix multissltest download of old OpenSSL (GH-19329) 2020-05-15 22:32:25 +02:00
romasku
382a5635bd
bpo-40607: Reraise exception during task cancelation in asyncio.wait_for() (GH-20054)
Currently, if asyncio.wait_for() timeout expires, it cancels
inner future and then always raises TimeoutError. In case
those future is task, it can handle cancelation mannually,
and those process can lead to some other exception. Current
implementation silently loses thoses exception.

To resolve this, wait_for will check was the cancelation
successfull or not. In case there was exception, wait_for
will reraise it.

Co-authored-by: Roman Skurikhin <roman.skurikhin@cruxlab.com>
2020-05-15 13:12:05 -07:00
Christian Heimes
c087a268a4
bpo-40515: Require OPENSSL_THREADS (GH-19953)
The ``ssl`` and ``hashlib`` modules now actively check that OpenSSL is
build with thread support. Python 3.7.0 made thread support mandatory and no
longer works safely with a no-thread builds.

Signed-off-by: Christian Heimes <christian@python.org>
2020-05-15 11:55:25 -07:00
Christian Heimes
62d618c06b
bpo-40479: Test with latest OpenSSL versions (GH-20108)
* 1.0.2u (EOL)
* 1.1.0l (EOL)
* 1.1.1g
* 3.0.0-alpha2 (disabled for now)

Build the FIPS provider and create a FIPS configuration file for OpenSSL
3.0.0.

Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: @tiran
2020-05-15 09:48:25 -07:00
Christian Heimes
16d4e6f6f5
bpo-40479: Fix hashlib issue with OpenSSL 3.0.0 (GH-20107)
OpenSSL 3.0.0-alpha2 was released today. The FIPS_mode() function has
been deprecated and removed. It no longer makes sense with the new
provider and context system in OpenSSL 3.0.0.

EVP_default_properties_is_fips_enabled() is good enough for our needs in
unit tests. It's an internal API, too.

Signed-off-by: Christian Heimes <christian@python.org>
2020-05-15 09:28:05 -07:00
Victor Stinner
6e57237faf
bpo-40055: test_distutils leaves warnings filters unchanged (GH-20095)
distutils.tests now saves/restores warnings filters to leave them
unchanged. Importing tests imports docutils which imports
pkg_resources which adds a warnings filter.
2020-05-15 18:06:23 +02:00
Andrew York
003708bcf8
Trivial typo fix in _tkinter.c (GH-19622)
Change spelling of a #define in _tkinter.c from HAVE_LIBTOMMAMTH to HAVE_LIBTOMMATH, since this is used to keep track of tclTomMath.h, not tclTomMamth.h. No other file seems to refer to this variable.
2020-05-15 03:43:58 -07:00
Chris Jerdonek
4a12d12186
Fix typo in code comment in main_loop label. (GH-20068) 2020-05-14 19:25:45 -07:00
Guido van Rossum
15bc9ab301
bpo-40612: Fix SyntaxError edge cases in traceback formatting (GH-20072)
This fixes both the traceback.py module and the C code for formatting syntax errors (in Python/pythonrun.c). They now both consistently do the following:

- Suppress caret if it points left of text
- Allow caret pointing just past end of line
- If caret points past end of line, clip to *just* past end of line

The syntax error formatting code in traceback.py was mostly rewritten; small, subtle changes were applied to the C code in pythonrun.c.

There's still a difference when the text contains embedded newlines. Neither handles these very well, and I don't think the case occurs in practice.

Automerge-Triggered-By: @gvanrossum
2020-05-14 19:22:48 -07:00
Chris Jerdonek
1aa8767baf
Update code comment re: location of struct _is. (GH-20067) 2020-05-14 19:11:00 -07:00
Filipe Laíns
6a78589b6b
bpo-40548: github actions: pass the changes check on no source changes (GH-20097)
Signed-off-by: Filipe Laíns <lains@archlinux.org>
2020-05-15 04:08:21 +02:00
Pablo Galindo
16ab07063c
bpo-40334: Correctly identify invalid target in assignment errors (GH-20076)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-05-15 02:04:52 +01:00
Joannah Nanjekye
7ba1f75f3f
bpo-38872: Document exec symbol for codeop.compile_command (GH-20047)
* Document exec symbol for codeop.compile_command

* Remove extra statements

Co-authored-by: nanjekyejoannah <joannah.nanjekye@ibm.com>
2020-05-14 21:59:46 -03:00
Victor Stinner
4b972faf60
bpo-40462: Fix typo in test_json (GH-20094) 2020-05-15 01:02:10 +02:00
Victor Stinner
edf2643bbb
bpo-40460: Fix typo in idlelib/zzdummy.py (GH-20093)
Replace ztest with ztext.
2020-05-15 00:51:51 +02:00
Filipe Laíns
75d7257b20
bpo-40548: GitHub Action workflow: skip jobs on doc only PRs (GH-19983)
Signed-off-by: Filipe Laíns <lains@archlinux.org>
2020-05-15 00:11:40 +02:00
Victor Stinner
d2dc827d16
bpo-40602: _Py_hashtable_set() reports rehash failure (GH-20077)
If _Py_hashtable_set() fails to grow the hash table (rehash), it now
fails rather than ignoring the error.
2020-05-14 22:44:32 +02:00
Lysandros Nikolaou
ce21cfca7b
bpo-40618: Disallow invalid targets in augassign and except clauses (GH-20083)
This commit fixes the new parser to disallow invalid targets in the
following scenarios:
- Augmented assignments must only accept a single target (Name,
  Attribute or Subscript), but no tuples or lists.
- `except` clauses should only accept a single `Name` as a target.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-14 21:13:50 +01:00
Pablo Galindo
bcc3036095
bpo-40619: Correctly handle error lines in programs without file mode (GH-20090) 2020-05-14 21:11:48 +01:00
Victor Stinner
a482dc500b
bpo-40602: Write unit tests for _Py_hashtable_t (GH-20091)
Cleanup also hashtable.c.
Rename _Py_hashtable_t members:

* Rename entries to nentries
* Rename num_buckets to nbuckets
2020-05-14 21:55:47 +02:00