122520 Commits

Author SHA1 Message Date
Matt Wozniski
32a0faba43
gh-119517: Fixes for pasting in pyrepl (#120253)
* Remove pyrepl's optimization for self-insert

This will be replaced by a less specialized optimization.

* Use line-buffering when pyrepl echoes pastes

Previously echoing was totally suppressed until the entire command had
been pasted and the terminal ended paste mode, but this gives the user
no feedback to indicate that an operation is in progress. Drawing
something to the screen once per line strikes a balance between
perceived responsiveness and performance.

* Remove dead code from pyrepl

`msg_at_bottom` is always true.

* Speed up pyrepl's screen rendering computation

The Reader in pyrepl doesn't hold a complete representation of the
screen area being drawn as persistent state. Instead, it recomputes it,
on each keypress. This is fast enough for a few hundred bytes, but
incredibly slow as the input buffer grows into the kilobytes (likely
because of pasting).

Rather than making some expensive and expansive changes to the repl's
internal representation of the screen, add some caching: remember some
data from one refresh to the next about what was drawn to the screen
and, if we don't find anything that has invalidated the results that
were computed last time around, reuse them. To keep this caching as
simple as possible, all we'll do is look for lines in the buffer that
were above the cursor the last time we were asked to update the screen,
and that are still above the cursor now. We assume that nothing can
affect a line that comes before both the old and new cursor location
without us being informed. Based on this assumption, we can reuse old
lines, which drastically speeds up the overwhelmingly common case where
the user is typing near the end of the buffer.

* Speed up pyrepl prompt drawing

Cache the `can_colorize()` call rather than repeatedly recomputing it.
This call looks up an environment variable, and is called once per
character typed at the REPL. The environment variable lookup shows up as
a hot spot when profiling, and we don't expect this to change while the
REPL is running.

* Speed up pasting multiple lines into the REPL

Previously, we were checking whether the command should be accepted each
time a line break was encountered, but that's not the expected behavior.
In bracketed paste mode, we expect everything pasted to be part of
a single block of code, and encountering a newline shouldn't behave like
a user pressing <Enter> to execute a command. The user should always
have a chance to review the pasted command before running it.

* Use a read buffer for input in pyrepl

Previously we were reading one byte at a time, which causes much slower
IO than necessary. Instead, read in chunks, processing previously read
data before asking for more.

* Optimize finding width of a single character

`wlen` finds the width of a multi-character string by adding up the
width of each character, and then subtracting the width of any escape
sequences. It's often called for single character strings, however,
which can't possibly contain escape sequences. Optimize for that case.

* Optimize disp_str for ASCII characters

Since every ASCII character is known to display as single width, we can
avoid not only the Unicode data lookup in `disp_str` but also the one
hidden in `str_width` for them.

* Speed up cursor movements in long pyrepl commands

When the current pyrepl command buffer contains many lines, scrolling up
becomes slow. We have optimizations in place to reuse lines above the
cursor position from one refresh to the next, but don't currently try to
reuse lines below the cursor position in the same way, so we wind up
with quadratic behavior where all lines of the buffer below the cursor
are recomputed each time the cursor moves up another line.

Optimize this by only computing one screen's worth of lines beyond the
cursor position. Any lines beyond that can't possibly be shown by the
console, and bounding this makes scrolling up have linear time
complexity instead.

---------

Signed-off-by: Matt Wozniski <mwozniski@bloomberg.net>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2024-06-11 16:42:10 +00:00
Pablo Galindo Salgado
ec3af291fe
gh-120346: Respect PYTHON_BASIC_REPL when running in interactive inspect mode (#120349) 2024-06-11 16:15:01 +00:00
Jelle Zijlstra
9b8611eeea
gh-119180: PEP 649 compiler changes (#119361) 2024-06-11 13:06:49 +00:00
Kirill Podoprigora
02c1dfff07
gh-120080: Mark test_round_with_none_arg_direct_call as cpython_only (#120328) 2024-06-11 09:56:38 +02:00
Michał Górny
7d2447137e
gh-120291: Fix a bashism in python-config.sh.in (#120292)
gh-120291: Fix bashisms in python-config.sh.in

Replace the use of bash-specific `[[ ... ]]` with POSIX-compliant
`[ ... ]` to make the `python-config` shell script work with non-bash
shells again.  While at it, use `local` in a safer way, since it is
not in POSIX either (though universally supported).

Fixes #120291
2024-06-11 10:11:13 +03:00
Nikita Sobolev
141babad9b
gh-120298: Fix use-after-free in list_richcompare_impl (#120303)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-06-11 10:04:27 +03:00
blhsing
9e9ee50421
gh-65454: avoid triggering call to a PropertyMock in NonCallableMock.__setattr__ (#120019) 2024-06-11 05:42:49 +00:00
Robert Collins
422c4fc855
gh-119600: mock: do not access attributes of original when new_callable is set (#119601)
In order to patch flask.g e.g. as in #84982, that
proxies getattr must not be invoked. For that,
mock must not try to read from the original
object. In some cases that is unavoidable, e.g.
when doing autospec. However, patch("flask.g",
new_callable=MagicMock) should be entirely safe.
2024-06-11 06:41:12 +01:00
AN Long
6efe346069
Fix the CODEOWNERS for _interpretersmodule.c (gh-120288) 2024-06-10 10:45:16 -06:00
Victor Stinner
7aff2de62b
gh-120057: Add os.environ.refresh() method (#120059) 2024-06-10 16:34:17 +00:00
Irit Katriel
56c3815ba1
gh-119786: copy compiler doc from devguide to InternalDocs and convert to markdown (#120134)
* gh-119876: move compiler doc from devguide to InternalDocs

Copy of 78fc0d7aa9

Co-Authored-By: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-Authored-By: Adam Turner <9087854+aa-turner@users.noreply.github.com>
Co-Authored-By: Brett Cannon <brett@python.org>
Co-Authored-By: Carol Willing <carolcode@willingconsulting.com>
Co-Authored-By: Daniel Porteous <danielporteous1@gmail.com>
Co-Authored-By: Dennis Sweeney <36520290+sweeneyde@users.noreply.github.com>
Co-Authored-By: Éric Araujo <merwok@netwok.org>
Co-Authored-By: Erlend Egeberg Aasland <erlend.aasland@innova.no>
Co-Authored-By: Ezio Melotti <ezio.melotti@gmail.com>
Co-Authored-By: Georg Brandl <georg@python.org>
Co-Authored-By: Guido van Rossum <guido@python.org>
Co-Authored-By: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-Authored-By: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
Co-Authored-By: Jeff Allen <ja.py@farowl.co.uk>
Co-Authored-By: Jim Fasarakis-Hilliard <d.f.hilliard@gmail.com>
Co-Authored-By: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Co-Authored-By: Lincoln <71312724+Lincoln-developer@users.noreply.github.com>
Co-Authored-By: Mariatta <Mariatta@users.noreply.github.com>
Co-Authored-By: Muhammad Mahad <mahadpy@gmail.com>
Co-Authored-By: Ned Deily <nad@acm.org>
Co-Authored-By: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
Co-Authored-By: Stéphane Wirtel <stephane@wirtel.be>
Co-Authored-By: Suriyaa ✌️️ <isc.suriyaa@gmail.com>
Co-Authored-By: Zachary Ware <zachary.ware@gmail.com>
Co-Authored-By: psyker156 <242220+psyker156@users.noreply.github.com>
Co-Authored-By: slateny <46876382+slateny@users.noreply.github.com>
Co-Authored-By: svelankar <17737361+svelankar@users.noreply.github.com>
Co-Authored-By: zikcheng <surfingbyte@gmail.com>

* convert to markdown

* add to index

* update more of the out of date stuff

---------

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Brett Cannon <brett@python.org>
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
Co-authored-by: Daniel Porteous <danielporteous1@gmail.com>
Co-authored-by: Dennis Sweeney <36520290+sweeneyde@users.noreply.github.com>
Co-authored-by: Éric Araujo <merwok@netwok.org>
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Co-authored-by: Georg Brandl <georg@python.org>
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Jeff Allen <ja.py@farowl.co.uk>
Co-authored-by: Jim Fasarakis-Hilliard <d.f.hilliard@gmail.com>
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Co-authored-by: Lincoln <71312724+Lincoln-developer@users.noreply.github.com>
Co-authored-by: Mariatta <Mariatta@users.noreply.github.com>
Co-authored-by: Muhammad Mahad <mahadpy@gmail.com>
Co-authored-by: Ned Deily <nad@acm.org>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Stéphane Wirtel <stephane@wirtel.be>
Co-authored-by: Suriyaa ✌️️ <isc.suriyaa@gmail.com>
Co-authored-by: Zachary Ware <zachary.ware@gmail.com>
Co-authored-by: psyker156 <242220+psyker156@users.noreply.github.com>
Co-authored-by: slateny <46876382+slateny@users.noreply.github.com>
Co-authored-by: svelankar <17737361+svelankar@users.noreply.github.com>
Co-authored-by: zikcheng <surfingbyte@gmail.com>
2024-06-10 16:15:12 +01:00
Pieter Eendebak
c3b6dbff2c
gh-115801: Only allow sequence of strings as input for difflib.unified_diff (GH-118333) 2024-06-10 14:06:18 +03:00
Victor Stinner
b90bd3e5bb
gh-120155: Fix Coverity issue in zoneinfo load_data() (#120232)
Declare the 'rv' varaible at the top of the load_data() function to
make sure that it's initialized before the first 'goto error' which
uses 'rv' (return rv).

Fix the Coverity issue:

Error: UNINIT (CWE-457):
Python-3.12.2/Modules/_zoneinfo.c:1233:5: skipped_decl: Jumping over declaration of ""rv"".
Python-3.12.2/Modules/_zoneinfo.c:1284:5: uninit_use: Using uninitialized value ""rv"".
  1282|       }
  1283|
  1284|->     return rv;
  1285|   }
  1286|
2024-06-10 11:54:35 +02:00
E. M. Bray
4829522b8d
bpo-24766: doc= argument to subclasses of property not handled correctly (GH-2487)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-06-10 08:55:49 +00:00
Clinton
e5a7bc6f2e
gh-120296: Fix format string of fcntl.ioctl() audit (#120301) 2024-06-10 08:17:50 +00:00
Carl Meyer
0ae8579b85
gh-119666: fix multiple class-scope comprehensions referencing __class__ (#120295) 2024-06-09 22:23:30 -04:00
Kirill Podoprigora
34f5ae69fe
gh-120268: Prohibit passing `None to _pydatetime.date.fromtimestamp` (#120269)
This makes the pure Python implementation consistent with the C implementation.
2024-06-08 16:45:57 -04:00
Clinton
7c016deae6
gh-120276: Fix incorrect email.header.Header maxlinelen default (GH-120277) 2024-06-08 20:18:58 +03:00
AN Long
5d59b870ef
gh-120121: Add InvalidStateError to concurrent.futures.__all__ (#120123)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2024-06-08 21:41:19 +05:30
neonene
38a25e9560
gh-120244: Fix re.sub() reference leak (GH-120245) 2024-06-08 10:22:07 +00:00
Saul Shanabrook
55402d3232
gh-119258: Eliminate Type Guards in Tier 2 Optimizer with Watcher (GH-119365)
Co-authored-by: parmeggiani <parmeggiani@spaziodati.eu>
Co-authored-by: dpdani <git@danieleparmeggiani.me>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
Co-authored-by: Ken Jin <kenjin@python.org>
2024-06-08 17:41:45 +08:00
Enrico Tröger
2080425154
bpo-37755: Use configured output in pydoc instead of pager (GH-15105)
If the Helper() class was initialized with an output, the topics, keywords
and symbols help still use the pager instead of the output.
Change the behavior so  the output is used if available while keeping the
previous behavior if no output was configured.
2024-06-08 09:19:13 +00:00
Nikita Sobolev
95f4db88d5
gh-120242: Fix handling of [setUp,tearDown]Class in test_datetime (#120243) 2024-06-08 10:51:09 +03:00
Irit Katriel
4fc82b6d3b
gh-120225: fix crash in compiler on empty block at end of exception handler (#120235) 2024-06-07 22:37:35 +01:00
Eric Snow
e6076d1e13
gh-119659: Get the datetime CAPI Tests Running Again (gh-120180)
The tests were accidentally disabled by 2da0dc0, which didn't handle classes correctly.

I considered updating no_rerun() to support classes, but the way test_datetime.py works would have made things fairly messy.  Plus, it looks like the refleaks we had encountered before have been resolved.
2024-06-07 11:44:56 -06:00
Barney Gale
242c7498e5
GH-116380: Move pathlib-specific code from glob to pathlib._abc. (#120011)
In `glob._Globber`, move pathlib-specific methods to `pathlib._abc.PathGlobber` and replace them with abstract methods. Rename `glob._Globber` to `glob._GlobberBase`. As a result, the `glob` module is no longer befouled by code that can only ever apply to pathlib.

No change of behaviour.
2024-06-07 17:59:34 +01:00
Victor Stinner
90b7540526
gh-120155: Fix copy/paste error in HAVE_SUBOFFSETS_IN_LAST_DIM() (#120228)
Don't hardcode 'dest' in HAVE_SUBOFFSETS_IN_LAST_DIM() macro of
memoryobject.c, but use its 'view' parameter instead.

Fix the Coverity issue:

Error: COPY_PASTE_ERROR (CWE-398):
Python-3.12.2/Objects/memoryobject.c:273:14: original: ""dest->suboffsets + (dest->ndim - 1)"" looks like the original copy.
Python-3.12.2/Objects/memoryobject.c:274:14: copy_paste_error: ""dest"" in ""src->suboffsets + (dest->ndim - 1)"" looks like a copy-paste error.
Python-3.12.2/Objects/memoryobject.c:274:14: remediation: Should it say ""src"" instead?
#  272|       assert(dest->ndim > 0 && src->ndim > 0);
#  273|       return (!HAVE_SUBOFFSETS_IN_LAST_DIM(dest) &&
#  274|->             !HAVE_SUBOFFSETS_IN_LAST_DIM(src) &&
#  275|               dest->strides[dest->ndim-1] == dest->itemsize &&
#  276|               src->strides[src->ndim-1] == src->itemsize);
2024-06-07 17:58:21 +02:00
Xi Ruoyao
a758424566
gh-120226: Fix test_sendfile_close_peer_in_the_middle_of_receiving on Linux >= 6.10 (#120227)
The worst case is that the kernel buffers 17 pages with a page size of 64k.
2024-06-07 08:51:32 -07:00
Nikita Sobolev
10fb1b8f36
gh-120200: Fix inspect.iscoroutinefunction(inspect) is True corner case (#120214) 2024-06-07 18:48:31 +03:00
Pieter Eendebak
9d6604222e
gh-114264: Optimize performance of copy.deepcopy by adding a fast path for atomic types (GH-114266) 2024-06-07 18:42:01 +03:00
Erlend E. Aasland
225aab7f70
gh-110383: Improve 'old string formatting' text in tutorial (#120219) 2024-06-07 15:37:18 +02:00
Irit Katriel
eca3f7762c
gh-93691: fix too broad source locations of with-statement instructions (#120125) 2024-06-07 14:06:24 +01:00
Serhiy Storchaka
d68a22e7a6
gh-120211: Fix tkinter.ttk with Tcl/Tk 9.0 (GH-120213)
* Use new methods for tracing Tcl variable.
* Fix Combobox.current() for empty combobox.
2024-06-07 10:49:07 +00:00
Aditya Borikar
6646a9da26
gh-110383: Clarify "non-integral" wording in pow() docs (#119688) 2024-06-07 11:44:42 +02:00
Kirill Podoprigora
d5ba4fc9bc
gh-120164: Fix test_os.test_win32_mkdir_700() (#120177)
Don't compare the path to avoid encoding issues.

Co-authored-by: Eryk Sun <eryksun@gmail.com>
2024-06-07 11:14:13 +02:00
Michael Allwright
47816f465e
gh-120154: Fix Emscripten/WASI pattern in case statement for LDSHARED (#120173)
Fix Emscripten/WASI pattern in case statement for LDSHARED
2024-06-07 10:38:15 +02:00
Xie Yanbo
6a97929a5a
Fix typos in comments (#120188) 2024-06-07 10:19:41 +02:00
Kirill Podoprigora
57ad769076
gh-120080: Accept `None as a valid argument for direct call of the int.__round__` (#120088)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2024-06-07 10:03:28 +02:00
Clinton
bd826b9c77
gh-120157: Remove unused code in concurrent.future (gh-120187) 2024-06-07 16:39:19 +09:00
Hugo van Kemenade
5c115567b1
Add Plausible for docs metrics (#119977)
Co-authored-by: Julien Palard <julien@palard.fr>
2024-06-07 00:38:31 -06:00
Jacob Walls
6b606522ca
gh-119577: Adjust DeprecationWarning when testing element truth values in ElementTree (GH-119762)
Adjust DeprecationWarning when testing element truth values in ElementTree, we're planning to go with the more natural True return rather than a disruptive harder to code around exception raise, and are deferring the behavior change for a few more releases.
2024-06-06 20:18:30 -07:00
Barney Gale
14e1506a6d
GH-119054: Add "Reading directories" section to pathlib docs (#119956)
Add a dedicated subsection for `Path.iterdir()`-related methods,
specifically `iterdir()`, `glob()`, `rglob()` and `walk()`.

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-06 23:27:39 +00:00
David Lowry-Duda
5bdc87b885
gh-120178: Documentation typo corrections (#120179) 2024-06-06 14:35:24 -07:00
Sam Gross
e21057b999
gh-117657: Fix TSAN race involving import lock (#118523)
This adds a `_PyRecursiveMutex` type based on `PyMutex` and uses that
for the import lock. This fixes some data races in the free-threaded
build and generally simplifies the import lock code.
2024-06-06 13:40:58 -04:00
Tian Gao
417bec733c
Add Tian Gao to CODEOWNERS and ACKS (GH-120166) 2024-06-06 10:20:37 -07:00
Serhiy Storchaka
2d7ff6e0e7
Restore decimal context after decimal doctests (GH-120149)
The modified context caused tests failures in several other tests.
2024-06-06 20:12:32 +03:00
Jelle Zijlstra
d50a7c478f
CODEOWNERS: Add myself to symtable and AST (#120139)
Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-06-06 16:25:05 +00:00
Victor Stinner
78634cfa3d
gh-120155: Initialize variables in _tkinter.c (#120156)
Initialize variables in _tkinter.c to make static analyzers happy.
2024-06-06 17:31:33 +02:00
Steve Dower
cccc9f63c6
gh-119679: Fix layout of PYD and DLL files on Windows when using PC/layout script (GH-120133) 2024-06-06 16:11:42 +01:00
Nice Zombies
fd104dfcb8
gh-120111: Don't use cirrus M1 macOS runners on fork (#120116)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2024-06-06 13:40:37 +02:00