107 Commits

Author SHA1 Message Date
Mark Shannon
a4a9f2e879
GH-96177: Move GIL and eval breaker code out of ceval.c into ceval_gil.c. (GH-96204) 2022-08-24 14:21:01 +01:00
Mark Shannon
3ef3c6306d
GH-95707: Fix uses of Py_TPFLAGS_MANAGED_DICT (GH-95854)
* Make sure that tp_dictoffset is correct with Py_TPFLAGS_MANAGED_DICT is set.

* Avoid traversing managed dict twice when subclassing class with Py_TPFLAGS_MANAGED_DICT set.
2022-08-15 12:29:27 +01:00
Mark Shannon
de388c0a7b
GH-95245: Store object values and dict pointers in single tagged pointer. (GH-95278) 2022-08-01 14:34:54 +01:00
Inada Naoki
f9c9354a7a
gh-92536: PEP 623: Remove wstr and legacy APIs from Unicode (GH-92537) 2022-05-12 14:48:38 +09:00
Mark Shannon
944fffee89
GH-88116: Use a compact format to represent end line and column offsets. (GH-91666)
* Stores all location info in linetable to conform to PEP 626.

* Remove column table from code objects.

* Remove end-line table from code objects.

* Document new location table format
2022-04-21 16:10:37 +01:00
Yu Liu
9300b6d729
gh-91595: fix the comparison of character and integer by using ord() (#91596)
* fix the comparison of character and integer by using ord()

* 📜🤖 Added by blurb_it.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2022-04-16 16:34:48 +02:00
Brandt Bucher
ef6a482b02
bpo-47177: Replace f_lasti with prev_instr (GH-32208) 2022-04-07 12:31:01 -07:00
Inada Naoki
52f6ce396d
Tools/gdb: Drop code to support Python 2. (GH-31717) 2022-03-15 17:04:11 +09:00
Inada Naoki
9833bb91e4
bpo-46845: Reduce dict size when all keys are Unicode (GH-31564) 2022-03-02 08:09:28 +09:00
Brandt Bucher
332e6b9725
bpo-45256: Don't track the exact depth of each InterpreterFrame (GH-30372) 2022-01-05 11:30:26 +00:00
Raymond Hettinger
29ea68bd1d
Revert "bpo-46131: add fastpath for PyFloat_Check() (GH-30200)" (GH-30208)
This reverts commit 2ef06d412531d1163dbc72877c88aedf3ed82a25.
2021-12-19 15:05:46 -06:00
Matti Picus
2ef06d4125
bpo-46131: add fastpath for PyFloat_Check() (#30200) 2021-12-19 14:24:30 -06:00
Mark Shannon
8319114fee
bpo-45947: Place dict and values pointer at fixed (negative) offset just before GC header. (GH-29879)
* Place __dict__ immediately before GC header for plain Python objects.

* Fix up lazy dict creation logic to use managed dict pointers.

* Manage values pointer, placing them directly before managed dict pointers.

* Convert hint-based load/store attr specialization target managed dict classes.

* Specialize LOAD_METHOD for managed dict objects.

* Remove unsafe _PyObject_GC_Calloc function.

* Remove unsafe _PyObject_GC_Malloc() function.

* Add comment explaning use of Py_TPFLAGS_MANAGED_DICT.
2021-12-07 16:02:53 +00:00
Pablo Galindo Salgado
4cf65240ae
Reactivate primary mechanism to retrieve frames in the gdb helpers (GH-29682) 2021-11-21 02:06:16 +00:00
Pablo Galindo Salgado
20205ad2b5
bpo-45637: Fix cframe-based fallback in the gdb helpers (GH-29515) 2021-11-10 13:41:22 +00:00
Pablo Galindo Salgado
f4c03484da
bpo-45637: Remove broken fallback in gdb helpers to obtain frame variable (GH-29257) 2021-11-09 11:19:47 +00:00
Mark Shannon
a8b9350964
bpo-45340: Don't create object dictionaries unless actually needed (GH-28802)
* Never change types' cached keys. It could invalidate inline attribute objects.

* Lazily create object dictionaries.

* Update specialization of LOAD/STORE_ATTR.

* Don't update shared keys version for deletion of value.

* Update gdb support to handle instance values.

* Rename SPLIT_KEYS opcodes to INSTANCE_VALUE.
2021-10-13 14:19:34 +01:00
Pablo Galindo Salgado
b4903afd4d
bpo-45256: Remove the usage of the C stack in Python to Python calls (GH-28488)
Ths commit inlines calls to Python functions in the eval loop and steals all the arguments in the call from the caller for
performance.
2021-10-09 16:51:30 +01:00
Mark Shannon
a7252f88d3
bpo-40116: Add insertion order bit-vector to dict values to allow dicts to share keys more freely. (GH-28520) 2021-10-06 13:19:53 +01:00
Mark Shannon
f9242d50b1
bpo-44990: Change layout of evaluation frames. "Layout B" (GH-27933)
Places the locals between the specials and stack. This is the more "natural" layout for a C struct, makes the code simpler and gives a slight speedup (~1%)
2021-08-25 13:44:20 +01:00
Mark Shannon
ae0a2b7562
bpo-44590: Lazily allocate frame objects (GH-27077)
* Convert "specials" array to InterpreterFrame struct, adding f_lasti, f_state and other non-debug FrameObject fields to it.

* Refactor, calls pushing the call to the interpreter upward toward _PyEval_Vector.

* Compute f_back when on thread stack, only filling in value when frame object outlives stack invocation.

* Move ownership of InterpreterFrame in generator from frame object to generator object.

* Do not create frame objects for Python calls.

* Do not create frame objects for generators.
2021-07-26 11:22:16 +01:00
Mark Shannon
0982ded179
bpo-44032: Move pointer to code object from frame-object to frame specials array. (GH-26771) 2021-06-18 11:00:29 +01:00
Eric Snow
2ab27c4af4
bpo-43693: Un-revert commits 2c1e258 and b2bf2bc. (gh-26577)
These were reverted in gh-26530 (commit 17c4edc) due to refleaks.

* 2c1e258 - Compute deref offsets in compiler (gh-25152)
* b2bf2bc - Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. (gh-26388)

This change fixes the refleaks.

https://bugs.python.org/issue43693
2021-06-07 12:22:26 -06:00
Pablo Galindo
17c4edc4e0
bpo-43693: Revert commits 2c1e2583fdc4db6b43d163239ea42b0e8394171f and b2bf2bc1ece673d387341e06c8d3c2bc6e259747 (GH-26530)
* Revert "bpo-43693: Compute deref offsets in compiler (gh-25152)"

This reverts commit b2bf2bc1ece673d387341e06c8d3c2bc6e259747.

* Revert "bpo-43693: Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. (gh-26388)"

This reverts commit 2c1e2583fdc4db6b43d163239ea42b0e8394171f.

These two commits are breaking the refleak buildbots.
2021-06-04 17:51:05 +01:00
Eric Snow
2c1e2583fd
bpo-43693: Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. (gh-26388)
A number of places in the code base (notably ceval.c and frameobject.c) rely on mapping variable names to indices in the frame "locals plus" array (AKA fast locals), and thus opargs.  Currently the compiler indirectly encodes that information on the code object as the tuples co_varnames, co_cellvars, and co_freevars.  At runtime the dependent code must calculate the proper mapping from those, which isn't ideal and impacts performance-sensitive sections.  This is something we can easily address in the compiler instead.

This change addresses the situation by replacing internal use of co_varnames, etc. with a single combined tuple of names in locals-plus order, along with a minimal array mapping each to its kind (local vs. cell vs. free).  These two new PyCodeObject fields, co_fastlocalnames and co_fastllocalkinds, are not exposed to Python code for now, but co_varnames, etc. are still available with the same values as before (though computed lazily).

Aside from the (mild) performance impact, there are a number of other benefits:

* there's now a clear, direct relationship between locals-plus and variables
* code that relies on the locals-plus-to-name mapping is simpler
* marshaled code objects are smaller and serialize/de-serialize faster

Also note that we can take this approach further by expanding the possible values in co_fastlocalkinds to include specific argument types (e.g. positional-only, kwargs).  Doing so would allow further speed-ups in _PyEval_MakeFrameVector(), which is where args get unpacked into the locals-plus array.  It would also allow us to shrink marshaled code objects even further.

https://bugs.python.org/issue43693
2021-06-03 10:28:27 -06:00
Mark Shannon
f8a95df84b
bpo-44206: Add a version number to dictionary keys (GH-26333)
* Store log2(size) instead of size in dict-keys.

* Use enum instead of function pointer to record kind of keys.

* Add version number to dict keys.
2021-05-28 09:54:10 +01:00
Mark Shannon
b11a951f16
bpo-44032: Move data stack to thread from FrameObject. (GH-26076)
* Remove 'zombie' frames. We won't need them once we are allocating fixed-size frames.

* Add co_nlocalplus field to code object to avoid recomputing size of locals + frees + cells.

* Move locals, cells and freevars out of frame object into separate memory buffer.

* Use per-threadstate allocated memory chunks for local variables.

* Move globals and builtins from frame object to per-thread stack.

* Move (slow) locals frame object to per-thread stack.

* Move internal frame functions to internal header.
2021-05-21 10:57:35 +01:00
Mark Shannon
fcb55c0037
bpo-27129: Use instruction offsets, not byte offsets, in bytecode and internally. (GH-25069)
* Use instruction offset, rather than bytecode offset. Streamlines interpreter dispatch a bit, and removes most EXTENDED_ARGs for jumps.

* Change some uses of PyCode_Addr2Line to PyFrame_GetLineNumber
2021-04-01 16:00:31 +01:00
Augusto Hack
b57ada98da
closes bpo-42726: gdb libpython: InstanceProxy support for py3 (GH-23912)
On Fedora 31 gdb is using python 3.7.9, calling `proxyval` on an instance with a dictionary fails because of the `dict.iteritems` usage. This PR changes the code to be compatible with py2 and py3.

This changed seemed small enough to not need an issue and news blurb, if one is required please let me know.

Automerge-Triggered-By: GH:benjaminp
2020-12-24 09:16:04 -08:00
Mark Shannon
877df851c3
bpo-42246: Partial implementation of PEP 626. (GH-23113)
* Implement new line number table format, as defined in PEP 626.
2020-11-12 09:43:29 +00:00
Victor Stinner
b9ee4af4c6
bpo-42208: Fix test_gdb for gc_collect_main() name (GH-23041)
The gcmodule.c collect() function was renamed to gc_collect_main():
update gdb/libpython.py (python-gdb.py).
2020-10-30 21:09:48 +01:00
Victor Stinner
7bf069b611
bpo-40019: Skip test_gdb if Python was optimized (GH-19081)
test_gdb now skips tests if it detects that gdb failed to read debug
information because the Python binary is optimized.
2020-03-20 08:23:26 +01:00
Victor Stinner
6d0ee60740
bpo-36184: Port python-gdb.py to FreeBSD (GH-18873)
python-gdb.py now checks for "take_gil" function name to check if a
frame tries to acquire the GIL, instead of checking for
"pthread_cond_timedwait" which is specific to Linux and can be a
different condition than the GIL.
2020-03-09 19:35:26 +01:00
Marc Hartmayer
6f53d34fb0 closes bpo-16637: libpython: construct integer object directly from gdbvalue (GH-15232)
This fixes the exception '`ValueError: invalid literal for int() with base 10`
if `str(gdbval)` returns a hexadecimal value (e.g. '0xa0'). This is the case if
the output-radix is set to 16 in gdb. See
https://sourceware.org/gdb/onlinedocs/gdb/Numbers.html for more information.
2019-09-23 20:34:12 -07:00
Jeroen Demeyer
7a6873cdb1 bpo-37151: remove special case for PyCFunction from PyObject_Call (GH-14684)
bpo-37151: remove special case for PyCFunction from PyObject_Call

Alse, make the undocumented function PyCFunction_Call an alias
of PyObject_Call and deprecate it.
2019-09-11 12:01:01 +01:00
Jeroen Demeyer
0d722f3cd6 bpo-36974: separate vectorcall functions for each calling convention (GH-13781) 2019-07-05 14:48:24 +02:00
Jeroen Demeyer
7e1a9aacff bpo-37151: remove _PyCFunction_FastCallDict (GH-14269) 2019-06-21 00:38:45 +09:00
Jeroen Demeyer
37788bc23f bpo-36974: rename _FastCallKeywords -> _Vectorcall (GH-13653) 2019-05-30 15:11:22 +02:00
Petr Viktorin
fecb75c1bb
bpo-36974: Fix GDB integration (GH-13665)
As it changes the way functions are called, the PEP 590 implementation
skipped the functions that the GDB integration is looking for
(by name) to find function calls.

Looking for the new helper `cfunction_call_varargs` hopefully fixes the
tests, and thus buildbots.

The changed frame nuber in test_gdb is due to there being fewer
C calls when calling a built-in method.
2019-05-29 22:45:41 +02:00
Lisa Roach
1ceb3a3d17
bpo-35132: Fixes missing target in gdb pep0393 check. (GH-11848) 2019-03-11 20:21:25 -07:00
Victor Stinner
2e438cc255
bpo-34989: python-gdb.py: fix current_line_num() (GH-9889)
python-gdb.py now handles errors on computing the line number
of a Python frame.

Changes:

* PyFrameObjectPtr.current_line_num() now catchs any Exception on
  calling addr2line(), instead of failing with a surprising "<class
  'TypeError'> 'FakeRepr' object is not subscriptable" error.
* All callers of current_line_num() now handle current_line_num()
  returning None.
* PyFrameObjectPtr.current_line() now also catchs IndexError on
  getting a line from the Python source file.
2018-10-15 23:19:57 +02:00
Victor Stinner
d22fc0bc7d
bpo-32962: python-gdb catchs UnicodeDecodeError (GH-7693)
python-gdb now catchs UnicodeDecodeError exceptions when calling
string().
2018-06-14 22:34:52 +02:00
Victor Stinner
019d33b7a4
bpo-32962: python-gdb catchs ValueError on read_var() (GH-7692)
python-gdb now catchs ValueError on read_var(): when Python has no
debug symbols for example.
2018-06-14 16:28:07 +02:00
Gregory P. Smith
53f67d401d
bpo-33312: update Tools/gdb/libpython.py to match. (GH-6548) 2018-04-20 11:32:08 -07:00
Łukasz Langa
5fe59f8e3a bpo-30983: [gdb] Fix py-bt, etc. for non-debug shared builds (#3153)
PEP 523 introduced _PyEval_EvalFrameDefault which inlines PyEval_EvalFrameEx on
non-debug shared builds.  This breaks the ability to use py-bt, py-up, and
a few other Python-specific gdb integrations.

This patch fixes the problem by only looking for _PyEval_EvalFrameDefault
frames.

test_gdb passes on both a debug and a non-debug build.

Original patch by Bruno "Polaco" Penteado.
2017-08-21 16:40:29 -07:00
Victor Stinner
7cc33998b8 bpo-30983: Revert changes which broke most buildbots (#3100)
* Revert "Add Bruno Penteado to ACKS (#3091)"

This reverts commit f978405b3f092e4005b92ba1dbaab15f609b3bb0.

* Revert "bpo-30983: eval frame rename in pep 0523 broke gdb's python extension (#2803)"

This reverts commit 2e0f4db114424a00354eab889ba8f7334a2ab8f0.
2017-08-16 11:02:05 +02:00
Bruno "Polaco" Penteado
2e0f4db114 bpo-30983: eval frame rename in pep 0523 broke gdb's python extension (#2803)
pep 0523 renames PyEval_EvalFrameEx to _PyEval_EvalFrameDefault while the gdb python extension only looks for PyEval_EvalFrameEx to understand if it is dealing with a frame.

Final effect is that attaching gdb to a python3.6 process doesnt resolve python objects. Eg. py-list and py-bt dont work properly.

This patch fixes that. Tested locally on python3.6
2017-08-14 15:14:17 -07:00
Jon Dufresne
3972628de3 bpo-30296 Remove unnecessary tuples, lists, sets, and dicts (#1489)
* Replaced list(<generator expression>) with list comprehension
* Replaced dict(<generator expression>) with dict comprehension
* Replaced set(<list literal>) with set literal
* Replaced builtin func(<list comprehension>) with func(<generator
  expression>) when supported (e.g. any(), all(), tuple(), min(), &
  max())
2017-05-18 07:35:54 -07:00
Lev Abalkin
661ca8843f Fixes bpo-29680: Older gdb does not have gdb.error. (#363)
This change is required to make python-dbg.py compatible with GDB versions before 7.3.
2017-03-01 13:16:23 -05:00
INADA Naoki
5566bbb8d5 Issue #29263: LOAD_METHOD support for C methods
Calling builtin method is at most 10% faster.
2017-02-03 07:43:03 +09:00