79607 Commits

Author SHA1 Message Date
Kevin Newton
1be64e34d0 [ruby/yarp] Alnum cannot be %-literal delimiters
https://github.com/ruby/yarp/commit/4ba6d5ca70
2023-09-15 22:59:48 +00:00
Kevin Newton
18780c22f6 [ruby/yarp] Properly handle invalid underscores in number literals
https://github.com/ruby/yarp/commit/35da3d1a4c
2023-09-15 22:38:58 +00:00
Kevin Newton
b848700ccf [ruby/yarp] Extract test listener to its own class
https://github.com/ruby/yarp/commit/c8caa997c0
2023-09-15 22:36:22 +00:00
Vinicius Stock
35960ce65e [ruby/yarp] Add node event dispatcher
This commit changes the node template to create a dispatcher class,
which can be used to walk an AST an emit events to all registered
listeners

https://github.com/ruby/yarp/commit/03a45f85e6

Co-authored-by: Kevin Newton <kddnewton@users.noreply.github.com>
2023-09-15 22:36:22 +00:00
Jemma Issroff
89bd1ebcb9 [ruby/yarp] Fix unexpected { after method call with args
https://github.com/ruby/yarp/commit/fe279d2d16
2023-09-15 22:25:33 +00:00
Kevin Newton
c2f7dae317 [ruby/yarp] Handle unterminated interpolated symbol
https://github.com/ruby/yarp/commit/9222faa1c6
2023-09-15 22:24:50 +00:00
Kevin Newton
d2c75bb937 [ruby/yarp] Handle missing terminators in parenthesized expression
https://github.com/ruby/yarp/commit/a8b54e8ed0
2023-09-15 22:24:38 +00:00
Kevin Newton
ffe77c022c [ruby/yarp] Handle infinite opt terms after missing case predicate
https://github.com/ruby/yarp/commit/d931e258d1
2023-09-15 22:24:25 +00:00
Andy Waite
bbf9f11ce6 [ruby/yarp] Fix behaviour of locations for comments
https://github.com/ruby/yarp/commit/b1ced67fba
2023-09-15 22:22:24 +00:00
Martin Emde
010017d86d [rubygems/rubygems] Reduce array allocations and concatenations in Index
Remove the default nested hash in Index entirely
Index#search_all now yields or returns enum since that's what caller
needs.

https://github.com/rubygems/rubygems/commit/c45ea3bbe2
2023-09-15 22:01:11 +00:00
Samuel Giddins
2cf5fe58fb [rubygems/rubygems] Avoid allocating empty hashes in Index
Since the hashes have a default proc that returns a (new) empty hash, we
can avoid allocating those empty hashes when we are only doing lookups.

Test from running `bundle update --bundler` against a rails app I have
lying around:

```
==> memprof.after.txt <==
Total allocated: 9.71 MB (68282 objects)
Total retained:  4.87 MB (33791 objects)

==> memprof.before.txt <==
Total allocated: 10.83 MB (100596 objects)
Total retained:  5.02 MB (34721 objects)
```

https://github.com/rubygems/rubygems/commit/8f7c9cb23e
2023-09-15 22:01:10 +00:00
Tim Morgan
4e8869c663 [ruby/yarp] Use templated constants for Regexp options
https://github.com/ruby/yarp/commit/38e1769c2e
2023-09-15 21:47:28 +00:00
Tim Morgan
101ac364a5 [ruby/yarp] Return Regexp options that match MRI for e, u, s, and n
https://github.com/ruby/yarp/commit/17dbf4ec46
2023-09-15 21:47:27 +00:00
Alan Wu
0996cf5593 YJIT: Fix and enable the unused_imports warning 2023-09-15 16:15:15 -04:00
Jemma Issroff
4012ba4ecf [ruby/yarp] Fix string delimiter with "@" confused for embedded
variable
(https://github.com/ruby/yarp/pull/1521)

https://github.com/ruby/yarp/commit/3ec0699622
2023-09-15 18:02:04 +00:00
Burdette Lamar
e50b7bf784
[DOC] Globals doc (#8445) 2023-09-15 13:59:48 -04:00
Samuel Giddins
c423d6e0e4 [rubygems/rubygems] Stop allocating the same settings keys repeatedly
Running `bundle update --bundler` on a rails app locally:

```
==> memprof.after.txt <==
Total allocated: 301.90 kB (3794 objects)
Total retained:  73.24 kB (698 objects)

==> memprof.before.txt <==
Total allocated: 14.47 MB (196378 objects)
Total retained:  25.93 kB (202 objects)
```

So for a slight increase in retained memory (all keys are now retained),
we go from about 200k allocations in the settings file to under 4k

https://github.com/rubygems/rubygems/commit/e64debb6ae
2023-09-15 17:54:01 +00:00
Adam Hess
4d86d932fd Free all heap pages at shutdown
previously heap_allocated_pages was decremented from heap_page_free causing only half the heap pages to be freed at shutdown
2023-09-15 13:24:32 -04:00
Aaron Patterson
a1dc1a3de9 Return line 0 for JIT frames
Frames pushed by YJIT have an unreliable PC.  The PC could be garbage,
and if we try to read the line number with a garbage PC, then the
program can crash.

This commit returns line 0 for programs where there is a `jit_return`
function.  If `jit_return` has been set then this frame was pushed by
the JIT, and we cannot trust the PC.

Here is a debugger session for a program that crashed due to a broken
PC:

```
(lldb) p ruby_current_vm_ptr->ractor.main_thread->ec->cfp->iseq->body->iseq_encoded
(VALUE *) $0 = 0x0000000118a30e00
(lldb) p/x ruby_current_vm_ptr->ractor.main_thread->ec->cfp->pc
(const VALUE *) $1 = 0x0000600000b02d00
(lldb) p/x ruby_current_vm_ptr->ractor.main_thread->ec->cfp->jit_return
(void *) $2 = 0x000000010622942c
```

You can see the PC is completely out of range, but there is a
`jit_return` pointer so we can avoid this crash.
2023-09-15 09:01:02 -07:00
Jean Boussier
efe2822708 Process.warmup: invoke malloc_trim if available
Similar to releasing free GC pages, releasing free malloc pages
reduce the amount of page faults post fork.
2023-09-15 17:45:21 +02:00
Kevin Newton
28fe88468d [ruby/yarp] Add more tests for implicit hash values
https://github.com/ruby/yarp/commit/1ddd74dc11
2023-09-15 15:41:02 +00:00
Kevin Newton
5f9f2fd72e [ruby/yarp] Add implicit nodes for ommitted hash values
https://github.com/ruby/yarp/commit/22130b3491
2023-09-15 15:41:02 +00:00
Kevin Newton
5a6eae0b0a [ruby/yarp] Keep value for invalid local variable write in tree
https://github.com/ruby/yarp/commit/eaafc829db
2023-09-15 15:14:40 +00:00
Kevin Newton
fab25082c2 [ruby/yarp] Ensure owned constants are marked as shared when overtaken
https://github.com/ruby/yarp/commit/d73f7c9cfa
2023-09-15 15:14:38 +00:00
Kevin Newton
b5084877c0 [ruby/yarp] Disallow numbered parameters in multiple scopes
https://github.com/ruby/yarp/commit/5fd4d3b89a
2023-09-15 15:14:37 +00:00
Kevin Newton
a4b4ebc7c1 [ruby/yarp] Error when numbered parameters are written
https://github.com/ruby/yarp/commit/65b536ba12
2023-09-15 15:14:36 +00:00
Kevin Newton
6031ab18c7 [ruby/yarp] Support parsing numbered parameters
https://github.com/ruby/yarp/commit/ffc8f35e56
2023-09-15 15:14:34 +00:00
Kevin Newton
236fe914af [ruby/yarp] Track explicit parameters on blocks
https://github.com/ruby/yarp/commit/99c91931e0
2023-09-15 15:14:33 +00:00
Kevin Newton
1badb09f61 [ruby/yarp] Properly handle missing method names
https://github.com/ruby/yarp/commit/4a30c69051
2023-09-15 15:14:19 +00:00
Nobuyoshi Nakada
89802078f9
[Bug #19882] Reject tokens invalid as symbols 2023-09-15 17:27:36 +09:00
dependabot[bot]
5a7f5bb0de Bump actions/checkout from 3.6.0 to 4.0.0
Bumps [actions/checkout](https://github.com/actions/checkout) from 3.6.0 to 4.0.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](f43a0e5ff2...3df4ab11eb)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-15 17:05:23 +09:00
Nobuyoshi Nakada
fe0225ff4d [Bug #19778] Add -I options for opt-dir to $INCFLAGS
These options have been separated from `$CFLAGS` already in the
other places.
2023-09-15 13:27:19 +09:00
dependabot[bot]
74277f9998 Bump ruby/setup-ruby from 1.152.0 to 1.153.0
Bumps [ruby/setup-ruby](https://github.com/ruby/setup-ruby) from 1.152.0 to 1.153.0.
- [Release notes](https://github.com/ruby/setup-ruby/releases)
- [Commits](250fcd6a74...5311f05890)

---
updated-dependencies:
- dependency-name: ruby/setup-ruby
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-15 12:20:35 +09:00
KJ Tsanaktsidis
0117a6d389
Fix Thread#native_thread_id being cached across fork (#8418)
The native thread ID can and does change on some operating systems (e.g.
Linux) after forking, so it needs to be re-queried.

[Bug #19873]
2023-09-15 10:33:32 +09:00
Aaron Patterson
f08cac066e Don't call malloc with 0
It seems not-uncommon for methods to have no IV, ISE, or ICVARC caches.
Calling malloc with 0 will actually allocate something, so if there
aren't any caches (`ISEQ_IS_SIZE(body) == 0`), then we can avoid
allocating memory by not calling malloc.  If there are no caches, then
theoretically nobody should be reading from the buffer anyway.

This saves about 1MB on Lobsters benchmark.
2023-09-14 16:15:53 -07:00
Takashi Kokubun
982d6503b9
YJIT: Skip Insn::Comment and format! if disasm is disabled (#8441)
* YJIT: Skip Insn::Comment and format!

if disasm is disabled

Co-authored-by: Alan Wu <alansi.xingwu@shopify.com>

* YJIT: Get rid of asm.comment

---------

Co-authored-by: Alan Wu <alansi.xingwu@shopify.com>
2023-09-14 15:49:40 -07:00
Alan Wu
0ba6c603bc YJIT: Remove UTF-8 BOM [ci skip]
/yjit/src/backend/x86_64/mod.rs Is also UTF-8 and it doesn't have the
marker. The standard recommends against it, so remove it.
2023-09-14 18:40:45 -04:00
Takashi Kokubun
07b615bb3b Skip renaming OpenSSL DLLs
to fix CI failures like:
https://github.com/ruby/ruby/actions/runs/6190995187/job/16808382442
2023-09-14 15:02:12 -07:00
Alan Wu
1961c5bb76
YJIT: Plug native stack overflow
Previously, TestStack#test_machine_stack_size failed pretty consistently
on ARM64 macOS, with Rust code and part of the interpreter used for
per-instruction fallback (rb_vm_invokeblock() and friends) touching the
stack guard page and crashing with SEGV. I've also seen the same test
fail on x64 Linux, though with a different symptom.
2023-09-14 17:18:45 -04:00
Martin Emde
66ffa15ce0 [rubygems/rubygems] Gemfile ruby file: covers more version formats
Increase test coverage and be explicit about what is and is not supported.

https://github.com/rubygems/rubygems/commit/a096397a00
2023-09-14 19:43:21 +00:00
Kevin Newton
0d33bc0cde [ruby/yarp] Simplify multi-target parsing
This simplifies how we handle multi-targets, and also fixes a bug we
had where for loops were always getting multi-targets, even when there
was only a single target.

https://github.com/ruby/yarp/commit/31eb8b7ad5
2023-09-14 19:16:39 +00:00
Kevin Newton
0a8f3670d1 [ruby/yarp] Introduce MatchWriteNode
This rarely used node holds information about the local variables
that need to get written in the case a regular expression is used on
the left-hand side of a =~ operator and it has named capture groups.

Note that we already "handled" these nodes by adding locals to the
AST, but we didn't actually expose this information, making it
difficult to compile.

The general idea behind this node is that it maintains the ability
for consumers to find all of the call nodes in the tree easily so
it's not flattening down. However, it should be okay because you
hopefully don't need any information in the call node to determine
what to compile because the locals list is on the top level.

https://github.com/ruby/yarp/commit/e136e7f9a8
2023-09-14 19:10:58 +00:00
Kevin Newton
9d2549ac31 [ruby/yarp] Write names should not underflow size_t
If the read_name is invalid, we shouldn't try to set a write name.

https://github.com/ruby/yarp/commit/06881c8ca7
2023-09-14 19:10:47 +00:00
Kevin Newton
63d1e05665 [ruby/yarp] Handle errors when operator writes on a call with a block
https://github.com/ruby/yarp/commit/93bec2c173
2023-09-14 18:32:16 +00:00
Kevin Newton
fb1328e467 [ruby/yarp] Fix multi target parentheses locations
https://github.com/ruby/yarp/commit/7f71527522
2023-09-14 18:31:55 +00:00
Kevin Newton
b098c4247b [ruby/yarp] Remove deprecated aliases
https://github.com/ruby/yarp/commit/db28e22363
2023-09-14 18:31:43 +00:00
Kevin Newton
de2fe8631c [ruby/yarp] Extract out parse_write_name
https://github.com/ruby/yarp/commit/0d3e8cdcd7
2023-09-14 18:30:11 +00:00
Kevin Newton
826bebb7d8 [ruby/yarp] Reject invalid call-operator-write
https://github.com/ruby/yarp/commit/d3a852dac2
2023-09-14 18:30:11 +00:00
Kevin Newton
7f6cf2d283 [ruby/yarp] Use bytes instead of bit fields
https://github.com/ruby/yarp/commit/890fa72fdd
2023-09-14 18:28:46 +00:00
Kevin Newton
581bda0cba [ruby/yarp] Add a frozen string flag
https://github.com/ruby/yarp/commit/09248a1f9e
2023-09-14 18:28:45 +00:00