83249 Commits

Author SHA1 Message Date
Jeremy Evans
0f90a24a81 Introduce Allocationless Anonymous Splat Forwarding
Ruby makes it easy to delegate all arguments from one method to another:

```ruby
def f(*args, **kw)
  g(*args, **kw)
end
```

Unfortunately, this indirection decreases performance.  One reason it
decreases performance is that this allocates an array and a hash per
call to `f`, even if `args` and `kw` are not modified.

Due to Ruby's ability to modify almost anything at runtime, it's
difficult to avoid the array allocation in the general case. For
example, it's not safe to avoid the allocation in a case like this:

```ruby
def f(*args, **kw)
  foo(bar)
  g(*args, **kw)
end
```

Because `foo` may be `eval` and `bar` may be a string referencing `args`
or `kw`.

To fix this correctly, you need to perform something similar to escape
analysis on the variables.  However, there is a case where you can
avoid the allocation without doing escape analysis, and that is when
the splat variables are anonymous:

```ruby
def f(*, **)
  g(*, **)
end
```

When splat variables are anonymous, it is not possible to reference
them directly, it is only possible to use them as splats to other
methods.  Since that is the case, if `f` is called with a regular
splat and a keyword splat, it can pass the arguments directly to
`g` without copying them, avoiding allocation.  For example:

```ruby
def g(a, b:)
  a + b
end

def f(*, **)
  g(*, **)
end

a = [1]
kw = {b: 2}

f(*a, **kw)
```

I call this technique: Allocationless Anonymous Splat Forwarding.

This is implemented using a couple additional iseq param flags,
anon_rest and anon_kwrest.  If anon_rest is set, and an array splat
is passed when calling the method when the array splat can be used
without modification, `setup_parameters_complex` does not duplicate
it.  Similarly, if anon_kwest is set, and a keyword splat is passed
when calling the method, `setup_parameters_complex` does not
duplicate it.
2024-01-24 18:25:55 -08:00
Jeremy Evans
b8516d6d01 Add pushtoarray VM instruction
This instruction is similar to concattoarray, but it takes the
number of arguments to push to the array, removes that number
of arguments from the stack, and adds them to the array now at
the top of the stack.

This allows `f(*a, 1)` to allocate only a single array on the
caller side (which can be reused on the callee side in the case of
`def f(*a)`). Prior to this commit, `f(*a, 1)` would generate
3 arrays:

* a dupped by splatarray true
* 1 wrapped in array by newarray
* a dupped again by concatarray

Instructions Before for `a = []; f(*a, 1)`:

```
0000 newarray                               0                         (   1)[Li]
0002 setlocal_WC_0                          a@0
0004 putself
0005 getlocal_WC_0                          a@0
0007 splatarray                             true
0009 putobject_INT2FIX_1_
0010 newarray                               1
0012 concatarray
0013 opt_send_without_block                 <calldata!mid:f, argc:1, ARGS_SPLAT|FCALL>
0015 leave
```

Instructions After for `a = []; f(*a, 1)`:

```
0000 newarray                               0                         (   1)[Li]
0002 setlocal_WC_0                          a@0
0004 putself
0005 getlocal_WC_0                          a@0
0007 splatarray                             true
0009 putobject_INT2FIX_1_
0010 pushtoarray                            1
0012 opt_send_without_block                 <calldata!mid:f, argc:1, ARGS_SPLAT|ARGS_SPLAT_MUT|FCALL>
0014 leave
```

With these changes, method calls to Ruby methods should
implicitly allocate at most one array.

Ignore typeprof bundled gem failure due to unrecognized instruction.
2024-01-24 18:25:55 -08:00
Jeremy Evans
6e06d0d180 Add concattoarray VM instruction
This instruction is similar to concatarray, but assumes the first
object is already an array, and appends to it directly.  This is
different than concatarray, which will create a new array instead
of appending to an existing array.

Additionally, for both concatarray and concattoarray, if the second
argument cannot be converted to an array, then just push it onto
the array, instead of creating a new array to wrap it, and then
using concat array.  This saves an array allocation in that case.

This allows `f(*a, *a, *1)` to allocate only a single array on the
caller side (which can be reused on the callee side in the case of
`def f(*a)`). Prior to this commit, `f(*a, *a, *1)` would generate
4 arrays:

* a dupped by splatarray true
* a dupped again by first concatarray
* 1 wrapped in array by third splatarray
* result of [*a, *a] dupped by second concatarray

Instructions Before for `a = []; f(*a, *a, *1)`:

```
0000 newarray                               0                         (   1)[Li]
0002 setlocal_WC_0                          a@0
0004 putself
0005 getlocal_WC_0                          a@0
0007 splatarray                             true
0009 getlocal_WC_0                          a@0
0011 splatarray                             false
0013 concatarray
0014 putobject_INT2FIX_1_
0015 splatarray                             false
0017 concatarray
0018 opt_send_without_block                 <calldata!mid:g, argc:1, ARGS_SPLAT|ARGS_SPLAT_MUT|FCALL>
0020 leave
```

Instructions After for `a = []; f(*a, *a, *1)`:

```
0000 newarray                               0                         (   1)[Li]
0002 setlocal_WC_0                          a@0
0004 putself
0005 getlocal_WC_0                          a@0
0007 splatarray                             true
0009 getlocal_WC_0                          a@0
0011 concattoarray
0012 putobject_INT2FIX_1_
0013 concattoarray
0014 opt_send_without_block                 <calldata!mid:f, argc:1, ARGS_SPLAT|ARGS_SPLAT_MUT|FCALL>
0016 leave
```
2024-01-24 18:25:55 -08:00
Jeremy Evans
22e488464a Add VM_CALL_ARGS_SPLAT_MUT callinfo flag
This flag is set when the caller has already created a new array to
handle a splat, such as for `f(*a, b)` and `f(*a, *b)`.  Previously,
if `f` was defined as `def f(*a)`, these calls would create an extra
array on the callee side, instead of using the new array created
by the caller.

This modifies `setup_args_core` to set the flag whenver it would add
a `splatarray true` instruction.  However, when `splatarray true` is
changed to `splatarray false` in the peephole optimizer, to avoid
unnecessary allocations on the caller side, the flag must be removed.
Add `optimize_args_splat_no_copy` and have the peephole optimizer call
that.  This significantly simplifies the related peephole optimizer
code.

On the callee side, in `setup_parameters_complex`, set
`args->rest_dupped` to true if the flag is set.

This takes a similar approach for optimizing regular splats that was
previiously used for keyword splats in
d2c41b1bff1f3102544bb0d03d4e82356d034d33 (via VM_CALL_KW_SPLAT_MUT).
2024-01-24 18:25:55 -08:00
KJ Tsanaktsidis
ef276858d9 Trigger postponed jobs on running_ec if that is available
Currently, any postponed job triggered from a non-ruby thread gets sent
to the main thread, but if the main thread is sleeping it won't be
checking ints. Instead, we should try and interrupt running_ec if that's
possible, and only fall back to the main thread if it's not.

[Bug #20197]
2024-01-25 13:10:35 +11:00
Hiroshi SHIBATA
2e18228dff Use echo with all platforms 2024-01-25 10:51:18 +09:00
Hiroshi SHIBATA
52085b66d6 Use echo command in mswin platform 2024-01-25 10:51:18 +09:00
Takashi Kokubun
34c688b163
Omit TestCompilePrism on s390x for now (#9694) 2024-01-24 16:03:05 -08:00
Alan Wu
2cc7a56ec7
YJIT: Avoid leaks by skipping objects with a singleton class
For receiver with a singleton class, there are multiple vectors YJIT can
end up retaining the object. There is a path in jit_guard_known_klass()
that bakes the receiver into the code, and the object could also be kept
alive indirectly through a path starting at the CME object baked into
the code.

To avoid these leaks, avoid compiling calls on objects with a singleton
class.

See: https://github.com/Shopify/ruby/issues/552

[Bug #20209]
2024-01-24 18:06:58 -05:00
Peter Zhu
f769d68a69 [PRISM] Fix getblockparamproxy for forwarding arguments 2024-01-24 17:32:43 -05:00
Max Prokopiev
a65b52b57f [ruby/prism] Add missing snapshot for the new file
https://github.com/ruby/prism/commit/459a9f544e
2024-01-24 20:32:02 +00:00
Max Prokopiev
e9f936df71 [ruby/prism] Extract snippet into its own file and skip in TruffleRuby
https://github.com/ruby/prism/commit/12649ee74c
2024-01-24 20:32:02 +00:00
Max Prokopiev
91a5093c15 [ruby/prism] Force encoding during deserialization of constants
otherwise we get failing tests if we have non-ascii
characters in fixtures/**/*.txt

https://github.com/ruby/prism/commit/9323243569
2024-01-24 20:32:01 +00:00
Max Prokopiev
6a63aaade9 [ruby/prism] Use char_is_identifier_start() to check for valid method name
alnum_char() only checks for alphanumeric characters while
ignoring other valid cases (like emoji codepoints for example)

https://github.com/ruby/prism/commit/d15958fd28
2024-01-24 20:32:01 +00:00
Peter Zhu
7d61454f2e [PRISM] Fix method calls in keyword arguments
Fixes ruby/prism#2248.
2024-01-24 15:18:06 -05:00
Aaron Patterson
94f3f9502c Fix repeated block param
Co-Authored-By: Matt Valentine-House <matt@eightbitraptor.com>
2024-01-24 12:04:44 -08:00
Aaron Patterson
1817af5074 Fix local table size / index for repeated kwrest
Co-Authored-By: Matt Valentine-House <matt@eightbitraptor.com>
2024-01-24 12:04:44 -08:00
Aaron Patterson
13c76e40d3 Fix local table size and variable offset for repeated keywords
Co-Authored-By: Matt Valentine-House <matt@eightbitraptor.com>
2024-01-24 12:04:44 -08:00
Aaron Patterson
8b7e78f156 Fix local table size and index for required post underscore
Co-Authored-By: Matt Valentine-House <matt@eightbitraptor.com>
2024-01-24 12:04:44 -08:00
Aaron Patterson
bb6af9287b Fix local table space for *_
We need to make sure there is enough room in the local table for
repeated `*_` parameters

Co-Authored-By: Matt Valentine-House <matt@eightbitraptor.com>
2024-01-24 12:04:44 -08:00
Aaron Patterson
03f76f098a Fix repeated optional _ parameters
Ensure there is enough space in the local table for repeated optional
parameters.

Co-Authored-By: Matt Valentine-House <matt@eightbitraptor.com>
2024-01-24 12:04:44 -08:00
Aaron Patterson
29c3ec3d49 Fix required positional repeated _ parameters
Co-Authored-By: Matt Valentine-House <matt@eightbitraptor.com>
2024-01-24 12:04:44 -08:00
Kevin Newton
44c337a397 [PRISM] Support __LINE__, __FILE__, and __ENCODING__ in patterns, fix pinned expressions 2024-01-24 15:02:32 -05:00
Kevin Newton
c888b8d63b [PRISM] Handle implicit lvar write in hash pattern 2024-01-24 14:18:33 -05:00
Peter Zhu
020fe6e0ca [PRISM] Fix forwarding from within block
Fixes ruby/prism#2253.
2024-01-24 14:18:25 -05:00
Peter Zhu
dc8fb7d97f [PRISM] Fix crash in anonymous block with forwarding arguments
Fixes ruby/prism#2262.
2024-01-24 13:42:21 -05:00
Kevin Newton
455fb32038 [ruby/prism] Add an implicit node for the target of a hash pattern
This simplifies compiling it, since you can now compile the value
as if it were always there.

https://github.com/ruby/prism/commit/bcfc74aacb
2024-01-24 18:37:36 +00:00
Takashi Kokubun
303fef8752
YJIT: Update yjit.md about mem size (#9687) 2024-01-24 10:26:57 -08:00
Maxime Chevalier-Boisvert
1702528258
YJIT: reduce default exec mem size to 48MiB (#9685)
YJIT: reduce default exec mem size to 48MiB based

Based on user feedback from @jhawthorn and others.
Better for small and memory-constrained deployments.

NOTE: This commit should be included in the next Ruby 3.3.x point
release. @xrxr should we tag someone specific?
2024-01-24 18:14:36 +00:00
Peter Zhu
6b9317c9d5 [PRISM] Don't skip test_ForwardingArgumentsNode
It seems to work with RUBY_ISEQ_DUMP_DEBUG=to_binary so we can try this
test again on CI.
2024-01-24 13:11:41 -05:00
Maxime Chevalier-Boisvert
23d4682926
YJIT: fix small bug causing jit_rb_int_rshift to side-exit (#9684)
YJIT: fix bug causing jit_rb_int_rshift to side-exit

The nqueens benchmark was showing zero performance improvement
because we immediately side-exited as soon as the shift amount
changed. If the amount changes, we want to fall back to the
C function call, not side-exit.
2024-01-24 13:09:08 -05:00
Aaron Patterson
578ff32611 Move filling in the rest of the locals to the end 2024-01-24 09:40:30 -08:00
Aaron Patterson
7a33a1aee2 Insert all locals in the locals index table
Prism provides an index (local_body_index) which is supposed to point at
the start of locals declared in the method body. Prism assumed that
method body locals would only occur _after_ parameter names.
Unfortunately this assumption is not correct, which meant that we would
in some cases not insert all locals in the local table.  This commit
iterates over locals a second time, inserting any that didn't get
inserted on the first pass.

Fixes: https://github.com/ruby/prism/issues/2245

Co-Authored-By: Matt Valentine-House <matt@eightbitraptor.com>
2024-01-24 09:40:30 -08:00
Peter Zhu
927928badb [PRISM] Fix anonymous keyword arguments
Fixes ruby/prism#2256.
2024-01-24 11:41:08 -05:00
Kevin Newton
4220bdba39 [PRISM] Remove dead code from prism_compile.c
Previously, we were using the main pm_compile_node switch to compile
target nodes. Eventually we switched to pm_compile_target_node and
updated all the call sites. As such, all of this has become dead
code.

However, the multi target code was reused for both parameters and
multi writes (it was just incorrect for multi writes). So that code
has now been moved into its own function that is more specific to
destructured parameters (and has more assertions to that effect).

Overall now you should not see target nodes when you call into
pm_compile_node, since they are more specialized and require more
special handling.
2024-01-24 11:18:58 -05:00
Kevin Newton
767f5d045f [PRISM] Use lvar depths for starting 2024-01-24 10:37:08 -05:00
Peter Zhu
529700d314 [PRISM] Nested MultiWriteNode with method calls
Fixes ruby/prism#2247.
2024-01-24 10:08:25 -05:00
Nobuyoshi Nakada
d86c4e553e
Define IO_WITHOUT_GVL macro 2024-01-24 20:51:50 +09:00
Nobuyoshi Nakada
0f417d640d
Initialize errno variables and fix maybe-uninitialized warnings 2024-01-24 19:33:25 +09:00
Hiroshi SHIBATA
4099b04c09 Pend make option test on mswin platform 2024-01-24 18:12:43 +09:00
Hiroshi SHIBATA
fb39128dce [rubygems/rubygems] Added test for rubygems
https://github.com/rubygems/rubygems/commit/64405147c9
2024-01-24 09:11:55 +00:00
Hiroshi SHIBATA
6191f4a169 [rubygems/rubygems] Simplified test-case for original issue
https://github.com/rubygems/rubygems/commit/89086c5458
2024-01-24 09:11:54 +00:00
Hiroshi SHIBATA
723deec9cf [rubygems/rubygems] Keep compatibility of past versions
https://github.com/rubygems/rubygems/commit/54b67fb251
2024-01-24 09:11:53 +00:00
Hiroshi SHIBATA
1018dca09a [rubygems/rubygems] Skip to load commented out words
https://github.com/rubygems/rubygems/commit/e6a9148ba2
2024-01-24 09:11:53 +00:00
Hiroshi SHIBATA
53d0cf442a Use exit 0 instead of true on windows platform 2024-01-24 18:00:12 +09:00
Hiroshi SHIBATA
c8355a8d1f
syslog is already extracted as bundled gems 2024-01-24 15:55:30 +09:00
Hiroshi SHIBATA
270ad76359
Revert "Ensure File.open applies default umask on gem extract"
This reverts commit 01f9766aa05182a7bbdc914a5dcd8a36ebade861.
2024-01-24 14:10:05 +09:00
Hiroshi SHIBATA
2f06e4f739
Revert "Pend tests on Travis CI"
This reverts commit 772afa00b5109f55c1d8359c688aca4063818549.
2024-01-24 14:09:58 +09:00
David Rodríguez
6bd997ecfd [rubygems/rubygems] Remove no longer necessary code
https://github.com/rubygems/rubygems/commit/b7d2de2ba8
2024-01-24 13:56:07 +09:00
なつき
1e519abe21 [rubygems/rubygems] Fix rake extension warning
https://github.com/rubygems/rubygems/commit/04973e349c
2024-01-24 13:55:42 +09:00