* BOM should not impact looking for the encoding string
* We should re-encode tokens when the encoding changes
* BOM should change the column of comments only
https://github.com/ruby/yarp/commit/119fc2d7b2
* Ensure the node gets initialized in case of failure with no assertions
* Include lambda and top-level nodes for scopes
* Small indentation fixes
https://github.com/ruby/yarp/commit/be29e07391
This commit creates a scope node, and exposes a method to get
ScopeNodes from other existing nodes. It still has TODOs around
creating ScopeNodes for other types of nodes, which will be
done in future commits.
https://github.com/ruby/yarp/commit/3b9ac5764d
* `le_len` to `eol_length`
* Braces on the same line as switch case
* `peek_addr` -> `peek_at`
* `peek_at` -> `peek_offset`
* `match_line_ending_addr` -> `match_eol_at`
* `match_line_ending_at` -> `match_eol_offset`
https://github.com/ruby/yarp/commit/d7ffa9e64e
Introduce three new inline helper functions:
- `match_line_ending`
- `match_line_ending_at`
- `match_line_ending_addr`
These functions are similar in signature to the `peek*` functions, but
return the length of the line ending being inspected (or 0 if no line
ending was found).
These functions are then used to simplify how we're detecting line
endings throughout "src/yarp.c".
Also:
- test coverage backfilled for `__END__` comments with CRLF line endings.
- error message for invalid `%` tokens updated to not include
the potential line endings.
- some small refactorings for readability along the way
https://github.com/ruby/yarp/commit/a00067386d
Previously, the following crashed due to use-after-free
with AArch64 Alpine Linux 3.18.3 (aarch64-linux-musl):
```ruby
str = 'a' * (32*1024*1024)
p({z: str})
```
32 MiB is the default for `GC_MALLOC_LIMIT_MAX`, and the crash
could be dodged by setting `RUBY_GC_MALLOC_LIMIT_MAX` to large values.
Under a debugger, one can see the `str2` of rb_str_buf_append()
getting prematurely collected while str_buf_cat4() allocates capacity.
Add GC guards so the buffer of `str2` lives across the GC run
initiated in str_buf_cat4().
[Bug #19792]
https://github.com/ruby/ruby/actions/runs/5956398507/job/16157091112
This has been extremely flaky on macOS GitHub Actions.
Benoit suggested to quarantine it if it's too problematic (it is) and
there's no reasonable fix in a short time (it already took too long).
So this commit follows the suggestion.
We should remove revert this once rb_cloexec_open() is fixed.
in Ripper EOL after whitespace is returned as a on_nl node with the whitespace as the content
k0kubun: This is a resync of 9aca3528aa1a1545468a508b02b77bc922bb7321.
git.ruby-lang.org had an old git version and the diff was wrong.
It now has a newer git, so this should succeed next time.
https://github.com/ruby/yarp/commit/be16d1deed
I was testing this script on git.ruby-lang.org to use its git version,
but it did not work because the server's default user doesn't have SSH
keys.
https works for everyone, so it's a safer default. You shouldn't need to
push to that remote from ruby/ruby anyway.
When Ripper encounters a method parameter that is the first
parameter and is an identifier and it shadows a local scope, it
incorrectly marks it as END|LABEL (because it think it's a local).
We need to account for that in the lex compat in order to properly
compare.
https://github.com/ruby/yarp/commit/15f725a1b1
Rack uses this. Speculate that the `obj` in `the_call(&obj)`
will be a proc when the compile-time sample is a proc.
Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
In many places in the code we use the idiom:
x < parser->end && *x == 'y'
which is essentially an extension of an existing pattern:
- `peek()` looks at `parser->current.end`
- `peek_at()` looks at `(parser->current.end + offset)`
This commit introduces a new inline function, `peek_addr`, which
accepts a pointer and encapsulates the address value check and
conditional dereferencing. The result is more readable code, and more
ubiquitous safety checks on pointer values, allowing us to rewrite the
above as:
peek_addr(parser, x) == 'y'
Also:
- change the type of `peek_at()`'s offset argument from `size_t` to
`ptrdiff_t` so that it can accept negative offsets.
- use `current_token_starts_line` in one place where the equivalent
code is inline.
- use `peek` or `peek_at` to replace inline code in a few places
These changes simplify the code and make it easier to visually spot
patterns, particularly around line-endings (which will be a subject of
a future pull request).
https://github.com/ruby/yarp/commit/4c608d53ea
