file
(https://github.com/ruby/yarp/pull/1371)
* refactor: move EOF check into yp_unescape_calculate_difference
parser_lex is a bit more readable when we can rely on that behavior
* fix: octal and hex digits at the end of a file
Previously this resulted in invalid memory access.
* fix: unicode strings at the end of a file
Previously this resulted in invalid memory access.
* Unterminated curly-bracket unicode is a syntax error
https://github.com/ruby/yarp/commit/21cf11acb5
This is an internal only function not exposed to the C extension API.
It's only use so far is from rb_vm_mark, where it's used to mark the
values in the vm->trap_list.cmd array.
There shouldn't be any reason why these cannot move.
This commit allows them to move by updating their references during the
reference updating step of compaction.
To do this we've introduced another internal function
rb_gc_update_values as a partner to rb_gc_mark_values.
This allows us to refactor rb_gc_mark_values to not pin
OpenBSD and Solaris behave differently here.
Linux does deliver the empty packet, which is questionable
as it's undistinguishable from a closed connection.
It seems that OpenBSD and Solaris simply drop it.
We could test the platform before doing the assertion, but
it would likely be fragile, and the entire web recommend
to not ever send an empty packet, so the value of this
assertion is low.
The old algorithm could calculate an undercount for the initial pages
due to two issues:
1. It did not take into account that some heap pages will have one less
slot due to alignment. It assumed that every heap page would be able
to be fully filled with slots. Pages that are unaligned with the slot
size will lose one slot. The new algorithm assumes that every page
will be unaligned.
2. It performed integer division, which truncates down. This means that
the number of pages might not actually satisfy the number of slots.
This can cause the heap to grow in `gc_sweep_finish_size_pool` after
allocating all of the allocatable pages because the total number of
slots would be less than the initial configured number of slots.
- Filter out files to be ignored first, then resolve conflicts.
- Add "added by gem" files, instead of hard-code paths to add.
- Remove gem specific patterns covered by more generic rules.
This commit changes RUBY_GC_HEAP_INIT_SIZE_{40,80,160,320,640}_SLOTS to
RUBY_GC_HEAP_{0,1,2,3,4}_INIT_SLOTS. This is easier to use because the
user does not need to determine the slot sizes (which can vary between
32 and 64 bit systems). They now just use the heap names
(`GC.stat_heap.keys`).
Also, a similar test and fix for interpolated regular expressions.
This snippet:
<<-A.g//,
A
/{/, ''\
previously created a regular expression node with inverted start and
end:
RegularExpressionNode(14...13)((14...15), (15...21), (12...13), ", ''", 0),
which failed an assertion during serialization.
After this change:
RegularExpressionNode(12...15)((14...15), (15...21), (12...13), ", ''", 0),
Found by the fuzzer.
https://github.com/ruby/yarp/commit/5fef572f95
The snippet added in this commit previously resulted in a CallNode
with inverted start and end locations:
> AssocNode(15...13)(
> CallNode(15...13)(
StringNode(15...17)((15...16), (16...16), (16...17), ""),
nil,
(12...13),
nil,
ArgumentsNode(12...13)([MissingNode(12...13)()]),
nil,
nil,
0,
"/"
),
MissingNode(13...13)(),
(13...13)
),
which failed an assertion during serialization.
After this change, it looks better:
> AssocNode(12...13)(
> CallNode(12...17)(
StringNode(15...17)((15...16), (16...16), (16...17), ""),
nil,
(12...13),
nil,
ArgumentsNode(12...13)([MissingNode(12...13)()]),
nil,
nil,
0,
"/"
),
MissingNode(13...13)(),
(13...13)
),
Found by the fuzzer.
https://github.com/ruby/yarp/commit/040aa63ad6
The presence of the heredocs in this snippet with invalid syntax:
for <<A + <<B
A
B
causes the MissingNode to have a location after other nodes in the
list, resulting in a StatementsNode with inverted start and end
locations:
[ForNode(0...14)(
MultiWriteNode(4...7)([InterpolatedStringNode(4...7)((4...7), [], (14...16))], nil, nil, nil, nil),
MissingNode(16...16)(),
> StatementsNode(16...14)(
[MissingNode(16...16)(), InterpolatedStringNode(10...13)((10...13), [], (16...18)), MissingNode(13...14)()]
),
(0...3),
(16...16),
nil,
(14...14)
)]
which failed an assertion during serialization.
With this fix, the node's locations are:
[ForNode(0...14)(
MultiWriteNode(4...7)([InterpolatedStringNode(4...7)((4...7), [], (14...16))], nil, nil, nil, nil),
MissingNode(16...16)(),
> StatementsNode(10...16)(
[MissingNode(16...16)(), InterpolatedStringNode(10...13)((10...13), [], (16...18)), MissingNode(13...14)()]
),
(0...3),
(16...16),
nil,
(14...14)
)]
Found by the fuzzer.
https://github.com/ruby/yarp/commit/09bcedc05e
