Hi,

This looks promising but I feel like the PEP is a bit short on explanations, especially how you came to that particular design.

Regarding some of the PEP details:

The performance improvements come from doing less work in the young generations (one collection per object, not two) and doing less work in the old generation due to the lower survivor rate from the young generations.

Where does the “lower survivor rate” come from? Is it simply the fact that the nursery is now twice as large, or something else? In other words, if you were to double the size of gens 0 and 1 in the legacy GC, such as to halve the number of non-full collections, would that erase the performance uplift of the incremental GC?

For small and short-lived applications, memory consumption is likely to increase due to the larger young generations, but the extra space is bounded at 24MB by default

How does the bounding work? In other words, what kind of metric is used to decide whether the 24MB limit is reached?

Now to the algorithm description:

work_to_do: int = -100_000

Why start at this value? Is it so that the interpreter startup phase isn’t interrupted by too many GC collections?

Also, how did you come to this value?

def scan_reachable(limit):

"""Move some reachable objects from pending to reachable and from

reachable to visited. They are reachable and cannot be garbage.

"""

moved_to_visited = 0

while reachable:

root = reachable.pop()

visited.append(root)

moved_to_visited += 1

for obj in gc.get_referents(root):

if obj in pending:

pending.remove(obj)

reachable.append(obj)

if moved_to_visited >= limit:

return moved_to_visited

return moved_to_visited

What does limit do here? Not only it does not seem to limit the number of values visited, but it also doesn’t affect the return the value (moved_to_visited is always returned).

The work_to_do updates are the particularly opaque part IMHO. This sentence seems to suggest that work_to_do should be roughly equal to len(pending_space), or a fraction thereof:

Increments are collected until sufficient objects have been scanned to keep up with the rate of objects being added to the old generation by the young collections

However, there is then this weird heuristic:

survivors = collect_cycles(increment)

work_to_do -= survivors

collected = candidates - survivors

# If we are collecting lots of objects, that means

# there is a lot of cycle garbage and we need to

# sweep the heap faster.

work_to_do += 2 * collected

(more concisely, work_to_do += 2 * candidates - 3 * survivors, which looks a bit magic)

Did you find this to be useful on some workloads? Detrimental on others? And on which metric? IMHO it would be really useful to have more explanations on how you decided to go with this.

Moreover, it seems that “we need to sweep the heap faster” could be applied in two different ways:

- we will do more work at each collection

- we will do collections more frequently

IIUC, this algorithm is implementing option 1 of doing more work at once, but wouldn’t an incremental collector be better served by option 2, so as to avoid ballooning GC pauses?

(that probably implies some kind of dynamic threshold for collections, based on observed collection behavior)