While the human's away, do the agents slip into foul play?
My overnight report called it a quiet night, then asked me twelve times to push work myself because the automatic pushes had failed.
A scheduled job starts AI coding assistants overnight, pushes their commits to the remote repository, and writes that report. I’d rebuilt it to leave me merge requests ready for review. Here’s the headline, runtime and first two “push failed” lines, with project names replaced and long lines wrapped:
**A quiet night** — no project had tasks ready to run.
_Runtime: 726s across 7 round(s) · 12 session(s) ·
time-box 0200-0500_
- project-a: work is committed locally but the push failed —
needs a manual push.
- project-b: work is committed locally but the push failed —
needs a manual push.The trouble began at launch. The job used a launcher command to start each assistant, but the assistant’s executable had moved the day before. A diagnosis written that evening quoted the launcher’s error:
exec: claude: not foundThe original output file is gone; the quotation survives in the diagnosis. Once a minute, the job checked whether the assistant’s process still existed. That was its only way to notice an exit: the first look found the launcher still alive; the next came sixty seconds later. An assistant that died in its first second was recorded as a one-minute run.
Each assistant got a fresh branch. Next the job asked git whether the assistant’s branch was on a different commit from its remote copy. Yes was supposed to mean the assistant had committed work that hadn’t been pushed. The check said yes. The job tried to push, failed, and added a “push failed” line to the report.
The job worked in rounds: one pass over every project that still had a task, starting one assistant per project. Each start counted as a session. That night, two projects had tasks. Nothing marked the tasks as attempted, so both projects kept qualifying. Six rounds of two sessions produced twelve sessions and twelve “push failed” lines. Twelve was the cap, so the seventh round started nothing. Those twelve waits of sixty seconds account for 720 of the report’s 726 seconds; everything else took six seconds.
The job chose its headline by counting sessions started. The code suggests the count changed in a separate shell process, so the part choosing the headline still saw zero.
The “push failed” lines weren’t new. Twelve real coding runs had received the same sentence on the first night the rebuilt job ran. Whether they left commits isn’t recorded. I hadn’t caught the problem: no fix or task questioned it until this failed night, two nights later.
I’d required a report even on an empty night, so its absence would tell me the job had died. The report arrived, but the check said every assistant had work to push, every night.
Give it nothing to do
Each assistant worked in a git worktree, a separate working directory with its own branch. The job checked whether that branch held unpushed work by comparing its current commit with its upstream, the remote branch it tracked. Except a fresh branch had no upstream yet.
This reproduction keeps the comparison. Save it as has-work.sh in a fresh
empty directory. It needs git and a POSIX shell, creates a tiny repository
and worktree beside itself, and uses no network. Its “child” stands in for
the assistant: whatever command you pass it; by default, one that doesn’t
exist.
#!/bin/sh
# Usage: sh has-work.sh [child command...]
# Default child: a command that does not exist.
set -u
[ "$#" -gt 0 ] || set -- ./no-such-assistant
g() { git -c user.name=me -c user.email=me@example.com "$@"; }
g init -q parent
g -C parent commit -q --allow-empty -m "start"
g -C parent worktree add -q -b night/work ../work
cd work
pre=$(git rev-parse HEAD)
t0=$(date +%s)
"$@" >../child.out 2>&1
secs=$(( $(date +%s) - t0 ))
echo "old test:"
if [ "$(git rev-parse HEAD 2>/dev/null)" != \
"$(git rev-parse 'night/work@{u}' 2>/dev/null || echo none)" ]; then
echo " work is committed locally but the push failed"
else
echo " the session produced no changes"
fi
echo "new test:"
post=$(git rev-parse HEAD)
if [ "$post" != "$pre" ]; then
echo " child committed: $(git log -1 --format=%s)"
elif [ "$secs" -lt 30 ]; then
echo " child failed to start (${secs}s): $(tail -n 1 ../child.out)"
else
echo " child ran ${secs}s and changed nothing"
fiI ran it with the nonexistent command. Here’s the actual output, wrapped:
$ sh has-work.sh
old test:
work is committed locally but the push failed
new test:
child failed to start (0s): has-work.sh: line 16:
./no-such-assistant: No such file or directoryNothing ran, yet the old comparison found committed work. Inside the
generated work/ directory, removing the redirection exposed why:
$ git rev-parse 'night/work@{u}'
fatal: no upstream configured for branch 'night/work'
$ echo $?
128Git’s exit status, 128, signalled failure. The old test discarded the error
with 2>/dev/null, then supplied none through || echo none. A commit
hash never equals none. The comparison turned a failed lookup into
evidence of work.
In the real job, that sent it into a push from a worktree deliberately barred from pushing, instead of the original checkout. The push failed, the sentence followed, and the job deleted that night’s branch with no new work to lose. The report couldn’t distinguish a productive assistant from one that had never started.
- Before trusting an overnight status, give it a night that should make it say something different.
What the repair can tell
Before starting the assistant, the repair saves the commit its branch is on. After the assistant exits, it reads the branch’s commit again: a different commit means the assistant committed; the same one means it didn’t. If the commit is unchanged, the job checks elapsed time. Under thirty seconds means a failed start, on the assumption that a real coding session takes longer. That assumption has an edge you can reach immediately.
I ran the script again in two fresh directories. The commit command uses your configured git identity:
$ sh has-work.sh git commit -q --allow-empty -m "night work"
old test:
work is committed locally but the push failed
new test:
child committed: night work
$ sh has-work.sh true
old test:
work is committed locally but the push failed
new test:
child failed to start (0s):The commit is recognised. But true starts, succeeds and exits immediately;
the repair calls it a failed start, with nothing to quote. It reads git and
a clock, not the reason for an empty result. An assistant that runs for
minutes without committing still gets reported as producing no changes,
whatever stopped it.
The repair stopped retrying quick failures and kept branches whose commits hadn’t reached the remote.
The next night’s own assistant wrote that repair. It didn’t govern the night that wrote it: the already-running job kept executing the old script. Only the following night did the rebuilt job open its first two merge requests, five days after I’d rebuilt it for that purpose.
Two changes ready for review. That’s a morning worth building for.