Anyone who has run scheduled jobs in production has learned this the hard way: a script can exit with status 0 and still fail to do its actual job. The wrapper finished. The database write happened. The file was created. But did the report actually get published? Did the API call actually succeed downstream? Exit code 0 tells you the process didn’t crash — it says nothing about whether the business goal was achieved.
A recent piece on Zenn tackles this exact problem from a test-design perspective, and the core insight is worth internalizing for anyone building batch pipelines, cron jobs, or automated publishing workflows: scheduling something to run is not the same as making it reliable.
The core problem: conflating process exit with business success
Most automated systems collapse two very different concepts into one signal. The first is process completion — did the script run to the end without an unhandled exception? The second is business completion — did the intended outcome actually happen in the world? These are not the same thing, and treating them as interchangeable is how you end up with silent partial failures that nobody notices until a customer complains.
The proposed fix is refreshingly concrete: stop using the process exit code as your success signal entirely. Instead, introduce a structured field — call it final_status — that explicitly records the actual business outcome, while the raw exit code (rc) is demoted to nothing more than a process-classification detail (did it crash, timeout, or run to completion). These are tracked separately and never merged into a single boolean.
To make this work, the article breaks a typical automated job into discrete, independently recorded stages: draft/content generation, quality inspection, external publication, and final verification of the published result (e.g., confirming the public URL is actually live and correct). Each stage gets its own recorded status. A job doesn’t get to claim “success” just because the last line of the script executed — it has to prove, stage by stage, that inputs were valid, processing produced a real artifact, that artifact passed quality checks, and the publication step was independently confirmed rather than assumed.
Why does this granularity matter so much? Because when something goes wrong at 3am and nobody’s watching, the next day’s processing shouldn’t have to guess what state the system is in based on incomplete logs. If yesterday’s publish step silently failed, today’s job needs to know that unambiguously — not infer it from the absence of an error, which is exactly the trap that relying on exit codes creates. Structured, stage-level status turns “probably fine” into “verifiably fine or explicitly not.”
This matters more than it might seem at first glance, because in most real systems, the gap between “the script ran” and “the outcome was achieved” is exactly where silent data corruption, missed notifications, and broken public-facing content come from. Nobody designs for these failures on purpose — they accumulate because exit code 0 became a proxy for correctness by default, and nobody questioned the proxy.
The human side of test design: knowing what NOT to test
A companion piece on test design, this time focused on AI-assisted testing, makes a complementary point that’s worth pairing with the exit-code discussion. The scenario: you ask an AI (in this case Claude) to generate test viewpoints for a feature, and it happily produces something like 100 angles to consider — edge cases, boundary conditions, error states, integration points, and so on. The problem isn’t generating enough ideas anymore; it’s dealing with the flood.
The article’s framing is sharp: AI is good at addition — exhaustively listing everything that could be tested. Humans are good at subtraction — deciding what actually should be tested, and being able to articulate why the other 65 items were cut. That’s the real skill gap. Anyone can now generate a comprehensive-looking checklist with an LLM prompt. Very few people can look at that checklist and make a principled, defensible call about which 30-40% deserves engineering time, and which items are redundant, low-risk, or effectively covered by other tests already in the suite.
This reframes test design skill away from “knowing what to test” (which AI now does reasonably well) toward “knowing what to justify cutting” — a much harder and more judgment-heavy skill, because it requires understanding risk, business priority, and the cost of a missed bug, not just technical coverage.
Why these two ideas belong in the same conversation
Both pieces are really about the same underlying discipline: refusing to accept a shallow signal as a substitute for genuine verification. In the job-design case, the shallow signal is exit code 0. In the AI test-generation case, the shallow signal is “we have 100 test cases, therefore we’re thorough.” Both are illusions of correctness that collapse under scrutiny — a process can exit cleanly while doing nothing useful, and a test suite can be enormous while still missing the failure modes that actually matter in production.
The practical takeaway for engineering teams: build structured, stage-level status tracking into automated jobs rather than trusting a single boolean-like signal, and treat AI-generated test coverage as a starting inventory to prune deliberately rather than a finished deliverable. In both cases, the hard, valuable work happens after the easy part (running the script, generating the list) is already done.