Initial text cleanup
This commit is contained in:
28
src/stats.jl
28
src/stats.jl
@@ -5,7 +5,7 @@
|
||||
# cannot even approximate it: every in-flight file sits in `spool/` regardless of
|
||||
# how far it has got, because stages route by enqueueing a reference rather than
|
||||
# by moving bytes (src/worker.jl header). A file's stage is a property of the
|
||||
# queue holding it, so only the pipeline itself can see it — there is no
|
||||
# queue holding it, so only the pipeline itself can see it. There is no
|
||||
# directory to poll.
|
||||
#
|
||||
# So each stage keeps four counters, all incremented in `worker_loop` (the one
|
||||
@@ -23,15 +23,15 @@
|
||||
# utilization = (Δbusy_ns - Δblocked_ns) / (Δt * workers) ~1.0 ⇒ bottleneck
|
||||
#
|
||||
# Utilization is the useful one. Throughput alone can't distinguish a stage that
|
||||
# is saturated from one that is merely starved by the stage ahead of it — both
|
||||
# show the same files/s — whereas utilization separates them: the saturated stage
|
||||
# is saturated from one that is merely starved by the stage ahead of it: both
|
||||
# show the same files/s. Utilization separates them: the saturated stage
|
||||
# sits near 1.0 with its queue backing up, the starved stage sits near 0.
|
||||
#
|
||||
# `blocked_ns` is what keeps that true downstream. Stages 1 and 3 apply blocking
|
||||
# backpressure: when the next queue is full the handler parks and retries rather
|
||||
# than dropping the file (see ROUTE_ENQUEUE_RETRY_SECONDS). That parked time is
|
||||
# inside the handler, so counting it as busy would show stage 1 pinned at 1.0
|
||||
# whenever stage 2 is the real bottleneck — every stage upstream of the jam would
|
||||
# whenever stage 2 is the real bottleneck, and every stage upstream of the jam would
|
||||
# look like the jam. Subtracting it leaves utilization meaning "doing its own
|
||||
# work", and the blocked share becomes its own signal: a stage blocked 90% of the
|
||||
# time is naming its successor as the bottleneck.
|
||||
@@ -41,7 +41,7 @@
|
||||
# a scrape is stateless and two readers can't disturb each other.
|
||||
#
|
||||
# Cost is a handful of atomic adds per file, against handlers that spawn
|
||||
# exiftool — unmeasurable in practice, and the counters are never read on the
|
||||
# exiftool: unmeasurable in practice, and the counters are never read on the
|
||||
# hot path.
|
||||
|
||||
# Stage identity lives here, in declaration order, so the report, the JSON and
|
||||
@@ -55,7 +55,7 @@ const STAGE_TITLES = (classify = "classify", enrich = "enrich",
|
||||
Counters for one pipeline stage. All fields are atomic and monotonic: workers
|
||||
only ever add, readers only ever read, so no lock is needed between them.
|
||||
|
||||
`in_flight` is the exception to monotonic — it goes up and down — and is the one
|
||||
`in_flight` is the exception to monotonic. It goes up and down, and is the one
|
||||
counter that is a *level* rather than a total.
|
||||
"""
|
||||
struct StageStats
|
||||
@@ -74,8 +74,8 @@ StageStats() = StageStats(Threads.Atomic{Int}(0), Threads.Atomic{Int}(0),
|
||||
"""
|
||||
Counters for HTTP intake, which has no worker loop to hang them off.
|
||||
|
||||
`files` counts what was actually spooled *and* queued — the same thing the 202
|
||||
body reports as `accepted` — so it lines up with stage 1's arrivals. Files
|
||||
`files` counts what was actually spooled *and* queued, the same thing the 202
|
||||
body reports as `accepted`, so it lines up with stage 1's arrivals. Files
|
||||
dropped because the queue was full are counted separately in `rejected`, since a
|
||||
run where intake outruns the pipeline should be visible as backpressure rather
|
||||
than as slow intake.
|
||||
@@ -96,7 +96,7 @@ from.
|
||||
|
||||
`since` matters to a reader computing rates from a single scrape: without it,
|
||||
"1000 files completed" has no denominator. Readers that scrape twice should use
|
||||
their own Δt instead — it excludes the time before they started watching.
|
||||
their own Δt instead, which excludes the time before they started watching.
|
||||
"""
|
||||
struct Metrics
|
||||
intake::IntakeStats
|
||||
@@ -117,7 +117,7 @@ const METRICS = Metrics()
|
||||
reset_metrics!()
|
||||
|
||||
Zero every counter and restart the measurement window. For tests, and for a
|
||||
benchmark that wants totals covering only its own run — though a harness that
|
||||
benchmark that wants totals covering only its own run, though a harness that
|
||||
scrapes before and after can subtract instead, which is safer against a server
|
||||
that is also serving someone else.
|
||||
"""
|
||||
@@ -153,14 +153,14 @@ the parked time to `stats.blocked_ns`.
|
||||
|
||||
This is the routing half of stages 1 and 3: a classified file is never dropped,
|
||||
so a full downstream queue means waiting, not failing. Wrapping the retry loop
|
||||
here — rather than repeating `while !enqueue! sleep end` at each call site — is
|
||||
here, rather than repeating `while !enqueue! sleep end` at each call site, is
|
||||
what makes that wait measurable at all, and keeps the three routing paths from
|
||||
drifting into three different backoff behaviours.
|
||||
|
||||
`stats` is `nothing` for callers that are not a pipeline stage — startup
|
||||
`stats` is `nothing` for callers that are not a pipeline stage. Startup
|
||||
recovery (`recover_dir!`) uses the same never-drop retry, but it runs outside any
|
||||
handler, so it has no `busy_ns` to charge against. Billing its wait to a stage's
|
||||
`blocked_ns` would make `/stats` utilization — `(Δbusy_ns - Δblocked_ns) / …` —
|
||||
`blocked_ns` would make `/stats` utilization, `(Δbusy_ns - Δblocked_ns) / …`,
|
||||
go *negative* after a long recovery. Skipping the charge keeps `blocked_ns`
|
||||
meaning exactly what its docs say: time a handler spent parked on its successor.
|
||||
"""
|
||||
@@ -183,7 +183,7 @@ end
|
||||
Read every counter into a plain value tree for `GET /stats`.
|
||||
|
||||
`queues` is a NamedTuple of `JobQueue`s keyed by `STAGE_KEYS`, supplying each
|
||||
stage's live depth and capacity — the counters above are about work *done*, and
|
||||
stage's live depth and capacity. The counters above are about work *done*, and
|
||||
a stage's depth is what says whether work is piling up in front of it.
|
||||
|
||||
The read is not atomic across stages: counters keep moving while we walk them,
|
||||
|
||||
Reference in New Issue
Block a user