# NJ School Performance Reports

Reporter's notebook for the SPR dataset: where it comes from, how we fetch it,
how it joins to our enrollment warehouse, and every oddity worth knowing before
you trust a number. Companion to the Fall Enrollment ledger
(`docs/source-packets/somsd-integration/SOURCES.md`).

```toml ergo
[dataset]
ergo = "0.4"
slug = "spr"
title = "NJ School Performance Reports"
publisher = "NJ Dept. of Education, Office of Performance Reports"
subject = "https://www.nj.gov/education/spr/"
source_url = "https://www.nj.gov/education/spr/"
pitfall = "NJ publishes two 4-yr graduation rates — State and Federal — but only State is populated across years (27,115 of 27,201 trend rows; Federal in just 5,423, effectively one year); plotting Federal as a trend, or comparing states on it, is the classic misread."
status = "live"
confidence = "A"
updated = "2026-08-03"
implementation = "https://github.com/lavallee/njschooldata"

[dataset.coverage]
years = "2015-16 → 2024-25 for graduation and chronic absenteeism, from all ten published editions; 2021-22 → 2024-25 for assessment and growth, which still read the current workbook's trend tabs only (see spr/trend-tabs-post-covid-only)"
grain = "school year × entity × student group"
entities = "every NJ public school, district, and the state"

[dataset.access]
keys = ["county_code", "district_code", "school_code", "school_year"]
```

## Where this data now comes from

njschooldata no longer downloads or parses this source. Aquifer owns the
acquisition, the schema-era parsers, and the immutable release; the issues below
that describe parsing or source format are recorded against the producer's own
registry and are monitored here rather than handled here. The consumer adapter
installs an exact offline pin, and the issues still marked as handled are the
ones this repo genuinely handles.

Rollback is no longer "re-run the local builder" — the local builder is gone.
Roll the pin back to a retained release and re-run the adapter.

## What it is

New Jersey's official annual "report card" for every public school and
district. ESSA-mandated accountability data: state assessments (NJSLA
ELA/Math/Science, NJGPA), student growth, graduation & dropout, chronic
absenteeism, discipline / HIB, college & career readiness (PSAT/SAT/ACT,
AP/IB/dual enrollment, CTE), staffing (experience, ratios, retention), and
ESSA accountability status.

This is the authoritative source for the academic-outcome modules we're
adding to school/district profiles (graduation first, then assessment +
absenteeism).

Contact: `reportcard@doe.nj.gov`. Granularity: school, district, and state —
every NJ public entity. Disaggregated by `StudentGroup` (all students +
race/ethnicity, ELL, econ-disadvantaged, disability, etc. — 17 categories
total; full list under Validation).

Logged as "acquisition in progress" through 2026-06; the builder now loads
five tables (`grad_rate`, `assessment`, `assessment_grade`, `absenteeism`,
`growth`) into the isolated SPR family before composition — see Builder.

## Access

The **landing page** is `https://www.nj.gov/education/spr/download/`, but
it's a JS app: a year `<select>` plus `script/download.js?v=11` that builds
the real file links. The files themselves live under a **different host
path** — `sprreports`, not `spr` (`spr/url-host-split`):

```
https://www.nj.gov/education/sprreports/download/DataFiles/<YEAR>/<file>
```

`<YEAR>` is hyphenated four-digit, e.g. `2024-2025`. Files (from
`download.js`):

| File | What |
|---|---|
| `Database_SchoolDetail.xlsx` | **school-level**, every report tab in one workbook |
| `Database_DistrictStateDetail.xlsx` | **district + state** level, same tabs |
| `Database_SchoolLayout.xlsx` / `Database_DistrictLayout.xlsx` | **data dictionary** — one sheet per tab (Field / Type / Description) |
| `…/Documents/<YEAR>/ReferenceGuide.pdf`, `DataPrivacyRules.pdf`, `FAQs.pdf` | prose docs |

`.zip` (zipped xlsx) and `.accdb` (Access) variants also exist **for older
years only**. The JS flags `isNewYearFormat = year >= "2024-2025"`: from
2024-25 on, **only the `.xlsx`** is offered — no zip, no Access, no
Reference Guide PDF (`spr/new-format-2024-25-xlsx-only`).

### Year coverage

Dropdown offers **2015-16 → 2024-25** (the JS pre-provisions
2025-26…2029-30).
- `2016-17 →` : full two-file (school + district/state) format.
- `2015-16` : special two-column layout.
- `2011-12 … 2014-15` : **legacy** — not in this URL scheme. The archive link
  this page used to give (`nj.gov/education/schoolperformance/archive/`) is a
  404; the live material sits under Title I accountability
  (`spr/legacy-archive-link-is-dead`).

### File sizes (2024-2025) — these are *large*

| File | Bytes | ~ |
|---|---|---|
| `Database_SchoolDetail.xlsx` | 346,029,712 | **330 MB** |
| `Database_DistrictStateDetail.xlsx` | 119,164,900 | **114 MB** |
| `Database_SchoolLayout.xlsx` | 214,014 | 209 KB |

One monolithic workbook packs **~80 report tabs**. Loading the whole thing
with openpyxl in default mode is heavy (`spr/monolithic-workbook-size`) — the
governed Aquifer capture uses streaming readers and pulls only the selected
sheets. (Contrast
Fall Enrollment: small per-year zips.)

Raw files are cached locally and are re-fetchable. Aquifer now owns governed
2024-25 edition captures, selected-tab manifests, literal suppression states,
immutable releases, and exact offline pins for chronic absence, graduation,
assessment headline, assessment by test, and student growth.
`tools/build_aquifer_spr_family.py` is the single consumer rebuild: it installs
`absenteeism`, `grad_rate`, `assessment`, `assessment_grade`, and `growth`, then
projects the pinned edition censuses into `absenteeism_trend_edition`,
`grad_rate_history`, and `grad_rate_edition`. It also projects literal cells and
physical workbook provenance from the assessment artifact's typed tables into
the consumer assessment mart used by participation-aware profile answers.
Rollback means selecting retained Aquifer releases in the five exact pins and
rerunning this same family adapter; there is no consumer-local workbook parser
or one-table compatibility CLI.

## Structure

Every tab repeats the same identity columns, then its own metrics:

```
CountyCode(Numeric) CountyName DistrictCode(Numeric) DistrictName
SchoolCode(Numeric) SchoolName SchoolYear StudentGroup  + <metrics>
```

Header is on **row 4**; data starts **row 5** (rows 1–3 are title/note).

## Joins

The layout types the code columns "Numeric," but the **actual data cells are
zero-padded TEXT** — `CountyCode`="13", `DistrictCode`="4900"
(`spr/layout-types-lie`) — i.e. they match our warehouse's TEXT keys
verbatim (county 2-wide, district 4-wide; school width to confirm from the
school file, expected 3). `SchoolYear` is `"2020-21"`, also matching the
warehouse format, though the layout's declared type for it disagrees tab to
tab (`spr/schoolyear-typing-inconsistent`). So SPR joins to `enrollment`
directly on `(district_code, school_code, school_year)` with no coercion.
Confirmed against the 2024-25 District/State file.

## Tabs we use

Confirmed columns from the 2024-25 School layout.

### Graduation — `GraduationRateTrends` (the one for `mod_graduation`)

Multi-year (**2020-21 … 2024-25** — five years), and carries school **+**
district **+** state in each row, so the trajectory line *and* its context
line come from one tab:

```
… SchoolYear StudentGroup 4YrCohort
4YrGraduationRate{State,Federal}_{School,District,State}
5Yr… 6Yr…  (same shape for 5- and 6-year cohorts)
```

- ⚠️ **Use the State calculation for the trajectory** (`spr/grad-state-vs-federal`):
  NJ publishes two 4-yr rates — State and Federal — but only the State rate
  is populated across all five years; the Federal rate appears only for the
  most recent year (2024-25).
  - `mod_graduation` plots `4YrGraduationRate**State**_{School|District}`
    over the five years, with `…State_State` as the statewide context line.
  - Headline the latest-year **Federal** 4-yr ACGR too where present, and
    note in the caption that NJ's State calc runs higher than the federal
    ACGR. Don't draw Federal as a trend — it has one point.
  - To extend earlier than 2020-21, pull older years' files (each carries ~5
    years of State rates). 5 years is enough for the proof-of-shape.
  - The local `grad_rate` compatibility table intentionally exposes the
    existing 4-year State/Federal and 5-year State columns. Aquifer's canonical
    pinned release additionally retains all 4/5/6-year State/Federal values at
    entity, district-context, and state-context grain with exact cohort and
    suppression literals.
- `FederalGraduationRates` (single year, Federal only) is redundant given the
  Trends tab's latest-year Federal columns.
- `GraduationCohortProfile` (graduated / continuing / non-continuing /
  persisting) is a richer follow-on, not yet loaded into the builder. Its
  `NonContinuing_District` and `Persisting_District` columns are typed
  **Text** while siblings are Numeric — a suppression tell
  (`spr/cohort-profile-text-suppression-tell`).

### Chronic absenteeism — `ChronicAbsenteeismTrends` → `mod_absenteeism` ✅ built

Years **2021-22 … 2024-25** (four); `ChronicAbsenteeismRate_{School,District,State}`
by StudentGroup. The "Alphanumeric" typing we flagged is real: cells are
`"8.1%"` strings, with prose suppression and `">90%"` extreme-capping mixed
in (`spr/rate-prose-suppression`). Loaded into the `absenteeism` table (rate
+ district/statewide context); applies to **all grade levels**, not
HS-gated. SOMSD: 12.6→15.6→11.2→8.1% vs NJ ~18→14%.

### Assessment — NJSLA ELA + Math → `mod_assessment` ✅ built (stacked-area, dual-lens)

Three tabs feed this, all **2021-22 → 2024-25** (`spr/trend-tabs-post-covid-only`):
- **`ELA`/`MathParticipationPerformance`** → table `assessment`: the
  official school-wide `MetExceededExpectations` NJ publishes (+
  participation), per entity·year·subject. Drives the headline % in each
  subject's lede.
- **`ELA`/`MathPerformanceByTest`** → table `assessment_grade`: per
  entity·year·subject·**grade-test**, the full **Level 1–5** distribution +
  mean scaled score + proficiency (numeric *and* raw string, so capped
  `<10%`/`>90%` show honestly). The honest grain — NJ never publishes a
  school-wide 1–5 distribution, only per test.
- **`StudentGrowthTrends`** → table `growth`: median **Student Growth
  Percentile** (1–99, 50 = typical) for ELA + Math with NJ's
  Low/Typical/High label, at school/district/state. The
  demographic-independent "is it good?" signal.

The module: **growth lead** (an SGP scale) → the exact NJDOE-published
school-wide All Students record → per subject, stacked-area 1–5 distributions
in two lenses. **Lens A** is "by grade, year to year" (a panel per grade).
**Lens B** lines up aggregate grade results along diagonals of the grade × year
matrix. It approximates a class path, but the rows are not linked student
records: enrollment and the tested population can change between years. All
panels are interactive (hover → level breakdown); single column on mobile.

Key design calls (settled through review):
- **Growth is the lead** where it exists; **high schools have no SGP**
  (`spr/hs-no-sgp` — it's grades 4-8 only) so they fall back to a
  proficiency frame and get **no cohort lens** (one grade can't be traced).
- **Per-grade, never school-wide-aggregated** levels
  (`spr/grade-aggregation-bias`) — aggregating across grade-tests is biased
  by suppression and mixes incomparable course populations.
- Math cohort tracking is **grades 3-8 only**
  (`spr/math-cohort-grades-3-8-only` — HS course tests aren't a clean
  grade+1 sequence).
- **Grade-mix nuance** (`spr/grade-mix-context`): a school's headline
  proficiency reflects only its tested grades, but district/state blend all
  grades 3-8 + HS — so a HS's math can sit *below* its district. Real, not a
  bug.
- Pre-COVID NJSLA (2018-19 + the no-spring-2020 gap) needs older-year files
  (deferred — `spr/trend-tabs-post-covid-only`).

## Issues

### Landing page and file host are different paths

```toml ergo
[issue]
id = "url-host-split"
title = "Files live under /education/sprreports/, not the /education/spr/ landing path"
effect = "breaks"
type = "availability"
status = "monitor"
discovered = "2026-06"
detection = "404s when constructing file URLs from the landing-page path instead of the sprreports host"

[issue.scope]
all = true
```

The landing page (`https://www.nj.gov/education/spr/download/`) is a JS app
whose `download.js?v=11` builds the real links:
`https://www.nj.gov/education/sprreports/download/DataFiles/<YEAR>/<file>`,
with hyphenated years (`2024-2025`). Easy to fetch the wrong path if you
build URLs from the landing page itself.

### The layout workbook's "Numeric" typing is misleading

```toml ergo
[issue]
id = "layout-types-lie"
title = "Code columns typed 'Numeric' in the layout are zero-padded TEXT in the data"
effect = "corrupts"
type = "format"
status = "monitor"
discovered = "2026-06"
detection = "CountyCode='13', DistrictCode='4900' — string cells with leading zeros preserved, despite the layout workbook's dictionary calling them Numeric"
misuse = "Coercing codes to integers strips leading zeros and silently breaks the warehouse join."

[issue.scope]
tables = ["grad_rate", "assessment", "assessment_grade", "absenteeism", "growth"]
columns = ["CountyCode", "DistrictCode", "SchoolCode"]
years = "all"
```

Confirmed against the 2024-25 District/State file: the data cells match the
warehouse's TEXT keys verbatim (county 2-wide, district 4-wide; school width
to confirm from the school file, expected 3), so the *correct* behavior is
to trust the data over the data dictionary.

### `SchoolYear`'s declared type disagrees tab to tab

```toml ergo
[issue]
id = "schoolyear-typing-inconsistent"
title = "SchoolYear is declared Numeric in some tabs' layout and Text in others, though the data is always the string \"2020-21\""
effect = "corrupts"
type = "format"
status = "monitor"
discovered = "2026-06"
detection = "layout workbook's Type column for SchoolYear varies by tab; every tab's actual SchoolYear cell reads as the hyphenated string, e.g. '2020-21'"
misuse = "Writing per-tab ingestion code that trusts each tab's declared SchoolYear type parses some tabs as numeric and others as text inconsistently; always treat SchoolYear as a string."

[issue.scope]
tables = ["grad_rate", "assessment", "assessment_grade", "absenteeism", "growth"]
columns = ["SchoolYear"]
years = "all"
```

Same lesson as the code columns (`spr/layout-types-lie`), but for
`SchoolYear`: the layout's per-tab data dictionary is not a reliable source
of column types. The builder's `s()` helper coerces every identity column to
a stripped string regardless of what the layout claims.

### Rate cells are strings, with prose suppression and capped extremes mixed in

```toml ergo
[issue]
id = "rate-prose-suppression"
title = "Rate cells carry % signs, prose suppression sentences, and >90%/<10% privacy caps"
effect = "breaks"
core = true
type = "suppression"
status = "monitor"
discovered = "2026-06"
detection = "value fails float() after stripping a trailing % — includes '>90%', '<10%', and prose sentences"
misuse = "Treating suppression NULLs as zero. Capped extremes also go NULL — fine for All Students (they only hit tiny subgroups), a real loss for small subgroups."

[issue.scope]
tables = ["grad_rate", "assessment", "assessment_grade", "absenteeism"]
columns = ["*Rate*", "*Percent*"]
years = "all"
```

Parse rule (`rate()` in the builder): a value is a number iff it parses
after stripping a trailing `%`; everything else → NULL. Four distinct
suppression sentences observed in the 2024-25 files: "Fewer than 10 students
were in the graduation cohort.", "Fewer than 10 valid scores", "Enrollment
for this group was less than 10 students.", "There is no data available for
this school year." — plus the two extreme caps, `">90%"` and `"<10%"`.

### Two graduation-rate calculations, one populated trend

```toml ergo
[issue]
id = "grad-state-vs-federal"
title = "State and Federal 4-yr graduation rates coexist; only State is populated across years"
effect = "misleads"
type = "measurement"
status = "monitor"
discovered = "2026-06"
detection = "4YrGraduationRateState_District non-empty in 27,115 of 27,201 trend rows; …Federal_District in only 5,423 (≈ one year)"
misuse = "Drawing the Federal rate as a trend line (it has one point), or comparing a State figure against another state's federal ACGR — NJ's State calc runs higher (SOMSD 2024-25: State 93.2% vs Federal 89.4%)."

[issue.scope]
tables = ["grad_rate"]
columns = ["4YrGraduationRate*", "5YrGraduationRate*"]
years = "all"
```

The builder keeps `grad4` (State calc) and `grad4_federal` (latest-year-only
ACGR) as separate columns rather than conflating them. `mod_graduation`
plots the State trajectory and headlines the latest-year Federal figure
separately, with the difference noted in the caption.

### Big monolithic workbooks need streaming, not a full load

```toml ergo
[issue]
id = "monolithic-workbook-size"
title = "The school-level workbook is ~330 MB with ~80 tabs — a full non-streaming load is impractical"
effect = "breaks"
type = "format"
status = "monitor"
discovered = "2026-06"
detection = "Database_SchoolDetail.xlsx is 346,029,712 bytes (330 MB); Database_DistrictStateDetail.xlsx is 119,164,900 bytes (114 MB)"

[issue.scope]
all = true
```

One monolithic workbook per year packs ~80 report tabs. Loading the whole
thing with openpyxl in default mode is heavy; the builder's `read_tab()`
opens with `read_only=True` and pulls only the sheet it needs.

### From 2024-25, only the xlsx is published

```toml ergo
[issue]
id = "new-format-2024-25-xlsx-only"
title = "From 2024-25 on, NJDOE drops the zip, Access, and Reference Guide PDF variants"
effect = "context"
type = "availability"
status = "open"
discovered = "2026-06"
detection = "download.js sets isNewYearFormat = year >= '2024-2025'; for those years only Database_*.xlsx is linked"
misuse = "Expecting a Reference Guide PDF, zip, or Access variant for 2024-25 and later — from 2024-25 on NJDOE publishes .xlsx only."

[issue.scope]
years = "2024-25 →"
```

Earlier years (`2016-17 →`) offer `.zip` (zipped xlsx) and `.accdb` (Access)
alongside the xlsx, plus a Reference Guide PDF, DataPrivacyRules PDF, and
FAQs PDF under `…/Documents/<YEAR>/`. `2015-16` uses a special two-column
layout; `2011-12 … 2014-15` are legacy and live outside this URL scheme
entirely — at a location that is **not** the one this page used to name
(`spr/legacy-archive-link-is-dead`).

### The legacy-years archive link is dead

```toml ergo
[issue]
id = "legacy-archive-link-is-dead"
title = "The legacy-years archive link is dead, and the live location is elsewhere"
effect = "breaks"
type = "availability"
status = "resolved"
discovered = "2026-07-30"
detection = "https://nj.gov/education/schoolperformance/archive/ returns 404, as do every year path beneath it and a PDF still indexed there; https://www.nj.gov/education/schoolperformance/ itself returns 200"
misuse = "Sending a reader to nj.gov/education/schoolperformance/archive/ for 2011-12 through 2014-15, which 404s, and concluding the material is gone."

[issue.scope]
years = "2011-12 → 2014-15"
```

This page sent readers to `https://nj.gov/education/schoolperformance/archive/`
for `2011-12 … 2014-15`. **That URL is a 404**, and so is every year path under
it and a PDF search engines still index there. The parent
`…/schoolperformance/` still resolves; only the archive branch is gone.

The material for those years does still exist, under Title I accountability
rather than school performance. It is the **ESEA Flexibility waiver
accountability archive**, five annual zips covering 2012 through 2016:

| year | file |
|---|---|
| 2012 | `https://www.nj.gov/education/title1/accountability/docs/archive/12.zip` |
| 2013 | `…/13.zip` |
| 2014 | `…/14.zip` |
| 2015 | `…/15.zip` |
| 2016 | `…/16.zip` |

Index page: <https://www.nj.gov/education/title1/accountability/>. Note that
the `docs/archive/` **directory** itself 404s — only the individual zips
resolve, so link the index or the files, never the folder.

Two cautions before treating this as a replacement. It is not the same data:
these are the state's accountability *determinations* by subgroup — met or
missed participation, met or missed the progress target — not the School
Performance Report tabs. And `11.zip`, `17.zip` and `18.zip` return 404, so the
recoverable span is 2012 to 2016 and stops where the waiver did.

Aquifer holds the five archives in append-only custody and publishes them as
`nj-esea-waiver-accountability-profiles`, with the per-year semantics recorded —
which matters here, because `YES*` in those files does not mean the school met
its target.

### `GraduationCohortProfile`'s Text-typed columns are a suppression tell

```toml ergo
[issue]
id = "cohort-profile-text-suppression-tell"
title = "NonContinuing/Persisting columns on GraduationCohortProfile are typed Text while their siblings are Numeric"
effect = "breaks"
type = "suppression"
status = "open"
discovered = "2026-06"
detection = "NonContinuing_District and Persisting_District are typed Text in the layout workbook; every other GraduationCohortProfile metric column is Numeric"

[issue.scope]
tables = ["GraduationCohortProfile"]
columns = ["NonContinuing_District", "Persisting_District"]
years = "all"
```

`GraduationCohortProfile` (graduated / continuing / non-continuing /
persisting) is a richer follow-on to the Trends tab, not yet loaded into the
builder. The Text typing on two of its columns — while every sibling metric
is Numeric — is the same tell as the rate columns elsewhere: these carry
suppression markers as strings, and will need the same `rate()`-style
handling whenever this tab is built.

### Trend tabs in the current file only go back to COVID

```toml ergo
[issue]
id = "trend-tabs-post-covid-only"
title = "The 2024-25 file's trend tabs start at 2020-21 or 2021-22, not earlier"
effect = "context"
type = "coverage"
status = "open"
discovered = "2026-06"
detection = "GraduationRateTrends carries 2020-21…2024-25 (five years); ELA/MathParticipationPerformance, PerformanceByTest, and StudentGrowthTrends carry 2021-22…2024-25 (four years) in the 2024-25 file"
misuse = "Assuming the current file's trend tabs cover full history — pre-COVID NJSLA (2018-19 and earlier, with the no-spring-2020 testing gap) requires pulling older years' files separately; deferred in this builder."

[issue.scope]
tables = ["grad_rate", "assessment", "assessment_grade", "absenteeism", "growth"]
years = "2024-25 file only shows 2020-21→2024-25 (grad) / 2021-22→2024-25 (assessment, absenteeism, growth)"
```

Each year's workbook carries a rolling window (five years for graduation,
four for assessment/absenteeism/growth), not the full published history.

**Resolved for graduation and chronic absenteeism, 2026-07-31.** The AQ-090
releases add a second artifact per family: a source-shaped census of all ten
editions NJDOE has published, so the older years no longer have to be pulled by
hand. Graduation now runs the classes of 2013 to 2025 and chronic absenteeism
2015-16 to 2024-25; assessment and growth still show the current workbook's
window only. The issue stays open because it is still true of those two.

### Ten editions are one measure, and that had to be proved before it was drawn

```toml ergo
[issue]
id = "the-nine-earlier-editions-are-the-same-measure"
title = "Older editions republish the same figures, and the overlap is the proof"
effect = "context"
type = "measurement"
status = "mitigated"
discovered = "2026-07-31"
detection = "the 2024-25 trend tab and each earlier year's own edition agree on 11,952 of 11,953 shared chronic-absence rows, and on 6,620 of 6,620 shared graduation rows"
misuse = "Splicing two publisher tabs into one line because they carry the same column name, without checking that they carry the same number where they meet."
instead = "Keep the overlap check in the adapter. A tab that stops agreeing is a different measure and must break the line rather than extend it."

[issue.scope]
tables = ["absence_history", "grad_rate_history"]
```

The edition census is **source-shaped and says so**: it preserves NJDOE's own
column names and asserts no cross-edition metric identity. Deciding that
`SchoolChronicAbsenteeism` (2015-16), `Chronic_Abs_Pct` (2017-18 to 2023-24) and
`ChronicAbsenteeismRate_School` (2024-25) are one measure is this repo's call,
and the adapters fail closed rather than take it on trust.

### The edition census is not a conformed mart

```toml ergo
[issue]
id = "edition-census-is-source-shaped-not-conformed"
title = "The edition census preserves publisher columns and asserts no metric identity"
effect = "context"
type = "measurement"
status = "mitigated"
discovered = "2026-07-31"
detection = "the artifact's own metadata: 'This layer asserts no cross-edition metric identity; mapping an edition's columns onto another edition's metrics is a conformance decision.'"
misuse = "Reading the census as though it were already conformed, and joining columns across editions because they sound alike."
instead = "Project it through the consumer's edition layer, which names the shape of every edition and refuses one it does not recognise."

[issue.scope]
tables = ["absence_history", "grad_rate_history", "absence_edition", "grad_rate_edition", "absenteeism_trend_edition"]
```

### The chronic-absenteeism trend tab is only two editions old

```toml ergo
[issue]
id = "the-trend-tab-is-two-editions-old"
title = "Only the last two editions publish a chronic-absenteeism trend tab at all"
effect = "context"
type = "coverage"
status = "mitigated"
discovered = "2026-07-31"
detection = "the census marks chronic-absenteeism-trends not-collected-this-edition in 15 of 20 edition/grain cells; the tab first appears in 2023-24"
misuse = "Concluding that New Jersey has only measured chronic absenteeism since 2021-22, when it is the trend *tab* that is new. The longer record is in each edition's own student-group snapshot."
instead = "Read the register in absenteeism_trend_edition, and take the earlier years from absence_history."

[issue.scope]
tables = ["absenteeism", "absenteeism_trend_edition"]
```

### Chronic absenteeism was not collected in 2019-20

```toml ergo
[issue]
id = "chronic-absenteeism-was-waived-in-2019-20"
title = "New Jersey collected no chronic absenteeism for 2019-20, by waiver"
effect = "corrupts"
type = "coverage"
status = "mitigated"
discovered = "2026-07-31"
detection = "the census marks 2019-2020 not-collected-this-edition at both grains; the edition ships no ChronicAbsenteeism sheet of any kind"
misuse = "Drawing a line straight through 2019-20, or reading a blank or a zero as a year in which nobody missed school."
instead = "Render the year as a labelled band carrying NJDOE's own words: 'The requirement to report on chronic absenteeism and related accountability measures for the 2019-2020 school year was waived. As a result, NJDOE will not report on chronic absenteeism and other attendance data for 2019-2020. Tables showing attendance data will be hidden in the 2019-2020 reports.' — 'Important 2019-2020 Notes' sheet, Database_DistrictStateDetail.xlsx, retrieved 2026-07-31."

[issue.scope]
tables = ["absence_history"]
years = ["2019-20"]
```

### The pandemic editions carry per-element cautions, in different words each

```toml ergo
[issue]
id = "pandemic-editions-carry-per-element-publisher-cautions"
title = "NJDOE cautions 2019-20 and 2020-21 element by element, not once"
effect = "misleads"
type = "measurement"
status = "mitigated"
discovered = "2026-07-31"
detection = "the 'Important 2019-2020 Notes' and 'Important 2020-2021 Notes' sheets inside the workbooks carry a status and a note per data element; graduation and chronic absenteeism each get their own, and neither matches the discipline wording"
misuse = "Reusing the discipline caution ('Due to school closures in spring 2020, discipline data ... may not be comparable') for graduation or attendance. It is not what the publisher says about either, and for 2019-20 absenteeism there is nothing to caution because nothing was collected."
instead = "Quote the element's own note. Chronic absenteeism 2020-21: 'because the number of days students spent in in-person and remote learning environments during the school year differed, the NJDOE recommends caution in comparing 2020-2021 attendance data between schools and districts and to prior or future school years.' Graduation 2019-20 and 2020-21: the graduation assessment requirement was waived for the classes of 2020 and 2021, and annual targets were not reported. Both retrieved 2026-07-31 from the workbooks' notes sheets."

[issue.scope]
tables = ["absence_history", "grad_rate_history"]
years = ["2019-20", "2020-21"]
```

### Graduation's rate-type column is missing from the release

```toml ergo
[issue]
id = "graduation-rate-type-is-recovered-from-the-statewide-column"
title = "Six editions lose GraduationRateType, and it is recovered rather than guessed"
effect = "corrupts"
type = "measurement"
status = "mitigated"
discovered = "2026-07-31"
detection = "the 2018-19 to 2023-24 workbooks carry a GraduationRateType column ('4-Year Rate' / '5-Year Rate'); the AQ-090 census records it in dimension_columns_json but source_rows.dimension_label carries only CohortYear, so those editions arrive with two indistinguishable rows per cohort"
misuse = "Taking either row as the four-year rate, or dropping six editions and rendering the classes of 2019 and 2020 as a hole New Jersey never published."
instead = "Recover the rate type from StatewideGraduationRate, which the same publisher prints on the same row: a cohort's statewide four-year rate is the literal carried by the newest cohort in an edition, the only one there with a single row. Check it forwards against the conformed 2024-25 mart and backwards against the 2017-18 edition's unambiguous 4YR_Rate."

[issue.scope]
tables = ["grad_rate_history"]
years = ["2018-19", "2019-20", "2020-21", "2021-22", "2022-23", "2023-24"]
```

### The All Students row is labelled four different ways

```toml ergo
[issue]
id = "the-all-students-row-is-labelled-four-ways"
title = "Chronic absence labels its all-students row four ways across ten editions"
effect = "corrupts"
type = "identity"
status = "mitigated"
discovered = "2026-07-31"
detection = "2015-16 has no student-group dimension and its row is unlabelled; 2016-17 to 2023-24 use 'Schoolwide' / 'Districtwide'; 2024-25 uses 'All Students'"
misuse = "Filtering on 'All Students' and silently losing eight editions off the chart."
instead = "Enumerate the labels. They are publisher facts, not values to normalise away."

[issue.scope]
tables = ["absence_history"]
```

### No statewide chronic-absence rate before 2021-22

```toml ergo
[issue]
id = "no-statewide-absence-rate-before-2021-22"
title = "Only the 2024-25 edition prints a statewide chronic-absence comparator"
effect = "context"
type = "coverage"
status = "mitigated"
discovered = "2026-07-31"
detection = "ChronicAbsenteeismRate_State exists in the 2024-25 edition only; earlier editions publish Target and MetTarget but no statewide rate, and no state entity row"
misuse = "Computing a statewide average from the district rows to fill the earlier years, and drawing it as though New Jersey had published it."
instead = "Let the state line start where the state started printing it, and say so in the caption."

[issue.scope]
tables = ["absence_history"]
```

### Two 2024-25 tabs disagree for one school

```toml ergo
[issue]
id = "two-2024-25-tabs-disagree-for-one-school"
title = "The 2024-25 student-group and trend tabs disagree for Bayonne Alternative High"
effect = "context"
type = "measurement"
status = "open"
discovered = "2026-07-31"
detection = "district 0220 school 010, 2024-25: ChronicAbsenteeismStudentGroup prints 90.0%, ChronicAbsenteeismTrends leaves the year blank. Both tabs agree on 87.4 for 2023-24. One row in 11,953."
misuse = "Widening the overlap tolerance until this stops failing, which would also stop the check catching a real divergence."
instead = "Pin it by name in ABSENCE_KNOWN_TAB_DISAGREEMENTS. It is the publisher disagreeing with itself inside one workbook, and if it stops reproducing the pin is stale and must be re-read."

[issue.scope]
tables = ["absence_history"]
years = ["2024-25"]
```

### Older editions join on a code, not on a verified school

```toml ergo
[issue]
id = "pre-2020-editions-join-on-an-unverified-code"
title = "Before 2020-08-11 a resolving CDS code proves the code, not the school"
effect = "misleads"
type = "identity"
status = "mitigated"
discovered = "2026-07-31"
detection = "the release's identity_resolution table marks every edition before 2020-08-11 outside its evidence window, with the caveat that a code which resolves 'says the tuple is a current NJDOE organization, and says nothing about what that tuple named in this edition's school year'"
misuse = "Presenting a ten-year line as one school's history when its older end is matched only by a code New Jersey may have reused."
instead = "Say which points are outside the evidence window, on the page, beside the chart."

[issue.scope]
tables = ["absence_history", "grad_rate_history"]
```

### The oldest graduation edition rounds to whole percent

```toml ergo
[issue]
id = "the-oldest-graduation-edition-rounds-to-whole-percent"
title = "2015-16 publishes graduation rates as bare whole numbers on a different sheet"
effect = "context"
type = "measurement"
status = "mitigated"
discovered = "2026-07-31"
detection = "the 2015-16 edition's ExtYearGradRate sheet prints '96', not '96.3'; every value it shares with a later edition is within 0.5 of that edition's one-decimal figure (806 shared school-cohort rows, max difference 0.5)"
misuse = "Reading a step between the 2013 and 2015 classes as a change in graduation when it is a change in rounding."
instead = "State the precision where it applies. It is the same measure at coarser precision, not a different one."

[issue.scope]
tables = ["grad_rate_history"]
years = ["2012-13", "2013-14"]
```

### Aggregating grade-tests silently biases the number

```toml ergo
[issue]
id = "grade-aggregation-bias"
title = "Summing grade-test proficiency into a school-wide figure is biased by suppression and course-mix — a capped Algebra I once inflated one"
effect = "misleads"
type = "measurement"
status = "monitor"
discovered = "2026-06"
detection = "a capped Algebra I test (values suppressed to '<10%'/'>90%') once inflated a home-grown school-wide aggregate by silently dropping its lowest cohort from the average"
misuse = "Aggregating assessment_grade's per-grade-test Level 1-5 distributions into a school-wide number instead of using NJ's own published school-wide MetExceededExpectations (table assessment) — suppression and incomparable course populations (Algebra I vs II) bias the result."

[issue.scope]
tables = ["assessment_grade"]
columns = ["l1", "l2", "l3", "l4", "l5", "proficiency", "prof_str"]
years = "all"
```

`build_by_test` keeps the per-grade-test grain and never aggregates across
grade-tests; NJ never publishes a school-wide 1-5 distribution, only per
test, and that's the honest grain. The school-wide headline % comes only
from NJ's own published `MetExceededExpectations` (table `assessment`),
never a home-grown recomputation.

### High schools have no Student Growth Percentile

```toml ergo
[issue]
id = "hs-no-sgp"
title = "StudentGrowthTrends has no rows for standalone high schools — SGP only exists for grades 4-8"
effect = "context"
type = "universe"
status = "open"
discovered = "2026-06"
detection = "growth table has zero rows for a standalone high school's own school_code; district/state context columns are still populated"
misuse = "Reading an absent growth row for a high school as missing or bad data — SGP is defined only for grades 4-8 by NJ's own methodology; high schools need a proficiency-only frame with no cohort lens."

[issue.scope]
tables = ["growth"]
entities = "standalone high schools"
years = "all"
```

Median Student Growth Percentile is NJ's demographic-independent "is it
adding value" signal (1-99, 50 = typical, with a Low/Typical/High label),
but it's computed only for grades 4-8. The profile module treats growth as
the lead metric where it exists and falls back to a proficiency frame — with
no cohort lens, since a single grade can't be traced — for high schools.

### Math cohort tracking only works for grades 3-8

```toml ergo
[issue]
id = "math-cohort-grades-3-8-only"
title = "The cohort-diagonal view (a class followed as it moves up) only works for grades 3-8 math"
effect = "context"
type = "coverage"
status = "open"
discovered = "2026-06"
misuse = "Trying to trace a high school math cohort year-over-year the way grades 3-8 are traced — HS course tests (Algebra I/II, Geometry, etc.) aren't a clean grade+1 sequence, so students in 'Algebra I' this year aren't reliably in 'Algebra II' next year."

[issue.scope]
tables = ["assessment_grade"]
rows = "grade_test rows for HS course tests (Algebra I/II, Geometry, etc.)"
years = "all"
```

Lens B of the assessment module ("by class, followed as it moved up,"
cohort diagonals of the grade × year matrix) is offered only for grades 3-8
math. High school course tests don't form a clean sequence, so no cohort
lens is offered for them.

### A school's headline proficiency reflects only its tested grades

```toml ergo
[issue]
id = "grade-mix-context"
title = "District/state context blends all tested grades; a school's value covers only its own"
effect = "context"
type = "definitional"
status = "open"
discovered = "2026-06"
misuse = "Reading a high school's math proficiency sitting below its district as a deficit — the district figure blends grades 3-8 with HS course tests over different populations."

[issue.scope]
tables = ["assessment", "assessment_grade"]
```

Real, not a bug: comparisons across aggregation levels compare different
grade mixes. Per-grade views (`assessment_grade`) are the honest grain.

## Validation

```toml ergo
[validation]
date = "2026-06"
method = "confirmed from real data — 2024-25 District/State file"
result = "codes are zero-padded TEXT (county 2-wide, district 4-wide) despite the layout's 'Numeric' declaration; join to enrollment is clean on (district_code, school_code, school_year) with no coercion"
```

```toml ergo
[validation]
date = "2026-06"
method = "confirmed from real data — 2024-25 District/State file"
result = "SchoolYear cells read as the string '2020-21' regardless of a tab's declared type, matching the warehouse's SchoolYear format"
```

```toml ergo
[validation]
date = "2026-06"
method = "confirmed from real data — 2024-25 District/State file"
result = "All-students label is 'All Students'. Full StudentGroup set: All Students, White, Black or African American, Hispanic/Latino, Asian Native Hawaiian or Pacific Islander, American Indian or Alaska Native, Two or More Races, Male, Female, Non-Binary/Undesignated Gender, Economically Disadvantaged Students, Multilingual Learners, Students with Disabilities, Migrant Students, Military-Connected Students, Students Experiencing Homelessness, Students in Foster Care"
```

```toml ergo
[validation]
date = "2026-06"
method = "confirmed from real data — 2024-25 District/State file"
result = "suppression is a prose sentence, not a flag; non-numeric metric values are prose suppression plus >90%/<10% extreme-caps; rates carry a literal %; all map to NULL via rate()"
```

```toml ergo
[validation]
date = "2026-06"
method = "confirmed from real data — 2024-25 District/State file"
result = "assessment + absenteeism trend tabs run 2021-22 → 2024-25 (post-COVID only); pre-COVID NJSLA needs older-year files (deferred)"
```

```toml ergo
[validation]
date = "2026-06"
method = "confirmed from real data — 2024-25 District/State file"
result = "assessment district/state context blends all tested grades; a school's value reflects only its own tested grades (grade-mix nuance)"
```

## Provenance

Latest published: 2024-25 (released May 2026); databases downloadable back
to 2015-16 in this URL scheme. The 2011-12 → 2014-15 legacy years are not here
and the archive link this page used to name is dead — the live location is the
Title I ESEA waiver accountability archive
(`spr/legacy-archive-link-is-dead`). Our working copy: 2024-25, fetched
2026-06; record re-pulls here.

## Changelog

```toml ergo
[change]
date = "2026-08-03"
note = "Consolidated the five conformed SPR projections and absence/graduation edition records behind one complete family adapter. Advanced the assessment headline, assessment-by-test, and growth pins from missing baseline directories to retained refresh-2 releases whose conformed source-table bytes match the prior exact pins. The consumer assessment mart now also projects literal cells and physical workbook provenance from the pinned typed tables, restoring participation-aware answers in a cold offline rebuild without a local workbook parser."
```

```toml ergo
[change]
date = "2026-07-30"
note = "Registered spr/legacy-archive-link-is-dead: the nj.gov/education/schoolperformance/archive/ link this page gave for 2011-12 to 2014-15 returns 404, as does every year path under it. Repointed at the live Title I ESEA Flexibility waiver accountability archive (12.zip through 16.zip), with the caveat that it holds accountability determinations rather than SPR tabs and stops at 2016."
```

```toml ergo
[change]
date = "2026-07-10"
note = "Converted to ergo format: 13 issues registered from the oddities list and tab notes; confirmed-from-real-data log became validation records."
```