coreval

Check your SDTM data against CDISC rules without leaving R.

You just finished writing your DM code. You have a data frame. Check it:

library(coreval)

check_dataset(dm)

── coreval — DM ────────────────────────────────────────────────────────────

9 problems across 6 records  (170 checks ran)

  wrong value         4   the data breaks the rule - start here
  missing required    2   the standard requires it
  missing optional    3   often legitimate: not collected, screen failure, ...

[wrong value]
Variable value is not in correct ISO 8601 date or datetime format
  2 records · RFSTDTC
    row 4     RFSTDTC = "2024-13-01"
    row 3     RFSTDTC = (empty)
    CORE-000547  · also SEND66, SEND67, SEND68, ...

[wrong value]
AGEU is missing when AGE is provided.
  1 record · AGE, AGEU
    row 3     AGE = "47", AGEU = (empty)
    CORE-000189  · also CG0665, TIG0699

  ... and 7 more here. See result$findings for all of them.

────────────────────────────────────────────────────────────────────────────
45 checks could not run.
  30 need other datasets (AE, AG, CM, DD, DS, EX, ...)
     → run check_study() on the whole folder to cover these
  15 need a define.xml

No standard declared, so rules from every standard ran.
  Narrow with  standard = "SDTMIG"  (or "SENDIG", "TIG", ...)

Fix what you can, then run this again.
To track the rest:  write_findings(result, "issues.xlsx")

It tells you what's wrong in words, which rows, which variables, and the actual values. No export, no upload, no waiting, no looking rule numbers up in a PDF.

About that ordering

CDISC Open Rules carry no severity field. I checked the source — there's nothing like Pinnacle 21's Notes / Minor / Major / Critical. That's P21's own layer, not CDISC's, so coreval can't report a CDISC severity and won't invent one.

What it does instead is separate the findings that are definitely wrong from the ones that might be perfectly fine:

That last row is the point. An empty RFSTDTC is not the same kind of problem as RFSTDTC = "2024-13-01", and sorting by "how many rows are affected" puts them in the wrong order. coreval sorts by this first, row count second — and within a problem, it shows you the row with the real bad value before the row that's merely empty.

This is coreval's own triage, not a regulatory grading. It's a triage column on every finding, so you can sort by it in the spreadsheet too.

The workflow this is built for

You're a programmer. You've just written the code that builds DM. You don't want to export transport files and open Pinnacle 21 to find out you left a month of 13 in a date.

result <- check_dataset(dm)   # 1. see what's wrong, in plain language
                              # 2. fix what you can
result <- check_dataset(dm)   # 3. run it again — it takes a second

write_findings(result, "dm_issues.xlsx")   # 4. track what's left

Step 4 gives you a spreadsheet with the problem described in words, plus empty Status, Owner and Notes columns to fill in — so "expected, see protocol deviation log" gets recorded next to the finding instead of in some other document.

Then the full run still happens: you or your study lead runs the whole study through your qualified tool. This doesn't replace that, and isn't trying to. It just means the expensive check finds far less, and you found the obvious things in seconds instead of half an hour.

Install

# install.packages("pak")
pak::pak("hrach-gevorgyan/coreval")

Needs R 4.1 or newer. Only data.table and haven to run. Two optional extras:

install.packages("xml2")     # to read Define-XML
install.packages("writexl")  # to write .xlsx

Checking one dataset

This is the one to reach for while you're writing code. Pass a data frame:

ae <- data.frame(
  STUDYID = "S1", DOMAIN = "AE", USUBJID = c("01", "01"),
  AESEQ   = c(1, 2),
  AETERM  = c("Headache", "Rash"),
  AESTDTC = c("2024-01-10", "2024-02-30")
)

result <- check_dataset(ae)

Or a file — .xpt, .sas7bdat or .csv:

result <- check_dataset("ae.xpt")

coreval works out the domain from your DOMAIN column, and falls back to the file name only when the data has no DOMAIN column at all. That order matters for a split dataset: ae1.xpt is checked as AE because its DOMAIN column says so — on a file with no DOMAIN column, the name ae1 is taken at face value. If it guesses wrong, tell it:

result <- check_dataset(ae, domain = "AE")

The catch, and it's an important one

Plenty of CDISC rules compare one dataset against another — an AE date against the subject's reference dates in DM, a visit against the trial design. Hand coreval a single dataset and those rules simply cannot be answered.

coreval doesn't guess. It skips them, and tells you which dataset it wanted:

head(result$skipped, 3)
#>       rule_id domain                                        reason
#> 1 CORE-000138     AE  needs DM, which was not supplied - check ...
#> 2 CORE-000140     AE  needs TV, which was not supplied - check ...
#> 3 CORE-000168     AE  needs SV, which was not supplied - check ...

Running them anyway would compare your data against columns that aren't there and report problems that don't exist. Better to say nothing than to make something up.

Most rules do still run — measured across AE, DM, LB and VS, 76–84% of the applicable rules work on the dataset alone. But the ones that can't are the cross-dataset checks, and those often matter most.

A short findings list here doesn't mean your data is clean. It's a quick first pass while you code. Run the whole study before you draw conclusions.

Checking a whole study

When you do have the full folder:

result <- check_study("path/to/study/sdtm")

A folder, not a file. coreval reads everything in it — XPT, SAS, CSV, whichever you have — and reading it all at once is what makes the cross-dataset rules possible. If there's a Define-XML in there, it finds it and uses it.

If you want to look at what was parsed, or check the same large study twice without re-reading it, do the read yourself:

study <- read_study("path/to/study/sdtm")
names(study$datasets)
#> [1] "AE" "CM" "DM" "EX" "LB" "VS"

check_study(study)

A study report is grouped by dataset, and tells you where the trouble is before showing you any detail:

── coreval ─────────────────────────────────────────────────────────────

27 problems across 12 records in 4 datasets  (689 checks ran)

  DM           9 problems    4 records
  AE           7 problems    4 records
  VS           7 problems    3 records
  STUDY        4 problems    1 record

── DM ──────────────────────────────────────────────────────────────────

SUBJID is not unique within study
  3 records · SUBJID
    not in the dataset: SUBJID
    CORE-000186
...

print(result, n = 20) shows more problems per dataset; rows = 5 shows more example records per problem.

Reading the results

You get two tables. Both matter.

Printing the result gives you the readable report above. When you want the raw rows — to filter, count, or feed somewhere else — they're in result$findings:

result$findings — what's wrong

head(result$findings[, c("Dataset", "Record", "Variable", "Value", "triage", "rule_id")])
#>  Dataset Record Variable          Value           triage     rule_id
#>       AE      2  AESTDTC     2024-02-30      wrong value CORE-000547
#>       AE      2  RFSTDTC Not in dataset      wrong value CORE-000547
#>       AE      1   AESTDY Not in dataset missing optional CORE-000328

(issue is dropped from that view only so the table fits the page — it is there on every row, and it is the column worth reading.)

One row per problem, pointing at the exact spot:

Not in dataset under Value means the rule wanted a variable you don't have — which is usually the point.

result$skipped — what couldn't be checked

head(result$skipped)
#>       rule_id domain                                    reason
#> 1 CORE-000916     AE  Match Datasets: unsupported join type...

This is the one people skip, and it's the one that bites. An empty findings table means one of two things: your data is clean, or half the rules never ran. Those look identical if you only read findings. coreval always shows you both.

Saving it, and tracking what you didn't fix

write_findings(result, "issues.xlsx")   # one workbook, several sheets
write_findings(result, "issues.csv")    # issues.csv + siblings

You get findings, skipped, an about sheet, and truncated if any rule matched more records than were kept. Both tables get written every time, for the reason above — and about carries the provenance with the file: which standard it was scoped to, how many checks ran, whether it was filtered before export, and whether any counts were capped. A shared spreadsheet outlives the console session that made it, and whoever opens it can't see what you saw.

The file has three empty columns — Status, Owner, Notes — for you to fill in once it's open. Not every finding is a bug you'll fix: some are expected, some belong to someone else, some are waiting on a data query. Those decisions belong next to the finding, not in a separate document nobody opens.

Dataset  Record  Variable  Value        issue                          rule_id      Status   Owner  Notes
DM       4       RFSTDTC   2024-13-01   Variable value is not in ...   CORE-000547
DM       3       AGE       47           AGEU is missing when AGE ...   CORE-000189

Pass tracking = FALSE if you're reading the file back into R and don't want the extra columns.

The whole API

Six functions. Three of them do the work:

The other three are there when you need them:

Plus print() and summary() on a result, which you get by typing the result's name.

That's it. If you only ever learn check_dataset() and write_findings(), you have most of the value.

Recipes

What does CORE-000547 actually mean?

rule <- list_rules(id = "CORE-000547")

rule$issue
#> [1] "Variable value is not in correct ISO 8601 date or datetime format"
rule$legacy_ids
#> [1] "SEND66, SEND67, SEND68, TIG0267, TIG0268, TIG0269"
rule$guidance
#> [1] "The SENDIG requires dates and times of day to be stored according to the
#>     international standard ISO 8601  (SENDIG v3.0 4.4)"
rule$standard
#> [1] "SENDIG, SENDIG-DART, SENDIG-GENETOX, TIG"

list_rules() always returns a data frame with one row per rule, so pick the columns you want off it. (t(rule) gives the whole row as a column, which is often easier to read for a single rule.)

Three things worth knowing here:

• legacy_ids are what Pinnacle 21 and the published Conformance Rules spreadsheets call the same rule. That's how you match a coreval finding to a P21 report — including to a severity CDISC itself doesn't publish. The console report shows them too: CORE-000189 · also CG0665, TIG0699.
• guidance is the sentence from the Implementation Guide the rule exists to enforce — the why, which no rule message carries. print(result, guidance = TRUE) shows it under each problem; it's off by default because it roughly doubles the report's length.
• All 797 rules carry both.

Just the things that are definitely wrong

filter_findings(result, triage = "wrong value")

Returns a result, so it prints as a report. Also takes dataset, rule and variable.

A three-line summary, for a script

summary(result)
#> 5 problems across 4 records  (111 checks ran, 21 could not)
#>   wrong value       2
#>   missing required  1
#>   missing optional  2

Which rules even apply to AE?

list_rules(domain = "AE")

Only the rules for my standard and IG version

result <- check_dataset(dm, standard = "SDTMIG", version = "3.4")

Rules are written per Implementation Guide version, so this genuinely narrows what runs — for DM: 134 rules for SDTMIG generally, 96 for 3.2, 132 for 3.4.

Worth knowing: this genuinely narrows what runs, and the report tells you how many rules it set aside. CDISC's coverage is uneven — the general "dates must be valid ISO 8601" rule is published for SENDIG and TIG but not for SDTMIG — so narrowing can mean a real problem stops being reported. Leave standard unset if you would rather see everything.

Only the fully-vetted rules, no drafts

subset(list_rules(), source == "published")

Which rule version am I running?

attr(list_rules(), "rules_version")
#> [1] "b540283d85e88fb8ee5f08ead5f03fac73eb1b8b"

write_findings() records it in every exported file, so you rarely need to ask.

That's the exact CDISC commit the bundled rules came from. Worth recording next to your results.

Just the AE findings

subset(result$findings, Dataset == "AE")

What's failing most?

sort(table(result$findings$rule_id), decreasing = TRUE)

How many records are actually affected?

nrow(unique(result$findings[, c("Dataset", "Record")]))

What's covered, and what isn't

CDISC publishes its rules in the open. Their repository has 1,348 rules in it. coreval ships 797. Here's exactly where the other 551 went, and why.

First, how anyone knows a rule works

For most rules, CDISC publishes three things: the rule itself, some example data, and an answer sheet — a file saying which exact rows a correct implementation should flag in that example.

That answer sheet is everything. It's how I prove my version of a rule does what CDISC's does, instead of just believing it. Every rule in coreval is run against CDISC's own examples and compared row by row.

Some rules ship without one. CDISC publishes the rule, sometimes even the example data, but never says what the right answer is. I can implement such a rule, and it will produce findings, and neither of us will have any way to know whether they're correct.

I don't ship those. A check you can't verify is worse than no check, because it looks exactly like a check that works.

Where the 1,348 rules go

Of the rules that are both readable and have an answer sheet, coreval has 767 of 767.

How many of the 797 are actually proven

Every rule falls into exactly one of four buckets. They add up to 797.

Of the rules that can be confirmed at all — 695 of 750, 93%.

The 37 nobody can confirm

One of those 6 deserves naming: CORE-000229 says RELSUB is a Special-Purpose dataset, while CDISC's own data model says it's a Relationship dataset. The two halves of its own scope can never both be true. I've left it alone rather than bend the model to fit — matching a rule against a class its own publisher disagrees with would be guessing.

These are never counted as passing. They're reported as skipped, by name, with the reason, every time you run.

The 10 blocked on data I don't ship

This is the honest "my fault" column, and it's 10 rules — all of them the same packaging decision, not bugs.

Put another way, the work that is actually left:

797 rules
 -37  nobody can confirm these, ever
 -10  blocked until I ship more reference data
 ---
 750  should end up confirmed
 695  are
 ---
  55  still to finish

The 55 that disagree

This is where the real work is, and I won't dress it up: for 55 rules, coreval flags different rows than CDISC's answer sheet says it should. 26 of them are deprecated rules that a check never actually runs, so 29 affect real output.

Every one of the 55 has been investigated individually and written down. 53 are cases where CDISC's example contradicts CDISC's own rule or is missing an answer it should have. The other 2 are split datasets: coreval finds the same problem but reports it against the file it is in, with that file's own row numbers, where CDISC reports it against the merged domain. That is deliberate — a finding has to point at a file you can open and a row you can find.

Some are bugs on my side. Others are cases where CDISC's own example data contradicts itself — a file whose stated answer doesn't match its own rows, usually because the data was edited after the answers were generated. Each one is investigated individually and written down rather than quietly ignored.

Either way, the honest reading is: treat a finding from those rules with more suspicion than the rest. That's why every finding carries its rule id — so you can look it up.

The ADaM question, since people ask

ADaM isn't missing because I skipped it. All 93 ADaM rules come with example data and zero answer sheets. Including them would mean shipping 93 checks that nobody — me, you, or CDISC — can confirm are right.

The day CDISC publishes answer sheets for them, they go in. Nothing else has to change.

Where it's genuinely weak

Three honest problems, none of them hidden from you at runtime:

• 9 rules need CDISC's terminology lists — the controlled vocabularies saying which codes are valid. That data is around 438 MB, which would turn a 0.6 MB install into a very large one, so it is not part of this release. Those 9 rules are reported as skipped, by name, with the reason, every time you run. They are never counted as passing.
• The agreement percentage is a floor, not a score. CDISC's examples are small and tidy. Real submissions are neither. Three separate bugs found in this package moved that percentage by exactly zero.

Keeping up with CDISC

The rules are pinned to one exact commit of CDISC's repository — recorded in the package, visible with attr(list_rules(), "rules_version"), and written into every file write_findings() saves. So a result is always traceable to the precise rule set that produced it.

Updating is deliberate, not automatic: a maintainer tool (data-raw/check_upstream.R) reports when CDISC's repository has moved, and re-pinning means re-extracting and re-running every rule against every example again. That way a new version can't silently change your results, and any rule that breaks shows up before release rather than in your data.

Not every rule carries the same weight either — list_rules() has a source column saying whether a rule is fully published, superseded, or still a draft.

How accurate is it?

CDISC publishes, for each rule, data that should trigger it, data that shouldn't, and the exact records their own engine flags. coreval replays all of it and compares record by record.

On published rules that ship reference data: 540 of 562, about 96%.

30 rules ship no reference data at all. CDISC publishes the rule but no examples, so there's nothing to compare against — they can't pass or fail. Counting them as failures understates things; hiding them overstates. So both are here.

Deprecated and draft rules are a weaker pool. Their examples predate CDISC's current conventions — some number records from the spreadsheet header row, so they expect a "record 5" in a four-row file. That's the example data being old, not coreval being wrong.

Most of the remaining disagreements are problems in the reference data, usually a file whose own stated values contradict its own rows — a sign the data was edited after the expected results were generated.

Please don't read 96% as a quality score. CDISC's examples are mostly simple, single-file datasets, so they don't exercise much of what real submissions do. I once found a bug that silently switched off a third of the rules on split-domain studies — it moved that number by exactly zero. It's a floor, not a ceiling. Which is the same reason the advice stays: run your qualified tool before you submit.

What this is, and isn't

I built coreval to stop wasting my own time, and I'm sharing it in case it saves yours.

It is an independent, personal open-source project. Not a CDISC product, not affiliated with CDISC, not endorsed by CDISC, and not a CORE-certified engine.

It is not qualified or validated software, and not a substitute for it. A clean run here doesn't mean your submission will be accepted, and a finding here doesn't mean it'll be rejected. It doesn't replace your organisation's own validation procedures.

What it is: a fast local check that catches the obvious problems while you're still writing the code, and that tells you honestly when it couldn't check something.

Status

Under active development, and the API may still change. Already useful for finding real problems in real data. See NEWS.md.

Contributing

Issues and pull requests welcome — especially a dataset that produces a wrong or missing finding. That's the most useful bug report there is. Please read the Code of Conduct first.

License

Package code is MIT (LICENSE.md). Bundled rule definitions come from cdisc-org/cdisc-open-rules and remain under CDISC's terms — see NOTICE.md.

Not affiliated with, endorsed by, or certified by CDISC.