Project 9076
Record schema
The structure every experiment record and extraction fragment is validated
against (machine-checked on every merge). Read straight from the files in
schema/ — this page always shows the current version.
FOIL 9076 experimental record
schema/record.schema.json updated 20 Aug 12:53One canonical record per experiment/lab batch. Built by merging fragments from all source documents. Finalized 2026-08-19 against the three primary sources: Old ELN PDFs (LAB JOURNAL v2.0 form), Project Expt Report table (28112025 9076 Expt Table, Step 1/2/3 layout), and the handwritten notebook OCR (9076 Exp 48-60). Every material value carries provenance. A scalar field may instead hold a conflict object when sources disagree.
| Field | Type | Description | |
|---|---|---|---|
| record_id | string | required | canonical id, e.g. exp-9076-048, exp-9076-024B, exp-9076-024C1 |
| project | always '9076' | required | |
| aliases | array of string | required | every ID string that refers to this experiment across the corpus (Lab Batch 24 B, LBN 24B, Batch-54A, NO-49, #48Q, ...) |
| stub | boolean | true when the record was created only to receive an unmatched fragment; needs linking review | |
| record_type | synthesis | blend | rework | unknown | synthesis = a lab reaction; blend = a letter-suffixed batch made by blending a parent batch's product with a solvent/additive (has NO synthesis parameters, only application testing — per the scientist's walkthrough, e.g. 22A = 22 + solvent, 21D = 21 + 2% AMP); rework = reprocessing of an earlier batch (e.g. 24CR) | |
| preparation | string | blend/rework recipe verbatim (e.g. 'LBN 21 + 2% AMP'); only for record_type blend/rework | |
| derived_from | array of string | parent batch record ids/aliases this blend/rework was made from | |
| date | conflictable_string | ||
| scientist | conflictable_string | ||
| aim | conflictable_string | ||
| eln_form | object | Experiment-level fields of the LAB JOURNAL v2.0 form that are metadata about the run rather than process data. | |
| └ journal_version | string | 'LAB JOURNAL - Version 2.0' banner | |
| └ equipment | string | 'Equipment Details' field, verbatim | |
| └ reaction_scheme | string | 'Reaction Scheme' field; often an image — store the printed text or 'image only' | |
| └ safety_precautions | string | ||
| └ other_fields | object | catch-all for rare/unexplained form labels (e.g. 'SPDu') so no printed field is ever dropped: label -> verbatim value | |
| materials | array of material | the ELN 'Input Raw Material' table (whole-experiment charge list); per-step charges go in process_steps[].materials | |
| process_steps | array of process_step | ||
| procedure_summary | string | ||
| conditions | object | experiment-level roll-up: the ELN 'Optimized Reaction Conditions' table (Temperature/N2-Vac/Time) or the summary export's T c/N2-Vac/Time columns | |
| └ temperature | conflictable_string | ||
| └ time | conflictable_string | ||
| └ atmosphere | conflictable_string | ||
| └ other | string | ||
| critical_process_parameters | array of critical_process_parameter | ||
| observations | string | ||
| samples | array of sample | ||
| analytical_summary | array of analytical_entry | experiment-level analytical results (e.g. amine value / acid value columns from the ELN summary) not yet tied to a specific sample id | |
| critical_parameters | array of measurement | the ELN 'Critical Parameters' table: parameter + reference (target) + measured value, phase 'target-check' | |
| conclusion | conflictable_string | verbatim Conclusion field (ELN form) / Conclusion column (summary export); outcome.status is derived from it | |
| outcome | object | ||
| └ status | ok | not ok | success | failure | partial | unknown | ||
| └ basis | string | verbatim statement the status came from | |
| └ citation | string | ||
| documents | array of linked_document | ||
| provenance | object | required | |
| └ fragments | array of string | required | fragment ids merged into this record |
| └ field_provenance | object | field path (dot/bracket notation) -> provenance | |
| └ last_merged | string | ||
| validation | object | ||
| └ validated_fields | object | field path -> {validated_by, date}. These fields are frozen: re-runs re-apply corrections.jsonl on top and never overwrite them. | |
| └ corrections_applied | array of string | correction ids from corrections.jsonl | |
| links | object | ||
| └ previous_experiment | string | null | ||
| └ next_experiment | string | null | ||
| └ related | array of string |
▶ Shared types used above (13) — conflict objects, provenance entries, …
conflictable_string string | conflict
conflictable_number number | string | null | conflict
conflict
Two or more sources disagree on this field. Human review resolves it; the resolution lands in corrections.jsonl.
| conflict | always True | required | |
| by_source | object | required | source key (short name of the source file) -> value as that source states it |
| resolved | string | number | null |
field_provenance
| source_file | string | required | path relative to workspace root |
| locator | string | page / sheet+row / slide that the value came from | |
| extractor | string | required | script name or qwen task prompt name |
| extractor_version | string | ||
| extracted_date | string | ||
| confidence | high | medium | low | ||
| human_checked | boolean |
material
One raw-material charge. Covers the ELN 'Input Raw Material' table (Sr.No/Raw Material/Sources/Cas No/RM No/Quantity gms/Purity/Rm %/Molecular Wt./No. of Moles/Molar Ratio), the Expt Table per-step Name/Qty(g) pairs, and the handwritten calc tables (Raw material/Weight/MW/Moles with HV/MW annotations).
| sr_no | integer | string | null | row number in the source table | |
| name | conflictable_string | ||
| descriptor | string | verbatim qualifier written with the name, e.g. 'Bu-EO-PO HV-62.3/MW.900', grade or spec annotations | |
| source | string | supplier/origin as written (e.g. FOIL, Loba) | |
| cas_no | string | ||
| rm_no | string | internal raw-material/lot code, e.g. 'R-53/07032024' | |
| quantity | conflictable_number | ||
| unit | string | verbatim unit; ELN and tables use g/gms | |
| purity_pct | number | string | null | ||
| rm_pct | number | string | null | 'Rm %' column of the ELN input table / '(85)'-style split in Step 3 rows | |
| molecular_wt | number | string | null | ||
| moles | number | string | null | ||
| molar_ratio | number | string | null | ||
| role | reactant | solvent | catalyst | intermediate | other | unknown | 'intermediate' for Step-N-product charges (e.g. 'Step-1 product (85)') | |
| calculated | boolean | true when the value is a spreadsheet/handwritten calculation rather than a weighed amount |
measurement
One measured/checked value: in-process checks embedded in procedure remarks and handwritten observations (AV, amine value, IV, HV, SAP, mono/bi, free acid, particle size), per-step Results columns (A.V., Other), and the ELN 'Critical Parameters' rows (Parameter/Reference/Value).
| parameter | string | required | e.g. 'A.V.', 'amine value', 'IV', 'HV', 'SAP', 'mono', 'bi', 'free acid', 'particle size', 'viscosity' |
| value | conflictable_number | ||
| unit | string | ||
| reference_value | number | string | null | target/spec it is judged against (ELN Critical Parameters 'Reference' column; e.g. aminev 30.1, av 43.2) | |
| phase | in-process | final | target-check | unknown | ||
| method | string | ||
| verbatim | string | the raw string the numbers were parsed from, kept verbatim (e.g. 'AV-0.3/ Amine 260/IV-34.5') |
procedure_event
One row of a process timeline: ELN 'Experimental Procedure' rows (Sr.No/Time hrs/Temperature C/Pressure-Vacuum-N2/Process Steps/Remarks) and handwritten 'Process' rows (Time/temperature/observation).
| seq | integer | string | null | Sr.No / row order | |
| time_h | number | string | null | duration or elapsed time, verbatim when a range ('1-5', '0.5') | |
| temperature | number | string | null | verbatim, ranges included ('65-75', 'RT-50') | |
| atmosphere | string | N2 / Vac / air, as written (handwritten combines it into the temperature cell, e.g. '75-80/N2' -> temperature '75-80', atmosphere 'N2') | |
| pressure | number | string | null | ELN Pressure/Vacuum-N2(kg/cm2) column | |
| action | string | the process-step text, verbatim (charging, feed, digestion, cooling...) | |
| observation | string | appearance/behaviour notes, verbatim | |
| measurements | array of measurement | in-process values stated on this row |
analytical_entry
| technique | string | required | 1H NMR | 13C NMR | IR | GC | PSD | amine value | acid value | ... |
| date | string | ||
| solvent | string | ||
| file | string | ||
| printed_result | string | number | null | verbatim printed value only; never an interpretation | |
| target_value | string | number | null | spec/target this is judged against, labelled as vendor/spec reference | |
| calculated | boolean |
application_test
| test | string | ||
| system | string | e.g. NC-PU ink | |
| pigment | string | Violet 23 | Violet 27 | Black | Blue | ... (APP DATA per-pigment blocks) | |
| conditions | string | ||
| result | string | number | null | ||
| vs_reference | string | null | comparison vs BYK-9076 standard, verbatim ('Eq. to Std', 'Ref as Std') | |
| verdict | ok | not ok | partial | unknown | None | ||
| trust_label | string | ||
| failure_reason | string | ||
| date | string | ||
| file | string |
sample
| sample_id | string | required | |
| sample_aliases | array of string | ||
| preparation | string | verbatim prep/blend recipe when the sample is not the neat batch product, e.g. 'LBN 21A + 3% AMP' (solvent blend) | |
| derived_from | array of string | record ids / batch aliases whose products went into this sample (blends reference their parent batches) | |
| analytical | array of analytical_entry | ||
| application_tests | array of application_test |
process_step
One synthesis step. The Expt Table's Step 1/2/3 blocks, the ELN's Step-A/B/C procedure phases, and the handwritten Step-I/II/III sections all land here (Step-I Phosphate ester, Step-II Acylate/Acetate intermediate, Step-III Neutralisation).
| step | integer | string | required | |
| name | string | verbatim step heading, e.g. 'Step-I) Phosphate', 'Step-III Neutralisation' | |
| description | string | ||
| date | string | date written on the step page (handwritten 'Date-1/11/25') | |
| materials | array of material | ||
| procedure | array of procedure_event | the step's timeline rows | |
| conditions | string | object | ||
| results | array of measurement | the step's Results columns (A.V., Other) and end-of-step checks | |
| remarks | string |
critical_process_parameter
One row of the ELN 'Critical Process Parameter' table (Sr No/Source/Time/Temp/Observation Details).
| sr_no | integer | string | null | ||
| source | string | ||
| time | number | string | null | ||
| temperature | number | string | null | ||
| observation | string |
linked_document
| file | string | required | |
| role | eln_report | handwritten_notes | analytical | application_test | summary_table | presentation | other | required | |
| link_basis | string | why this doc is linked to this record (which alias matched) | |
| link_status | rule | proposed | user-confirmed | ||
| pages | string | page range within the document that belongs to this record (handwritten notebook spans many experiments) | |
| watermark | string | export watermark / e-sign stamp printed on the document (e.g. 'satish_warghude@fineorganics.com 2026-8-04 12:26:49') |
▶ Raw JSON schema
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "record.schema.json",
"title": "FOIL 9076 experimental record",
"description": "One canonical record per experiment/lab batch. Built by merging fragments from all source documents. Finalized 2026-08-19 against the three primary sources: Old ELN PDFs (LAB JOURNAL v2.0 form), Project Expt Report table (28112025 9076 Expt Table, Step 1/2/3 layout), and the handwritten notebook OCR (9076 Exp 48-60). Every material value carries provenance. A scalar field may instead hold a conflict object when sources disagree.",
"type": "object",
"additionalProperties": false,
"required": [
"record_id",
"project",
"aliases",
"provenance"
],
"$defs": {
"conflictable_string": {
"oneOf": [
{
"type": "string"
},
{
"$ref": "#/$defs/conflict"
}
]
},
"conflictable_number": {
"oneOf": [
{
"type": [
"number",
"string",
"null"
]
},
{
"$ref": "#/$defs/conflict"
}
]
},
"conflict": {
"type": "object",
"description": "Two or more sources disagree on this field. Human review resolves it; the resolution lands in corrections.jsonl.",
"additionalProperties": false,
"required": [
"conflict",
"by_source"
],
"properties": {
"conflict": {
"const": true
},
"by_source": {
"type": "object",
"description": "source key (short name of the source file) -> value as that source states it",
"minProperties": 2
},
"resolved": {
"type": [
"string",
"number",
"null"
]
}
}
},
"field_provenance": {
"type": "object",
"additionalProperties": false,
"required": [
"source_file",
"extractor"
],
"properties": {
"source_file": {
"type": "string",
"description": "path relative to workspace root"
},
"locator": {
"type": "string",
"description": "page / sheet+row / slide that the value came from"
},
"extractor": {
"type": "string",
"description": "script name or qwen task prompt name"
},
"extractor_version": {
"type": "string"
},
"extracted_date": {
"type": "string"
},
"confidence": {
"enum": [
"high",
"medium",
"low"
]
},
"human_checked": {
"type": "boolean"
}
}
},
"material": {
"type": "object",
"description": "One raw-material charge. Covers the ELN 'Input Raw Material' table (Sr.No/Raw Material/Sources/Cas No/RM No/Quantity gms/Purity/Rm %/Molecular Wt./No. of Moles/Molar Ratio), the Expt Table per-step Name/Qty(g) pairs, and the handwritten calc tables (Raw material/Weight/MW/Moles with HV/MW annotations).",
"additionalProperties": true,
"properties": {
"sr_no": {
"type": [
"integer",
"string",
"null"
],
"description": "row number in the source table"
},
"name": {
"$ref": "#/$defs/conflictable_string"
},
"descriptor": {
"type": "string",
"description": "verbatim qualifier written with the name, e.g. 'Bu-EO-PO HV-62.3/MW.900', grade or spec annotations"
},
"source": {
"type": "string",
"description": "supplier/origin as written (e.g. FOIL, Loba)"
},
"cas_no": {
"type": "string"
},
"rm_no": {
"type": "string",
"description": "internal raw-material/lot code, e.g. 'R-53/07032024'"
},
"quantity": {
"$ref": "#/$defs/conflictable_number"
},
"unit": {
"type": "string",
"description": "verbatim unit; ELN and tables use g/gms"
},
"purity_pct": {
"type": [
"number",
"string",
"null"
]
},
"rm_pct": {
"type": [
"number",
"string",
"null"
],
"description": "'Rm %' column of the ELN input table / '(85)'-style split in Step 3 rows"
},
"molecular_wt": {
"type": [
"number",
"string",
"null"
]
},
"moles": {
"type": [
"number",
"string",
"null"
]
},
"molar_ratio": {
"type": [
"number",
"string",
"null"
]
},
"role": {
"enum": [
"reactant",
"solvent",
"catalyst",
"intermediate",
"other",
"unknown"
],
"description": "'intermediate' for Step-N-product charges (e.g. 'Step-1 product (85)')"
},
"calculated": {
"type": "boolean",
"description": "true when the value is a spreadsheet/handwritten calculation rather than a weighed amount"
}
}
},
"measurement": {
"type": "object",
"description": "One measured/checked value: in-process checks embedded in procedure remarks and handwritten observations (AV, amine value, IV, HV, SAP, mono/bi, free acid, particle size), per-step Results columns (A.V., Other), and the ELN 'Critical Parameters' rows (Parameter/Reference/Value).",
"additionalProperties": true,
"required": [
"parameter"
],
"properties": {
"parameter": {
"type": "string",
"description": "e.g. 'A.V.', 'amine value', 'IV', 'HV', 'SAP', 'mono', 'bi', 'free acid', 'particle size', 'viscosity'"
},
"value": {
"$ref": "#/$defs/conflictable_number"
},
"unit": {
"type": "string"
},
"reference_value": {
"type": [
"number",
"string",
"null"
],
"description": "target/spec it is judged against (ELN Critical Parameters 'Reference' column; e.g. aminev 30.1, av 43.2)"
},
"phase": {
"enum": [
"in-process",
"final",
"target-check",
"unknown"
]
},
"method": {
"type": "string"
},
"verbatim": {
"type": "string",
"description": "the raw string the numbers were parsed from, kept verbatim (e.g. 'AV-0.3/ Amine 260/IV-34.5')"
}
}
},
"procedure_event": {
"type": "object",
"description": "One row of a process timeline: ELN 'Experimental Procedure' rows (Sr.No/Time hrs/Temperature C/Pressure-Vacuum-N2/Process Steps/Remarks) and handwritten 'Process' rows (Time/temperature/observation).",
"additionalProperties": true,
"properties": {
"seq": {
"type": [
"integer",
"string",
"null"
],
"description": "Sr.No / row order"
},
"time_h": {
"type": [
"number",
"string",
"null"
],
"description": "duration or elapsed time, verbatim when a range ('1-5', '0.5')"
},
"temperature": {
"type": [
"number",
"string",
"null"
],
"description": "verbatim, ranges included ('65-75', 'RT-50')"
},
"atmosphere": {
"type": "string",
"description": "N2 / Vac / air, as written (handwritten combines it into the temperature cell, e.g. '75-80/N2' -> temperature '75-80', atmosphere 'N2')"
},
"pressure": {
"type": [
"number",
"string",
"null"
],
"description": "ELN Pressure/Vacuum-N2(kg/cm2) column"
},
"action": {
"type": "string",
"description": "the process-step text, verbatim (charging, feed, digestion, cooling...)"
},
"observation": {
"type": "string",
"description": "appearance/behaviour notes, verbatim"
},
"measurements": {
"type": "array",
"items": {
"$ref": "#/$defs/measurement"
},
"description": "in-process values stated on this row"
}
}
},
"analytical_entry": {
"type": "object",
"additionalProperties": true,
"required": [
"technique"
],
"properties": {
"technique": {
"type": "string",
"description": "1H NMR | 13C NMR | IR | GC | PSD | amine value | acid value | ..."
},
"date": {
"type": "string"
},
"solvent": {
"type": "string"
},
"file": {
"type": "string"
},
"printed_result": {
"type": [
"string",
"number",
"null"
],
"description": "verbatim printed value only; never an interpretation"
},
"target_value": {
"type": [
"string",
"number",
"null"
],
"description": "spec/target this is judged against, labelled as vendor/spec reference"
},
"calculated": {
"type": "boolean"
}
}
},
"application_test": {
"type": "object",
"additionalProperties": true,
"properties": {
"test": {
"type": "string"
},
"system": {
"type": "string",
"description": "e.g. NC-PU ink"
},
"pigment": {
"type": "string",
"description": "Violet 23 | Violet 27 | Black | Blue | ... (APP DATA per-pigment blocks)"
},
"conditions": {
"type": "string"
},
"result": {
"type": [
"string",
"number",
"null"
]
},
"vs_reference": {
"type": [
"string",
"null"
],
"description": "comparison vs BYK-9076 standard, verbatim ('Eq. to Std', 'Ref as Std')"
},
"verdict": {
"enum": [
"ok",
"not ok",
"partial",
"unknown",
null
]
},
"trust_label": {
"type": "string"
},
"failure_reason": {
"type": "string"
},
"date": {
"type": "string"
},
"file": {
"type": "string"
}
}
},
"sample": {
"type": "object",
"additionalProperties": true,
"required": [
"sample_id"
],
"properties": {
"sample_id": {
"type": "string"
},
"sample_aliases": {
"type": "array",
"items": {
"type": "string"
}
},
"preparation": {
"type": "string",
"description": "verbatim prep/blend recipe when the sample is not the neat batch product, e.g. 'LBN 21A + 3% AMP' (solvent blend)"
},
"derived_from": {
"type": "array",
"items": {
"type": "string"
},
"description": "record ids / batch aliases whose products went into this sample (blends reference their parent batches)"
},
"analytical": {
"type": "array",
"items": {
"$ref": "#/$defs/analytical_entry"
}
},
"application_tests": {
"type": "array",
"items": {
"$ref": "#/$defs/application_test"
}
}
}
},
"process_step": {
"type": "object",
"description": "One synthesis step. The Expt Table's Step 1/2/3 blocks, the ELN's Step-A/B/C procedure phases, and the handwritten Step-I/II/III sections all land here (Step-I Phosphate ester, Step-II Acylate/Acetate intermediate, Step-III Neutralisation).",
"additionalProperties": true,
"required": [
"step"
],
"properties": {
"step": {
"type": [
"integer",
"string"
]
},
"name": {
"type": "string",
"description": "verbatim step heading, e.g. 'Step-I) Phosphate', 'Step-III Neutralisation'"
},
"description": {
"type": "string"
},
"date": {
"type": "string",
"description": "date written on the step page (handwritten 'Date-1/11/25')"
},
"materials": {
"type": "array",
"items": {
"$ref": "#/$defs/material"
}
},
"procedure": {
"type": "array",
"items": {
"$ref": "#/$defs/procedure_event"
},
"description": "the step's timeline rows"
},
"conditions": {
"oneOf": [
{
"type": "string",
"description": "legacy flattened form ('Temp oC: 45-50 | 70-75; Time h: 0.5 | 4.5; Vac/N2: N2')"
},
{
"type": "object",
"additionalProperties": true,
"properties": {
"temperature": {
"$ref": "#/$defs/conflictable_string"
},
"time": {
"$ref": "#/$defs/conflictable_string"
},
"atmosphere": {
"$ref": "#/$defs/conflictable_string"
},
"pressure": {
"type": "string"
},
"other": {
"type": "string"
}
}
}
]
},
"results": {
"type": "array",
"items": {
"$ref": "#/$defs/measurement"
},
"description": "the step's Results columns (A.V., Other) and end-of-step checks"
},
"remarks": {
"type": "string"
}
}
},
"critical_process_parameter": {
"type": "object",
"description": "One row of the ELN 'Critical Process Parameter' table (Sr No/Source/Time/Temp/Observation Details).",
"additionalProperties": true,
"properties": {
"sr_no": {
"type": [
"integer",
"string",
"null"
]
},
"source": {
"type": "string"
},
"time": {
"type": [
"number",
"string",
"null"
]
},
"temperature": {
"type": [
"number",
"string",
"null"
]
},
"observation": {
"type": "string"
}
}
},
"linked_document": {
"type": "object",
"additionalProperties": false,
"required": [
"file",
"role"
],
"properties": {
"file": {
"type": "string"
},
"role": {
"enum": [
"eln_report",
"handwritten_notes",
"analytical",
"application_test",
"summary_table",
"presentation",
"other"
]
},
"link_basis": {
"type": "string",
"description": "why this doc is linked to this record (which alias matched)"
},
"link_status": {
"enum": [
"rule",
"proposed",
"user-confirmed"
]
},
"pages": {
"type": "string",
"description": "page range within the document that belongs to this record (handwritten notebook spans many experiments)"
},
"watermark": {
"type": "string",
"description": "export watermark / e-sign stamp printed on the document (e.g. 'satish_warghude@fineorganics.com 2026-8-04 12:26:49')"
}
}
}
},
"properties": {
"record_id": {
"type": "string",
"pattern": "^exp-9076-[0-9]{3}[A-Z0-9]*$",
"description": "canonical id, e.g. exp-9076-048, exp-9076-024B, exp-9076-024C1"
},
"project": {
"const": "9076"
},
"aliases": {
"type": "array",
"items": {
"type": "string"
},
"description": "every ID string that refers to this experiment across the corpus (Lab Batch 24 B, LBN 24B, Batch-54A, NO-49, #48Q, ...)"
},
"stub": {
"type": "boolean",
"description": "true when the record was created only to receive an unmatched fragment; needs linking review"
},
"record_type": {
"enum": [
"synthesis",
"blend",
"rework",
"unknown"
],
"description": "synthesis = a lab reaction; blend = a letter-suffixed batch made by blending a parent batch's product with a solvent/additive (has NO synthesis parameters, only application testing \u2014 per the scientist's walkthrough, e.g. 22A = 22 + solvent, 21D = 21 + 2% AMP); rework = reprocessing of an earlier batch (e.g. 24CR)"
},
"preparation": {
"type": "string",
"description": "blend/rework recipe verbatim (e.g. 'LBN 21 + 2% AMP'); only for record_type blend/rework"
},
"derived_from": {
"type": "array",
"items": {
"type": "string"
},
"description": "parent batch record ids/aliases this blend/rework was made from"
},
"date": {
"$ref": "#/$defs/conflictable_string"
},
"scientist": {
"$ref": "#/$defs/conflictable_string"
},
"aim": {
"$ref": "#/$defs/conflictable_string"
},
"eln_form": {
"type": "object",
"description": "Experiment-level fields of the LAB JOURNAL v2.0 form that are metadata about the run rather than process data.",
"additionalProperties": false,
"properties": {
"journal_version": {
"type": "string",
"description": "'LAB JOURNAL - Version 2.0' banner"
},
"equipment": {
"type": "string",
"description": "'Equipment Details' field, verbatim"
},
"reaction_scheme": {
"type": "string",
"description": "'Reaction Scheme' field; often an image \u2014 store the printed text or 'image only'"
},
"safety_precautions": {
"type": "string"
},
"other_fields": {
"type": "object",
"description": "catch-all for rare/unexplained form labels (e.g. 'SPDu') so no printed field is ever dropped: label -> verbatim value",
"additionalProperties": {
"type": [
"string",
"number",
"null"
]
}
}
}
},
"materials": {
"type": "array",
"items": {
"$ref": "#/$defs/material"
},
"description": "the ELN 'Input Raw Material' table (whole-experiment charge list); per-step charges go in process_steps[].materials"
},
"process_steps": {
"type": "array",
"items": {
"$ref": "#/$defs/process_step"
}
},
"procedure_summary": {
"type": "string"
},
"conditions": {
"type": "object",
"description": "experiment-level roll-up: the ELN 'Optimized Reaction Conditions' table (Temperature/N2-Vac/Time) or the summary export's T c/N2-Vac/Time columns",
"additionalProperties": false,
"properties": {
"temperature": {
"$ref": "#/$defs/conflictable_string"
},
"time": {
"$ref": "#/$defs/conflictable_string"
},
"atmosphere": {
"$ref": "#/$defs/conflictable_string"
},
"other": {
"type": "string"
}
}
},
"critical_process_parameters": {
"type": "array",
"items": {
"$ref": "#/$defs/critical_process_parameter"
}
},
"observations": {
"type": "string"
},
"samples": {
"type": "array",
"items": {
"$ref": "#/$defs/sample"
}
},
"analytical_summary": {
"type": "array",
"items": {
"$ref": "#/$defs/analytical_entry"
},
"description": "experiment-level analytical results (e.g. amine value / acid value columns from the ELN summary) not yet tied to a specific sample id"
},
"critical_parameters": {
"type": "array",
"items": {
"$ref": "#/$defs/measurement"
},
"description": "the ELN 'Critical Parameters' table: parameter + reference (target) + measured value, phase 'target-check'"
},
"conclusion": {
"$ref": "#/$defs/conflictable_string",
"description": "verbatim Conclusion field (ELN form) / Conclusion column (summary export); outcome.status is derived from it"
},
"outcome": {
"type": "object",
"additionalProperties": false,
"properties": {
"status": {
"enum": [
"ok",
"not ok",
"success",
"failure",
"partial",
"unknown"
]
},
"basis": {
"type": "string",
"description": "verbatim statement the status came from"
},
"citation": {
"type": "string"
}
}
},
"documents": {
"type": "array",
"items": {
"$ref": "#/$defs/linked_document"
}
},
"provenance": {
"type": "object",
"additionalProperties": false,
"required": [
"fragments"
],
"properties": {
"fragments": {
"type": "array",
"items": {
"type": "string"
},
"description": "fragment ids merged into this record"
},
"field_provenance": {
"type": "object",
"description": "field path (dot/bracket notation) -> provenance",
"additionalProperties": {
"$ref": "#/$defs/field_provenance"
}
},
"last_merged": {
"type": "string"
}
}
},
"validation": {
"type": "object",
"additionalProperties": false,
"properties": {
"validated_fields": {
"type": "object",
"description": "field path -> {validated_by, date}. These fields are frozen: re-runs re-apply corrections.jsonl on top and never overwrite them.",
"additionalProperties": {
"type": "object",
"additionalProperties": false,
"required": [
"validated_by",
"date"
],
"properties": {
"validated_by": {
"type": "string"
},
"date": {
"type": "string"
}
}
}
},
"corrections_applied": {
"type": "array",
"items": {
"type": "string"
},
"description": "correction ids from corrections.jsonl"
}
}
},
"links": {
"type": "object",
"additionalProperties": false,
"properties": {
"previous_experiment": {
"type": [
"string",
"null"
]
},
"next_experiment": {
"type": [
"string",
"null"
]
},
"related": {
"type": "array",
"items": {
"type": "string"
}
}
}
}
}
}
FOIL 9076 record fragment
schema/fragment.schema.json updated 19 Aug 12:52What ONE source document says about ONE experiment. Emitted by every parser and every Qwen extraction task; the merge step resolves target_aliases to a canonical record and folds the payload in. This is the universal envelope: spreadsheets, ELN PDFs, test reports, spectra headers and slides all arrive as fragments.
| Field | Type | Description | |
|---|---|---|---|
| fragment_id | string | required | stable id: <extractor>--<source-slug>--<alias-slug> so re-runs overwrite rather than duplicate |
| source_file | string | required | path relative to workspace root (the ORIGINAL document, not the ocr/ derivative) |
| derived_from | string | the ocr/ or converted file actually parsed | |
| locator | string | sheet+rows / page / slide within the source | |
| extractor | string | required | |
| extractor_version | string | ||
| extracted_date | string | ||
| target_aliases | array of string | required | ID strings, verbatim as they appear in the document, that identify which experiment this fragment belongs to |
| document_role | eln_report | handwritten_notes | analytical | application_test | summary_table | presentation | other | ||
| payload | object | required | Partial record: any subset of the record schema's content fields (date, scientist, aim, materials, process_steps, conditions, observations, samples, analytical_summary, outcome). Same shapes as record.schema.json; validated after merge. |
| field_confidence | object | payload field path -> high|medium|low | |
| notes | string | extractor's caveats, e.g. cells that could not be read |
▶ Raw JSON schema
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "fragment.schema.json",
"title": "FOIL 9076 record fragment",
"description": "What ONE source document says about ONE experiment. Emitted by every parser and every Qwen extraction task; the merge step resolves target_aliases to a canonical record and folds the payload in. This is the universal envelope: spreadsheets, ELN PDFs, test reports, spectra headers and slides all arrive as fragments.",
"type": "object",
"additionalProperties": false,
"required": [
"fragment_id",
"source_file",
"extractor",
"target_aliases",
"payload"
],
"properties": {
"fragment_id": {
"type": "string",
"description": "stable id: <extractor>--<source-slug>--<alias-slug> so re-runs overwrite rather than duplicate"
},
"source_file": {
"type": "string",
"description": "path relative to workspace root (the ORIGINAL document, not the ocr/ derivative)"
},
"derived_from": {
"type": "string",
"description": "the ocr/ or converted file actually parsed"
},
"locator": {
"type": "string",
"description": "sheet+rows / page / slide within the source"
},
"extractor": {
"type": "string"
},
"extractor_version": {
"type": "string"
},
"extracted_date": {
"type": "string"
},
"target_aliases": {
"type": "array",
"minItems": 1,
"items": {
"type": "string"
},
"description": "ID strings, verbatim as they appear in the document, that identify which experiment this fragment belongs to"
},
"document_role": {
"enum": [
"eln_report",
"handwritten_notes",
"analytical",
"application_test",
"summary_table",
"presentation",
"other"
]
},
"payload": {
"type": "object",
"description": "Partial record: any subset of the record schema's content fields (date, scientist, aim, materials, process_steps, conditions, observations, samples, analytical_summary, outcome). Same shapes as record.schema.json; validated after merge."
},
"field_confidence": {
"type": "object",
"description": "payload field path -> high|medium|low",
"additionalProperties": {
"enum": [
"high",
"medium",
"low"
]
}
},
"notes": {
"type": "string",
"description": "extractor's caveats, e.g. cells that could not be read"
}
}
}