Skip to content

Validation & Queries

Comprehensive data quality management system for ensuring data integrity.

Validation Types

Real-Time Validation

Field-level validation during data entry:

const schema = z.object({
  age: z.number().min(18).max(120),
  email: z.string().email(),
  weight: z.number().positive(),
  visit_date: z.date().max(new Date()), // Cannot be in future
});

Validation rules: - Required fields - Must be filled - Data type - Number, date, text, etc. - Range checks - Min/max values - Format validation - Email, phone, regex - Dependency rules - Conditional requirements

Post-Submission Validation

Batch validation after data submission:

  • Cross-field validation - Internal consistency
  • Outlier detection - Statistical anomalies
  • Completeness checks - Missing required data
  • Logic checks - Business rule violations

Custom Validation Rules

Researchers can define study-specific rules:

{
  "rule_name": "bmi_consistency",
  "condition": "weight / (height ** 2) != bmi",
  "severity": "warning",
  "message": "Calculated BMI does not match entered BMI"
}

Quality Flags

Data records can have multiple quality flags:

Flag Description Action
✓ Valid Passes all validation None required
⚠️ Warning Non-critical issue Review recommended
❌ Error Critical issue Correction required
❓ Query Discrepancy identified Response required
🔍 Review Pending review SDV needed
✅ Verified Source-verified No action

Query Management

Creating Queries

await api.createQuery({
  study_id: study_id,
  participant_id: participant_id,
  field_name: "systolic_bp",
  query_type: "clarification",
  severity: "medium",
  message: "Value seems unusually high (180 mmHg). Please verify.",
  assigned_to: coordinator_id,
});

Query Workflow

1. Open       → 2. Assigned → 3. Responded → 4. Resolved
                    ↓
                Escalated (if needed)

Query states: - Open - Query created, not yet assigned - Assigned - Assigned to data entry staff - Responded - Staff provided response - Resolved - Reviewer accepted response - Escalated - Requires PI/sponsor review

Query Dashboard

Portal provides query management interface:

  • List all queries by status
  • Filter by participant, study, severity
  • Bulk assign queries
  • Export query log

Source Data Verification (SDV)

SDV Process

  1. Select records for verification (random or targeted)
  2. Compare data to source documents
  3. Mark as verified or create query
  4. Track SDV completion percentage

SDV Modes

  • 100% SDV - All data verified (critical studies)
  • Risk-based SDV - Targeted verification
  • Random SDV - Sample-based approach

SDV Tracking

{
  "study_id": "STUDY-001",
  "target_sdv_percentage": 20,
  "current_sdv_percentage": 18.5,
  "records_total": 1000,
  "records_verified": 185,
  "records_with_queries": 12
}

Data Monitoring

Real-Time Monitoring Dashboard

  • Enrollment rate
  • Data completeness
  • Query rate
  • SDV progress
  • Protocol deviations

Automated Alerts

Configure alerts for:

{
  "alert_type": "data_quality",
  "condition": "error_rate > 5%",
  "recipients": ["coordinator@study.org"],
  "frequency": "daily"
}

Data Cleaning

Batch Operations

Clean data across multiple records:

// Remove trailing spaces from all text fields
await api.cleanData({
  study_id: study_id,
  operation: "trim_text",
  dry_run: true, // Preview changes first
});

// Standardize date formats
await api.cleanData({
  study_id: study_id,
  operation: "standardize_dates",
  target_format: "YYYY-MM-DD",
});

Data Transformations

Apply transformations to specific fields:

  • Text normalization - Case, whitespace
  • Date standardization - Format consistency
  • Unit conversion - Imperial ↔ metric
  • Calculated fields - Derived values (BMI, eGFR)

Quality Metrics

Track data quality over time:

{
  "period": "2026-01",
  "metrics": {
    "completeness": 0.98,
    "accuracy": 0.95,
    "error_rate": 0.02,
    "query_rate": 0.05,
    "sdv_rate": 0.20,
    "time_to_resolution_days": 2.3
  }
}

Validation Reports

Generate reports for monitoring:

  • Data Quality Report - Overall quality metrics
  • Query Log - All queries and resolutions
  • SDV Report - Verification status
  • Outlier Report - Statistical anomalies
  • Completeness Report - Missing data

Integration with REDCap

Validation rules sync with REDCap:

  • Import validation rules from REDCap
  • Export queries to REDCap
  • Sync data quality indicators

See REDCap Integration for details.

Next Steps