ICMR Framework

MIDAS 2.0: Assessing health dataset readiness for safe, trustworthy AI

MIDAS 2.0, the Metric-based Integrity and Data Assessment System, gives institutions a common method to check data quality, documentation, representativeness, interoperability, governance, and privacy before a dataset is shared, reused, or used to build AI tools.

MIDAS at a Glance
15
Quality Domains
2
Lite + Technical Versions
0-100
Clear quality and privacy scoring outputs
Quality Scoring Privacy Risk Independent Validation
What MIDAS Is

A practical framework for deciding whether a biomedical dataset can be trusted and responsibly reused

If you are new to MIDAS, think of it as a practical system that helps you understand whether a health dataset is truly ready for research, repository onboarding, and AI development, not just whether the files exist.

Why It Exists

Many health datasets are useful, but not equally ready for AI

Useful datasets are often held back by uneven documentation, inconsistent metadata, weak interoperability, or unclear privacy safeguards. MIDAS 2.0 exists to replace guesswork with a consistent, evidence-based way to judge whether a dataset is strong enough to be trusted and reused.

What It Measures

It looks beyond file quality to real-world trustworthiness

MIDAS 2.0 evaluates whether a dataset is not only complete, but also understandable, reusable, representative, and safe.

  • Data quality: annotation fidelity, metadata, documentation, and completeness
  • Operational readiness: interoperability, AI-readiness, representativeness, and sustainability
  • Trust safeguards: governance, ethics, security, consent, and privacy risk
What It Helps Decide

It supports improvement planning, certification, and access control

The framework helps centres identify gaps, improve weak areas, compare datasets on a common scale, and decide how confidently a dataset can be shared, governed, and reused. It also supports repository onboarding and future benchmark AI work.

Why MIDAS 2.0

Why MIDAS 2.0 had to go beyond MIDAS 1.0

MIDAS 1.0 proved that high-quality, standardized biomedical datasets could be built in a structured way. MIDAS 2.0 takes the next necessary step. If datasets are going to be compared across centres, onboarded into trusted repositories, and reused for AI, they must be assessed with clearer evidence, stronger privacy safeguards, better interoperability checks, and a framework that works beyond imaging alone.

What MIDAS 1.0 Proved

It established the foundation for structured dataset curation

MIDAS 1.0 showed that biomedical datasets could be built with stronger annotation discipline, richer metadata, and more consistent curation standards instead of being assembled in an ad hoc way. That early work created the foundation for a more mature national framework.

Its biggest contribution was proving that dataset quality could be treated as a scientific and operational priority, not as an afterthought.
Why MIDAS 2.0 Matters

It adds the measurable rules needed for trusted reuse at scale

MIDAS 1.0 was an important beginning, but it was not enough for a national-quality ecosystem. A stronger framework was needed because datasets today must do more than look well curated inside one institution. They must be comparable across centres, usable across data types, and safe enough to support responsible sharing and AI development.

  • It introduces reproducible quantitative scoring through the Composite Quality Index and Privacy-Risk Score.
  • It extends assessment beyond imaging to multimodal biomedical and health data, including text, voice, and EHR-linked datasets.
  • It adds stronger checks for interoperability, governance, representativeness, and privacy readiness.
  • It supports defensible decisions about dataset quality, safeguards, and readiness for wider reuse.
How It Works

From self-assessment to verified release decision

MIDAS 2.0 uses a two-stage pathway so dataset custodians can first complete the Lite Version themselves and then undergo an independent Technical review before the final CQI and PRS are assigned. This keeps the process practical for centres while still requiring evidence-based validation before broader sharing or repository onboarding.

1

Centre completes the Lite Version

The dataset custodian performs a structured self-assessment and records the current state of the dataset against the MIDAS domains.

2

Evidence is assembled and retained

Metadata, SOPs, validation logs, consent information, and privacy documentation are kept ready so every claim can be verified rather than assumed.

3

Nodal Centre performs the Technical review

An independent reviewer checks the submission in greater detail, asks for clarifications if needed, and computes the final quality and privacy scores.

4

Scores guide release and improvement

The final CQI and PRS determine whether the dataset is ready for broader use, needs targeted improvement, or requires stronger sharing restrictions.

Scoring Logic

How MIDAS turns evidence into decisions

The framework produces two outputs: one score for overall dataset quality and one score for residual privacy risk. Together, they show whether a dataset is strong enough to support trustworthy reuse and what level of safeguards it still needs. This is what turns MIDAS 2.0 from a checklist into a decision framework for certification, repository onboarding, and responsible reuse.

📊

Composite Quality Index

Summarises how complete, reusable, representative, and well-governed a dataset is across the 15 MIDAS domains.

In simple terms, CQI answers: "How strong and trustworthy is this dataset overall?" A higher CQI means the dataset is better documented, easier to reuse, and more suitable for high-quality research and AI development.

Quality Index Formula
CQI = (Sum of domain scores / 60) × 100
6-Tier Grading Ladder
  • Diamond ≥ 95 — Global exemplar
  • Platinum 85–94 — Best-practice dataset
  • Gold 70–84 — High-quality dataset
  • Silver 50–69 — Permissible, needs improvement
  • Bronze 25–49 — Embargoed for enhancement
  • Remediation < 25 — Iterative QA required
🔒

Privacy-Risk Score

Estimates how much re-identification or sensitive-attribute risk remains after privacy protections have been applied.

PRS answers a different question: "Even after de-identification and other controls, how much privacy risk still remains?" A lower PRS supports wider reuse, while a higher PRS signals the need for tighter controls.

Example Baseline for Tabular Data
BaselineRisktabular = 100 × p
Example Baseline for Differential Privacy
BaselineRiskDP = min(100, 20 × ε)

The baseline is then adjusted for how sensitive the data are. Higher sensitivity means stricter handling requirements.

PRS = round(AdjustedRisk)
Low  0–15 Moderate  16–40 High  41–70 Very High  71–100
Expert Validation

Why this portal includes Delphi review

Before MIDAS 2.0 is used more widely, ICMR is asking experts to review whether the framework is clear, scientifically sound, complete, and practical across different biomedical data contexts. The goal is not only to validate the wording, but also to ensure the framework can be applied consistently before it is used for wider implementation and dataset certification.

01

What experts are reviewing

  • Whether the 15 domains cover the main dimensions of dataset quality, trustworthiness, and AI-readiness.
  • Whether the wording and scoring ladders are clear enough to be used consistently across centres.
  • Whether any domains, criteria, or evidence requirements need refinement before broader rollout.
02

How reviewers score items

Each statement is rated on a 5-point scale, and lower scores must be explained so unclear or impractical sections can be revised.

1
Very unclear
2
Unclear
3
Needs clarification
4
Clear
5
Exceptionally clear
⚠  Ratings of 1–3 must be accompanied by explanatory comments.
03

How agreement is measured

Item-level agreementHow many experts rated an item 4 or 5. Target: $I\text{-}CVI \ge 0.78$
Scale-level agreementAverage agreement across the full framework. Target: $S\text{-}CVI/Ave \ge 0.90$
Modified Kappa ($k^*$)Adjusts for agreement that might happen by chance. $\ge 0.74$ indicates excellent consensus.

Ready to review or continue your assessment?

If you are an invited expert, you can log in and continue the validation workflow. If you are new to MIDAS, start with the Delphi Proposal to read the full review document and scoring context.