Comprehensive benchmarking of tools for nanopore-based detection of DNA methylation
TL;DR
A comprehensive benchmark of nanopore DNA methylation tools reveals that older models like Dorado v4r1 and RockFish are best for CpG sites, while newer Dorado v5 models excel at non-CpG and 6mA detection.
Problem / question
While Oxford Nanopore sequencing can directly detect DNA modifications, it is unclear which computational tools perform best on the latest R10 chemistry, especially for non-CpG methylation, 6-methyladenine (6mA), and 4-methylcytosine (4mC).
Methods
Researchers generated whole-genome nanopore (R10.4.1) and EMSeq (ground truth) data for plants (Arabidopsis, rice), mouse (brain, embryonic stem cells), and five bacterial species, plus human HG002 data. They benchmarked tools including Dorado (v4, v5, v5.2), DeepBAM, DeepMod2, DeepPlant, f5C, and RockFish, evaluating F1 scores, Pearson correlation, computational speed, and the impact of neighboring modifications and read quality.
Key findings
For CpG methylation, Dorado v4r1 and RockFish achieved the highest accuracy, with RockFish reaching an F1 score of 0.996 at stringent thresholds. For non-CpG sites, Dorado v5r3 performed best overall, while DeepPlant was highly accurate for plants but failed on mouse data. For bacterial 6mA, Dorado v5r1 was the most robust (F1 greater than 0.95), whereas 4mC models lagged behind (F1 0.8 to 0.9). The study also found that neighboring methylated bases cause false positives or negatives in many models, and that DeepPlant consumed over 84 GB of RAM compared to Dorado's 12 to 14 GB.
Why it matters
By detailing the specific strengths, sequence biases, and memory requirements of state-of-the-art models, this study provides researchers with concrete guidelines to accurately profile diverse epigenetic marks and avoid tool-induced errors.
Limitations
The study relied on EMSeq as the ground truth for 5mC (which can have conversion biases), did not evaluate 5-hydroxymethylcytosine (5hmC), and had limited sequence contexts for 4mC due to its rarity.
Takeaway
Use Dorado v4r1 or RockFish for CpG methylation, Dorado v5r3 for non-CpG methylation, and ensure a minimum sequencing depth of 20x and a read quality score above q20 for reliable results.