When you build or calibrate a bioinformatics tool, there is a common risk of overfitting ("teaching to the test"): if you tweak your code, thresholds, and equations while looking at all 12 datasets at once, you can accidentally tailor the rules just to make those specific 12 datasets pass—only for the tool to lose accuracy on a 13th dataset. We saw this firsthand in our early Biosynthetic Gene Cluster (BGC) work (Discovery #60), where rules tuned on an initial genome set dropped in recall when tested on new genomes.

• Step 1 — Development Cohort (The Workbench): During development, we inspect errors and refine the Rust code on the Development set (e.g., DEV-01 to DEV-06).
• Step 2 — Parameter Freeze (Pencils Down): Once the module matches the external reference software on the Development set, all weights, thresholds, and formulas are locked.
• Step 3 — Untouched Reserve Cohort (The Unseen Test): We run the frozen module on the Reserve datasets (e.g., RES-01 to RES-06) that were not used during development to verify that it generalizes cleanly.

In Tool Benchmark #17 (bio-suite mk-compare vs. IQ-TREE v2.4.0, 2.31 ms vs. 5,738 ms) or Tool Benchmark #16 (relax-compare vs. MACSE/PAML), three factors explain the wall-clock difference:

1. Zero OS Process Spawns & Zero Disk I/O: To score 12 cohorts across 2 models (MK and MK+ASC), an external CLI harness has to spawn iqtree2.exe 24 separate times on Windows—loading DLLs, reading .phy/.nwk files, and writing 168 output files to disk (~230 ms of OS/disk overhead per launch). Our Rust engine evaluates all 12 cohorts in memory inside a single process via rayon::par_iter().
2. Closed-Form Analytical Math: For symmetric models like Lewis (2001) M_k, the transition probability matrix simplifies to a closed-form analytical exponential equation rather than general numerical matrix diagonalization, and scores both MK and MK+ASC in a single tree pass.
3. Important Caveat (Small-Matrix Loops vs. Large De Novo Searches): Avoiding subprocess and disk startup overhead yields large speedups when scoring fixed trees or chaining multiple stages in memory. On a single long-running de novo tree search where IQ-TREE launches once and runs in RAM for minutes, process startup overhead is negligible and the algorithmic/parallelism speedup is in the 5×–28× range.

In most bioinformatics workflows, researchers have to glue together separate command-line tools in different languages using ad-hoc shell scripts. Our long-term goal with RuNIE is to build a unified Rust framework where calibrated native modules share typed in-memory data structures so analyses can be chained across Tier 1 (Exploratory), Tier 2 (Validation), and Tier 3 (Scientific) workflows:

• Current Scope (Work in Progress): Benchmarking individual native Rust modules against established external tools (such as MAFFT, IQ-TREE, PAML, MACSE, HMMER3, and ViennaRNA) on Development and Untouched Reserve cohorts to ensure our baseline implementations match reference behavior.
• Multi-Stage Chaining Goal: Connecting sequence/repeat mechanics, evolutionary selection, 2D/3D structural biophysics, and cellular/clinical phenotypes into reproducible multi-stage pipelines without intermediate file-parsing bottlenecks.
• Public Development Log: Engineering notes and calibration fixes are tracked in RuNIE & Changelog and RUNIE_CHANGELOG.md.

Our initial baseline Rust aligner only used nucleotide-level gap penalties, which shifted the downstream reading frame when encountering 1-bp deletions or single-base indels inside homopolymer tracts (such as a 6G run in ABCA4 or a 5A run in HBB). Established tools like MACSE solve this by aligning in 3-frame amino-acid space with explicit frameshift states (!).

In entry RN-2026-008, we updated our native Rust module in src/pseudo_loss.rs to align in 3-frame codon space and check downstream purifying selection (d_N/d_S ≪ 1), bringing our native implementation into 100% agreement with MACSE v2.07 and PAML v4.10.10 across the 12 test cohorts so it can be chained cleanly into multi-stage workflows.

• 🧭 Tier 1: Exploratory: Seeks out and explores a mix of new, classic, or unseen datasets (datasets we had not previously evaluated in the suite), or uses an existing validation set in a new analytical way when it makes sense as part of a calibration sweep or tool benchmark.
• 🎯 Tier 2: Validation: Uses curated validation datasets (tracked in validation-datasets-registry.json, VAL-SET-001 through VAL-SET-008) that we have repeatedly replicated and validated—building out structured metadata around Development vs. Untouched Reserve splits, primary literature anchors, and cross-tool concordance as part of our pipeline validation process (such as Discovery #61 and Discovery #62).
• 🔬 Tier 3: Scientific: Combines a mix of new and established validation data—building on previous results by adding new datasets or testing new hypotheses—and must arrive at a new scientific interpretation, conclusion, or result.

• 0🧪 Excluded / Calibration Reject (zero-star-calibration-dump.json): Runs that fail quality gates—such as collapsed barcoding gaps (≤ 0.0), single-copy housekeeping operons lacking biosynthetic synthases, or shuffled negative controls. Logged to verify that our gates reject false positives.
• 🧪 1/5 Baseline / Review Note: Small test datasets (e.g., single-locus or 60–100 kb single-region scans), intragenomic paralog baselines, or known reference rediscoveries (MIBiG Jaccard ≥ 0.80).
• 🧪🧪 2/5 Technical Validation: Reproducible multi-locus (≥ 5 loci) or multi-clade benchmarks confirming established biological or structural parameters.
• 🧪🧪🧪 3/5 Comparative Synthesis: Cross-clade (≥ 12–25 taxa/cohorts or ≥ 250 kb) quantitative synthesis uncovering structured trade-offs across engines.
• 🧪🧪🧪🧪 4/5 Strong Emergent Signal & 🧪🧪🧪🧪🧪 5/5 Significant Biological Signal: Reserved for larger, well-powered datasets (≥ 250 kb to ≥ 1.0 Mb, ≥ 3 chained pipelines, Z ≥ 3.50–4.50 SD) with verified cross-engine concordance.

• Bacterial 16S rRNA Paralog Collapse: A single E. coli cell carries seven 16S copies (rrsA–rrsH) that differ more from each other (0.0091) than E. coli differs from Salmonella (0.0078). Rescue: Switch to single-copy rpoB/gyrB (Discovery #29) or whole-contig tetranucleotide D_KL for plasmid resistance islands.
• Vertebrate 18S Floor & Mitochondrial COX1 NUMTs: Nuclear 18S is 99.89% identical between mouse and rat, while COX1 PCR primers often co-amplify non-functional nuclear pseudogenes (NUMTs). Rescue: Pair full-length COX1 + MT-RNR2 (16S mtDNA) and filter out sequences where 2nd-codon entropy > 0.05 bits or d_N/d_S ≥ 0.10 (Discovery #30).
• Morphological Ascertainment Bias (Mk+ASC): Paleontologists typically score only anatomical traits that vary across their fossil sample. Without Lewis (2001) +ASC conditioning (dividing by 1 − L_const) and invariant-column filtering, branch lengths inflate or software rejects constant columns (Discovery #62).

All native aligners, phylogenetic tree builders, profile HMM scanners, pocket dockers, and candle-core tensor routines are written in Rust. You can run the native pipelines offline with a single binary (bio-suite.exe on Windows or ./target/release/bio-suite on Linux/macOS), while optional CLI flags (like --tree-builder iqtree or --selection paml) allow side-by-side comparisons against installed external tools.

Click 📋 Copy for Excel (TSV) on any table in a Lab Discovery report to copy tab-separated values with one number per cell—ready to paste into Excel, R, or Python pandas. Every figure and table also includes a bottom Computational Tool Attribution footer (🔧 Generated by: ...) and a Provenance Ledger linking the CLI command and downloadable JSON artifact in /labs/releases/.

We do not conduct wet-lab synthesis or clinical trials. When our pipelines analyze human knockout targets (PCSK9, ANGPTL3, HSD17B13), pharmacogenomic variants (MT-RNR2 m.1555A>G, CYP2D6), or somatic repeat expansions (HTT), our goal is to provide reproducible sequence and structural informatics that downstream wet-lab biologists and clinicians can evaluate experimentally.