Improve annotation: custom ortholog species support

[ivokwee]

Summary

• Add an Ortholog species selector to the dataset upload/compute panel, replacing the fixed human-only ortholog mapping. Users can now pick the ortholog target species (Human, Drosophila melanogaster, Arabidopsis thaliana, C. elegans, S. cerevisiae, E. coli) when computing a PGX; it's passed through to playbase::pgx.createPGX() as ortholog_species (bin/pgxcreate_op.R, upload_module_computepgx.R).
• Add an Include default genesets checkbox (include_default_gmt) so users can choose whether the built-in gene sets are merged in alongside an uploaded custom GMT file.
• Rename human_ortholog → ortholog across ~13 board modules (compare, correlation, dataview, enrichment, expression, featuremap, pathway, signature, singlecell), matching the corresponding column rename in playbase now that the ortholog target isn't hardcoded to human.
• Remove the unused "Probe annotation (alpha)" upload UI (ENABLE_ANNOT / upload_annot_table_ui) — dead code.
• Comment out the "download example GMT" link/handler (kept for reference, currently unused).
• Minor layout tweaks in the compute-options and Copilot board UI (spacing, row heights).

Compatibility: human_ortholog → ortholog column rename

The pgx$genes$human_ortholog column is renamed to pgx$genes$ortholog (it's no longer necessarily human — it tracks whatever ortholog_species was used). Any downstream code (scripts, exports, other repos) that still reads human_ortholog directly needs to be updated to ortholog.
Old PGX objects are not broken: pgx.initialize() in playbase normalizes legacy columns on load via colnames(pgx$genes) <- gsub("^human_orth","orth", colnames(pgx$genes)) (R/pgx-init.R), so existing datasets get their column renamed automatically the next time they're loaded through the platform. Anything that reads pgx$genes outside pgx.initialize() (raw .pgx files, external scripts, caches) will still see the old name until re-normalized.

Dependency

Requires the matching playbase PR: bigomics/playbase#520 (same branch name improve-annot, base edgy) — this PR calls pgx.createPGX(..., ortholog_species =, include_default_gmt =), which only exist on that playbase branch. Please merge/land both together.

Note

• dataview_module_geneinfo.R's candidate-column list now lists "ortholog" twice (a leftover of the mechanical human_ortholog→ortholog rename) — harmless but worth a small follow-up cleanup.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CpXxxrjbEhE4n4bpXbJxhK

[ESCRI11]

Nice cleanup — the rename is mechanically complete (no human_ortholog left anywhere in R code) and the two new params are threaded correctly through bin/pgxcreate_op.R. But the generalisation from "human" to "any species" is only half-done, and I think three things block the merge.

I checked each one against origin/edgy in both repos to be sure it's introduced here rather than pre-existing, and each has a runnable reprex below so you can confirm and re-check the fix.

---

1. The backward-compat claim doesn't hold (bigomics/playbase#520, R/pgx-init.R)

Both PR descriptions say old PGX objects are safe because pgx.initialize() normalizes human_ortholog → ortholog on load. The rename is real, but it runs at :234 — 42 lines after the column is read at :192 and :205. On edgy those reads named human_ortholog, which legacy objects actually have, so this is new.

Effect on every load of every existing dataset:

• genes comes out length 0, so pgx$families is built from nothing → every family falls below famsize >= 10 and is dropped, leaving only <all>. FeatureMap's family filter and pgx.getFeatureSets() go empty.
• all(is.na(...)) is TRUE, so getHumanOrtholog() fires a full network ortholog lookup and overwrites the column that was correct all along.

## playbase#520, R/pgx-init.R -- pgx.initialize() reads $ortholog at :192/:205
## but only renames the legacy column at :234, 42 lines later.

## a LEGACY pgx exactly as it sits in an existing .pgx file
genes_df <- data.frame(
  symbol         = c("Trp53", "Myc", "Jun"),
  gene_name      = c("Trp53", "Myc", "Jun"),
  human_ortholog = c("TP53", "MYC", "JUN")
)

## R/pgx-init.R:192 -- $ortholog does not exist yet, so this is length 0
genes <- ifelse(!is.na(genes_df$ortholog), genes_df$ortholog, genes_df$gene_name)
length(genes)   # expected 3
#> [1] 0

## R/pgx-init.R:205 -- so this is TRUE, and getHumanOrtholog() fires on every
## load of every legacy dataset, overwriting the column that was there all along
all(is.na(genes_df$ortholog)) || all(genes_df$ortholog == "")
#> [1] TRUE

## R/pgx-init.R:221 -- families are built from the empty `genes`
intersect(c("TP53", "MYC", "JUN"), genes)   # -> every family empty, dropped by famsize >= 10
#> character(0)

## R/pgx-init.R:234 -- the compat rename, too late to help any of the above
colnames(genes_df) <- gsub("^human_orth", "orth", colnames(genes_df))
colnames(genes_df)
#> [1] "symbol"    "gene_name" "ortholog"

Fix: move the colnames(pgx$genes) <- gsub("^human_orth","orth",colnames(pgx$genes)) line to the top of pgx.initialize(), before anything reads the column. (Also worth fixing the comment above it — it currently reads "Rename 'ortholog' to 'ortholog'".)

I also ran the same block extracted verbatim from origin/edgy vs this branch against a 300-gene mouse table seeded from real playdata::FAMILIES: before → length(genes)=300, 1 family kept; after → length(genes)=0, 0 families kept.

---

2. Two of the six dropdown values can't be resolved, and the failure throws

ORTHOLOG_SPECIES (upload_module_computepgx.R:233) goes straight to getOrtholog(target_species=), which resolves the string against the live g:Profiler organism list via .map_gprofiler_id(). Two entries don't match anything:

• "Drosphila melanogaster" — typo, should be Drosophila
• "Escherichia coli" — not in g:Profiler's core organism list at all (and HomoloGene/babelgene are eukaryote-only, so there's no fallback backend for it either)

The NULL isn't caught. .convert_orthologs() resolves target_species a second time, so .map_gprofiler_id(NULL) runs and throws; nothing between there and getGeneAnnotation() wraps it in try(), so the whole PGX computation aborts.

This is only reachable now: on edgy there is no getOrtholog(), only getHumanOrtholog(), with target_species = "hsapiens" hardcoded and no caller overriding it.

## opg#1891 upload_module_computepgx.R:233 + playbase#520 R/pgx-annot-utils.R:407
## The new "Ortholog species" dropdown feeds getOrtholog(target_species=), which
## resolves the string with .map_gprofiler_id() -- pasted verbatim below.

.map_gprofiler_id <- function(species) {
  orgs <- jsonlite::fromJSON("https://biit.cs.ut.ee/gprofiler/api/util/organisms_list")
  ## exact match
  exact.species <- paste0("^", species, "$")
  i <- which(
    grepl(exact.species, orgs$id, ignore.case = TRUE) |
      grepl(exact.species, orgs$scientific_name, ignore.case = TRUE) |
      grepl(exact.species, orgs$display_name, ignore.case = TRUE)
  )
  ## internal match
  if (length(i) == 0) {
    i <- which(grepl(species, orgs$scientific_name, ignore.case = TRUE) |
      grepl(species, orgs$display_name, ignore.case = TRUE))
  }
  if (length(i) == 0) return(NULL)
  orgs[i[1], "id"]
}

## the dropdown, verbatim from upload_module_computepgx.R:233
ORTHOLOG_SPECIES <- c(
  "Human", "Drosphila melanogaster", "Arabidopsis thaliana",
  "Caenorhabditis elegans", "Saccharomyces cerevisiae", "Escherichia coli"
)
sapply(ORTHOLOG_SPECIES, function(s) {
  id <- .map_gprofiler_id(s)
  if (is.null(id)) "NULL -- unresolved" else id
})
#>                    Human   Drosphila melanogaster     Arabidopsis thaliana 
#>               "hsapiens"     "NULL -- unresolved"              "athaliana" 
#>   Caenorhabditis elegans Saccharomyces cerevisiae         Escherichia coli 
#>       "caelegprjna13758"            "scerevisiae"     "NULL -- unresolved"

## getOrtholog() passes that NULL on to .convert_orthologs(), which resolves it
## a second time -- .map_gprofiler_id(NULL). Nothing wraps this in try().
.map_gprofiler_id(NULL)
#> Error in `grepl()`:
#> ! invalid 'pattern' argument

Also visible above: "Caenorhabditis elegans" resolves to caelegprjna13758 (WormBase ParaSite) rather than celegans — there are two exact matches and i[1] picks the wrong one.

Fix: correct the typo; drop E. coli (or wire up a backend that covers it); pin C. elegans to celegans. Ideally build the dropdown from resolvable ids rather than free-text species names, and make getOrtholog() degrade gracefully instead of throwing on an unknown target.

---

3. Seven call sites still require the ortholog to be human

These were only renamed, but each one depends on the column holding human symbols. On edgy that was guaranteed by construction; with a user-chosen species it isn't, and nothing errors — the boards just come back empty.

• pathway_plot_reactome_graph.R:94, pathway_plot_wikipathway_graph.R:105 — Reactome/WikiPathways node ids are human symbols (the wikipathway comment at :102 still says "Rename to human orthologs")
• dataview_plot_tissue.R:69 — the variable is literally hgnc.gene; GTEx tissue lookup needs HGNC
• featuremap_server.R:300, singlecell_plot_markersplot.R:126,129 — playdata::FAMILIES are human symbols matched against the ortholog column
• signature_server.R:183 — map_probes(column = "ortholog") against human signature gene lists
• compare_server.R:224 — cross-organism compare uses ortholog as the bridge; this now only works if both datasets chose the same ortholog_species, where human used to be a guaranteed common bridge

## opg#1891 -- these call sites were only renamed, but they still need the
## ortholog to be HUMAN. On edgy that held by construction (getHumanOrtholog
## only, target_species="hsapiens" hardcoded). With the new dropdown it doesn't.

human_family <- c("TP53", "MYC", "JUN")   # e.g. playdata::FAMILIES[["Kinases (KEA)"]]

genes_human <- data.frame(symbol = c("Trp53", "Myc", "Jun"),
                          ortholog = c("TP53", "MYC", "JUN"))
genes_yeast <- data.frame(symbol = c("Trp53", "Myc", "Jun"),
                          ortholog = c("YBR112C", "YGL025C", "YOR028C"))

## featuremap_server.R:300, verbatim
genes_human$symbol[match(human_family, genes_human$ortholog, nomatch = 0)]
#> [1] "Trp53" "Myc"   "Jun"
genes_yeast$symbol[match(human_family, genes_yeast$ortholog, nomatch = 0)]
#> character(0)

## pathway_plot_reactome_graph.R:94 / pathway_plot_wikipathway_graph.R:105
## -- fc names must be human symbols for the Reactome/WikiPathways node ids
sum(genes_human$ortholog %in% human_family)
#> [1] 3
sum(genes_yeast$ortholog %in% human_family)
#> [1] 0

The inverse shows up too: enrichment_table_genes_in_geneset.R:42, signature_table_genes_in_signature.R:39 and expression_table_genetable.R:107 still drop the column with if (organism == "Human") df$ortholog <- NULL — for a human dataset mapped to, say, yeast orthologs that now hides real information.

This one needs a design call rather than a patch: does the platform still need a guaranteed-human bridge column for pathways / tissue / families / cross-dataset compare? Two options I can see — keep human_ortholog alongside a new species-specific ortholog, or guard all seven sites on pgx$ortholog_species (it is stored on the object) and fall back to the old behaviour when it isn't human.

---

Non-blocking

• upload_module_computepgx.R:601 — create_ai_infographics default flipped FALSE → TRUE. Not in the PR description, and it contradicts both the tooltip ("adds extra compute time and cost") and the now-stale comment at :1240 ("defaults FALSE — image generation has real per-slot provider cost/time"). Every upload now opts into paid image generation. Intentional?
• include_default_gmt has no NULL guard. bin/pgxcreate_op.R passes params$include_default_gmt; any params.RData written before this change (a job queued across a deploy) or by an external caller yields NULL → pgx.add_GMT's if (has.px && include_default_gmt) → argument is of length zero. ortholog_species is guarded (if (is.null(...)) "Human"); worth doing the same here.
• Unchecking "Include default genesets" without uploading a GMT: playbase#520 also changed species_go to !(organism %in% c("Human","Mouse","Rat")), so on a human dataset that combination yields a completely empty pgx$GMT. A UI guard or validation message would help.
• dataview_module_geneinfo.R:70 — "ortholog", "ortholog" duplicated (you flagged this yourself).
• dataview_table_rawdata.R:119-120 — the block is now a no-op rename of ortholog to ortholog; can go.
• components/app/R/global.R:229 and dev/board.launch/global.R:171 — ENABLE_ANNOT is dead now that the UI it gated is removed.
• upload_module_computepgx.R:604-608, 1630-1638 — commented-out download_gmt link and handler; git has the history, so these can be deleted rather than kept commented.
• correlation_table_corr.R:72-75 — the if (pgx$organism %in% c("Human","human")) branch is now identical to the line above it (pre-existing, but touched here).
• No tooltip on the new "Ortholog species" select, unlike its neighbours.

---

Happy to help with any of these — 1 and 2 are small, 3 is the one I'd want your read on first.

🤖 Generated with Claude Code

https://claude.ai/code/session_01F8nidF6wudy9VzfzmpqAF1

[phisanti]

Loading a pre-PR dataset — three defects, one root cause. The shim at pgx-init.R:234 sits 42 lines below the three things that need it, so on all 48 versioned objects: genes collapses to logical(0) → every gene family is dropped (mouse 64→1, arabidopsis 4→1, measured); the :205 branch fires and pays a full remote lookup that reports total ratio mapped = 99.5718% and then discards it, because ortho$human is NULL after getHumanOrtholog renamed its columns; and :209 appends a duplicate ortholog column holding toupper(symbol) (Pzp;A2m → PZP;A2M).

The fix is ~5 lines of reordering. Move the gsub to just after the obj.needed check, drop a pre-existing ortholog column first so no duplicate can form, and fix ortho$human/ortho$humans → ortho$ortholog/ortho$orthologs. With the rename first, ortholog is already populated on every legacy object, so :205 never fires — no lookup, no clobber, families intact. Three bugs, one reordering.

Worth stressing to the CTO: toupper(symbol) reproduces the real human mapping in 94% of mouse rows but 0% of C. elegans, 3.1% of arabidopsis, 8.3% of fly. Testing on mouse would have hidden it almost perfectly.

On recording the model organism — three answers:

• Source organism: yes. pgx$organism (pgx-compute.R:422), on all 52, mirrored in datasets-info.csv. But it's stored un-normalised — the fleet has "Human", "human", "Homo sapiens", "Mouse", "mouse", "Mus musculus" as distinct strings. That's exactly what breaks species_go <- !(pgx$organism %in% c("Human","Mouse","Rat")) at pgx-compute.R:1170: "Homo sapiens" takes the TRUE branch and pays a full AnnotationHub GO download.
• Ortholog target: new, and incomplete. The PR adds pgx$ortholog_species at pgx-compute.R:430 — but 0/52 legacy objects have it, pgx.initialize() never sets it, and only two places read it (both inside create/GMT). It's not in datasets-info.csv and not attached to the gene table.
• The gap that matters: before this PR the column name was the guarantee — human_ortholog meant human. After the rename nothing in the codebase knows what pgx$genes$ortholog contains, and the nine sites that index human-keyed resources from it (CMAP, GENE_SUMMARY, FAMILIES ×2, sigdb, TileDB ×3, playdata GMT, collapse_by_humansymbol) never consult ortholog_species.

[ivokwee]

guys. about the ortholog above, it seems the "best" solution for now is to keep both $human_ortholog and the new $ortholog  in the gene annotation? In a later PR we can think better how to really handle non-human-ortholog cases.  What do you think?

[ivokwee]

Thanks both for the thorough review — this took a few passes to get right. Summary of what changed, mapped to your findings, plus a request for a critical second look before merge.

Issue 1 — init-order bug (ESCRI11 + phisanti's live-tested repro)

Fixed in playbase@36084f573. The human_ortholog→ortholog rename now runs before anything reads the column, guards against a duplicate ortholog column, and fixes the ortho$human/ortho$humans → ortho$ortholog/ortho$orthologs typo phisanti found (confirmed via reprex: legacy 3-gene mouse table now keeps length(genes)==3, no spurious re-lookup, full family intersect).

playbase@71a508d6e also backfills pgx$ortholog_species to "Human" on load when missing — phisanti flagged this was set at compute time only, 0/52 legacy objects had it.

Issue 2 — dropdown typo / unresolvable species / crash on unknown species

Fixed in playbase@36084f573 + omicsplayground@71aa48af2. Typo corrected (Drosphila→Drosophila), Escherichia coli dropped from the dropdown (confirmed against the live g:Profiler organism list — not present, and the fallback backends are eukaryote-only), C. elegans now resolves to the core celegans id instead of a WormBase ParaSite strain (fixed the ambiguous-match tie-break in .map_gprofiler_id), and getOrtholog()/.map_gprofiler_id() degrade gracefully on an unresolvable species instead of throwing and aborting the whole PGX computation.

Non-blocking list

All addressed in omicsplayground@71aa48af2 + playbase@36084f573: reverted the accidental create_ai_infographics default flip, added the include_default_gmt NULL guard, added a warning when neither default nor custom genesets would be included, removed the dead ENABLE_ANNOT flag / no-op rename blocks / commented-out download_gmt handler / redundant organism branch, and added a tooltip to the ortholog species select.

Issue 3 — the "which sites need which column" question

We went with Option A (keep a guaranteed-human human_ortholog bridge column alongside the new species-configurable ortholog) rather than gating every consumer on pgx$ortholog_species — the latter turned out to be a much bigger refactor than originally scoped (see below), so we deferred it as a tracked follow-up rather than block this merge on it.

playbase@e13dcc06e adds human_ortholog back, computed alongside ortholog in getGeneAnnotation() (skips the extra remote lookup when ortholog_species is already Human, so the default path pays nothing extra). Every human-keyed consumer we could find is repointed at it: default genesets (GSETxGENE), TileDB (3 sites), GENE_SUMMARY/GeneCards lookups, CMAP/L1000 drug connectivity, sigdb connectivity scores, collapse_by_humansymbol(), pgx.getFeatureSets(), WGCNA preservation enrichment, rename_by/rename_by2 fallback chains, and — on the omicsplayground side (omicsplayground@76338ae88) — Reactome/WikiPathways, GTEx tissue, FeatureMap, singlecell markers, custom signature search, and the cross-dataset compare bridge.

Turns out the affected-site list is meaningfully bigger than either of your reviews scoped (phisanti's 9-site list plus TileDB, pgx.getFeatureSets(), WGCNA enrichment, and a couple of rename_by fallback chains) — that's exactly why we didn't attempt the full per-site ortholog_species gating in this pass. Filed as #1892 for follow-up, along with bigomics/playbase#522 for the separate pgx$organism normalization issue phisanti flagged ("Human"/"human"/"Homo sapiens" etc. as distinct strings breaking species_go).

The species-configurable ortholog column stays wired to display purposes (raw-data table, gene-info panel) and its one genuine analytical consumer: pgx.add_GMT()'s species_go block, which fetches GO terms for pgx$ortholog_species and maps them back via auto column-detection.

Bug found during manual verification

While browser-testing the fix (installed the local playbase build, drove the app through Cluster Features / Test Signatures / Pathway Analysis / Drug Connectivity / Cell Profiling / PCSF / Standard WGCNA against the example dataset), hit a real crash: Error: 'map_probes' is not an exported object from 'namespace:playbase'. Turned out map_probes() lost its @export tag in the original 7b74313b7 commit — confirmed via git diff origin/edgy -- NAMESPACE that it was exported there and silently dropped when NAMESPACE got regenerated. Fixed in playbase@cfb84c207. Not something either of your static reviews would have caught — only surfaced by actually running the app.

Request: critical second check

I'd like a skeptical second pass specifically on:

1. The human-keyed vs. species-specific split itself — did we get every site right? The determination for each site was "does it match against a resource that's fundamentally human-keyed" (GeneCards, GENE_SUMMARY, GSETxGENE, Reactome/WikiPathways, GTEx, playdata::FAMILIES, L1000/CMAP) vs. genuinely wanting the user's chosen species (only pgx.add_GMT()'s species_go block, as far as we could tell). If there's a site we miscategorized, it'll silently produce wrong-but-plausible-looking results rather than an error.
2. Cross-organism compare_server.R — reverted to human_ortholog for the bridge column, but I didn't have a second-organism dataset available to actually exercise that code path in the browser session. Only the same-organism path got real verification.
3. The dual-lookup cost/correctness tradeoff — getGeneAnnotation() now does a second getOrtholog() call for human_ortholog whenever ortholog_species != "Human". Worth checking this doesn't quietly double compute time/API load in ways that matter for large datasets.
4. Anything in playbase@e13dcc06e's file list I should have caught but didn't (it touched 12 files across the codebase — happy to walk through the reasoning per-file if useful).

🤖 Generated with Claude Code

https://claude.ai/code/session_01NdKjbG6mq44txcD2jbmgid

[ESCRI11]

@ivokwee last comment sounds good; can you actually push the commit?

[ESCRI11]

also please target devel branch; edgy is a thing of the past now